Hi Nicolas,

I have not been able to reproduce the issue with your poc but it sounds
like the issue that was fixed in 7.19.0.2. If so the only affected
version was 7.19.0.1.

Release
announcement: http://curl.haxx.se/mail/curlpython-2013-10/0000.html

Fix:
https://github.com/pycurl/pycurl/commit/b01a04fbd7797c70c5397a1b04abb09e6e4c8a36

As a side note, pycurl currently requires libcurl 7.19.0 or better.

Oleg

On Mon, 18 Nov 2013 16:27:38 +0100
Nicolas Collignon <nicolas.collignon_at_synacktiv.com> wrote:

> Hi,
>
> I've identified a bug in PyCURL library concerning "setopt +
> pycurl.POSTFIELDS".
>
> client.setopt(pycurl.POSTFIELDS, data)
>
> The bug may be classified as a "use-after-free". The "data" pointer
> (post fields data) is passed as-is to the libcurl C API without
> increasing the associated Python object reference.
>
>
> file pycurl.c, line 1704:
> -----------------------------------------
> case CURLOPT_POSTFIELDS:
> if (PyString_AsStringAndSize(obj, &str, &len) != 0)
> return NULL;
> /* automatically set POSTFIELDSIZE */
> if (len <= INT_MAX) {
> res = curl_easy_setopt(self->handle,
> CURLOPT_POSTFIELDSIZE, (long)len);
> } else {
> res = curl_easy_setopt(self->handle,
> CURLOPT_POSTFIELDSIZE_LARGE, (curl_off_t)len);
> }
> [...]
> /* Call setopt */
> res = curl_easy_setopt(self->handle, (CURLoption)option, str);
> -----------------------------------------
>
> Therefore as soon as the "data" object is dereferenced by the Python
> code, its memory allocation may be reused by any Python code. It
> results in garbage being sent to the server.
> The bug is quite easy to reproduce with the provided python script.
>
> I confirmed that by increasing the python object reference count
> (variable "obj"), the problem is solved however it leads to a memory
> leak since there is no way of tracking the python object usage
> without adding a curl-specific garbage collector.
>
> The problem may be solved with 3 different approaches:
>
> 1) keep track of the data object and unref the object when the curl
> object is destructed or unset (which seems like a bad idea since the
> POST size is set at the same time as the POST data. the POST size may
> change if the application changes the object content).
>
> 2) copy the data object and free it when the curl object is
> destructed or unset (require adding a new field to PyCURL object).
>
> 3) ask libcurl C API to copy the data by converting POSTFIELDS to
> COPYPOSTFIELDS at the Python API level (however the COPYPOSTFIELDS
> option has been merged in libcurl 7.18.1).
>
> By the way, this issue is solved by using approaches 2 + 3 in
> php-curl.
>
> So i have 2 questions:
> - Do you agree with my analysis ?
> - Do you want me to "pull-request" the code that perform POSTFIELDS
> -> COPYPOSTFIELDS conversion for libcurl >= 7.18.1 and keep the bug
> for libcurl < 7.18.1 ?
>
> -- Nicolas Collignon

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-python
Received on 2013-11-18