cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Bug in libcurl in multithreaded program

From: Avery Fay <avery_fay_at_symantec.com>
Date: Tue, 4 Jun 2002 10:32:06 -0400

Alright, here's sample program with most everything unneeded removed.

On linux with an http url (server is localhost running apache):

Retrieved 536870912B in 7 seconds at a rate of 76695844B/s

On linux with an ftp url (server is localhost running wu-ftpd in inetd
mode):

Retrieved 536870912B in 7 seconds at a rate of 76695844B/s

Note that the above numbers are the same because I'm only using second
resolution.

On windows with an http url (server is localhost running IIS5):

Retrieved 534768128B in 12 seconds at a rate of 44564010B/s

On windows with an ftp url (server is localhost running IIS5):

Retrieved 534768128B in 619 seconds at a rate of 863922B/s

Note that the different in bytes retrieved on windows and linux is due to a
slightly different test file.

Also, I have tried the same test with IIS5 on a different machine and
results are similar (but a lot slower since the program is limited by 100Mb
ethernet).

Program is attached.

Avery Fay

(See attached file: main.c)

|---------+---------------------------->
| | Daniel Stenberg |
| | <daniel_at_haxx.se> |
| | |
| | 06/04/2002 07:45 |
| | AM |
| | |
|---------+---------------------------->
>------------------------------------------------------------------------------------------------------------------------------|
  | |
  | To: Avery Fay <avery_fay_at_symantec.com> |
  | cc: libcurl Mailing list <curl-library_at_lists.sourceforge.net> |
  | Subject: Re: Bug in libcurl in multithreaded program |
>------------------------------------------------------------------------------------------------------------------------------|

On Mon, 3 Jun 2002, Avery Fay wrote:

> On windows: HTTP and HTTPS work fine regardless of number of threads. FTP
> is ok until about 4 threads or so and then everything grinds nearly to a
> halt.
>
> Anyone have any ideas about this? Has anyone gotten a mulithreaded
libcurl
> application on windows doing FTP transfers to preform well?

I haven't heard anyone with these kinds of problems, nor have I ever tried
this myself.

> If there are no suggestions, I'll see if I can whip up a little sample
> program that causes the above mentioned slow transfers.

Would be very useful. Would it be possible to run the program using kind
kind
of profiling or something to see where all the time is spent? Using a
network
analyzer here could be helpful too, to see if the problem is incoming or
outgoing data or whatever.

> On a different note, I'm interested in using the multi interface but I
have
> a question. If I set CURLOPT_WRITEFUNCTION and it is called by libcurl
how
> do I know which connection it corresponds to? In a multithreaded
> application I can just do a lookup in a table by the thread ID, but
> obviously in a situation with multiple connections per thread I can't do
> that.

You set a custom pointer with CURLOPT_WRITEDATA to contain whatever you
like.
Most likely you make each easy handle's "writedata" point to a unique
instance of a particular struct that you init accordingly. Then the
callback
function can easily read from that to identify itself.

--
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/

_______________________________________________________________

Don't miss the 2002 Sprint PCS Application Developer's Conference
August 25-28 in Las Vegas -- http://devcon.sprintpcs.com/adp/index.cfm

  • application/octet-stream attachment: main.c
Received on 2002-06-04