Daniel Stenberg (daniel-curl_at_haxx.se) wrote:
>
> On Mon, 21 Feb 2005, Nayeem Syed wrote:
>
> >> Do you have any curl-related logging enabled that shows what curl does or
> >> doesn't do when it fails?
> >
> > in the php class we made we make a log of what curl returns and we get a
> > blank string logged when this happens. i dont know of any ways to log curl
> > at a lower level.
>
> In the PHP/CURL binding the abilities are very weak and if you have no other
> means, it'll be very hard to understand you're issues. Unfortunately, there
> doesn't seem to be anyone interested enough to add proper tracing ability to
> it.

>
> If you can't use the command line's --trace-ascii option or set the
> CURLOPT_DEBUGFUNCTION in the library, it will be next to impossible to solve
> this.
>
I will check if theres a way in the curlmulti in php to enable
CURLOPT_DEBUGFUNCTION. Is there any way we can use an external curl to fetch
the documents? We just need an ability to fetch multiple documents
simultaneously. Can it be done using plain sockets?

> >> Do you need 200 threads to make it fail or does it happen with less as
> >> well?
> >
> > no sometimes it happens with much less threads. but an apache restart seems
> > to fix it when it happens with less processes. today we had two occurances
> > of this during the day, both times the server hit 190 simultaneous users.
> > earlier in the day we had over 500 simultaneous users but had no problems
> > with curl.
>
> Can it be an out-of-memory situation? 500 x 9 connections is gonna use a whole
> lot of memory, especially if you download data to memory...
I checked the server's status when we are getting blanks and the server is not
heavily loaded and still has free memory left, but I will check again next
time. Out of 4GB RAM we seem to have 3.9GB used pretty much all the time,
theres a lot of swap memory free. is it usual?
>
> >> How many sockets/connections can you handle in a single process (all
> >> threads taken together)?
> >
> > i am not sure where I should look for it. We are running apache2 and there
> > doesnt seem to be any httpd.conf on it.
>
> If you're running apache2 with forks instead of threads, then it shouldn't be
> a problem and since you say you _can_ use 500 x 9 without problems it proves
> that this isn't an issue.

ok we changed apache to use forks instead of threads.

>
> > does curl create processes or threads?
>
> Neither. It doesn't create processes nor threads.

then how does it fetch multiple documents simultaneously?

>
> --
> Daniel Stenberg -- http://curl.haxx.se -- http://daniel.haxx.se
> Dedicated custom curl help for hire: http://haxx.se/curl.html
>
Received on 2005-02-22