On Sat, Aug 10, 2013 at 7:28 AM, Daniel Stenberg <daniel_at_haxx.se> wrote:
> While I think that's a rather sever limitation of a timer functionality,
> calling curl_multi_socket_action() a little bit too often won't cause any
> harm at all. It'll just not be as optimal as otherwise.

It is definitely an annoying limitation. I'll work on wrapping around
one of the two methods a bit later to see if I can get it behaving
properly, but until then, it's good to know that it isn't hurting
anything. I know that Apple provides this functionality in the
higher-level NSOperation class, but in the interests of keeping this
simple/lightweight, I'll try working it out on my own.

> The default name resolver is synchronous so it'll block until done so it
> doesn't use a timer.

That would be why, then! :)

> A very strange effect. I won't rule out the risk that this is actually a
> libcurl bug so that perhaps you've not been told to wait for the correct
> socket action and the 15 minutes timeout that occurs is probably something
> the server end has implemented...

I checked the manual for pure-ftpd, and 15 minutes is indeed the
default idle timeout, so libcurl is definitely making no attempt to
send any data (or to tell me to do so) after opening the second
socket.

> I'll see if I can get some time soon to write an event-based ftp upload
> application in plain C with something like libevent and verify how that
> works. Unfortunately we don't have any event-based tests in the test suite.
> I have that on my TODO for the coming months but I can't promise anything.
> (And I'll bring this up in a separate topic on the list in a short while.)

Ok, and I'll keep bashing on it from this side as well. I poked around
in the libcurl source a little bit the other day, but it'll take a
while to become familiar enough with it to really be able to make a
dent in it. Also, I think I'll try reversing the polarity (heh) and
see if a download completes successfully. That will at least give some
direction to my poking.

>> Finally, I'm seeing a very odd and extremely intermittent issue: every
>> once in a while, when I perform the upload (with no variation in the
>> process, at least that *I'm* causing), rather than just getting the normal
>> messages about reading on a socket, I see a massive amount of writes.
>
>
> Are those happening when libcurl has told the app to wait for a socket
> writability and the socket turns writable? Or who's telling that socket
> write thing? It seems wrong to wait for a writable socket when libcurl waits
> for a command response...

As far as I can tell, libcurl is instructing me to watch that socket
just like the others. It happens on the same socket as the normal
exchange of commands, so I'm not sure what's prompting it to register
for write availability. On a normal run it only registers for
CURL_POLL_IN, but on the anomolous runs it also registers for
CURL_POLL_OUT (and this happens before registering for CURL_POLL_IN,
if that makes a difference). The writes start immediately when the
socket is opened and the Curl_addHandleToPipeline happens, before the
welcome message has even arrived from the server, so something seems
to be getting very confused.

> There's a chance that the problems you see and have are depending on timing
> so perhaps every once in a while it'll act different due to the timing and
> things happening slighlty slower or faster or in another order...

Timing was definitely my first thought, just because there seems to be
no rhyme or reason as to when it happens. I think I might try to set
up some packet-sniffing software and see if I can't get a handle on
exactly what is getting sent where when it happens. I'll also switch
on verbose logging on my test FTP server to see if that has any clues.

As I was finishing up the paragraph on the strange writes, I noticed
something: when it opens the data connection to the server, it's
registering for CURL_POLL_IN. Shouldn't this be CURL_POLL_OUT for an
upload? That would explain why it stops, it's waiting for data to
arrive when none ever will, instead of trying to send it. I verified
that this isn't me mis-interpreting the value passed to
socket_callback as the action somehow, it is in fact what libcurl is
asking for.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-08-10