cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: how to reduce the number of TCP session generated by an FTP download using libcurl

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Sun, 23 Oct 2016 23:28:23 +0200 (CEST)

On Sun, 23 Oct 2016, Maurice Lemarin wrote:

> I need to read portions of a remote large file using FTP and seek at a
> different offset in that file for each new portion read.
>
> The easy interface is not adapted to that, so I used the multi interface

I don't see how the two interfaces differ in any significant way for this sort
of operation.

> when I capture the network exchanges, I see of course a lot of TCP data
> session as expected, but also a lot of FTP control session one after the
> other, while a single one should be sufficient to my understanding.

One control connection should be enough if you're doing the requests serially
and if the server keeps the connection open.

> If I keep the easy handle added in the multi handle while I modify the
> CURLOPT_RESUME_FROM_LARGE option on the easy handle, the new offset for the
> next read is not taken into account. So what I do is to first to
> curl_multi_remove_handle() then modify the easy handle with
> CURLOPT_RESUME_FROM_LARGE then curl_multi_add_handle() back the easy handle.

It is documented that you should set the options before you add the handle so
I don't think this contradicts the documentation. Until you remove the handle,
it is still sort of stuck on the first transfer and you want to change the
options for the next transfer.

> Am I doing something wrong? Is there a way to keep the same control session
> by doing another way or using options I missed seeing on the easy or multi
> handles?

libcurl tries to find an existing connection to use from the connection cache
whenever it is about to setup a connection so if there's an idle FTP control
connection there it should use it. You need to explictily tell libcurl not to,
to prevent that from happening. Or the server perhaps closes the connection.

If you enable CURLOPT_VERBOSE, libcurl will actually tell you if it leaves the
connection open when its done and it will show clearly when it needs to create
a new connection or when it reuses an existing one for new transfers.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-library
Etiquette:  https://curl.haxx.se/mail/etiquette.html
Received on 2016-10-23