cURL / Mailing Lists / curl-library / Single Mail

curl-library

non-blocking FTP

From: Jeff Weber <jweber_at_amsc.com>
Date: Tue, 13 May 2008 17:13:49 -0500

I need to make nonblocking uploads of multiple files using either the
scp:// or ftp:// protocols. In some cases, the code must upload via
scp://, in other cases ftp://.

Given the comment in the libcurl-multi.3 -- man page, that ftp://
still contains some blocking code, even in the multi interface, I
decided to use the easy interface, and split my transfer into
pthreads. Most of the code is in the parent thread, and only the
curl_easy_perform() in a child thread. If the child thread "takes too
long", I cancel it. Regardless, the parent always calls
curl_easy_cleanup(), curl_global_cleanup(). Unfortunately, cancelling
curl_easy_perform() leads to memory leaks. Details below for a test
uploading to ftp:// and scp:// that blocked.

Are there any other suggestions for implementing nonblocking
ftp:// transfers?

I'm using curl-7.18.1, libssh2-0.18.

    TIA,
        Jeff

canceled upload to ftp:// valgrind leak report:
==00:00:00:22.195 4112== 72 bytes in 2 blocks are definitely lost in loss record
 2 of 4
 ==00:00:00:22.195 4112== at 0x401B504: malloc (vg_replace_malloc.c:149)
 ==00:00:00:22.196 4112== by 0x804E7A9: curl_domalloc (memdebug.c:136)
 ==00:00:00:22.196 4112== by 0x807E4CF: ftp_statemach_act (ftp.c:2731)
 ==00:00:00:22.196 4112== by 0x807EBC9: ftp_easy_statemach (ftp.c:2984)
 ==00:00:00:22.196 4112== by 0x807EE18: ftp_connect (ftp.c:3100)
 ==00:00:00:22.196 4112== by 0x805750B: Curl_protocol_connect (url.c:2795)
 ==00:00:00:22.196 4112== by 0x80599A4: SetupConnection (url.c:4281)
 ==00:00:00:22.196 4112== by 0x8059AD7: Curl_connect (url.c:4355)
 ==00:00:00:22.196 4112== by 0x8062FCE: connect_host (transfer.c:2275)
 ==00:00:00:22.196 4112== by 0x806314D: Curl_perform (transfer.c:2354)
 ==00:00:00:22.196 4112== by 0x804DE90: curl_easy_perform (easy.c:509)
 ==00:00:00:22.197 4112== by 0x804C320: AMSC::CFileXferImple::blockingUpload(void*) (amscfilexfer.cpp:435)
 ==00:00:00:22.197 4112== by 0x415D4EA: pthread_start_thread (in /lib/libpthre

canceled upload to scp:// valgrind leak report:
==00:00:00:18.169 3440== 16,958 (6,420 direct, 10,538 indirect) bytes in 1 block
s are definitely lost in loss record 5 of 6
==00:00:00:18.169 3440== at 0x401B504: malloc (vg_replace_malloc.c:149)
==00:00:00:18.170 3440== by 0x804E73D: curl_domalloc (memdebug.c:136)
==00:00:00:18.170 3440== by 0x806D748: libssh2_malloc (ssh.c:281)
==00:00:00:18.170 3440== by 0x8087983: libssh2_session_init_ex (session.c:448
)
==00:00:00:18.170 3440== by 0x8071269: ssh_connect (ssh.c:2025)
==00:00:00:18.170 3440== by 0x805749F: Curl_protocol_connect (url.c:2795)
==00:00:00:18.170 3440== by 0x8059938: SetupConnection (url.c:4281)
==00:00:00:18.170 3440== by 0x8059A6B: Curl_connect (url.c:4355)
==00:00:00:18.170 3440== by 0x8062F62: connect_host (transfer.c:2275)
==00:00:00:18.170 3440== by 0x80630E1: Curl_perform (transfer.c:2354)
==00:00:00:18.170 3440== by 0x804DE24: curl_easy_perform (easy.c:509)
==00:00:00:18.171 3440== by 0x804C2B6: AMSC::CFileXferImple::blockingUpload(void*) (amscfilexfer.cpp:427)
==00:00:00:18.171 3440== by 0x415D4EA: pthread_start_thread (in /lib/libpthre
ad-0.10.so)
Received on 2008-05-14