cURL / Mailing Lists / curl-library / Single Mail

curl-library

libcurl: SFTP Sample Code with LARGE operations

From: Paul Romero <paulr_at_rcom-software.com>
Date: Sat, 03 Jul 2010 17:37:30 -0700

Dear User Group:

I could not get an SFTP test program--generated with curl,
to work correctly in my environment which is a Debian Linux
i686 Lenny machine with Pentium 4 CPU. It also has version
4.1.2 of the GCC GNU compiler installed.

I was able to make it work by replacing the following
"LARGE" operations with non-LARGE operations: CURLOPT_INFILE_SIZE_LARGE,

and CURLOPT_RESUME_FROM_LARGE.

I don't think this should be necessary and may have used the
wrong compile flags. However, the E-Mail floating around
suggests it is problem with the configure. In particular,
it appears that libcurl was compiled for a 64 bit machine.

Is that true, and if so, what is the correct way to run
the configure for 32 bit machines.

The sections with the relevant information are as follows:

A - How the test program was generated.
B - How the test program was compliled and linked.
C - The way configure was executed.
D - A suspicious section of the config.log
E - The orginial test program.

Best Regards,

Paul R.

A) The application program was generated as follows and
   it the curl command worked correctly.

./curl --key ~paulr/.ssh/id_rsa --pubkey ~paulr/.ssh/id_rsa.pub \
 -u paulr: sftp://build:22/home/paulr/x --libcurl ex.c

B) It was compiled and linked as follows:

 cc -c -Wall -I/home/paulr/libcurl/curl-7.21.0/include ex.c
 cc ex.o -L/home/paulr/libssh2/libssh2-1.2.6/src/.libs \
  -L/home/paulr/libcurl/curl-7.21.0/lib/.libs -lcurl -lidn -lssh2 \
  -lssl -lcrypto -lrt -lz

C) This is way I executed configure.

env PKG_CONFIG_PATH=/home/paulr/libssh2/libssh2-1.2.6 \
./configure --with-libssh2 \
CFLAGS="-I/home/paulr/libssh2/libssh2-1.2.6/include" \
LDFLAGS="-L/home/paulr/libssh2/libssh2-1.2.6/src/.libs"

D) This is a snippet from the config.log file generated
by configure. While my i686 machine is not a 64 bit machine
I checked the size of off_t and discovered it is an 8 byte variable.

configure:5293: checking for 64-bit curl_off_t data type
configure:5343: gcc -c -I/home/paulr/libssh2/libssh2-1.2.6/include
conftest.c >&5
configure:5343: $? = 0
configure:5370: gcc -E conftest.c
configure:5370: $? = 0
configure:5405: result: int64_t
configure:5640: checking size of curl_off_t
configure:5642: result: 8
configure:5645: checking formatting string directive for curl_off_t
configure:5685: result: "lld"
configure:5688: checking formatting string directive for unsigned
curl_off_t
configure:5690: result: "llu"
configure:5694: checking constant suffix string for curl_off_t
configure:5761: gcc -c -I/home/paulr/libssh2/libssh2-1.2.6/include
conftest.c >&5
configure:5761: $? = 0
configure:5775: result: LL
configure:5777: checking constant suffix string for unsigned curl_off_t
configure:5779: result: ULL

E) This is the unmodified sample code generated with the curl command
which fails with error 36--resume error, and the following diagnostic:
Offset (577732303141208064) was beyond file size (12).

/********* Sample code generated by the curl command line tool
**********
 * Lines with [REMARK] below might need to be modified to make this code

 * usable. Add error code checking where appropriate.
 * Compile this with a suitable header include path. Then link with
 * libcurl.
 * If you use any *_LARGE options, make sure your compiler figure
 * out the correct size for the curl_off_t variable.
 * Read the details for all curl_easy_setopt() options online on:
 * http://curlm.haxx.se/libcurl/c/curl_easy_setopt.html
 ************************************************************************/

#define _FILE_OFFSET_BITS 64 /* for pre libcurl 7.19.0 curl_off_t magic
*/
#include <curl/curl.h>

int main(int argc, char *argv[])
{
  CURLcode ret;
  CURL *hnd = curl_easy_init();
  /* curl_easy_setopt(hnd, CURLOPT_WRITEDATA, 0xbfda05e8); [REMARK] */
  /* curl_easy_setopt(hnd, CURLOPT_WRITEFUNCTION, 0x804e0d0); [REMARK]
*/
  /* curl_easy_setopt(hnd, CURLOPT_READDATA, 0xbfda0628); [REMARK] */
  /* curl_easy_setopt(hnd, CURLOPT_READFUNCTION, 0x804e060); [REMARK] */

  /* curl_easy_setopt(hnd, CURLOPT_SEEKDATA, 0xbfda0628); [REMARK] */
  /* curl_easy_setopt(hnd, CURLOPT_SEEKFUNCTION, 0x804e020); [REMARK] */

  curl_easy_setopt(hnd, CURLOPT_INFILESIZE_LARGE, -1);
  curl_easy_setopt(hnd, CURLOPT_URL, "sftp://build:22/home/paulr/x");
  curl_easy_setopt(hnd, CURLOPT_PROXY, NULL);
  curl_easy_setopt(hnd, CURLOPT_NOPROGRESS, 1);
  curl_easy_setopt(hnd, CURLOPT_HEADER, 0);
  curl_easy_setopt(hnd, CURLOPT_FAILONERROR, 0);
  curl_easy_setopt(hnd, CURLOPT_UPLOAD, 0);
  curl_easy_setopt(hnd, CURLOPT_DIRLISTONLY, 0);
  curl_easy_setopt(hnd, CURLOPT_APPEND, 0);
  curl_easy_setopt(hnd, CURLOPT_NETRC, 0);
  curl_easy_setopt(hnd, CURLOPT_FOLLOWLOCATION, 0);
  curl_easy_setopt(hnd, CURLOPT_UNRESTRICTED_AUTH, 0);
  curl_easy_setopt(hnd, CURLOPT_TRANSFERTEXT, 0);
  curl_easy_setopt(hnd, CURLOPT_USERPWD, "paulr:");
  curl_easy_setopt(hnd, CURLOPT_PROXYUSERPWD, NULL);
  curl_easy_setopt(hnd, CURLOPT_NOPROXY, NULL);
  curl_easy_setopt(hnd, CURLOPT_RANGE, NULL);
  /* curl_easy_setopt(hnd, CURLOPT_ERRORBUFFER, 0xbfda046c); [REMARK] */

  curl_easy_setopt(hnd, CURLOPT_TIMEOUT, 0);
  curl_easy_setopt(hnd, CURLOPT_REFERER, NULL);
  curl_easy_setopt(hnd, CURLOPT_AUTOREFERER, 0);
  curl_easy_setopt(hnd, CURLOPT_USERAGENT, "curl/7.21.0
(i686-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8g zlib/1.2.3.3
libidn/0.6.5 libssh2/1.2.6");
  curl_easy_setopt(hnd, CURLOPT_FTPPORT, NULL);
  curl_easy_setopt(hnd, CURLOPT_LOW_SPEED_LIMIT, 0);
  curl_easy_setopt(hnd, CURLOPT_LOW_SPEED_TIME, 0);
  curl_easy_setopt(hnd, CURLOPT_MAX_SEND_SPEED_LARGE, 0);
  curl_easy_setopt(hnd, CURLOPT_MAX_RECV_SPEED_LARGE, 0);
  curl_easy_setopt(hnd, CURLOPT_RESUME_FROM_LARGE, 0);
  curl_easy_setopt(hnd, CURLOPT_COOKIE, NULL);
  curl_easy_setopt(hnd, CURLOPT_HTTPHEADER, NULL);
  curl_easy_setopt(hnd, CURLOPT_SSLCERT, NULL);
  curl_easy_setopt(hnd, CURLOPT_SSLCERTTYPE, NULL);
  /* curl_easy_setopt(hnd, CURLOPT_SSLKEY, 0x8079490); [REMARK] */
  curl_easy_setopt(hnd, CURLOPT_SSLKEYTYPE, NULL);
  curl_easy_setopt(hnd, CURLOPT_KEYPASSWD, NULL);
  curl_easy_setopt(hnd, CURLOPT_SSH_PRIVATE_KEYFILE,
"/home/paulr/.ssh/id_rsa");
  curl_easy_setopt(hnd, CURLOPT_SSH_PUBLIC_KEYFILE,
"/home/paulr/.ssh/id_rsa.pub");
  curl_easy_setopt(hnd, CURLOPT_SSH_HOST_PUBLIC_KEY_MD5, NULL);
  curl_easy_setopt(hnd, CURLOPT_SSL_VERIFYHOST, 2);
  curl_easy_setopt(hnd, CURLOPT_SSH_KNOWNHOSTS,
"/home/paulr/.ssh/known_hosts");
  curl_easy_setopt(hnd, CURLOPT_MAXREDIRS, 50);
  curl_easy_setopt(hnd, CURLOPT_CRLF, 0);
  curl_easy_setopt(hnd, CURLOPT_QUOTE, NULL);
  curl_easy_setopt(hnd, CURLOPT_POSTQUOTE, NULL);
  curl_easy_setopt(hnd, CURLOPT_PREQUOTE, NULL);
  curl_easy_setopt(hnd, CURLOPT_HEADERDATA, NULL);
  curl_easy_setopt(hnd, CURLOPT_COOKIEFILE, NULL);
  curl_easy_setopt(hnd, CURLOPT_COOKIESESSION, 0);
  curl_easy_setopt(hnd, CURLOPT_SSLVERSION, 0);
  curl_easy_setopt(hnd, CURLOPT_TIMECONDITION, 0);
  curl_easy_setopt(hnd, CURLOPT_TIMEVALUE, 0);
  curl_easy_setopt(hnd, CURLOPT_CUSTOMREQUEST, NULL);
  /* curl_easy_setopt(hnd, CURLOPT_STDERR, 0xb7f09580); [REMARK] */
  curl_easy_setopt(hnd, CURLOPT_HTTPPROXYTUNNEL, 0);
  curl_easy_setopt(hnd, CURLOPT_INTERFACE, NULL);
  curl_easy_setopt(hnd, CURLOPT_KRBLEVEL, NULL);
  curl_easy_setopt(hnd, CURLOPT_TELNETOPTIONS, NULL);
  curl_easy_setopt(hnd, CURLOPT_RANDOM_FILE, NULL);
  curl_easy_setopt(hnd, CURLOPT_EGDSOCKET, NULL);
  curl_easy_setopt(hnd, CURLOPT_CONNECTTIMEOUT, 0);
  curl_easy_setopt(hnd, CURLOPT_ENCODING, NULL);
  curl_easy_setopt(hnd, CURLOPT_FTP_CREATE_MISSING_DIRS, 0);
  curl_easy_setopt(hnd, CURLOPT_IPRESOLVE, 0);
  curl_easy_setopt(hnd, CURLOPT_FTP_ACCOUNT, NULL);
  curl_easy_setopt(hnd, CURLOPT_IGNORE_CONTENT_LENGTH, 0);
  curl_easy_setopt(hnd, CURLOPT_FTP_SKIP_PASV_IP, 0);
  curl_easy_setopt(hnd, CURLOPT_FTP_FILEMETHOD, 0);
  curl_easy_setopt(hnd, CURLOPT_FTP_ALTERNATIVE_TO_USER, NULL);
  curl_easy_setopt(hnd, CURLOPT_SSL_SESSIONID_CACHE, 1);
  /* curl_easy_setopt(hnd, CURLOPT_SOCKOPTFUNCTION, 0x804d8a0); [REMARK]
*/
  /* curl_easy_setopt(hnd, CURLOPT_SOCKOPTDATA, 0xbfda0190); [REMARK] */

  curl_easy_setopt(hnd, CURLOPT_POSTREDIR, 0);
  ret = curl_easy_perform(hnd);
  curl_easy_cleanup(hnd);
  return (int)ret;
}
/**** End of sample code ****/

--
Paul Romero
RCOM Communications Software
Phone/Fax: (510)339-2628
E-Mail: paulr_at_rcom-software.com
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2010-07-04