Re: curl post fails.

From: jian he via curl-library <curl-library_at_lists.haxx.se>
Date: Tue, 4 Oct 2022 12:13:34 +0530

Hi.


On Mon, Oct 3, 2022 at 3:31 AM Ray Satiro via curl-library <
curl-library_at_lists.haxx.se> wrote:

> On 10/2/2022 2:09 PM, jian he via curl-library wrote:
>
> Follow this url (https://www.backblaze.com/b2/docs/b2_upload_file.html)
> The command line way will work.
>
>> FILE_TO_UPLOAD=/home/jian/helloc/curl_1.c
>> MIME_TYPE=text/plain
>> SHA1_OF_FILE=$(openssl dgst -sha1 $FILE_TO_UPLOAD | awk '{print $2;}')
>> UPLOAD_URL=
>> https://pod-040-2009-17.backblaze.com/b2api/v2/b2_upload_file/daed35e922928ea38f340519/c004_v0402009_t0052
>>
>> UPLOAD_AUTHORIZATION_TOKEN="4_004ad5922e3f4590000000000_01a7620b_106677_upld_sg7Wnh52ocxqngTkjBLSfzE_ZzM="
>> curl \
>> -v \
>> -H "Authorization: $UPLOAD_AUTHORIZATION_TOKEN" \
>> -H "X-Bz-File-Name: $FILE_TO_UPLOAD" \
>> -H "Content-Type: $MIME_TYPE" \
>> -H "X-Bz-Content-Sha1: $SHA1_OF_FILE" \
>> -H "X-Bz-Info-Author: unknown" \
>> -H "X-Bz-Server-Side-Encryption: AES256" \
>> --data-binary "_at_$FILE_TO_UPLOAD" \
>> $UPLOAD_URL
>>
>
> I am trying to write it in libcurl. Then it fails.
> /*
> gcc -Wall -Wextra -pedantic -Wshadow -Ofast curl_upload.c -lcurl &&./a.out
> valgrind ./a.out
> openssl dgst -sha1 "/home/jian/helloc/curl_stmp1.c" | awk '{print $2;}'
> 03cc73740b9672c51a0cfe1b382c11bbb6a706c7
> */
> #include <stdio.h>
> #include <curl/curl.h>
> #include <string.h>
> #include <stdlib.h>
> #include <errno.h>
> #include <sys/stat.h>
> #include<stdlib.h>
> const char access_token[] =
> "4_004ad5922e3f4590000000000_01a7620b_106677_upld_sg7Wnh52ocxqngTkjBLSfzE_ZzM=";
> const char upload_url[] = "
> https://pod-040-2009-17.backblaze.com/b2api/v2/b2_upload_file/daed35e922928ea38f340519/c004_v0402009_t0052
> ";
> const char buckId[] = "daed35e922928ea38f340519";
> int main(void)
> {
> // unsigned char buffer[BUFSIZ];
> CURL *curl_handle = curl_easy_init();
> struct stat file_info;
> char *filename = "curl_1.c";
> char *path = realpath(filename,NULL);
> stat(filename,&file_info);
>
> printf("%s file size is %zu bytes\n",path,file_info.st_size);
> FILE *fp = fopen(path,"rb");
> if(fp == NULL){
> perror("failed");
> return 1;
> }
>
> struct curl_slist *chunk = NULL;
> chunk = curl_slist_append(chunk,"Content-Type: text/plain");
> chunk = curl_slist_append(chunk,"X-Bz-File-Name:
> /home/jian/helloc/curl_1.c");
> chunk = curl_slist_append(chunk,"Content-Length: 799");
> chunk = curl_slist_append(chunk,"Content-Length: 799");
> chunk = curl_slist_append(chunk,"X-Bz-Server-Side-Encryption: AES256");
> chunk = curl_slist_append(chunk,"X-Bz-Content-Sha1:
> 4ce3ebcd1ac59818d2884b4e96123b794ba69c4c");
> chunk = curl_slist_append(chunk,"X-Bz-Info-Author: unknown");
>
> /* Content-Length: [NUMBER_OF_BYTES_IN_FILE] */
> char size[200];
> snprintf(size,200,"Content-Length: %lld",(long long
> int)file_info.st_size);
> printf("size=%s\n",size);
>
> char auth[300];
> strcat(auth,"Authorization: ");
> strcat(auth,access_token);
> printf("auth:%s\n\n\n",auth);
> chunk = curl_slist_append(chunk,auth);
> curl_easy_setopt(curl_handle,CURLOPT_VERBOSE,1L);
> curl_easy_setopt(curl_handle,CURLOPT_HTTPHEADER,chunk);
> curl_easy_setopt(curl_handle, CURLOPT_HTTP_VERSION,
> CURL_HTTP_VERSION_1_1);
> curl_easy_setopt(curl_handle,CURLOPT_URL,upload_url);
> curl_easy_setopt(curl_handle,CURLOPT_POST,1L);
> curl_easy_setopt(curl_handle,CURLOPT_READDATA,fp);
> CURLcode res = curl_easy_perform(curl_handle);
> if(res != CURLE_OK){
> fprintf(stderr,"curl_easy_perform() failed:
> %s\n",curl_easy_strerror(res));
> }else{
> printf("\n\nSuccess\n");
> }
> curl_easy_cleanup(curl_handle);
> return 0;
> }
>
> The verbose message:
>
>> /home/jian/helloc/curl_1.c file size is 759 bytes
>> size=Content-Length: 759
>> auth:Authorization:
>> 4_004ad5922e3f4590000000000_01a7620b_106677_upld_sg7Wnh52ocxqngTkjBLSfzE_ZzM=
>>
>>
>> * Trying 149.137.133.93:443...
>> * Connected to pod-040-2009-17.backblaze.com (149.137.133.93) port 443
>> (#0)
>> * found 384 certificates in /etc/ssl/certs
>> * GnuTLS ciphers: NORMAL:-ARCFOUR-128:-CTYPE-ALL:+CTYPE-X509:-VERS-SSL3.0
>> * ALPN, offering http/1.1
>> * SSL connection using TLS1.3 / ECDHE_RSA_AES_128_GCM_SHA256
>> * server certificate verification OK
>> * server certificate status verification SKIPPED
>> * common name: backblaze.com (matched)
>> * server certificate expiration date OK
>> * server certificate activation date OK
>> * certificate public key: RSA
>> * certificate version: #3
>> * subject: CN=backblaze.com
>> * start date: Mon, 26 Sep 2022 21:40:41 GMT
>> * expire date: Sun, 25 Dec 2022 21:40:40 GMT
>> * issuer: C=US,O=Let's Encrypt,CN=R3
>> * ALPN, server did not agree to a protocol
>> > POST
>> /b2api/v2/b2_upload_file/daed35e922928ea38f340519/c004_v0402009_t0052
>> HTTP/1.1
>> Host: pod-040-2009-17.backblaze.com
>> Accept: */*
>> Transfer-Encoding: chunked
>> Content-Type: text/plain
>> X-Bz-File-Name: /home/jian/helloc/curl_1.c
>> Content-Length: 799
>> Content-Length: 799
>> X-Bz-Server-Side-Encryption: AES256
>> X-Bz-Content-Sha1: 4ce3ebcd1ac59818d2884b4e96123b794ba69c4c
>> X-Bz-Info-Author: unknown
>> Authorization:
>> 4_004ad5922e3f4590000000000_01a7620b_106677_upld_sg7Wnh52ocxqngTkjBLSfzE_ZzM=
>> Expect: 100-continue
>>
>> * Mark bundle as not supporting multiuse
>> < HTTP/1.1 400
>> < Content-Type: text/html;charset=utf-8
>> < Content-Language: en
>> < Content-Length: 435
>> < Date: Sun, 02 Oct 2022 18:02:23 GMT
>> < Connection: close
>> <
>> * Closing connection 0
>> <!doctype html><html lang="en"><head><title>HTTP Status 400 – Bad
>> Request</title><style type="text/css">body
>> {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b
>> {color:white;background-color:#525D76;} h1 {font-size:22px;} h2
>> {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;}
>> .line
>> {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP
>> Status 400 – Bad Request</h1></body></html>
>>
>> Success
>>
>
> I am not so sure why it failed.
> Interesting part: if I only have one chunk =
> curl_slist_append(chunk,"Content-Length: 799");
> then it will fail error message is like:
> {
> "code": "bad_request",
> "message": "Missing header: Content-Length",
> "status": 400
> * Closing connection 0
> }
>
> If I have two lines chunk = curl_slist_append(chunk,"Content-Length:
> 799"); then it will fail because of Bad Request.
> Why did I set Length to 799? 759 + 40 = 799.
> In the doc: https://www.backblaze.com/b2/docs/b2_upload_file.html aout
> Content length:
>
>> The number of bytes in the file being uploaded. Note that this header is
>> required; you cannot leave it out and just use chunked encoding.
>> When sending the SHA1 checksum at the end, the Content-Length should be
>> set to the size of the file plus the 40 bytes of hex checksum.
>>
>
> Since the only message is "HTTP Status 400 – Bad Request" then I don't
> know how to debug it any more...
>
>
> You can use curl option --libcurl [1] to get the source code equivalent.
> That will show you that CURLOPT_POSTFIELDS and CURLOPT_POSTFIELDSIZE_LARGE
> are used to respectively set the already read in data and its size.
>
> In your example you have chosen instead to use CURLOPT_POST [2] and
> CURLOPT_READDATA to read the data from a file. That's fine, however you
> should also set CURLOPT_SEEKFUNCTION [3] (basically forward to fseek; read
> the doc for pitfalls) and CURLOPT_SEEKDATA to the same pointer as READDATA.
> That way if the first post request fails (like, authentication required)
> the file can be properly rewound and read again from the beginning.
>
> Your example also sets the content length header twice, but it should not
> do that even once. By using the read function you are by default sending
> data using chunked encoding, and then you are adding the content length
> header. The bad request looks like this:
>
> Transfer-Encoding: chunked
> Content-Type: text/plain
> X-Bz-File-Name: /home/jian/helloc/curl_1.c
> Content-Length: 799
> Content-Length: 799
>
> You could set CURLOPT_POSTFIELDSIZE_LARGE to set the content length
> instead of using chunked encoding. These are things you need to review the
> documentation for, review the links at the bottom of this e-mail.
>
> There are other problems with your example. You are not checking the
> return code of stat and you are concatenating to an uninitialized string
> auth[] which will likely cause memory corruption.
>
>
> [1]: https://curl.se/docs/manpage.html#--libcurl
> [2]: https://curl.se/libcurl/c/CURLOPT_POST.html
> [3]: https://curl.se/libcurl/c/CURLOPT_SEEKFUNCTION.html
>
> --
> Unsubscribe: https://lists.haxx.se/listinfo/curl-library
> Etiquette: https://curl.se/mail/etiquette.html
>


Thanks for the command line --libcurl trick. By following through example
code, the CURLOPT_POSTFIELDS option is easier to understand compared to
CURLOPT_READDATA.

How does the curl command line encode the PNG file data to (char *postdata)
[0]?


[0]: https://curl.se/libcurl/c/CURLOPT_POSTFIELDS.html

-- 
 I recommend David Deutsch's <<The Beginning of Infinity>>
  Jian

-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2022-10-04