cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: PATCH: A fix to allow OpenVMS to (HTTP) POST data larger than64K...

From: Tim Sneddon <tsneddon_at_bsd.infomedia.com.au>
Date: Fri, 05 Nov 2004 14:34:13 +0800

Daniel Stenberg wrote:
> Hm. I understand and sympathize for the need of this fix. I'm only wondering
> if it can't be made in a slightly nicer way. Have you tried to lower the limit
> libcurl uses to determine weather the full POST should be sent in the first
> initial send() ? libcurl checks the size of the full POST and if it is larger
> than 100K, it will split it up and send it in multiple pieces and when doing
> this, it will never use pieces larger than 64K. So, if you try to lower the
> 100K limit down to 60K, does that work for you? (the check is done at
> lib/http.c:1891 in my version)
>
> The 100K limit is not scientificly determined, we could easily move it around
> to better suit us and having it set to 60K shouldn't hurt anyone much and
> would in my eyes provide a much simpler and more elegant solution to your
> problem.
>

I have finally gotten the time to sit down and provide a patch :-) I
have followed the suggestion above and created a pre-processor
definition MAX_INITIAL_POST_SIZE. Below is a list of files that have
been edited and why.

    o. lib/http.c Modified to use MAX_INITIAL_POST_SIZE.

    o. lib/http.h Modified to check if MAX_INITIAL_POST_SIZE
                         has been defined. If not it is set to 100
                         (to retain the previous behaviour).

    o. packages/vms/config-vms.h_without_ssl
                         Modified to define MAX_INITIAL_POST_SIZE and
                         HAVE_BASENAME. The lack of HAVE_BASENAME was
                         creating a compiler warning. This isn't really
                         an issue, but it does produce linker warnings
                         anytime that the library is linked against,
                         and can cause problems when using shareable
                         images.

    o. packages/vms/config-vms.h_with_ssl
                         Same as above.

Please consider and let me know what you think.

Regards, Tim.

-- 
Tim Sneddon                                tsneddon[at]infomedia.com.au
Systems Programmer                  Infomedia Business Systems Division
                         Level 3, 823 Wellington St, West Perth WA 6005
                          Phone: +61 8 9217 5000   Fax: +61 8 9217 5055

diff -ur ../curl/lib/http.c ./lib/http.c
--- ../curl/lib/http.c Fri Nov 5 09:41:25 2004
+++ ./lib/http.c Fri Nov 5 09:59:37 2004
@@ -1925,14 +1925,15 @@
       if(data->set.postfields) {
 
         if((data->state.authhost.done || data->state.authproxy.done )
- && (postsize < (100*1024))) {
+ && (postsize < (MAX_INITIAL_POST_SIZE*1024))) {
           /* If we're not done with the authentication phase, we don't expect
              to actually send off any data yet. Hence, we delay the sending of
              the body until we receive that friendly 100-continue response */
 
- /* The post data is less than 100K, then append it to the header.
- This limit is no magic limit but only set to prevent really huge
- POSTs to get the data duplicated with malloc() and family. */
+ /* The post data is less than MAX_INITIAL_PORT_SIZE, then append it
+ to the header. This limit is no magic limit but only set to
+ prevent really huge POSTs to get the data duplicated with
+ malloc() and family. */
 
           result = add_buffer(req_buffer, "\r\n", 2); /* end of headers! */
           if(result)
diff -ur ../curl/lib/http.h ./lib/http.h
--- ../curl/lib/http.h Fri Nov 5 09:41:26 2004
+++ ./lib/http.h Fri Nov 5 09:57:18 2004
@@ -57,5 +57,14 @@
    public curl/curl.h header. */
 #define CURLAUTH_PICKNONE (1<<30) /* don't use auth */
 
+/* MAX_INITIAL_POST_SIZE indicates the number of kilobytes that will be
+ sent in the initial part of a multi-part POST message. This is primarily
+ for OpenVMS where the maximum number of bytes allowed per I/O is 64K.
+ For other systems that do not define this, the default is (as it was
+ previously) 100. */
+#ifndef MAX_INITIAL_POST_SIZE
+#define MAX_INITIAL_PORT_SIZE 100
+#endif
+
 #endif
 #endif
diff -ur ../curl/packages/vms/config-vms.h_with_ssl ./packages/vms/config-vms.h_with_ssl
--- ../curl/packages/vms/config-vms.h_with_ssl Fri Nov 5 09:42:36 2004
+++ ./packages/vms/config-vms.h_with_ssl Fri Nov 5 12:34:24 2004
@@ -2,6 +2,7 @@
 /* MSK, 03/09/04, Seems to work for all platforms I've built on so far. */
 /* Added HAVE_SYS_IOCTL_H, IOCTL_3_ARGS and SIZEOF_CURL_OFF_T defines */
 /* MSK, 06/04/04, Added HAVE_INET_NTOP */
+/* TES, 11/05/04, Added MAX_INITIAL_POST_SIZE, HAVE_BASENAME */
 
 /* Define cpu-machine-OS */
 #ifdef __ALPHA
@@ -14,6 +15,12 @@
 #endif
 #endif
 
+/* Define to set number of kilobytes in initial POST message. On VMS systems
+ this is important as the default is 100 and the maximum amount of data
+ transferable in a VMS $QIO is 64K. All VMS versions prior to Alpha/I64
+ V8.2 and TCP/IP V5.5 are affected by this. */
+#define MAX_INITIAL_POST_SIZE 60
+
 /* Define if you have the ANSI C header files. */
 #define STDC_HEADERS 1
 
@@ -31,6 +38,9 @@
 
 /* Define if you have the geteuid function. */
 #define HAVE_GETEUID 1
+
+/* Define if you have the basename function. */
+#define HAVE_BASENAME 1
 
 /* Define if you have the gethostbyaddr function. */
 #define HAVE_GETHOSTBYADDR 1
diff -ur ../curl/packages/vms/config-vms.h_without_ssl ./packages/vms/config-vms.h_without_ssl
--- ../curl/packages/vms/config-vms.h_without_ssl Fri Nov 5 09:42:37 2004
+++ ./packages/vms/config-vms.h_without_ssl Fri Nov 5 12:33:49 2004
@@ -2,6 +2,7 @@
 /* MSK, 03/09/04, Seems to work for all platforms I've built on so far. */
 /* Added HAVE_SYS_IOCTL_H, IOCTL_3_ARGS and SIZEOF_CURL_OFF_T defines */
 /* MSK, 06/04/04, Added HAVE_INET_NTOP */
+/* TES, 10/06/04, Added MAX_INITIAL_POST_SIZE, HAVE_BASENAME */
 
 /* Define cpu-machine-OS */
 #ifdef __ALPHA
@@ -14,6 +15,12 @@
 #endif
 #endif
 
+/* Define to set the number of kilobytes per POST message. On VMS systems
+ this is important as the default is 100 and the maximum amount of data
+ transferable in a VMS QIO is 64K. All VMS versions prior to Alpha/I64 V8.2
+ and TCP/IP V5.5 are affected by this. */
+#define MAX_INITIAL_POST_SIZE 60
+
 /* Define if you have the ANSI C header files. */
 #define STDC_HEADERS 1
 
@@ -31,6 +38,9 @@
 
 /* Define if you have the geteuid function. */
 #define HAVE_GETEUID 1
+
+/* Define if you have the basename function. */
+#define HAVE_BASENAME 1
 
 /* Define if you have the gethostbyaddr function. */
 #define HAVE_GETHOSTBYADDR 1
Received on 2004-11-05