Daniel Stenberg ha scritto:

> On Mon, 28 Nov 2005, Legolas wrote:
>
>> I am new to this mailing list, please excuse me if I'm posting wrongly.
>
>
> Since it is a libcurl issue, we do the talk better on the curl-library
> list. I'm cc'ing my reply there.
>
>> I would like to help on implementing a zero copy interface (as
>> mentioned in the TODO list or in the Hiper aims) for libcurl, but I
>> do not know how the code design or build is planned: is somebody
>> (perhaps Daniel himself) already working on it?
>
>
> Nope, I am not working on it (yet), but I've been thinking about it
> and been discussing it previously over the years so I think I have a
> fairly good idea of what to do.
>
> It would basically be a way for the application to provide an area and
> a size for libcurl to store downloaded data into and then libcurl
> calls a callback to request further buffers.
>
> Similiarly, for uploads, the read callback should not have to copy
> data to an internal buffer of libcurl but instead offer a pointer and
> a length to a buffer that libcurl can use.
>
> I haven't yet worked out any details, but if you have any ideas or
> plans, feel free to pass them on!
>
This is exactly the way I was thinking about (it looks like being the
best one), but I will show you a more interesting solution.

I am currently developing an *RWops* struct with its C methods &
functions with a clear API, the code needs some sharpening and clearing
but it works great, I am currently debugging it and I think it will be
useful for enhancing libcurl.
The aim of the RWops struct is to encapsulate into one object any kind
of input/output with its custom read/write/close/seek/tell/flush
handlers; I have started coding using the SDL_RWops.h header (not the
code) and now my object can read from files, file pointers, memory
streams and also read (performing GET) from CURL objects and write
(performing a POST and reading the response). Here is the basic struct
(most comments have been wiped out to make this mail shorter):

- - - - from RWops.h - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - -
typedef struct _RW_mem{
            char *base;
             char *here;
            char *stop;
            int ahead;
            char *stop_dy;
} _RW_mem;

typedef struct RWops {
    int (*seek)(struct RWops *context, int offset, int whence);
    int (*read)(struct RWops *context, void *ptr, int size, int maxnum);
    int (*write)(struct RWops *context, const void *ptr, int size, int num);
    int (*flush)(struct RWops *);

    int type;
    union {
        struct {
             FILE *fp;
            int autoclose,
                    autoflush; /* set to
1 for console outputs */
        } stdio;
        struct _RW_mem mem;
        struct {
            struct _RW_mem mem_get, /* GET memory stream */
                                            mem_post; /* POST
memory stream */
            int shared; /* curl
object usage policy */
            const char *url; /* url is
kept if curl handler is shared */
            CURL *curl; /* associated
CURL, maybe shared */
        } url;
    } hidden;

} RWops;

RWops *RWFromFile(const char *file, const char *mode);
RWops *RWFromFP(FILE *fp, int autoclose, int autoflush);
RWops *RWFromMem(void *memory, int size, int autofree);
RWops *RWFromConstMem(const void *memory, int size, int dynamic);
RWops *RWFromUrl(const char *url, CURL *open_curl, int post_enabled);
int RWAssertMem(RWops *rw, int dynamic);
long RWsize(struct RWops *rw);
int RW_printf(RWops *rw, char *format, ...);
int RW_vprintf(RWops *rw, const char *format, va_list args);
RWops *RWAlloc(void);
void RWFree(RWops *area);

#define RWseek(ctx, offset, whence) (ctx)->seek(ctx, offset, whence)
#define RWtell(ctx) (ctx)->seek(ctx, 0, SEEK_CUR)
#define RWread(ctx, ptr, size, n) (ctx)->read(ctx, ptr, size, n)
#define RWwrite(ctx, ptr, size, n) (ctx)->write(ctx, ptr, size, n)
#define RWclose(ctx) (ctx)->close(ctx)
#define RWflush(ctx) (ctx)->flush(ctx)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - -

There are also other methods which are private of course.
An RWops object can be of type 'FILE', 'MEM' or 'URL', the URL type is
basically a double-stream
memory RWops. FILE RWops can also be stream-lined into memory with
RWAssertMem(..).
The advantage of using this struct is easy to see (give a look again to
the macros).

Now my question is:

*Do you think using this object would be useful?*
Or else do you want me to extract the memory streams interface code from
the main project?

I think that using an RWops input/output interface directly into libcurl
would be great, RWops already has
memory-on-demand functions to get more buffer space but if the user is
writing directly to a file, 0 memory
copies will be made (not technically, I know...) and data would be
written directly from the socket buffer to the file.
I am also upgrading the RWops struct to support actually two streams per
object (a write stream, if enabled, and a read
stream, if enabled), but it is a much more complex talk and I will not
start it here.

If some points are not clear, I will be glad to explain them in the next
reply.
Received on 2005-11-29