Re: Curl handle as a FILE *

From: Timothe Litt <litt_at_acm.org>
Date: Tue, 6 Dec 2022 12:20:08 -0500

On 06-Dec-22 11:42, Patrick Monnerat via curl-library wrote:
>
> On 12/6/22 15:50, Timothe Litt via curl-library wrote:
>>
>> On 06-Dec-22 09:24, Patrick Monnerat via curl-library wrote:
>>
>> It's a good idea.
>>
> Thanks!
>>
>> I don't think your seek() semantics are correct. seek(SEEK_END)
>> terminates connection, which isn't what seek does, or what a C
>> program would expect.  seek just positions.  I think that what you're
>> doing maps to fclose().
>>
> No, I know it is not !
>
> The reasons why I did it this way is there is no way to get an
> internal handle (struct cookie *) from a FILE * thus is it not
> possible to implement a user-callable flush function (without close)
> elsewhere.

Sure there is.  When you process a curl_fopen, keep a linked list of
your cookies; you already put the handle in cookie_fdopen as p->fp. 
When you need the the handle, walk the list until the p->fp matches the
handle you're looking for.  If you think there may be a lot of these,
use a hash table (e.g hcreate/hsearch).  In either case, use a lock when
touching the list so you're thread safe.

delete the cookie in fclose - or simply mark it invalid and re-use it on
the next curl_fopen.  Or put it on a free list.

Plenty of room for optimizations.

Plus, isn't libc handling buffering (including flush) for you?


> But a caller might want to flush without close, to check if no error
> occurred. The standard fflush drains the output buffer, but does not
> wait for request completion. In addition, a real full-featured fseek
> is meaningless, so I decide to use it. I wish glibc designers have
> provided a "void *fcookie(FILE *fp)" to avoid this problem. We have to
> do with what is provided.
>
> Fclose() should not be waiting (or at least not able to hang). In
> addition, you cannot retrieve the error status after the close, so
> you'll never know if the request was successful or not.
>
Yes, you can.  fclose will return EOF on error, so if you set errno, the
usual semantics apply.  If you want to do more than that, you'll have to
invent another mechanism.  If you keep the cookie around, you could get
status before it's recycled.

It's unfortunate that there's no ioctl function in the cookie API.  But
again, you could implement one using a reverse lookup.


> I'm aware this deviates from the standard semantics, but I did not
> find yet any other elegant way to do it.
>
>> seek should return errno = EBADF if not implemented.  EINVAL is for a
>> bad 'whence' .
>>
> Thanks for pointing to it. In fact, it IS implemented but in certain
> cases it should rather return ESPIPE.

If seek is performed on a non-seekable fd, it may also be ENXiO. See
https://pubs.opengroup.org/onlinepubs/9699919799/

(click "systemInterfaces", scroll down and click "fseek") for a complete
list of the defined errors.

>>
>> The idea of doing range requests based on seek from fcurll's issues
>> has merit.  May need some support in the library.
>>
> Why not! But this will rather be an FTP-like RESTART command than a
> real range.

With  possibly some support from the library, you should be able to do
range requests.  You might want to use mode modifiers to specify this -
e.g. "r%R" for read with ranges (or random).  You can extend this as
needed, as long as the defaults look like a sequential read/write.

>>
>> You might also provide these as curl_xxx(), using curl handles. While
>> transparency is nice, you can't always hook the C RTL. curl_xxx could
>> be the portable version - it's a pretty simple search-and-replace for
>> users.  And your hooked version, if available at configure time, can
>> be used where it's available.
>>
> Well... this is fcurl!
But with a bit of work, you can replace fcurl with your project.
>> You may also want to consider other key functions, including delete
>> and rename.  With ftp, at least: chmod, chown...
> I also had this ikind of idea. But I'm stuck with the missing
> fcookie(fp) problem here too! And chmod, chown... are file systems
> API, not IO.

ftp lets you do remote chmod, chown.  libcurl supports ftp.


>>
>> There's quite a bit to making network operations look exactly like
>> local files.  But you're definitely on to a useful subset.
>>
> Perfection cannot be reached with "patched pieces" :-)
>
>
Perfection isn't required.
> Many thanks for your comments,
>
> Patrick
>

-- 
Unsubscribe: https://lists.haxx.se/listinfo/curl-library
Etiquette:   https://curl.se/mail/etiquette.html

Received on 2022-12-06