Bugs item #2565128, was opened at 2009-02-04 20:51
Message generated for change (Comment added) made by bagder
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=2565128&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: http
Group: wrong behaviour
>Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Jocelyn (jocelynj)
Assigned to: Daniel Stenberg (bagder)
Summary: "not new enough" file when using CURL_TIMECOND_IFMODSINCE

Initial Comment:
Hi,

I'm using libcurl 7.18.2-8 in a C program, and I get the message "The requested document is not new enough" even if the server sends a file with response 200 OK. The problem I have with this behaviour is that the program depends on the HTTP response code to know if there was a more recent file or not on the server, and expects to receive a file when getting 200 OK. But in this case, curl doesn't modify the http headers, and makes a 0-byte file.

This behaviour happens if the server doesn't have a correct time, and is in the past. If the client sends a request with an If-Modified-Since header with a future date for the server, then the server can send a 200 OK response. I'm joining a file to show one transfer like this.

In my opinion, curl shouldn't check the file time in addition to the check done by the server, except if there is a good reason. I have checked curl code, and it seems that if there is a 304 http response received, libcurl already behaves correctly, and there is an additional code to check the filetime returned by the server (line 498 of lib/transfer.c on revision 7.19.3)

Thanks !

----------------------------------------------------------------------

>Comment By: Daniel Stenberg (bagder)
Date: 2009-02-12 09:05

Message:
Thanks, case closed!

----------------------------------------------------------------------

Comment By: Jocelyn (jocelynj)
Date: 2009-02-11 23:47

Message:
Please forget what I have written earlier: I had linked with the old
library instead of the new library. Now that I'm using the correct library,
I can see that CURLINFO_CONDITION_UNMET info variable is set to FALSE when
I'm not using CURL_TIMECOND_IFMODSINCE.

----------------------------------------------------------------------

Comment By: Jocelyn (jocelynj)
Date: 2009-02-11 23:34

Message:
I have tried your new code, and it seems to work correctly with my
application. Only one question: is CURLINFO_CONDITION_UNMET info variable
supposed to be set to 0 if I'm not using CURL_TIMECOND_IFMODSINCE ? And
where is this info variable cleared in the library code ?

It looks like that the info variable is always set when I don't set
CURL_TIMECOND_IFMODSINCE.

For reference, my real name is Jocelyn Jaubert.

Thanks.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-02-11 23:04

Message:
CURLINFO_CONDITION_UNMET was committed just now anyway, so if you'd like to
try it out and give it a check I'll let this entry remain open for a while
more.

Thanks for the report!

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-02-11 14:30

Message:
How about CURLINFO_CONDITION_UNMET ? I want to keep it open as "condition"
as then it could serve for other (related) conditions in the future or so.

If you tell me your real name I'll give you proper credit for this work in
the changelog!

----------------------------------------------------------------------

Comment By: Jocelyn (jocelynj)
Date: 2009-02-10 21:29

Message:
Apart from the name of the new variable that I don't really like, your
patch looks good :)

A better name could be CURLINFO_CONDITIONAL_FAILED or
CURLINFO_UNMET_TIME_CONDITION, or even CURLINFO_FAILED_TIMECOND (which is a
name similar to the option one)

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-02-09 23:26

Message:
Yes, that's what I mean. I've just uploaded condition.patch which
introduces the CURLINFO_CONDITIONAL info variable that returns 1 if libcurl
didn't transfer a file due to a condition you specified.

Existing libcurl versions of course already exist and need to be dealt
with, but then you would have to no matter how this gets fixed. And yes,
you'd have to do something conditional to make your app works with both
this new stuff and older libcurls.

> If libcurl fails to download the file because of the time condition,
will there be a guarantee that the target file
> won't be modified ?

Mostly libcurl doesn't touch target files at all, as your app would deal
with that. But yes, if the condition prevents a file to get transfered it
will not call the write callback (not even the built-in if no custom one
has been set).

----------------------------------------------------------------------

Comment By: Jocelyn (jocelynj)
Date: 2009-02-09 22:11

Message:
I suppose that means that the application won't have to check the http
response code, but only this new information variable. This variable would
then be almost like a "valid" bit, telling the application if the server's
file is more recent than the local one, and if libcurl has updated the file
or not. If that is what you propose, I think that it is a good idea

There is still the problem on how to handle already existing applications,
as they will get the bug I reported. I suppose that you want the programmer
to modify the application, and use only a recent version of libcurl (or use
#ifdef to know which libcurl version he is using).

If libcurl fails to download the file because of the time condition, will
there be a guarantee that the target file won't be modified ?

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-02-09 16:41

Message:
Thinking about this further, we have the exact same problem with FTP but
for that protocol there is no 304 or similar so in order to signal this
back to the app we need something new.

I'm now more thinking of a new info-variable to be readable after a
transfer that was done with a time-condition, as I think it isn't quite an
error so returning an error-code would be a bit weird methinks. It would
then presumably also get set when a 304 is returned on a time-conditional
request.

What do you think about that idea?

----------------------------------------------------------------------

Comment By: Jocelyn (jocelynj)
Date: 2009-02-04 22:05

Message:
Well, there are two ways of seeing things:

 - we trust everything that the server sends, so libcurl doesn't have to
check the time, and only returns what the server sent (http response code
anf file if one was sent)

 - we don't trust the server, so libcurl do another check on time. In this
case, I think that libcurl shouldn't mislead the application in thinking
that the server sent a 0-byte file.

If I understand your point, you prefer the second possibility. In this
case, shouldn't libcurl sends http response code 304 instead of an error ?

If you prefer not sending CURLE_OK, that's fine - I suppose that the
application will still be able to understand what happened by looking at
the error code.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-02-04 21:44

Message:
I beg to differ.

The option you pass to libcurl is about requesting a document modified
since a specific date, and libcurl takes measures to only deliver the
document if newer (or older) than the specified time. Why would you like to
get the document that hasn't changed even though you explicitly asked you
didn't want it then?

What I think could be argued is how this is signaled to the application,
as just returning CURLE_OK doesn't feel quite right to me as that would
imply a normal transfer which with it isn't exactly...

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=2565128&group_id=976
Received on 2009-02-12