Bugs item #1999181, was opened at 2008-06-20 22:29
Message generated for change (Comment added) made by yangtse
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1999181&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: compile or build problem
Group: wrong behaviour
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: John Lightsey (jlightsey)
Assigned to: Daniel Stenberg (bagder)
Summary: CLOCK_MONOTONIC always fails on some systems

Initial Comment:
I've seen several Centos 3 i386 systems with this particular problem.

clock_gettime(CLOCK_MONOTONIC) compiles and links correctly, but at runtime it will always fail and return -1. This causes curl to timeout before opening a socket when creating a https connection.

verbose output:
root@centos3edge [~]# /opt/curlssl/bin/curl -G https://www.google.com -v
* About to connect() to www.google.com port 443 (#0)
* Connection time-out
* Closing connection #0
curl: (28) Connection time-out

The attached patch checks for broken CLOCK_MONOTONIC support at configure time so that curl will fallback on gettimeofday() on these systems.

----------------------------------------------------------------------

Comment By: Yang Tse (yangtse)
Date: 2008-07-03 16:32

Message:
Logged In: YES
user_id=1590006
Originator: NO

Relative to this issue, CVS repository has finally been updated 2 Jun 2008
relative to curl and libcurl 7.18.2 with the following changes:

*) Configure script tries do detect a monotonic clock_gettime the same way
it did in 7.18.2 without using the POSIX feature test macro. It verifies if
it is compilable and linkable for all systems. Additionally when not
cross-compiling a run-time check is now also done to verify if it actually
works.

*) Even when the configure process has truly detected monotonic clock
availability, it might happen that it is not actually available at
run-time. When this occurs curl and libcurl will simply fallback to
gettimeofday() or time().

This re-enables the use of the monotonic clock on systems where the more
strict POSIX check had disabled it. On current curl auto-builds the strict
POSIX check limited the use of the monotonic clock in such a way that it
was only being used for Solaris systems. With the current checks it is now
also used for Linux systems.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2008-07-01 20:43

Message:
Logged In: YES
user_id=1110
Originator: NO

Yes, that's what I mean.

----------------------------------------------------------------------

Comment By: Yang Tse (yangtse)
Date: 2008-07-01 02:57

Message:
Logged In: YES
user_id=1590006
Originator: NO

Daniel,

[...] I'm a bit split on this but since the check actually _can_ be
made run-time and thus avoid a complete havoc I figure we better make a
best effort [...]

Do you mean something like the following and also removing the use of
feature test macro _POSIX_MONOTONIC_CLOCK in the configure script and
keeping the AC_RUN_IFELSE verification when not cross-compiling ?

Index: lib/timeval.c
===================================================================
RCS file: /cvsroot/curl/curl/lib/timeval.c,v
retrieving revision 1.31
diff -u -r1.31 timeval.c
--- lib/timeval.c 12 May 2008 02:04:22 -0000 1.31
+++ lib/timeval.c 1 Jul 2008 02:41:52 -0000
@@ -52,9 +52,19 @@
   */
   struct timeval now;
   struct timespec tsnow;
- (void)clock_gettime(CLOCK_MONOTONIC, &tsnow);
- now.tv_sec = tsnow.tv_sec;
- now.tv_usec = tsnow.tv_nsec / 1000;
+ if(0 == clock_gettime(CLOCK_MONOTONIC, &tsnow)) {
+ now.tv_sec = tsnow.tv_sec;
+ now.tv_usec = tsnow.tv_nsec / 1000;
+ }
+#ifdef HAVE_GETTIMEOFDAY
+ else
+ (void)gettimeofday(&now, NULL);
+#else
+ else {
+ now.tv_sec = (long)time(NULL);
+ now.tv_usec = 0;
+ }
+#endif
   return now;
 }

----------------------------------------------------------------------

Comment By: xbx (xbx___)
Date: 2008-06-30 16:13

Message:
Logged In: YES
user_id=1452265
Originator: NO

> However, aren't you fooling yourself then? We're doing quite a lot
> of checks in the configure scripts that kind of makes the output
> rather specific for the system and environment it was built on.

my mistake, then: I thought the checks were about compilation environment
and available libraries features.
(and not about runtime features.)

I'll have to get a closer look at the configure checks, then... ;)

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2008-06-30 13:35

Message:
Logged In: YES
user_id=1110
Originator: NO

I can see how you want this to work in a run-time dynamic way and I can
sympathize with that.

However, aren't you fooling yourself then? We're doing quite a lot of
checks in the configure scripts that kind of makes the output rather
specific for the system and environment it was built on. Moving the build
libcurl from one system to another where enough factors are different will
cause trouble, and if this is the only thing that hurt you then I think you
should consider yourself lucky.

This said, I'm a bit split on this but since the check actually _can_ be
made run-time and thus avoid a complete havoc I figure we better make a
best effort, like this suggestion.

----------------------------------------------------------------------

Comment By: xbx (xbx___)
Date: 2008-06-30 11:12

Message:
Logged In: YES
user_id=1452265
Originator: NO

hi,

I had a quick look to the fix (using http://cool.haxx.se/cvs.cgi/curl, so
I may have misread)
but I understand this is a configure time fix.

I think this isn't enough: if the lib is used on a different computer as
the one it has been compiled in, it may fail.

I had a similar issue on one program, that is distributed as a binary, and
while clock_gettime(CLOCK_MONOTONIC) did compile and did work on some
machines, on others (some older BSD or older linuxes) it didn't.

I had to do a runtime check, something along the lines of

struct timespec TimerResolution = {0};
if (clock_getres(CLOCK_MONOTONIC, &TimerResolution) != 0) {
  // I think errno should be EINVAL if the timer isn't supported
  s_UseLegacyClock = True;
}
// maybe check the resolution is "sensible?"
if (TimerResolution.tv_sec > 0 || TimerResolution.tv_nsec > 100*1000*1000)

{
  s_UseLegacyClock = True;
}

if (s_UseLegacyClock) {
  gettimeofday()
} else {
  clock_gettime(CLOCK_MONOTONIC)
}

----------------------------------------------------------------------

Comment By: Yang Tse (yangtse)
Date: 2008-06-29 03:34

Message:
Logged In: YES
user_id=1590006
Originator: NO

John, could you verify if what I have just committed to the CVS repository
solves this issue for you ?

----------------------------------------------------------------------

Comment By: John Lightsey (jlightsey)
Date: 2008-06-25 16:59

Message:
Logged In: YES
user_id=679418
Originator: YES

Sorry for taking so long to get back with you. You're absolutely right.
sysconf(_SC_MONOTONIC_CLOCK) does give an accurate appraisal of
CLOCK_MONOTONIC support at runtime.

On CentOS 3:

Monotonic support: -1

On CentOS 5:

Monotonic support: 200112

----------------------------------------------------------------------

Comment By: Yang Tse (yangtse)
Date: 2008-06-23 02:00

Message:
Logged In: YES
user_id=1590006
Originator: NO

You will have noticed that I'm certainly trying to avoid the monotonic
clock runtime check at configuration time. Yes it is true, and it is for
the following reason...

Runtime availability check at configuration time only grants that the
built software will properly run on that machine where it is being built.
As soon as you move the libcurl library or curl executable to another
machine it might fail, or it could work, but no one will be able to grant
that beforehand.

As a matter of fact if _POSIX_MONOTONIC_CLOCK is defined to zero what it
actually means is that it might be available or not at runtime. And the
program has to check at runtime if it is actually available or not using
sysconf(), pathconf() or fpathconf(). This means that it would need to be
the library the one making the sysconf() check inmediately before using the
monotonic clock, eeek. If it was available at configuration time with
_POSIX_MONOTONIC_CLOCK defined to zero, it does not imply that it will be
available when finally run, specially if built library or program is moved
to a different machine and dynamic linking is being used.

The above is the reason for which I propose to only use the monotonic
clock on configure capable systems when _POSIX_MONOTONIC_CLOCK is defined
to a value greater than zero, and simply fallback to gettimeofday() when
_POSIX_MONOTONIC_CLOCK is not defined or defined to zero.

I would agree on a configuration runtime check that would run on systems
that claim full support (_POSIX_MONOTONIC_CLOCK > 0) to actually verify if
it truly works. Keeping in mind that when cross compiling this check won't
be possible.

It is true that with this approach there will be systems on which the
monotonic clock could work that no longer will be used. But if the system
libraries does not grant that it works, why should we take the risk and
assume that our configuration runtime check will still hold true once that
the library is moved to another system ?

The internal monotonic clock use was introduced in curl and libcurl 7.18.2
so I don't think there will be any problem if next version falls back again
to gettimeofday() on some OS's, and keeps using the monotonic clock on
those that actually claim full support.

It seems that FreeBSD fully supports the monotonic clock since version 7.0
and NetBSD since 2.0.

John, I think that instead of sysconf(_POSIX_MONOTONIC_CLOCK) you should
have used sysconf(_SC_MONOTONIC_CLOCK), you could get a very different
result.

Agreements, disagreements, comments ?

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2008-06-22 17:58

Message:
Logged In: YES
user_id=1110
Originator: NO

Yeps, it seems my systems that have it defined all define it to 0... I do
have an old redhat 7.2 that doesn't define it at all.

So, I think a AC_RUN_IFELSE really is needed...

----------------------------------------------------------------------

Comment By: John Lightsey (jlightsey)
Date: 2008-06-22 16:31

Message:
Logged In: YES
user_id=679418
Originator: YES

Sorry, I didn't notice that yangtse had said it would require a positive
value for _POSIX_MONOTONIC_CLOCK. That would indeed fix these Centos 3
systems, but it would also disable clock_gettime on Debian Sid, Centos 5
and probably every other Linux system where it currently works.

----------------------------------------------------------------------

Comment By: John Lightsey (jlightsey)
Date: 2008-06-22 16:25

Message:
Logged In: YES
user_id=679418
Originator: YES

_POSIX_MONOTONIC_CLOCK is defined on those Centos 3 systems..

root_at_centos3edge [~]# grep -r _POSIX_MONOTONIC /usr/include 2>/dev/null
/usr/include/bits/posix_opt.h:#define _POSIX_MONOTONIC_CLOCK 0
root_at_centos3edge [~]# grep -r bits/posix_opt.h /usr/include 2>/dev/null
/usr/include/unistd.h:#include <bits/posix_opt.h>

That OpenGroup link you sent earlier explains that 0 means it's not
guaranteed to be available, but it looks like even querying for support at
runtime gives confusing results..

"If a symbolic constant is defined with the value zero, all headers, data
types, and functions shall be present. The application can check at runtime
to see whether the option is supported by calling fpathconf(), pathconf(),
or sysconf() with the indicated name parameter."

root_at_centos3edge [~]# cat test.c
#include <unistd.h>
#include <sys/types.h>
#include <time.h>
#include <stdio.h>
int main() {
        struct timespec ts;
        printf("Monotonic support: %ld\n",
sysconf(_POSIX_MONOTONIC_CLOCK));
        if (clock_gettime(CLOCK_MONOTONIC, &ts) == -1) {
                perror("clock_gettime");
        }
        exit(0);
}
root_at_centos3edge [~]# gcc -o test test.c -lrt
root_at_centos3edge [~]# ./test
Monotonic support: 131072
clock_gettime: Invalid argument

----------------------------------------------------------------------

Comment By: Yang Tse (yangtse)
Date: 2008-06-22 02:48

Message:
Logged In: YES
user_id=1590006
Originator: NO

From this page
http://www.opengroup.org/onlinepubs/009695399/basedefs/unistd.h.html I
understand the following:

When feature test macro _POSIX_MONOTONIC_CLOCK is defined with a value
greater than zero the implementation shall always support it when executed,
there is no need to use sysconf() at runtime to actually verify system
support.

So, if a stricter check is done which requires a positive
_POSIX_MONOTONIC_CLOCK then no runtime check would be needed, and this
would be fully standard compliant.

The proposed patch then would be the following one.

Index: acinclude.m4
===================================================================
RCS file: /cvsroot/curl/curl/acinclude.m4,v
retrieving revision 1.158
diff -u -r1.158 acinclude.m4
--- acinclude.m4 20 Jun 2008 18:09:48 -0000 1.158
+++ acinclude.m4 22 Jun 2008 02:30:11 -0000
@@ -1986,6 +1986,9 @@
 #ifdef HAVE_SYS_TYPES_H
 #include <sys/types.h>
 #endif
+#ifdef HAVE_UNISTD_H
+#include <unistd.h>
+#endif
 #ifdef HAVE_SYS_TIME_H
 #include <sys/time.h>
 #ifdef TIME_WITH_SYS_TIME
@@ -1997,8 +2000,13 @@
 #endif
 #endif
     ]],[[
+#if (defined(_POSIX_SOURCE) || defined(_POSIX_C_SOURCE)) && \
+ defined(_POSIX_MONOTONIC_CLOCK) && (_POSIX_MONOTONIC_CLOCK > 0)
       struct timespec ts;
       (void)clock_gettime(CLOCK_MONOTONIC, &ts);
+#else
+ HAVE_CLOCK_GETTIME_MONOTONIC shall not be defined.
+#endif
     ]])
   ],[
     AC_MSG_RESULT([yes])

It would be interesting if John Lightsey reported back the definition of
_POSIX_MONOTONIC_CLOCK from unistd.h on those systems where clock_gettime()
with CLOCK_MONOTONIC is failing at runtime.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2008-06-21 20:19

Message:
Logged In: YES
user_id=1110
Originator: NO

The monotonic clock support is "optional" according to that opengroup.org
page, so to me it seems that a POSIX-compliant OS can still fail at
run-time
even though clock_gettime is present.

Is the _POSIX_MONOTONIC_CLOCK mandatory for POSIX systems that implement
CLOCK_MONOTONIC? If not, I think a run-time check is unavoidable...

----------------------------------------------------------------------

Comment By: Yang Tse (yangtse)
Date: 2008-06-21 03:09

Message:
Logged In: YES
user_id=1590006
Originator: NO

I still maintain what I've said before. But...

It could also be possible that libcurl's configuration compilation check
should also be verifying definition of _POSIX_MONOTONIC_CLOCK.

Can you verify if the following patch improves the situation or if it does
not change ?

Index: acinclude.m4
===================================================================
RCS file: /cvsroot/curl/curl/acinclude.m4,v
retrieving revision 1.158
diff -u -r1.158 acinclude.m4
--- acinclude.m4 20 Jun 2008 18:09:48 -0000 1.158
+++ acinclude.m4 21 Jun 2008 02:57:58 -0000
@@ -1986,6 +1986,9 @@
 #ifdef HAVE_SYS_TYPES_H
 #include <sys/types.h>
 #endif
+#ifdef HAVE_UNISTD_H
+#include <unistd.h>
+#endif
 #ifdef HAVE_SYS_TIME_H
 #include <sys/time.h>
 #ifdef TIME_WITH_SYS_TIME
@@ -1997,8 +2000,13 @@
 #endif
 #endif
     ]],[[
+#if !defined(CLOCK_MONOTONIC) || \
+ !defined(_POSIX_MONOTONIC_CLOCK) || (_POSIX_MONOTONIC_CLOCK < 0)
+ HAVE_CLOCK_GETTIME_MONOTONIC shall not be defined.
+#else
       struct timespec ts;
       (void)clock_gettime(CLOCK_MONOTONIC, &ts);
+#endif
     ]])
   ],[
     AC_MSG_RESULT([yes])

----------------------------------------------------------------------

Comment By: Yang Tse (yangtse)
Date: 2008-06-21 01:18

Message:
Logged In: YES
user_id=1590006
Originator: NO

My point of view...

Linking of libcurl using function clock_gettime() is only possible if the
system libraries actually provide an implementation of the clock_gettime(),
otherwise linking would fail and curl would not build.

Compilation of libcurl using function clock_gettime() and clock_id
CLOCK_MONOTONIC is only possible if the system header files provide both a
prototype for function clock_gettime() and the clock_id definition for
CLOCK_MONOTONIC, otherwise compilation would fail.

Both checks mentioned above are done at libcurl configuration time, and if
any of them fail libcurl will be configured to not use the clock_gettime()
function, and simply use gettimeofday().

If a system provides bogus information in its system header files relative
to the actual capabilities of its libraries implementation it is a system
library or system header file issue. No application will be capable of
building properly using that system. The OS provider can fix the header
files or fix the system library implementation. Or not fix it and expect
the rest of the world to workaround the system bugs.

It could also be possible that there is no problem on the OS provided
system header files and the OS provided system libraries. But it could
happen that someone could have 'ugraded' the system header files without
'upgrading' the system libraries creating and unexpected problem for any
application that is built on that 'unbalanced' system.

The IEEE and The Open Group [1] are very clear at this point. The
clock_gettime() function shall fail if the clock_id argument does not
specify a known clock.

From what you are describing the system header files do define a clock_id
for CLOCK_MONOTONIC, that implies that the system headers _know_ the
CLOCK_MONOTONIC clock, but later on the system libraries do not _know_ it
and fail at runtime.

Could you verify/discard if its just an accidental mismatch between system
libraries and header files ?

If not, then, could you provide further information on which system
library version and provider has the problem ? And if it has already been
fixed in a further version ? Or a pointer to the bug/fix record ?

[1]
http://www.opengroup.org/onlinepubs/009695399/functions/clock_getres.html

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=1999181&group_id=976
Received on 2008-07-03