On Fri, 3 Apr 2009, Stefan Teleman wrote:

> I am sorry, but this is not an explanation of the problem. To me, this is a
> description of a symptom: "using off_t has bitten far too many users for us
> to be happy with it".

Yes I agree. The "bitten" part is quite clearly about the sympthoms of the
problem.

> 1. 32-bit CURL_OFF_T on Solaris can have two different sizes: 4 bytes,
> or 8 bytes.
> 2. Having a 32-bit 8-byte CURL_OFF_T on Solaris is desirable.
> 3. The 32-bit size of CURL_OFF_T is determined by compiler flags, at
> compile-time.

> 4. Overflow problems for CURL_OFF_T can occur in applications or libraries
> which are consumers of CURL, in the specific case where the 32-bit compiler
> flags used to build the CURL binaries do not determine the same size for
> CURL_OFF_T as the consumer.

Yes. And the reason for that problem is of course that the API is silly and
doesn't allow the lib to detect this situation and accept either 32bit or
64bit depending on what the app happens to use. But changing that is
undoubtably going to break the ABI all over...

> [4] Can happen regardless of whether or not CURL_OFF_T is a typedef
> for off_t, or it is a typedef for long/long long conditional on

Yes. We're gambling here that most people on systems that feature 64bit file
operations will want to have those features enabled. That's basically what the
"potential" part is about in the "potential risk" claim.

But for systems that are capable of running configure, such as I would assume
most systems that can use the Sun Compiler, configure will determine the type
and do the necessary typedef and put that in a public header file to be used
by applications. That typedef _does not_ include that super-huge #ifdef thing.
At least it shouldn't in most circumstances.

> If a consumer of the CURL binaries compiled as described above compiles
> their own 32-bit application without -D_FILE_OFFSET_BITS=64
> -D_LARGEFILE_SOURCE -D__EXTENSIONS__, their typedef for CURL_OFF_T will be 4
> bytes in size. You are back to square one: the same exact overflow problem
> occurs.

No. If you just build with configure and use build your app using the headers
that were output as a result of that libcurl build, they *should* use the
exact same type of the exact same size. If they don't, it's a bug.

There should be no risk at all for overflows or "mis-alignments".

> You can simplify the typedefs and macros, and avoid using the restricted
> availability long long type in curlbuild.h

The "restricted availability" is checked for by configure.

> by only using one semi-native datatype: off_t. In 32-bit, this datatype
> changes in size, depending on the compiler flags indicated above. It does
> all the typedef'ing for you behind the scenes.

And as we've mentioned (like 3 or 4 times) this is what we used to do, and we
had MANY users get into trouble with this since the app had to use the same
off_t size as the lib and there's no really good way the lib or the lib's
headers can make the app do this. Thus lots of users got into this un-aligned
situation and the behaviors the users then experience are not easily
understood to be due to this (by "mere" users I mean, us devs could see this
pretty quickly).

> And this still has not fixed the problem 100%: CURL still needs a way to
> communicate the relevant compiler flags to its consumers, otherwise the same
> CURL_OFF_T size mismatch can occur.

That communication of compiler flags to app compiling users turned out really
hard and we failed miserably during a long time until we decided we should try
something else.

I read you loud and clear that you believe simply reverting all these efforts
to the pre-7.19.0 state is the right way, but doing that will just bring back
the old problems and instead of keeping the current ones. And I'm not seeing
how they were any more fun than the more theoretical current ones - and in
fact we've had very few complaints about this switch.

-- 
  / daniel.haxx.se