Menu

#1401 OPENSSL_config causes app to exit with bad conf file

closed-fixed
5
2015-01-20
2014-07-27
Jan-E
No

I got this error after upgrading to libcurl 7.37.1 and compiling php_curl.dll with it on Windows

C:\phpdev\php55nts.x32>php -v
Auto configuration failed
7140:error:0E065068:configuration file routines:STR_COPY:variable has no value:.
\crypto\conf\conf_def.c:618:line 37

To analyze what went wrong, I went back to libcurl 7.37.0 and applied the openssl patches since that release:

7.37.0 unpatched: OK
7.37.0 with the 'Fix uninitialized variable use in NPN callback': still OK
7.37.0 with the additional 'call OPENSSL_config for initing engines' patch: error

So this commit broke it on my server:
https://github.com/bagder/curl/commit/c50ce859187cabecee5470a95a51c35bf73d3c47

OS: Windows 2008 R2
OpenSSL 1.0.1h
PHP-versions: 5.3-29-dev, 5.4.31, 5.5.15
So it happened both with PHP compiled with VC9 (5.3, 5.4) as with PHP compiled with VC11 (5.5).

Other point that is relevant: it only happened on the production server. I could not reproduce it on the developmet server, which runs Windows 2008 R2 as well. And also on my laptop with Windows 7 it did not happen.

Furthermore: it went wrong with every php-extension that used libcurl (php_http.dll for instance).

Discussion

  • Jan-E

    Jan-E - 2014-07-27

    x86 or x64 does not make any difference either. I tested it with a PHP 5.4.31 NTS x64 with only php_curl.dll (7.37.1) loaded. The same error keeps PHP from running: STR_COPY:variable has no value.

    Here is a screenshot of PHP 5.5.15 NTS with php_curl 7.37.1 and 7.36.0:
    http://elijst.nl/curl7371.gif

     

    Last edit: Jan-E 2014-07-27
  • Jan-E

    Jan-E - 2014-07-27

    Additional info: curl.exe has the same behaviour.

    C:\phpdev\php55bad.x32>curl --version
    Auto configuration failed
    6960:error:0E065068:configuration file routines:STR_COPY:variable has no value:.\crypto\conf\conf_def.c:618:line 37
    

    My laptop shows the version fine:

    C:\>curl --version
    curl 7.37.1 (i386-pc-win32) libcurl/7.37.1 OpenSSL/1.0.1h zlib/1.2.8 WinIDN libssh2/1.4.3
    Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
    Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM SPNEGO SSL SSPI libz
    

    I have added curl.exe plus bin/lib/include to http://elijst.nl/php55bad.x32.zip which contains all the files involved. If you are a Windows user: unzip it, go to the php-dir and run either 'php -v' or 'curl --version'.

     
  • Daniel Stenberg

    Daniel Stenberg - 2014-07-27

    The error message seems to be OpenSSL telling you that your config file is wrong (on line 37) so that it can't parse it correctly or similar.

    But also, that error shouldn't cause anything to stop ("It ignores all errors silently" the documentation says[*]).

    If you try to get a HTTPS page with curl, what exactly fails?

    [*] = http://wiki.openssl.org/index.php/Manual:OPENSSL_config%283%29

     
  • Daniel Stenberg

    Daniel Stenberg - 2014-07-27
    • assigned_to: Daniel Stenberg
     
  • Jan-E

    Jan-E - 2014-07-27

    There was a OPENSSL_CONF environment variable that pointed to a openssl.cfg which was apparently wrong. I changed the name of the variable to BAD_OPENSSL_CONF, opened a new DOS-prompt and the error was gone. Before changing that, I could not do anything with curl.

    This issue is a real tricky one. The openssl.cfg was put there by OpenVPN, but OpenVPN has not been running for ages. I guess a lot of people have left-overs of previously used programs, not only on Windows but also on Linux.

    If OpenSLL does not ignore those errors silently, many people will have broken systems without knowing where to look. In my case Apache would not restart because the PHP for mod_php was broken. Luckily I have a setup, where I can instantly switch to a working PHP version (/php55nts.x86 in stead of /php55nts.x32), so I could get Apache up'n'running again within seconds.

    Thanks for the help!

    Microsoft Windows [Version 6.1.7601]
    Copyright (c) 2009 Microsoft Corporation.  All rights reserved.
    
    C:\Users\Administrator>cd \phpdev\php55bad.x32
    
    C:\phpdev\php55bad.x32>set | find "openssl"
    BAD_OPENSSL_CONF=C:\Program Files (x86)\OpenVPN\bin\OpenSSL-Win64\bin\openssl.cfg
    
    C:\phpdev\php55bad.x32>curl https://sourceforge.net/
    curl: (60) SSL certificate problem: unable to get local issuer certificate
    More details here: http://curl.haxx.se/docs/sslcerts.html
    
    curl performs SSL certificate verification by default, using a "bundle"
     of Certificate Authority (CA) public keys (CA certs). If the default
     bundle file isn't adequate, you can specify an alternate file
     using the --cacert option.
    If this HTTPS server uses a certificate signed by a CA represented in
     the bundle, the certificate verification probably failed due to a
     problem with the certificate (it might be expired, or the name might
     not match the domain name in the URL).
    If you'd like to turn off curl's verification of the certificate, use
     the -k (or --insecure) option.
    
    C:\phpdev\php55bad.x32>php -v
    PHP 5.5.15 (cli) (built: Jul 23 2014 15:03:28)
    Copyright (c) 1997-2014 The PHP Group
    Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
    
     
  • Jan-E

    Jan-E - 2014-07-27

    If you take a look at the OpenSSL sources there are a lot of CONFerr() statements. CONFerr is defined in ./err/err.h and calls ERR_PUT_error(). The only way to silently ignore conf errors is to compile OpenSSL with a OPENSSL_NO_ERR flag. But then you suppress all errors, not only the config ones.

    If OpenSSL really wants to silently ignore conf errors this line:

    #define CONFerr(f,r) ERR_PUT_error(ERR_LIB_CONF,(f),(r),__FILE__,__LINE__)
    

    has to be changed into

    #define CONFerr(f,r) ERR_PUT_error(ERR_LIB_CONF,(f),(r),NULL,0)
    

    Now that I have a situation with a fatal config error, I can test it. I will recompile OpenSSL with the changes above and see if it removes the fatality.

     
  • Jan-E

    Jan-E - 2014-07-28

    No luck. I could suppress the error messages, but that was only suppressing the symptoms. If I did not suppress the messages, but removed the exit(1) in conf_sap.c the output became really weird:

    C:\phpdev\php55bad.x32>php -v
    Auto configuration failed
    7152:error:0E065068:configuration file routines:STR_COPY:variable has no value:.
    \crypto\conf\conf_def.c:618:line 37
    PHP 5.5.15 (cli) (built: Jul 23 2014 15:03:28)
    Copyright (c) 1997-2014 The PHP Group
    Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
    

    But trying to do some SSL-operations failed.

    My conclusion at the moment: as long as no program loads the faulty openssl.cnf everything works OK. But as soon as a program executes a OPENSSL_config(NULL) things get screwed up. curl.exe and php_curl.dll do not use openssl.cnf, so is not better to avoid laoding it?

    The situation with the known bug #83 seems better to me than breaking php when people have a faulty openssl.cnf.

     
  • Jan-E

    Jan-E - 2014-07-28

    As an afterthought, curl already has the option

    -K, --config FILE   Specify which config file to read
    

    How about this idea:
    1. If the option is not set, do not execute OPENSSL_config(NULL)
    2. It it is set as --config NULL execute OPENSSL_config(NULL)
    3. If it is set as --config FILE execute OPENSSL_config(FILE)

    Would not that be a solution for the original known bug #83 without breaking setups like mine?

    Edit: Oops, that seems to be another config file. Well, an extra option --sslconfig could achieve the same thing.

     

    Last edit: Jan-E 2014-07-28
  • Daniel Stenberg

    Daniel Stenberg - 2014-07-28

    In my opinion, this being a fatal problem or not is because you have a bad setup and OpenSSL insists on that being very bad - so bad it exits with failure. I see nothing in this that says curl or libcurl do things wrong.

    curl needs to call OPENSSL_config() to work properly in other areas so going back to the previous state is not a solution either.

    Why do you insist to be able to run with a broken setup? and why is it curl's job to ignore it?

    As you mention, --config cannot be used for this and while curl/libcurl could in theory add such an option I've A) not seen a very convincing argument for it and B) I fight against every option that is SSL-library specific and this is another such one.

     
  • Jan-E

    Jan-E - 2014-07-28

    Up until 7.37.0 curl did not load openssl.cnf, now it does. Even with a openssl.cnf without errors this is a change of behaviour that could break things. Take a look at drook's comment in #83: https://sourceforge.net/p/curl/bugs/1208/#488e

    If a PHP extension deliberately loads another than the default openssl.cnf it now depends on the extension loading order which openssl.cnf is actually loaded. drook's intention was to find a solution "regardless of the module loading order". As far as I can see, with libcurl 7.37.1 the loading order now matters.

    PHP ignores any OPENSSL_CONF environment setting, see https://bugs.php.net/bug.php?id=64501. PHP is distributed with a openssl.cnf in the extras/ssl directory. What if php-curl tries to load the OPENSSL_CONF one and other extensions the one from extras/ssl? Which one will prevail? Maybe Pierre Joye can shine a light on this. I will point him to this discussion.

    Edit: see your own comment in http://curl.haxx.se/mail/lib-2014-06/0004.html as well

     

    Last edit: Jan-E 2014-07-28
  • Daniel Stenberg

    Daniel Stenberg - 2014-07-29

    The OpenSSL wiki page says this: "It is strongly recommended that all new applications call OPENSSL_config()". strongly recommended they say.

    Arguing that curl didn't "load openssl.cnf" before 7.37.1 isn't that interesting. Whenever curl has a bug that we fix, there was something it didn't do until we fixed it. Not calling OPENSSL_config() clearly was a bug since adding the call fixed one or more problems.

    We knew there was a possibility for problems with this function as that had been reported before. But we also didn't have any other means to for example fix the engine problem.

    I can't answer to what PHP does and what it should or shouldn't do. I need to focus on what libcurl should do when it uses OpenSSL. So please in this bug tracker we focus on what we think libcurl should do. libcurl is used by thousands of other applications and users, not just PHP.

    I still believe you're mostly identifying problems in OPENSSL_config(). Do you agree? Have you discussed this matter with the OpenSSL team?

    Finally, I would also like to point out that libcurl has a flag to curl_global_init() that makes it avoid "initing" openssl, which is intended to be used by users who use openssl in several places. Perhaps that's for you?

     
  • Daniel Stenberg

    Daniel Stenberg - 2014-07-30
    • summary: STR_COPY:variable has no value --> OPENSSL_config causes app to exit with bad conf file
     
  • Jan-E

    Jan-E - 2014-08-06

    I still owed you an answer to your questions. As a starter: yes, it is mainly on OpenSSL issue. With that in mind, you could close the bug as 'will not fix'.

    In the mean time, I checked what was wrong with the OpenSSL.cnf. Actually it was not entirely wrong, but libcurl opened it out-of-context. The OpenSSL.cnf was installed for Easy-RSA as part of OpenVPN 2.2.2. A current version of the config file can be found here:
    https://github.com/OpenVPN/easy-rsa/blob/master/easyrsa3/openssl-1.0.cnf

    As you can see, quite a lot of $ENV variables have to be set before this openssl.cnf can be opened wthout errors. Curl did (of course) not set those variables and OpenSSL did not silently ignore the errors. Result: PHP with the curl extension exited immediately and the Apache that used PHP as mod_php did not start as well.

    OpenSSL strongly recommends "that all new applications call OPENSSL_config() or the more sophisticated functions such as CONF_modules_load() during initialization (that is before starting any threads)." In your message http://curl.haxx.se/mail/lib-2014-06/0004.html you were still looking if any "more sophisticated function" could do the job. You asked Дмитрий Фалько a question about that, got no answer and then pushed the change. A remarkable chain of events.

    Anyway: close the bug if you want to. And let us hope that not many people have installed easy-rsa as their only OpenSSL client.

     
  • Jan-E

    Jan-E - 2014-08-06

    I now see that you got an answer to your question to Дмитрий Фалько:
    http://curl.haxx.se/mail/lib-2014-06/0005.html

    He tried

    CONF_modules_load_file(NULL, NULL, CONF_MFLAGS_DEFAULT_SECTION|CONF_MFLAGS_IGNORE_MISSING_FILE)
    

    But it seems to drill down to the same: loading OpenSSL.cnf.

     

    Last edit: Jan-E 2014-08-06
  • Jan-E

    Jan-E - 2014-08-06

    I recompiled PHP with

    CONF_modules_load_file(NULL, NULL, CONF_MFLAGS_DEFAULT_SECTION|CONF_MFLAGS_IGNORE_MISSING_FILE);
    

    in stead of

    OPENSSL_config(NULL);
    

    That did not break PHP. See the output below. This is with CONF_modules_load_file:

    C:\php55bad.x32>php -v
    PHP 5.5.15 (cli) (built: Jul 23 2014 15:03:28)
    Copyright (c) 1997-2014 The PHP Group
    Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
    C:\php55bad.x32>php curl_v.php
    7.37.1
    

    And this with OPENSSL_config:

    C:\php55bad.x32>php -v
    Auto configuration failed
    5536:error:0E065068:configuration file routines:STR_COPY:variable has no value:.
    \crypto\conf\conf_def.c:618:line 37 
    C:\php55bad.x32>php curl_v.php
    Auto configuration failed
    6576:error:0E065068:configuration file routines:STR_COPY:variable has no value:.
    \crypto\conf\conf_def.c:618:line 37
    

    For reference, this is curl_v.php:

    C:\php55bad.x32>type curl_v.php
    <?php $c = curl_version(); echo $c['version'];?>
    

    If this really solves Дмитрий's problem like he said in http://curl.haxx.se/mail/lib-2014-06/0005.html you should use CONF_modules_load_file(NULL, NULL, CONF_MFLAGS_DEFAULT_SECTION|CONF_MFLAGS_IGNORE_MISSING_FILE) as well.

     
  • Jan-E

    Jan-E - 2014-08-07

    Looking at OpenSSL's code explains why OPENSSL_config(NULL) fails with a 'bad' OpenSSL.cnf:
    https://github.com/openssl/openssl/blob/master/crypto/conf/conf_sap.c#L91

    It exits in those cases with 'Auto configuration failed'. OPENSSL_config(NULL) does not silently ignore errors, but the more sophisticated function CONF_modules_load_file(NULL, NULL, CONF_MFLAGS_DEFAULT_SECTION|CONF_MFLAGS_IGNORE_MISSING_FILE) does ignore them.

    I will make an issue of this at OpenSSL. Either they should change the documentation at http://wiki.openssl.org/index.php/Manual:OPENSSL_config(3) or they should not exit with the 'Auto configuration failed' error.

    In the mean time, for Curl it seems better to use:

    CONF_modules_load_file(NULL, NULL, CONF_MFLAGS_DEFAULT_SECTION|CONF_MFLAGS_IGNORE_MISSING_FILE);
    

    in stead of

    OPENSSL_config(NULL);
    

    Edit: typo's

     

    Last edit: Jan-E 2014-08-07
  • Daniel Stenberg

    Daniel Stenberg - 2014-08-07

    Yeps, I have come to the same conclusion in the mean time and I've run all tests with a patch here. I also emailed openssl about their use of exit() in OPENSSL_config which I think is wrong.

    Fix just pushed now!

     
  • Jan-E

    Jan-E - 2014-08-07

    OK, thanks. Not that I am struggling with this bug anymore, but for all those others...

     
  • Daniel Stenberg

    Daniel Stenberg - 2014-08-07
    • status: open --> closed-fixed
     
  • Daniel Stenberg

    Daniel Stenberg - 2014-08-07

    Closing this issue as it is now fixed. Thanks!

     
  • Jan-E

    Jan-E - 2014-08-07

    Your contact with OpenSSL has everything of a culture clash. Thanks for trying. Maybe they will adjust the documentation.