From vd@datamax.bg Mon Apr 18 15:23:53 2005 Return-Path: Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4373516A4CE for ; Mon, 18 Apr 2005 15:23:53 +0000 (GMT) Received: from jengal.datamax.bg (jengal.datamax.bg [82.103.104.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5931343D49 for ; Mon, 18 Apr 2005 15:23:52 +0000 (GMT) (envelope-from vd@datamax.bg) Received: from sinanica.bg.datamax (sinanica.bg.datamax [192.168.10.1]) by jengal.datamax.bg (Postfix) with QMQP id B1B1E87CB for ; Mon, 18 Apr 2005 18:23:50 +0300 (EEST) Received: (nullmailer pid 84877 invoked by uid 1004); Mon, 18 Apr 2005 15:23:50 -0000 Message-Id: <1113837830.709566.84876.nullmailer@sinanica.bg.datamax> Date: Mon, 18 Apr 2005 18:23:50 +0300 From: Vasil Dimov Reply-To: Vasil Dimov To: FreeBSD-gnats-submit@freebsd.org Subject: Bug in OpenSSL's sk_insert() causes segfaults in other OpenSSL functions. X-Send-Pr-Version: 3.113 X-GNATS-Notify: >Number: 80074 >Category: bin >Synopsis: [patch] openssl(1): Bug in OpenSSL's sk_insert() causes segfaults in other OpenSSL functions. >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Apr 18 15:30:24 GMT 2005 >Closed-Date: Fri Jan 25 21:54:36 UTC 2008 >Last-Modified: Fri Jan 25 21:54:36 UTC 2008 >Originator: Vasil Dimov >Release: FreeBSD 4.11-STABLE i386 >Organization: DataMax >Environment: System: FreeBSD f4.bg.datamax 4.11-STABLE FreeBSD 4.11-STABLE #0: Tue Apr 12 13:06:12 GMT 2005 root@f4.bg.datamax:/usr/obj/usr/src/sys/F4 i386 >Description: This PR is also sent to the OpenSSL developers via their Request Tracker system as it is not FreeBSD related. It can be located at: http://marc.theaimsgroup.com/?l=openssl-dev&m=111383480516715&w=2 http://www.aet.TU-Cottbus.DE/rt2/Ticket/Display.html?id=1047 There is a bug in sk_insert() function which (sometimes) causes segfaults. sk_insert() breaks consistency of the stack in which it inserts elements by not NULL-terminating it when element is inserted on the last position (over the last (NULL) element). This then causes bogus pointers dereferencing and signal 11 to be received by the program. for example imagine the following stack: st->num = 3 A B C NULL bogus bogus ... after insering element D at position 4 stack looks like this: st->num = 4 A B C D bogus bogus ... but should be like this: st->num = 4 A B C D NULL bogus ... The reason crashes do not occur every time is that memory returned by OPENSSL_realloc in most cases is initialized with zeroes (bogus==NULL in the above example). ---- output from openssl's make report { OpenSSL self-test report: OpenSSL version: 0.9.7d Last change: Fix null-pointer assignment in do_change_cipher_spec() ... Options: no-krb5 OS (uname): FreeBSD f4.bg.datamax 4.11-STABLE FreeBSD 4.11-STABLE #0: Tue Apr 12 13:06:12 GMT 2005 root@f4.bg.datamax:/usr/obj/usr/src/sys/F4 i386 OS (config): i386-pc-freebsd4.11 Target (default): FreeBSD-elf Target: dist Compiler: Using builtin specs. gcc version 2.95.4 20020320 [FreeBSD] Test passed. ---- } output from openssl's make report As I see this would also be a problem in the latest version 0.9.7g. ---- backtrace { Core was generated by `php'. Program terminated with signal 11, Segmentation fault. #0 0x283cf537 in engine_table_select (table=0x2840cc80, nid=1) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/engine/eng_table.c:306 306 if((ret->funct_ref > 0) || !(table_flags & ENGINE_TABLE_FLAG_NOINIT)) #0 0x283cf537 in engine_table_select (table=0x2840cc80, nid=1) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/engine/eng_table.c:306 #1 0x2839c7de in ENGINE_get_default_RSA () at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/engine/tb_rsa.c:106 #2 0x2839a165 in RSA_new_method (engine=0x0) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/rsa/rsa_lib.c:156 #3 0x28399f58 in RSA_new () at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/rsa/rsa_lib.c:77 #4 0x283c0c71 in rsa_cb (operation=0, pval=0xbfbfdcf4, it=0x28414874) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/rsa/rsa_asn1.c:80 #5 0x283d227c in asn1_item_ex_combine_new (pval=0xbfbfdcf4, it=0x28414874, combine=0) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/asn1/tasn_new.c:160 #6 0x283d1fe4 in ASN1_item_ex_new (pval=0xbfbfdcf4, it=0x28414874) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/asn1/tasn_new.c:83 #7 0x283cc488 in ASN1_item_ex_d2i (pval=0xbfbfdcf4, in=0xbfbfdda0, len=137, it=0x28414874, tag=16, aclass=0, opt=0 '\000', ctx=0xbfbfdcf8) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/asn1/tasn_dec.c:317 #8 0x283cbd54 in ASN1_item_d2i (pval=0xbfbfdcf4, in=0xbfbfdda0, len=140, it=0x28414874) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/asn1/tasn_dec.c:115 #9 0x283c0d7c in d2i_RSAPublicKey (a=0x0, in=0xbfbfdda0, len=140) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/rsa/rsa_asn1.c:111 #10 0x283b62b2 in d2i_PublicKey (type=6, a=0x0, pp=0xbfbfdda0, length=140) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/asn1/d2i_pu.c:93 #11 0x283b465a in X509_PUBKEY_get (key=0x81c89d0) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/asn1/x_pubkey.c:194 #12 0x283b2fe5 in X509_get_pubkey (x=0x81c2a80) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/x509/x509_cmp.c:362 #13 0x283311f2 in X509_get_pubkey_parameters (pkey=0x0, chain=0x81c6d20) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/x509/x509_vfy.c:902 #14 0x2832fe71 in X509_verify_cert (ctx=0xbfbfdea4) at /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/x509/x509_vfy.c:295 #15 0x282c2ecc in ssl_verify_cert_chain (s=0x81c7e00, sk=0x81c6700) at /usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/ssl_cert.c:494 #16 0x282a8b4c in ssl3_get_server_certificate (s=0x81c7e00) at /usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:833 #17 0x282a78e1 in ssl3_connect (s=0x81c7e00) at /usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:275 #18 0x282bad4d in SSL_connect (s=0x81c7e00) at /usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/ssl_lib.c:824 #19 0x282a5e99 in ssl23_get_server_hello (s=0x81c7e00) at /usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s23_clnt.c:486 #20 0x282a54cc in ssl23_connect (s=0x81c7e00) at /usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s23_clnt.c:187 #21 0x282bad4d in SSL_connect (s=0x81c7e00) at /usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/ssl_lib.c:824 #22 0x2827eb4c in Curl_SSLConnect (conn=0x817d000, sockindex=0) at ssluse.c:1504 #23 0x2826b8a3 in Curl_http_connect (conn=0x817d000, done=0xbfbfe2b6 "") at http.c:1262 #24 0x282784be in Curl_protocol_connect (conn=0x817d000, protocol_done=0xbfbfe2b6 "") at url.c:2099 #25 0x2827ab51 in SetupConnection (conn=0x817d000, hostaddr=0x81794a0, protocol_done=0xbfbfe2b6 "") at url.c:3513 #26 0x2827ac4e in Curl_connect (data=0x818c000, in_connect=0xbfbfe2f4, asyncp=0xbfbfe2b7 "", protocol_done=0xbfbfe2b6 "") at url.c:3569 #27 0x28287ec0 in Curl_connect_host (data=0x818c000, conn=0xbfbfe2f4) at transfer.c:2054 #28 0x2828810c in Curl_perform (data=0x818c000) at transfer.c:2142 #29 0x28288ab8 in curl_easy_perform (curl=0x818c000) at easy.c:474 #30 0x2825956d in zif_curl_exec (ht=1, return_value=0x81843ec, this_ptr=0x0, return_value_used=0) at /usr/ports/ftp/php4-curl/work/php-4.3.11/ext/curl/curl.c:1127 #31 0x8102425 in execute (op_array=0x818340c) at /usr/ports/lang/php4-cli/work/php-4.3.11/Zend/zend_execute.c:1654 #32 0x80f0bad in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /usr/ports/lang/php4-cli/work/php-4.3.11/Zend/zend.c:926 #33 0x80c0753 in php_execute_script (primary_file=0xbfbffb54) at /usr/ports/lang/php4-cli/work/php-4.3.11/main/main.c:1745 #34 0x8108087 in main (argc=2, argv=0xbfbffbd0) at /usr/ports/lang/php4-cli/work/php-4.3.11/sapi/cli/php_cli.c:828 ---- } backtrace (gdb) ins *ret Cannot access memory at address 0x726f7020. (gdb) The "goto trynext;" loop walks on the stack waiting for NULL element to indicate "stack end" but hits a invalid pointer instead and crashes while trying to dereference it. >How-To-Repeat: Huh, really complicated, probably there is an easier way. On a FreeBSD4.11 i386 machine do the following: $ cd /usr/ports/lang/php4 && make install $ cd /usr/ports/ftp/php4-curl && make install $ cd /usr/ports/databases/php4-mysql && make install $ cd /usr/ports/security/php4-openssl && make install then make sure /usr/local/etc/php/extensions.ini contains extension=curl.so extension=mysql.so extension=openssl.so exactly in that order. Other orders do not produce the crash - OPENSSL_realloc() returns 0x0-initialized memory. create foo.php that contains: and then execute it: $ php foo.php Segmentation fault (core dumped) $ Notice that even changing the order of shared objects in extensions.ini does not produce a crash so this will (probably) be very difficult to reproduce on a machine with different setup. I suggest altering malloc/realloc functions to initialize returned memory with non-zeros - than the bug will be easier to reproduce. Thanks to Toni Viemero for finding and reporting this problem to the FreeBSD GNATS system: http://www.FreeBSD.org/cgi/query-pr.cgi?pr=79617 >Fix: ---- stack.c.patch { --- stack.c.orig Mon Apr 18 09:49:02 2005 +++ stack.c Mon Apr 18 14:06:14 2005 @@ -172,6 +172,7 @@ st->data[loc]=data; } st->num++; + st->data[st->num]=NULL; st->sorted=0; return(st->num); } ---- } stack.c.patch >Release-Note: >Audit-Trail: From: Kris Kennaway To: Vasil Dimov Cc: FreeBSD-gnats-submit@FreeBSD.org Subject: Re: bin/80074: Bug in OpenSSL's sk_insert() causes segfaults in other OpenSSL functions. Date: Mon, 18 Apr 2005 23:12:51 -0700 On Mon, Apr 18, 2005 at 06:23:50PM +0300, Vasil Dimov wrote: > This PR is also sent to the OpenSSL developers via their Request Tracker > system as it is not FreeBSD related. It can be located at: > http://marc.theaimsgroup.com/?l=openssl-dev&m=111383480516715&w=2 > http://www.aet.TU-Cottbus.DE/rt2/Ticket/Display.html?id=1047 Usually this means you shouldn't submit it here, then. Once they fix it, it will be imported into FreeBSD with a future release. Kris From: Vasil Dimov To: Kris Kennaway Cc: bug-followup@FreeBSD.org Subject: Re: bin/80074: Bug in OpenSSL's sk_insert() causes segfaults in other OpenSSL functions. Date: Tue, 19 Apr 2005 12:43:53 +0300 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Usually this means you shouldn't submit it here, then. Once they fix > it, it will be imported into FreeBSD with a future release. Why should we wait for openssl developers to fix the problem and even more - wait for the next openssl release? Except from using openssl developers as a competent bugfix approvers. -----BEGIN PGP SIGNATURE----- iD8DBQFCZNLZFw6SP/bBpCARAqwyAKDB7OWBJsa9NbR5gW+BCSYe/uGEyACdGQPS NvObSAGSZRhgPsvGVbEQcGU= =g0G3 -----END PGP SIGNATURE----- From: Kris Kennaway To: Vasil Dimov Cc: Kris Kennaway , bug-followup@FreeBSD.org Subject: Re: bin/80074: Bug in OpenSSL's sk_insert() causes segfaults in other OpenSSL functions. Date: Tue, 19 Apr 2005 10:49:49 -0700 On Tue, Apr 19, 2005 at 12:43:53PM +0300, Vasil Dimov wrote: > > Usually this means you shouldn't submit it here, then. Once they fix > > it, it will be imported into FreeBSD with a future release. > > Why should we wait for openssl developers to fix the problem and even > more - wait for the next openssl release? > > Except from using openssl developers as a competent bugfix approvers. Because generally FreeBSD does not develop the third party software like OpenSSL, we only import the results of the work of the OpenSSL developers. Kris From: Vasil Dimov To: Kris Kennaway Cc: bug-followup@FreeBSD.org Subject: Re: bin/80074: Bug in OpenSSL's sk_insert() causes segfaults in other OpenSSL functions. Date: Wed, 20 Apr 2005 10:22:14 +0300 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 It seems that this issue is already solved (in a different way) in OpenSSL 0.9.7g. I suggest we should apply differences between 0.9.7d and 0.9.7g's crypto/stack/stack.c to our /usr/src/crypto/openssl/crypto/stack/stack.c or just sk_value() and sk_set() implementations? http://marc.theaimsgroup.com/?t=111383482000004&r=1&w=2 -----BEGIN PGP SIGNATURE----- iD8DBQFCZgMmFw6SP/bBpCARAgvVAKDPSYOzvl/QoCEMZlLuZG6ItGicqACdFVit kAc22/zfBCKMmhk9csMiFBc= =zqO7 -----END PGP SIGNATURE----- State-Changed-From-To: open->closed State-Changed-By: linimon State-Changed-When: Fri Jan 25 21:54:07 UTC 2008 State-Changed-Why: This bug is believed to be fixed in the version of OpenSSL that is currently in FreeBSD. http://www.freebsd.org/cgi/query-pr.cgi?pr=80074 >Unformatted: