From nobody@FreeBSD.org Wed Mar 16 14:43:42 2011 Return-Path: Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D054F1065670 for ; Wed, 16 Mar 2011 14:43:42 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22]) by mx1.freebsd.org (Postfix) with ESMTP id BD79C8FC24 for ; Wed, 16 Mar 2011 14:43:42 +0000 (UTC) Received: from red.freebsd.org (localhost [127.0.0.1]) by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p2GEhgwE005383 for ; Wed, 16 Mar 2011 14:43:42 GMT (envelope-from nobody@red.freebsd.org) Received: (from nobody@localhost) by red.freebsd.org (8.14.4/8.14.4/Submit) id p2GEhg3H005370; Wed, 16 Mar 2011 14:43:42 GMT (envelope-from nobody) Message-Id: <201103161443.p2GEhg3H005370@red.freebsd.org> Date: Wed, 16 Mar 2011 14:43:42 GMT From: Heribert Steuer To: freebsd-gnats-submit@FreeBSD.org Subject: select() does not return EBADF on closed file descriptor X-Send-Pr-Version: www-3.1 X-GNATS-Notify: >Number: 155606 >Category: kern >Synopsis: [libc] select() does not return EBADF on closed file descriptor >Confidential: no >Severity: serious >Priority: medium >Responsible: kib >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Mar 16 14:50:08 UTC 2011 >Closed-Date: Sun Nov 27 19:45:58 UTC 2011 >Last-Modified: Sun Nov 27 19:45:58 UTC 2011 >Originator: Heribert Steuer >Release: 8.1-RELEASE >Organization: >Environment: FreeBSD opusv2p.lr.invalid 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:55:53 UTC 2010 root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 >Description: I have tracked down a problem that came up while using the quickfix C++ library on a FreeBSD 8.1 server. We found that the select() system call does not return with a value < 0 and errno set when called on a closed socket. This seems to be related to a threaded environment. Our logs stated, that this happens, whenever a thread context switch occurs between the close() and the select() system calls. If there is no context switch in between, select() returns -1 and errno is set to 9. This is the expected behaviour. The process is reproducable. Unfortunately I cannot provide example code to reproduce this scenario. >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: From: Florian Smeets To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/155606: [libc] select() does not return EBADF on closed file descriptor Date: Fri, 07 Oct 2011 23:27:23 +0200 This is reproducible with nspr's regression testsuite. The test is called selct_er. Just go to devel/nspr and run make && make test, the selct_er test will hang indefinitely. cognet found out the commit causing this behavior is supposedly: http://svnweb.freebsd.org/base/head/sys/kern/sys_generic.c?r1=195259&r2=195258&pathrev=195259 From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/155606: commit references a PR Date: Sun, 13 Nov 2011 10:28:15 +0000 (UTC) Author: kib Date: Sun Nov 13 10:28:01 2011 New Revision: 227485 URL: http://svn.freebsd.org/changeset/base/227485 Log: To limit amount of the kernel memory allocated, and to optimize the iteration over the fdsets, kern_select() limits the length of the fdsets copied in by the last valid file descriptor index. If any bit is set in a mask above the limit, current implementation ignores the filedescriptor, instead of returning EBADF. Fix the issue by scanning the tails of fdset before entering the select loop and returning EBADF if any bit above last valid filedescriptor index is set. The performance impact of the additional check is only imposed on the (somewhat) buggy applications that pass bad file descriptors to select(2) or pselect(2). PR: kern/155606, kern/162379 Discussed with: cognet, glebius Tested by: andreast (powerpc, all 64/32bit ABI combinations, big-endian), marius (sparc64, big-endian) MFC after: 2 weeks Modified: head/sys/kern/sys_generic.c Modified: head/sys/kern/sys_generic.c ============================================================================== --- head/sys/kern/sys_generic.c Sun Nov 13 06:39:49 2011 (r227484) +++ head/sys/kern/sys_generic.c Sun Nov 13 10:28:01 2011 (r227485) @@ -831,6 +831,54 @@ sys_select(struct thread *td, struct sel NFDBITS)); } +/* + * In the unlikely case when user specified n greater then the last + * open file descriptor, check that no bits are set after the last + * valid fd. We must return EBADF if any is set. + * + * There are applications that rely on the behaviour. + * + * nd is fd_lastfile + 1. + */ +static int +select_check_badfd(fd_set *fd_in, int nd, int ndu, int abi_nfdbits) +{ + char *addr, *oaddr; + int b, i, res; + uint8_t bits; + + if (nd >= ndu || fd_in == NULL) + return (0); + + oaddr = NULL; + bits = 0; /* silence gcc */ + for (i = nd; i < ndu; i++) { + b = i / NBBY; +#if BYTE_ORDER == LITTLE_ENDIAN + addr = (char *)fd_in + b; +#else + addr = (char *)fd_in; + if (abi_nfdbits == NFDBITS) { + addr += rounddown(b, sizeof(fd_mask)) + + sizeof(fd_mask) - 1 - b % sizeof(fd_mask); + } else { + addr += rounddown(b, sizeof(uint32_t)) + + sizeof(uint32_t) - 1 - b % sizeof(uint32_t); + } +#endif + if (addr != oaddr) { + res = fubyte(addr); + if (res == -1) + return (EFAULT); + oaddr = addr; + bits = res; + } + if ((bits & (1 << (i % NBBY))) != 0) + return (EBADF); + } + return (0); +} + int kern_select(struct thread *td, int nd, fd_set *fd_in, fd_set *fd_ou, fd_set *fd_ex, struct timeval *tvp, int abi_nfdbits) @@ -845,14 +893,26 @@ kern_select(struct thread *td, int nd, f fd_mask s_selbits[howmany(2048, NFDBITS)]; fd_mask *ibits[3], *obits[3], *selbits, *sbp; struct timeval atv, rtv, ttv; - int error, timo; + int error, lf, ndu, timo; u_int nbufbytes, ncpbytes, ncpubytes, nfdbits; if (nd < 0) return (EINVAL); fdp = td->td_proc->p_fd; - if (nd > fdp->fd_lastfile + 1) - nd = fdp->fd_lastfile + 1; + ndu = nd; + lf = fdp->fd_lastfile; + if (nd > lf + 1) + nd = lf + 1; + + error = select_check_badfd(fd_in, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); + error = select_check_badfd(fd_ou, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); + error = select_check_badfd(fd_ex, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); /* * Allocate just enough bits for the non-null fd_sets. Use the _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" State-Changed-From-To: open->patched State-Changed-By: kib State-Changed-When: Sun Nov 27 18:45:23 UTC 2011 State-Changed-Why: Take. Responsible-Changed-From-To: freebsd-bugs->kib Responsible-Changed-By: kib Responsible-Changed-When: Sun Nov 27 18:45:23 UTC 2011 Responsible-Changed-Why: Take. http://www.freebsd.org/cgi/query-pr.cgi?pr=155606 From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/155606: commit references a PR Date: Sun, 27 Nov 2011 18:49:39 +0000 (UTC) Author: kib Date: Sun Nov 27 18:49:16 2011 New Revision: 228032 URL: http://svn.freebsd.org/changeset/base/228032 Log: MFC r227485: To limit amount of the kernel memory allocated, and to optimize the iteration over the fdsets, kern_select() limits the length of the fdsets copied in by the last valid file descriptor index. If any bit is set in a mask above the limit, current implementation ignores the filedescriptor, instead of returning EBADF. Fix the issue by scanning the tails of fdset before entering the select loop and returning EBADF if any bit above last valid filedescriptor index is set. The performance impact of the additional check is only imposed on the (somewhat) buggy applications that pass bad file descriptors to select(2) or pselect(2). PR: kern/155606, kern/162379 Approved by: re (bz) Modified: stable/9/sys/kern/sys_generic.c Directory Properties: stable/9/sys/ (props changed) Modified: stable/9/sys/kern/sys_generic.c ============================================================================== --- stable/9/sys/kern/sys_generic.c Sun Nov 27 17:51:13 2011 (r228031) +++ stable/9/sys/kern/sys_generic.c Sun Nov 27 18:49:16 2011 (r228032) @@ -831,6 +831,54 @@ sys_select(struct thread *td, struct sel NFDBITS)); } +/* + * In the unlikely case when user specified n greater then the last + * open file descriptor, check that no bits are set after the last + * valid fd. We must return EBADF if any is set. + * + * There are applications that rely on the behaviour. + * + * nd is fd_lastfile + 1. + */ +static int +select_check_badfd(fd_set *fd_in, int nd, int ndu, int abi_nfdbits) +{ + char *addr, *oaddr; + int b, i, res; + uint8_t bits; + + if (nd >= ndu || fd_in == NULL) + return (0); + + oaddr = NULL; + bits = 0; /* silence gcc */ + for (i = nd; i < ndu; i++) { + b = i / NBBY; +#if BYTE_ORDER == LITTLE_ENDIAN + addr = (char *)fd_in + b; +#else + addr = (char *)fd_in; + if (abi_nfdbits == NFDBITS) { + addr += rounddown(b, sizeof(fd_mask)) + + sizeof(fd_mask) - 1 - b % sizeof(fd_mask); + } else { + addr += rounddown(b, sizeof(uint32_t)) + + sizeof(uint32_t) - 1 - b % sizeof(uint32_t); + } +#endif + if (addr != oaddr) { + res = fubyte(addr); + if (res == -1) + return (EFAULT); + oaddr = addr; + bits = res; + } + if ((bits & (1 << (i % NBBY))) != 0) + return (EBADF); + } + return (0); +} + int kern_select(struct thread *td, int nd, fd_set *fd_in, fd_set *fd_ou, fd_set *fd_ex, struct timeval *tvp, int abi_nfdbits) @@ -845,14 +893,26 @@ kern_select(struct thread *td, int nd, f fd_mask s_selbits[howmany(2048, NFDBITS)]; fd_mask *ibits[3], *obits[3], *selbits, *sbp; struct timeval atv, rtv, ttv; - int error, timo; + int error, lf, ndu, timo; u_int nbufbytes, ncpbytes, ncpubytes, nfdbits; if (nd < 0) return (EINVAL); fdp = td->td_proc->p_fd; - if (nd > fdp->fd_lastfile + 1) - nd = fdp->fd_lastfile + 1; + ndu = nd; + lf = fdp->fd_lastfile; + if (nd > lf + 1) + nd = lf + 1; + + error = select_check_badfd(fd_in, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); + error = select_check_badfd(fd_ou, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); + error = select_check_badfd(fd_ex, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); /* * Allocate just enough bits for the non-null fd_sets. Use the _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/155606: commit references a PR Date: Sun, 27 Nov 2011 19:01:07 +0000 (UTC) Author: kib Date: Sun Nov 27 19:00:52 2011 New Revision: 228034 URL: http://svn.freebsd.org/changeset/base/228034 Log: MFC r227485: To limit amount of the kernel memory allocated, and to optimize the iteration over the fdsets, kern_select() limits the length of the fdsets copied in by the last valid file descriptor index. If any bit is set in a mask above the limit, current implementation ignores the filedescriptor, instead of returning EBADF. Fix the issue by scanning the tails of fdset before entering the select loop and returning EBADF if any bit above last valid filedescriptor index is set. The performance impact of the additional check is only imposed on the (somewhat) buggy applications that pass bad file descriptors to select(2) or pselect(2). PR: kern/155606, kern/162379 Approved by: re (bz) Modified: releng/9.0/sys/kern/sys_generic.c Directory Properties: releng/9.0/sys/ (props changed) Modified: releng/9.0/sys/kern/sys_generic.c ============================================================================== --- releng/9.0/sys/kern/sys_generic.c Sun Nov 27 18:56:04 2011 (r228033) +++ releng/9.0/sys/kern/sys_generic.c Sun Nov 27 19:00:52 2011 (r228034) @@ -831,6 +831,54 @@ sys_select(struct thread *td, struct sel NFDBITS)); } +/* + * In the unlikely case when user specified n greater then the last + * open file descriptor, check that no bits are set after the last + * valid fd. We must return EBADF if any is set. + * + * There are applications that rely on the behaviour. + * + * nd is fd_lastfile + 1. + */ +static int +select_check_badfd(fd_set *fd_in, int nd, int ndu, int abi_nfdbits) +{ + char *addr, *oaddr; + int b, i, res; + uint8_t bits; + + if (nd >= ndu || fd_in == NULL) + return (0); + + oaddr = NULL; + bits = 0; /* silence gcc */ + for (i = nd; i < ndu; i++) { + b = i / NBBY; +#if BYTE_ORDER == LITTLE_ENDIAN + addr = (char *)fd_in + b; +#else + addr = (char *)fd_in; + if (abi_nfdbits == NFDBITS) { + addr += rounddown(b, sizeof(fd_mask)) + + sizeof(fd_mask) - 1 - b % sizeof(fd_mask); + } else { + addr += rounddown(b, sizeof(uint32_t)) + + sizeof(uint32_t) - 1 - b % sizeof(uint32_t); + } +#endif + if (addr != oaddr) { + res = fubyte(addr); + if (res == -1) + return (EFAULT); + oaddr = addr; + bits = res; + } + if ((bits & (1 << (i % NBBY))) != 0) + return (EBADF); + } + return (0); +} + int kern_select(struct thread *td, int nd, fd_set *fd_in, fd_set *fd_ou, fd_set *fd_ex, struct timeval *tvp, int abi_nfdbits) @@ -845,14 +893,26 @@ kern_select(struct thread *td, int nd, f fd_mask s_selbits[howmany(2048, NFDBITS)]; fd_mask *ibits[3], *obits[3], *selbits, *sbp; struct timeval atv, rtv, ttv; - int error, timo; + int error, lf, ndu, timo; u_int nbufbytes, ncpbytes, ncpubytes, nfdbits; if (nd < 0) return (EINVAL); fdp = td->td_proc->p_fd; - if (nd > fdp->fd_lastfile + 1) - nd = fdp->fd_lastfile + 1; + ndu = nd; + lf = fdp->fd_lastfile; + if (nd > lf + 1) + nd = lf + 1; + + error = select_check_badfd(fd_in, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); + error = select_check_badfd(fd_ou, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); + error = select_check_badfd(fd_ex, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); /* * Allocate just enough bits for the non-null fd_sets. Use the _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/155606: commit references a PR Date: Sun, 27 Nov 2011 19:36:48 +0000 (UTC) Author: kib Date: Sun Nov 27 19:36:36 2011 New Revision: 228038 URL: http://svn.freebsd.org/changeset/base/228038 Log: MFC r227485: To limit amount of the kernel memory allocated, and to optimize the iteration over the fdsets, kern_select() limits the length of the fdsets copied in by the last valid file descriptor index. If any bit is set in a mask above the limit, current implementation ignores the filedescriptor, instead of returning EBADF. Fix the issue by scanning the tails of fdset before entering the select loop and returning EBADF if any bit above last valid filedescriptor index is set. The performance impact of the additional check is only imposed on the (somewhat) buggy applications that pass bad file descriptors to select(2) or pselect(2). PR: kern/155606, kern/162379 Modified: stable/8/sys/kern/sys_generic.c Directory Properties: stable/8/sys/ (props changed) Modified: stable/8/sys/kern/sys_generic.c ============================================================================== --- stable/8/sys/kern/sys_generic.c Sun Nov 27 19:13:45 2011 (r228037) +++ stable/8/sys/kern/sys_generic.c Sun Nov 27 19:36:36 2011 (r228038) @@ -829,6 +829,54 @@ select(struct thread *td, struct select_ NFDBITS)); } +/* + * In the unlikely case when user specified n greater then the last + * open file descriptor, check that no bits are set after the last + * valid fd. We must return EBADF if any is set. + * + * There are applications that rely on the behaviour. + * + * nd is fd_lastfile + 1. + */ +static int +select_check_badfd(fd_set *fd_in, int nd, int ndu, int abi_nfdbits) +{ + char *addr, *oaddr; + int b, i, res; + uint8_t bits; + + if (nd >= ndu || fd_in == NULL) + return (0); + + oaddr = NULL; + bits = 0; /* silence gcc */ + for (i = nd; i < ndu; i++) { + b = i / NBBY; +#if BYTE_ORDER == LITTLE_ENDIAN + addr = (char *)fd_in + b; +#else + addr = (char *)fd_in; + if (abi_nfdbits == NFDBITS) { + addr += rounddown(b, sizeof(fd_mask)) + + sizeof(fd_mask) - 1 - b % sizeof(fd_mask); + } else { + addr += rounddown(b, sizeof(uint32_t)) + + sizeof(uint32_t) - 1 - b % sizeof(uint32_t); + } +#endif + if (addr != oaddr) { + res = fubyte(addr); + if (res == -1) + return (EFAULT); + oaddr = addr; + bits = res; + } + if ((bits & (1 << (i % NBBY))) != 0) + return (EBADF); + } + return (0); +} + int kern_select(struct thread *td, int nd, fd_set *fd_in, fd_set *fd_ou, fd_set *fd_ex, struct timeval *tvp, int abi_nfdbits) @@ -843,14 +891,26 @@ kern_select(struct thread *td, int nd, f fd_mask s_selbits[howmany(2048, NFDBITS)]; fd_mask *ibits[3], *obits[3], *selbits, *sbp; struct timeval atv, rtv, ttv; - int error, timo; + int error, lf, ndu, timo; u_int nbufbytes, ncpbytes, ncpubytes, nfdbits; if (nd < 0) return (EINVAL); fdp = td->td_proc->p_fd; - if (nd > fdp->fd_lastfile + 1) - nd = fdp->fd_lastfile + 1; + ndu = nd; + lf = fdp->fd_lastfile; + if (nd > lf + 1) + nd = lf + 1; + + error = select_check_badfd(fd_in, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); + error = select_check_badfd(fd_ou, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); + error = select_check_badfd(fd_ex, nd, ndu, abi_nfdbits); + if (error != 0) + return (error); /* * Allocate just enough bits for the non-null fd_sets. Use the _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" State-Changed-From-To: patched->closed State-Changed-By: kib State-Changed-When: Sun Nov 27 19:45:38 UTC 2011 State-Changed-Why: Merged to HEAD, 9 and 8. http://www.freebsd.org/cgi/query-pr.cgi?pr=155606 >Unformatted: