From nobody@FreeBSD.org Mon Feb 14 10:58:52 2005 Return-Path: Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF82A16A4CE for ; Mon, 14 Feb 2005 10:58:52 +0000 (GMT) Received: from www.freebsd.org (www.freebsd.org [216.136.204.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id B2C5643D1D for ; Mon, 14 Feb 2005 10:58:52 +0000 (GMT) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.13.1/8.13.1) with ESMTP id j1EAwq4N029862 for ; Mon, 14 Feb 2005 10:58:52 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.13.1/8.13.1/Submit) id j1EAwqqX029861; Mon, 14 Feb 2005 10:58:52 GMT (envelope-from nobody) Message-Id: <200502141058.j1EAwqqX029861@www.freebsd.org> Date: Mon, 14 Feb 2005 10:58:52 GMT From: Travis Morgan To: freebsd-gnats-submit@FreeBSD.org Subject: freebsd 5.3 + bash process substitution X-Send-Pr-Version: www-2.3 >Number: 77493 >Category: kern >Synopsis: [pipe] freebsd 5.3 + bash process substitution fails due to bug in named pipes >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Feb 14 11:00:40 GMT 2005 >Closed-Date: Mon Feb 22 17:46:31 UTC 2010 >Last-Modified: Mon Feb 22 17:46:31 UTC 2010 >Originator: Travis Morgan >Release: 5.3-RELEASE >Organization: BigFiber.net >Environment: FreeBSD pliosaur.tera-byte.com 5.3-RELEASE-p4 FreeBSD 5.3-RELEASE-p4 #0: Sun Jan 9 22:01:00 MST 2005 root@pliosaur.tera-byte.com:/usr/src/sys/i386/compile/IMBEZOL i386 FreeBSD home.ecn.ab.ca 5.3-RELEASE-p5 FreeBSD 5.3-RELEASE-p5 #0: Sat Feb 12 12:07:46 MST 2005 root@mail.ecn.ab.ca:/usr/obj/usr/src/sys/IMBEZOL i386 >Description: process substitution gives garbled results while read line ; do echo $line ; done < <(cat /etc/passwd) For some reason under 5.3 it loses a chunk off the front of each line producing garbage results >How-To-Repeat: while read line ; do echo $line ; done < <(cat /etc/passwd) >Fix: Not sure if this is a bash issue or a kernel issue but this is not a problem on any of the 4.x boxes I've tried it on. Also, the version of bash is not a factor.. it happens with 2.x and 3.x >Release-Note: >Audit-Trail: From: Travis Morgan To: freebsd-gnats-submit@FreeBSD.org Cc: Subject: Re: misc/77493: freebsd 5.3 + bash process substitution Date: Wed, 23 Mar 2005 12:29:19 -0700 I filed this bug with the bash bugs team as well and received the following response from them. According to Chet this would appear to be a FreeBSD problem, not a bash problem. Travis http://bigfiber.net Date: Wed, 9 Mar 2005 21:45:17 -0500 From: Chet Ramey To: travis@bigfiber.net Subject: Re: process substitution data corruption on FreeBSD 5.3 Cc: bug-bash@gnu.org, chet@po.cwru.edu Reply-To: chet.ramey@case.edu In-Reply-To: Message from travis@bigfiber.net of Sat, 05 Mar 2005 20:32:19 +0000 (id <20050305203219.69980.qmail@home.ecn.ab.ca>) Read-Receipt-To: chet.ramey@case.edu > Machine Type: i386-portbld-freebsd5.3 > > Bash Version: 3.0 > Patch Level: 16 > Release Status: release > > Description: > Process substitution doesn't work on FreeBSD 5.3 due to corruption of the data. I've tested on other versions of FreeBSD (4.x) and Linux and there are no problems. On all of the 5.3 boxes I have tried though with Bash 3.x and Bash 2.x it does not work properly. > I don't know if this is a bash bug or a FreeBSD bug. I submitted a bug report to FreeBSD some time ago but haven't seen any activity on it. > http://www.freebsd.org/cgi/query-pr.cgi?pr=77493 I think this is a FreeBSD bug. I did some testing today using 5.4-PRERELEASE. The bash `read' builtin, unless it's reading from a pipe, uses buffered reads. When reading from stdin, upon reading a newline, it must seek the input file pointer back to the offset of the newline to avoid potentially stealing input from other programs. This seeking backward does not work when applied to named pipes on FreeBSD 5.[34], though there is no indication that it fails. I don't know how FreeBSD implements named pipes, but they're not pipes: lseek doesn't return failure when applied to them. So the following happens: bash reads 128 bytes from the named pipe connected to the process running `cat /etc/passwd'. It reads 74 bytes or so (on my system), finds a newline, returns the newline, and attempts to seek backwards 54 bytes. lseek appears to work and sets the file offset to 74. The next read, however, gets another 128 bytes of data beginning at offset 129. You end up missing large chunks of the file. It worked on FreeBSD 4.x because that version had /dev/fd, and so bash didn't use named pipes. FreeBSD 5 doesn't support /dev/fd by default for any file descriptors other than 0, 1, and 2. Chet From: Mark Linimon To: Travis Morgan Cc: freebsd-bugs@FreeBSD.org, Subject: Re: misc/77493: freebsd 5.3 + bash process substitution Date: Wed, 23 Mar 2005 13:45:35 -0600 (CST) From your description, this sounds like a kernel bug in the way named pipes are handled. I have changed the Synopsis to reflect this and reclassified the bug from 'misc' to 'kern' in the hopes that this will help attract some attention to it. mcl From: Bruce Evans To: Travis Morgan Cc: freebsd-gnats-submit@freebsd.org Subject: Re: misc/77493: freebsd 5.3 + bash process substitution Date: Thu, 24 Mar 2005 21:06:23 +1100 (EST) On Wed, 23 Mar 2005, Travis Morgan wrote: > I filed this bug with the bash bugs team as well and received the following response from them. According to Chet this would appear to be a FreeBSD problem, not a bash problem. > Date: Wed, 9 Mar 2005 21:45:17 -0500 > From: Chet Ramey > ... > I think this is a FreeBSD bug. I did some testing today using > 5.4-PRERELEASE. > > The bash `read' builtin, unless it's reading from a pipe, uses > buffered reads. When reading from stdin, upon reading a newline, it > must seek the input file pointer back to the offset of the newline to > avoid potentially stealing input from other programs. > > This seeking backward does not work when applied to named pipes on > FreeBSD 5.[34], though there is no indication that it fails. > > I don't know how FreeBSD implements named pipes, but they're not pipes: > lseek doesn't return failure when applied to them. > ... This (the non-failure of lseek() on named pipes) was fixed in 1996 in rev.1.52 of vfs_syscalls.c and associated changes, but was broken in 2003 in rev.1.319 of vfs_syscalls.c and associated changes. See PR 65786 for details. (This PR is about the affect of lseek()'s bug on dd.) Bruce State-Changed-From-To: open->closed State-Changed-By: jh State-Changed-When: Mon Feb 22 17:46:30 UTC 2010 State-Changed-Why: Fixed in r150033. http://www.freebsd.org/cgi/query-pr.cgi?pr=77493 >Unformatted: