From nobody@FreeBSD.org Fri Aug 12 21:12:23 2011 Return-Path: Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B6C10106564A for ; Fri, 12 Aug 2011 21:12:23 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22]) by mx1.freebsd.org (Postfix) with ESMTP id A59AC8FC15 for ; Fri, 12 Aug 2011 21:12:23 +0000 (UTC) Received: from red.freebsd.org (localhost [127.0.0.1]) by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p7CLCNtW003787 for ; Fri, 12 Aug 2011 21:12:23 GMT (envelope-from nobody@red.freebsd.org) Received: (from nobody@localhost) by red.freebsd.org (8.14.4/8.14.4/Submit) id p7CLCN51003786; Fri, 12 Aug 2011 21:12:23 GMT (envelope-from nobody) Message-Id: <201108122112.p7CLCN51003786@red.freebsd.org> Date: Fri, 12 Aug 2011 21:12:23 GMT From: Stefano Lattarini To: freebsd-gnats-submit@FreeBSD.org Subject: make(1) in parallel mode fails report failure of @-prefixed recipe using "set +e" X-Send-Pr-Version: www-3.1 X-GNATS-Notify: >Number: 159730 >Category: bin >Synopsis: make(1) in parallel mode fails report failure of @-prefixed recipe using "set +e" >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Aug 12 21:20:10 UTC 2011 >Closed-Date: >Last-Modified: Sun Aug 14 12:20:10 UTC 2011 >Originator: Stefano Lattarini >Release: 8.2-RELEASE >Organization: >Environment: FreeBSD bigio.localdomain 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Fri Feb 18 02:24:46 UTC 2011 root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 >Description: When used in parallel mode ("make -jN ..."), FreeBSD make sometimes report successful exit status from recipes that should fail, in case such recipes are prefixed with `@' and unset the `errexit' shell flag. Example: $ echo 'all: ; @set +e; false' | make -j2 -f-; echo status: $? status: 0 I found this problem while testing Automake; see automake bug#9245: This issue causes automake-generated Makefiles using the newer parallel-tests harness to always report success if run with "make -jN check"; a really nasty case of spurious success IMHO, which caused me to label this bug as "serious". >How-To-Repeat: Here is how to reproduce the issue: $ echo 'all: ; @set +e; false' | make -j2 -f-; echo status: $? status: 0 Notice that all of `-j', `@' and `set +e' seems to be required to trigger the bug: $ echo 'all: ; set +e; false' | make -j2 -f- set +e; false *** Error code 1 1 error $ echo 'all: ; @false' | make -j2 -f- *** Error code 1 1 error $ echo 'all: ; @set +e; false' | make -f- *** Error code 1 Stop in /tmp. >Fix: >Release-Note: >Audit-Trail: From: Jilles Tjoelker To: bug-followup@FreeBSD.org Cc: harti@FreeBSD.org Subject: Re: bin/159730: make(1) in parallel mode fails report failure of @-prefixed recipe using "set +e" Date: Sun, 14 Aug 2011 14:11:20 +0200 Stefano Lattarini reports: > [echo 'all: ; @set +e; false' | make -j2 -f- fails to detect error] I can reproduce this. The cause is that make(1) runs all commands for a target in a single shell when -j is in effect. This shell has the -e and -v options initially enabled and reads the script from standard input. In that case, the '@' modifier is handled by surrounding the command with a line 'set -' and a line 'set -v', and filtering the 'set -' from the output. (Similarly, the '+' modifier temporarily disables '-e'.) The 'set -v' has exit status 0 and masks the exit status from the command that was supposed to be tested. More generally, using a single shell in this manner will lead to results unexpected from the POSIX specification because changes to the shell environment remain in effect. A possible fix is to put each command (line) in a subshell environment with parentheses. To keep proper output, this requires replacing 'set -v' with explicit printf commands. If the command ends with an external program, this does not result in extra forks with our sh, although there will be more copy-on-write faults. By the way, having sh read the script from standard input will lead to suboptimal performance. One reason is that our sh has an optimization for -c that skips a fork for a final external program (with a few exceptions such as if there are trap handlers) but there is no such optimization for scripts named on the command line or commands passed via standard input. Another reason is that POSIX requires sh, when it is reading commands from standard input, to position the file pointer directly after each command before executing it. Our sh currently does not do this, however. If {ARG_MAX} permits, it is more efficient to pass the commands via -c. -- Jilles Tjoelker >Unformatted: