From nobody@FreeBSD.org Thu Jun 27 17:29:53 2002 Return-Path: Received: from www.freebsd.org (www.FreeBSD.org [216.136.204.117]) by hub.freebsd.org (Postfix) with ESMTP id D74A237B4F3 for ; Thu, 27 Jun 2002 17:29:46 -0700 (PDT) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.12.4/8.12.4) with ESMTP id g5S0SLOT059821 for ; Thu, 27 Jun 2002 17:28:21 -0700 (PDT) (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.12.4/8.12.4/Submit) id g5S0SL84059820; Thu, 27 Jun 2002 17:28:21 -0700 (PDT) Message-Id: <200206280028.g5S0SL84059820@www.freebsd.org> Date: Thu, 27 Jun 2002 17:28:21 -0700 (PDT) From: Guolin Cheng To: freebsd-gnats-submit@FreeBSD.org Subject: /usr/sbin/periodic sends thousands of emails X-Send-Pr-Version: www-1.0 >Number: 39940 >Category: bin >Synopsis: [patch] /usr/sbin/periodic sends thousands of emails >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Thu Jun 27 17:30:01 PDT 2002 >Closed-Date: Wed Feb 28 07:40:35 GMT 2007 >Last-Modified: Wed Feb 28 07:40:35 GMT 2007 >Originator: Guolin Cheng >Release: FreeBSD 4.6, but same for all 4.* version >Organization: ALexa Internet. Inc. >Environment: FreeBSD rontmp.alexa.com 4.6-RELEASE FreeBSD 4.6-RELEASE #0: Fri Jun 21 20:37:42 GMT 2002 root@rontmp.alexa.com:/usr/src/sys/compile/ALEXA i386 >Description: The problems are: 1, /usr/sbin/periodic sends tons of email, when running hourly running scripts whether or not the latter generate outputs. Then email server will be overwhelmed if we have hundreds or thousands FreeBSD boxes. 2, periodic can not tell normal shell scripts from backup files ended with '~' and ',' , which are normally created by CVS/RVS/Perforce. it tries to run all executable files under base directory specified. 3, the coufiguration file */periodic.conf, which contain variable definition entries including _{show|output|...}, will confuse Bourne Shell interpreter when the basedir contains character '.', which is normal for directory name but not appropriate for shell variable name substitution. >How-To-Repeat: just create a directory /etc/periodic/hourly, and create a simply output-nothing shell script hahaha.sh under it: #! /bin/sh # do nothing and output nothing # echo "hahahaa... wrong :)" Set its executable mode bit and run the periodic with hourly directory as its argument.. sh -x /usr/sbin/periodic hourly Then copy the hahaha.sh to hahaha.sh~ and run the same command above, you will find that both shell scripts are processed! at last change the hourly directory name to cron.hourly name, and it will report problem even at beginning when processing the periodic.conf file! >Fix: To fix the problem, we need to tell periodic 1, skip shell scripts ended with '~' and '.', 2, substitute the '.' character to '_' in the periodic.conf and inform users about that special case, so that no base diretories like cron.daily and cron_daily exists at the same time. or if so, then the setting will overlaps. 3, send emails only when necessary, normally that means scripts under base directory generating outputs. The diff between original edited version of /usr/sbin/periodic are attached. shell> diff -c /usr/sbin/periodic /var/tmp/periodic *** /usr/sbin/periodic Mon Jun 10 21:19:56 2002 --- /var/tmp/periodic Thu Jun 27 15:10:53 2002 *************** *** 28,33 **** --- 28,34 ---- host=`hostname` export host tmp_output=`mktemp ${TMPDIR:-/tmp}/periodic.XXXXXXXXXX` + tmp_email_output=`mktemp ${TMPDIR:-/tmp}/periodic.XXXXXXXXXX` # Execute each executable file in the directory list. If the x bit is not # set, assume the user didn't really want us to muck with it (it's a *************** *** 35,42 **** for arg do # Where's our output going ? ! eval output=\$${arg##*/}_output case "$output" in /*) pipe="cat >>$output";; "") pipe=cat;; --- 36,47 ---- for arg do + # for special cases that the base directory has a '.' character contained, + # and difficult to change, then we can replace '.' with '_' in */period.conf, + # to pass the shell parameter name check for */periodic.conf + arg_in_conf=`echo $arg | tr '.' '_'` # Where's our output going ? ! eval output=\$${arg_in_conf##*/}_output case "$output" in /*) pipe="cat >>$output";; "") pipe=cat;; *************** *** 46,52 **** success=YES info=YES badconfig=NO # Defaults when ${run}_* aren't YES/NO for var in success info badconfig do ! case $(eval echo "\$${arg##*/}_show_$var") in [Yy][Ee][Ss]) eval $var=YES;; [Nn][Oo]) eval $var=NO;; esac --- 51,57 ---- success=YES info=YES badconfig=NO # Defaults when ${run}_* aren't YES/NO for var in success info badconfig do ! case $(eval echo "\$${arg_in_conf##*/}_show_$var") in [Yy][Ee][Ss]) eval $var=YES;; [Nn][Oo]) eval $var=NO;; esac *************** *** 72,79 **** processed=0 for dir in $dirlist do ! for file in $dir/* do if [ -x $file -a ! -d $file ] then output=TRUE --- 77,91 ---- processed=0 for dir in $dirlist do ! for file in $dir/*[^~,] ; ## Ignore *~ and *, scripts do + # Don't run *.{rpmsave,rpmorig,rpmnew,swp} scripts + [ "${file%.rpmsave}" != "${file}" ] && continue + [ "${file%.rpmorig}" != "${file}" ] && continue + [ "${file%.rpmnew}" != "${file}" ] && continue + [ "${file%.swp}" != "${file}" ] && continue + [ "${file%,v}" != "${file}" ] && continue + if [ -x $file -a ! -d $file ] then output=TRUE *************** *** 93,106 **** fi done done ! if [ $empty = TRUE ] ! then ! [ $processed = 1 ] && plural= || plural=s ! echo "No output from the $processed file$plural processed" ! else ! echo "" ! echo "-- End of $arg output --" ! fi ! } | eval $pipe done rm -f $tmp_output --- 105,124 ---- fi done done ! ## If there is no output from scripts processed, then we will not sendmail ! #if [ $empty = TRUE ] ! #then ! # [ $processed = 1 ] && plural= || plural=s ! # echo "No output from the $processed file$plural processed" ! #else ! # echo "" ! # echo "-- End of $arg output --" ! #fi ! } > $tmp_email_output ! if [ -s $tmp_email_output ] ; ! then ! cat $tmp_email_output | eval $pipe ! fi done rm -f $tmp_output + rm -f $tmp_email_output >Release-Note: >Audit-Trail: From: Peter Pentchev To: Guolin Cheng Cc: bug-followup@FreeBSD.org Subject: Re: bin/39940: /usr/sbin/periodic sends thousands of emails Date: Fri, 28 Jun 2002 13:39:54 +0300 On Thu, Jun 27, 2002 at 05:28:21PM -0700, Guolin Cheng wrote: > > >Number: 39940 > >Category: bin > >Synopsis: /usr/sbin/periodic sends thousands of emails > >Originator: Guolin Cheng > >Release: FreeBSD 4.6, but same for all 4.* version > >Organization: > ALexa Internet. Inc. > >Environment: > FreeBSD rontmp.alexa.com 4.6-RELEASE FreeBSD 4.6-RELEASE #0: Fri Jun 21 20:37:42 GMT 2002 root@rontmp.alexa.com:/usr/src/sys/compile/ALEXA i386 > > >Description: > The problems are: > > 1, /usr/sbin/periodic sends tons of email, when running hourly running > scripts whether or not the latter generate outputs. Then email server > will be overwhelmed if we have hundreds or thousands FreeBSD boxes. This can be worked around easily: make your scripts return an appropriate exit code to periodic(8) as described in its manual page; notably, use an exit code of 0 for scripts that produce no output, and set the hourly_show_success variable to 'no' in /etc/periodic.conf. Actually, this is the way it is *designed* to work :) > 2, periodic can not tell normal shell scripts from backup files ended > with '~' and ',' , which are normally created by CVS/RVS/Perforce. it > tries to run all executable files under base directory specified. Uhm.. so.. don't leave any? :) In the case of source-controlled script directories, I usually keep my working copies somewhere else, and use the 'real' scripts dir only as a checked-out copy; since it is never modified, it should never create any conflicts, so no temporary/backup files should ever be created. > 3, the coufiguration file */periodic.conf, which contain variable > definition entries including _{show|output|...}, will confuse > Bourne Shell interpreter when the basedir contains character '.', > which is normal for directory name but not appropriate for shell > variable name substitution. One might argue that this may be solved by the "Don't do that" approach.. An underscore character would serve just as well as a word separator in the directory name. G'luck, Peter -- Peter Pentchev roam@ringlet.net roam@FreeBSD.org PGP key: http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553 I am the meaning of this sentence. State-Changed-From-To: open->analyzed State-Changed-By: vs State-Changed-When: Thu Jul 29 09:12:42 GMT 2004 State-Changed-Why: Issue understood. It's a feature. http://www.freebsd.org/cgi/query-pr.cgi?pr=39940 State-Changed-From-To: analyzed->closed State-Changed-By: remko State-Changed-When: Wed Feb 28 07:40:33 UTC 2007 State-Changed-Why: The issue described and explained by roam should be enough reasoning to close the ticket. http://www.freebsd.org/cgi/query-pr.cgi?pr=39940 >Unformatted: