From faber@ISI.EDU Mon Apr 19 11:19:03 1999 Return-Path: Received: from zephyr.isi.edu (zephyr.isi.edu [128.9.160.160]) by hub.freebsd.org (Postfix) with ESMTP id 6EE091563A for ; Mon, 19 Apr 1999 11:18:30 -0700 (PDT) (envelope-from faber@ISI.EDU) Received: from vermouth.isi.edu (vermouth.isi.edu [128.9.160.247]) by zephyr.isi.edu (8.8.7/8.8.6) with ESMTP id LAA10698 for ; Mon, 19 Apr 1999 11:16:01 -0700 (PDT) Received: (from faber@localhost) by vermouth.isi.edu (8.9.2/8.8.5) id LAA02511; Mon, 19 Apr 1999 11:16:04 -0700 (PDT) Message-Id: <199904191816.LAA02511@vermouth.isi.edu> Date: Mon, 19 Apr 1999 11:16:04 -0700 (PDT) From: Ted Faber Reply-To: faber@ISI.EDU To: FreeBSD-gnats-submit@freebsd.org Subject: MFS does not sync from reboot syscall X-Send-Pr-Version: 3.2 >Number: 11222 >Category: kern >Synopsis: MFS does not sync from reboot syscall >Confidential: no >Severity: non-critical >Priority: medium >Responsible: freebsd-bugs >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Apr 19 11:20:01 PDT 1999 >Closed-Date: Tue Jan 4 11:55:15 PST 2000 >Last-Modified: Tue Jan 4 11:55:40 PST 2000 >Originator: Ted Faber >Release: FreeBSD 3.1-RELEASE i386 >Organization: USC/ISI >Environment: FreeBSD 3.1-RELEASE running XFree86-3.3.3.1 >Description: If there are dirty buffers from an MFS file system when the reboot syscall is made, those buffers are never cleaned. Therefore all filesystems (including real disk-based ones) are not unmounted cleanly, requiring a full fsck on reboot. Internally the problem is that the MFS strategy routine, which is called by the sync calls in reboot (syscall), queues buffers for the MFS process itself to write, and wakes that process up. The reboot syscall never releases the processor, so those buffers remain dirty. The enclosed patch adds code to the reboot syscall to detect such MFS mounted buffers and if they are present, release the processor to allow the MFS to clean up after itself. The patch also raises the MFS process's sleep priority to make sure that it runs when reboot releases the processor. The patch is relative to /sys on a 3.1-RELEASE system. It modifies /sys/kern/kern_shutdown.c and /sys/ufs/mfs/nfs_vfsops.c >How-To-Repeat: I was able to consistently reproduce the problem by rebooting from a complex X session that uses /tmp (an MFS) extensively. The default root X session does not tickle the bug. Although I didn't try it, I would bet that rebooting while one or more processes was rewriting a file to an MFS file system would also work. >Fix: The enclosed patch allows the buffers to be cleaned. Without the patch, users can reboot their system by using shutdown and manually umounting /tmp (the MFS) before rebooting. The patch is relative to /sys on a 3.1-RELEASE system. It modifies /sys/kern/kern_shutdown.c and /sys/ufs/mfs/nfs_vfsops.c *** kern/kern_shutdown.c.orig Mon Dec 28 15:03:00 1998 --- kern/kern_shutdown.c Sun Apr 18 14:35:02 1999 *************** *** 196,201 **** --- 196,202 ---- if (!cold && (howto & RB_NOSYNC) == 0 && waittime < 0) { register struct buf *bp; int iter, nbusy; + int mfs = 0; /* Number of buffers on MFS filesystems */ waittime = 0; printf("\nsyncing disks... "); *************** *** 218,228 **** --- 219,241 ---- /* bawrite(bp);*/ nbusy++; } + if ( major(bp->b_dev) == 0xff && + bp->b_dev != NODEV) + mfs++; } if (nbusy == 0) break; printf("%d ", nbusy); sync(&proc0, NULL); + if ( mfs ) { + /* + * there are MFS bufferes to sync. Give + * up the processor so the MFS code can + * clean the buffers + */ + mfs = 0; + tsleep(&proc0, PWAIT, "rmfswait",1); + } DELAY(50000 * iter); } /* *** ufs/mfs/mfs_vfsops.c.orig Thu Dec 31 20:14:11 1998 --- ufs/mfs/mfs_vfsops.c Sun Apr 18 14:29:24 1999 *************** *** 367,373 **** } ! static int mfs_pri = PWAIT | PCATCH; /* XXX prob. temp */ /* * Used to grab the process and keep it in the kernel to service --- 367,373 ---- } ! static int mfs_pri = PRIBIO | PCATCH; /* XXX prob. temp */ /* * Used to grab the process and keep it in the kernel to service >Release-Note: >Audit-Trail: From: Ted Faber To: FreeBSD-gnats-submit@freebsd.org Cc: Subject: Re: kern/11222: MFS does not sync from reboot syscall Date: Tue, 6 Jul 1999 11:44:29 -0700 (PDT) >Submitter-Id: current-users >Originator: Ted Faber >Organization: USC/ISI >Confidential: no >Synopsis: MFS does not sync from reboot syscall >Severity: non-critical >Priority: medium >Category: kern >Release: FreeBSD 3.2-RELEASE i386 >Class: sw-bug >Environment: Followup to kern/11222 >Description: see kern/11222 >How-To-Repeat: see kern/11222 >Fix: New patch, not MFS-specific, replaces DELAY with tsleep in reboot syscall *** ufs/mfs/mfs_vfsops.c.orig Thu Dec 31 20:14:11 1998 --- ufs/mfs/mfs_vfsops.c Sun Apr 18 14:29:24 1999 *************** *** 367,373 **** } ! static int mfs_pri = PWAIT | PCATCH; /* XXX prob. temp */ /* * Used to grab the process and keep it in the kernel to service --- 367,373 ---- } ! static int mfs_pri = PRIBIO | PCATCH; /* XXX prob. temp */ /* * Used to grab the process and keep it in the kernel to service *** kern/kern_shutdown.c.orig Mon Dec 28 15:03:00 1998 --- kern/kern_shutdown.c Thu Apr 29 09:15:19 1999 *************** *** 178,183 **** --- 178,184 ---- int howto; { sle_p ep; + int to; /* to check tsleep timeouts */ #ifdef SMP if (smp_active) { *************** *** 223,229 **** break; printf("%d ", nbusy); sync(&proc0, NULL); ! DELAY(50000 * iter); } /* * Count only busy local buffers to prevent forcing --- 224,240 ---- break; printf("%d ", nbusy); sync(&proc0, NULL); ! /* DELAY(50000 * iter); */ ! ! /* ! * Ensure that to is not zero (we don't want ! * to sleep forever) ! */ ! if ( (hz * 5 *iter ) / 100 > 0 ) ! to = (hz * 5 * iter ) / 100 ; ! else ! to = 1; ! tsleep(&proc0, PWAIT, "reboot", to); } /* * Count only busy local buffers to prevent forcing *************** *** 259,265 **** (long)bp->b_lblkno); } } ! DELAY(5000000); /* 5 seconds */ #endif } else { printf("done\n"); --- 270,277 ---- (long)bp->b_lblkno); } } ! /* DELAY(5000000); 5 seconds */ ! tsleep(&proc0, PWAIT, "reboot", 5 * hz ); #endif } else { printf("done\n"); *************** *** 269,275 **** if (panicstr == 0) vfs_unmountall(); } ! DELAY(100000); /* wait for console output to finish */ } /* --- 281,292 ---- if (panicstr == 0) vfs_unmountall(); } ! /*DELAY(100000); wait for console output to finish */ ! if ( hz / 10 > 0 ) to = hz/10; ! else to = 1; ! ! tsleep(&proc0, PWAIT, "reboot", to); ! } /* *************** *** 314,320 **** PANIC_REBOOT_WAIT_TIME); for (loop = PANIC_REBOOT_WAIT_TIME * 10; loop > 0; --loop) { ! DELAY(1000 * 100); /* 1/10th second */ /* Did user type a key? */ if (cncheckc() != -1) break; --- 331,342 ---- PANIC_REBOOT_WAIT_TIME); for (loop = PANIC_REBOOT_WAIT_TIME * 10; loop > 0; --loop) { ! /* DELAY(1000 * 100); 1/10th second */ ! if ( hz/10 > 0 ) to = hz/10; ! else to = 1; ! ! tsleep(&proc0, PWAIT, "reboot", to); ! /* Did user type a key? */ if (cncheckc() != -1) break; *************** *** 330,336 **** } die: printf("Rebooting...\n"); ! DELAY(1000000); /* wait 1 sec for printf's to complete and be read */ /* cpu_boot(howto); */ /* doesn't do anything at the moment */ cpu_reset(); for(;;) ; --- 352,360 ---- } die: printf("Rebooting...\n"); ! /*DELAY(1000000); wait 1 sec for printf's to complete and be read */ ! tsleep(&proc0, PWAIT, "reboot", hz); ! /* cpu_boot(howto); */ /* doesn't do anything at the moment */ cpu_reset(); for(;;) ; State-Changed-From-To: open->closed State-Changed-By: dillon State-Changed-When: Tue Jan 4 11:55:15 PST 2000 State-Changed-Why: Fix committed kern_shutdown.c 1.42 (actually committed a while ago) >Unformatted: