From nobody@FreeBSD.org Sun Sep 24 16:38:05 2006 Return-Path: Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0EE8316A40F for ; Sun, 24 Sep 2006 16:38:05 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [216.136.204.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6438043D49 for ; Sun, 24 Sep 2006 16:38:04 +0000 (GMT) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.13.1/8.13.1) with ESMTP id k8OGc4tW088580 for ; Sun, 24 Sep 2006 16:38:04 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.13.1/8.13.1/Submit) id k8OGc3uA088579; Sun, 24 Sep 2006 16:38:03 GMT (envelope-from nobody) Message-Id: <200609241638.k8OGc3uA088579@www.freebsd.org> Date: Sun, 24 Sep 2006 16:38:03 GMT From: Alexey Illarionov To: freebsd-gnats-submit@FreeBSD.org Subject: [ipfilter] ipf -D cause kernel panic X-Send-Pr-Version: www-2.3 >Number: 103569 >Category: kern >Synopsis: [ipfilter] ipf -D cause kernel panic >Confidential: no >Severity: non-critical >Priority: low >Responsible: oleg >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sun Sep 24 16:40:23 GMT 2006 >Closed-Date: Thu Oct 26 12:02:34 GMT 2006 >Last-Modified: Thu Oct 26 12:02:34 GMT 2006 >Originator: Alexey Illarionov >Release: 6.2-PRERELEASE #2 >Organization: >Environment: FreeBSD ls2.orionet.ru 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE #2: Sun Sep 24 15:46:21 MSD 2006 root@ls2.orionet.ru:/usr/obj/usr/src/sys/LS_DEBUG i386 IP Filter: v4.1.13 >Description: Executing "ipf -D" cause kernel panic with new version of ipfilter. It is compiled as loadable kernel module, but this command worked in FreeBSD 6.1 with ipfilter v4.1.8. Or at least it did not cause a kernel panic. Crash debug (kernel compiled with WITNESS option): # kgdb -n 0 kgdb: kvm_nlist(_stopped_cpus): kgdb: kvm_nlist(_stoppcbs): [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: panic: lock (sleep mutex) ipf filter load/unload mutex not locked @ /usr/src/sys/modules/ipfilter/../../contrib/ipfilter/netinet/ip_fil_freebsd.c:630 KDB: stack backtrace: kdb_backtrace(100,c19ae000,c19ae000,c172eac0,c19ae078,...) at kdb_backtrace+0x29 panic(c079f8a1,c07b3a0e,c172a6d7,c172a768,276,...) at panic+0xa8 witness_unlock(c172eac0,8,c172a768,276) at witness_unlock+0xbc _mtx_unlock_flags(c172eac0,0,c172a768,276,c079f334,...) at _mtx_unlock_flags+0x28 iplioctl(c1682600,80047248,c159d7b0,3,c19ae000,...) at iplioctl+0xba devfs_ioctl_f(c169e2d0,80047248,c159d7b0,c19f8600,c19ae000) at devfs_ioctl_f+0xaf ioctl(c19ae000,c8308d04) at ioctl+0x344 syscall(3b,3b,3b,2806bcf0,bfbfec70,...) at syscall+0x22f Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (54, FreeBSD ELF32, ioctl), eip = 0x28197bc7, esp = 0xbfbfe9bc, ebp = 0xbfbfe9d8 --- KDB: enter: panic panic: from debugger Uptime: 3m51s Dumping 63 MB (2 chunks) chunk 0: 1MB (160 pages) ... ok chunk 1: 63MB (16128 pages) 48 32 16 #0 doadump () at pcpu.h:165 165 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc05a089a in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc05a0af3 in panic (fmt=0xc0777992 "from debugger") at /usr/src/sys/kern/kern_shutdown.c:565 #3 0xc0453835 in db_panic (addr=-1067748277, have_addr=0, count=-1, modif=0xc8308894 "") at /usr/src/sys/ddb/db_command.c:438 #4 0xc04537cc in db_command (last_cmdp=0xc083aee4, cmd_table=0x0, aux_cmd_tablep=0xc07c11d4, aux_cmd_tablep_end=0xc07c11d8) at /usr/src/sys/ddb/db_command.c:350 #5 0xc0453894 in db_command_loop () at /usr/src/sys/ddb/db_command.c:458 #6 0xc045543d in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_main.c:221 #7 0xc05b76c7 in kdb_trap (type=3, code=0, tf=0xc83089d4) at /usr/src/sys/kern/subr_kdb.c:473 #8 0xc073f390 in trap (frame= {tf_fs = -936378360, tf_es = -1067778008, tf_ds = -1065811928, tf_edi = 1, tf_esi = -1065748319, tf_ebp = -936343020, tf_isp = -936343040, tf_ebx = -936342976, tf_edx = 0, tf_ecx = -1056878592, tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -1067748277, tf_cs = 32, tf_eflags = 662, tf_esp = -936342988, tf_ss = -1067840861}) at /usr/src/sys/i386/i386/trap.c:594 #9 0xc072e3da in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #10 0xc05b744b in kdb_enter (msg=0x12
) at cpufunc.h:60 #11 0xc05a0aa3 in panic (fmt=0xc079f8a1 "lock (%s) %s not locked @ %s:%d") at /usr/src/sys/kern/kern_shutdown.c:549 #12 0xc05c0d54 in witness_unlock (lock=0xc172eac0, flags=8, file=0xc172a768 "/usr/src/sys/modules/ipfilter/../../contrib/ipfilter/netinet/ip_fil_freebsd.c", line=630) at /usr/src/sys/kern/subr_witness.c:1237 #13 0xc0598bd4 in _mtx_unlock_flags (m=0xc172eac0, opts=0, file=0xc172a768 "/usr/src/sys/modules/ipfilter/../../contrib/ipfilter/netinet/ip_fil_freebsd.c", line=630) at /usr/src/sys/kern/kern_mutex.c:315 #14 0xc171ffda in ?? () #15 0xc172eac0 in ?? () #16 0x00000000 in ?? () #17 0xc172a768 in ?? () #18 0x00000276 in ?? () #19 0xc079f334 in ?? () #20 0x00000000 in ?? () #21 0xc0847100 in witness_spin_warn () #22 0x00000000 in ?? () #23 0xc079f334 in ?? () #24 0x000006a9 in ?? () #25 0xc088cff4 in w_locklistdata () #26 0xc8308adc in ?? () #27 0xc05c0e4e in witness_unlock (lock=0x4, flags=-2147192248, file=0xc159d7b0 "", line=3) at /usr/src/sys/kern/subr_witness.c:1285 #28 0xc0559913 in devfs_ioctl_f (fp=0xc169e2d0, com=2147775048, data=0xc159d7b0, cred=0xc19f8600, td=0xc19ae000) at /usr/src/sys/fs/devfs/devfs_vnops.c:407 #29 0xc05c24c8 in ioctl (td=0xc19ae000, uap=0xc8308d04) at file.h:264 #30 0xc073fb13 in syscall (frame= {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = 671530224, tf_esi = -1077941136, tf_ebp = -1077941800, tf_isp = -936342172, tf_ebx = 2, tf_edx = 0, tf_ecx = 672916548, tf_eax = 54, tf_trapno = 12, tf_err = 2, tf_eip = 672758727, tf_cs = 51, tf_eflags = 535, tf_esp = -1077941828, tf_ss = 59}) at /usr/src/sys/i386/i386/trap.c:983 #31 0xc072e42f in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200 #32 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) dmesg: Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.2-PRERELEASE #2: Sun Sep 24 15:46:21 MSD 2006 root@ls2.orionet.ru:/usr/obj/usr/src/sys/LS_DEBUG WARNING: WITNESS option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Pentium II/Pentium II Xeon/Celeron (233.87-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x634 Stepping = 4 Features=0x80f9ff real memory = 67108864 (64 MB) avail memory = 56127488 (53 MB) cpu0 on motherboard pcib0: pcibus 0 on motherboard pir0: on motherboard pci0: on pcib0 agp0: mem 0xd0000000-0xd1ffffff at device 0.0 on pci0 pcib1: at device 1.0 on pci0 pci1: on pcib1 pci1: at device 0.0 (no driver attached) isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 7.1 on pci0 ata0: on atapci0 ata1: on atapci0 uhci0: port 0xd000-0xd01f irq 10 at device 7.2 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 usbd_get_string: getting lang failed, using 0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered pci0: at device 7.3 (no driver attached) pci0: at device 8.0 (no driver attached) xl0: <3Com 3cSOHO100-TX OfficeConnect> port 0xe800-0xe87f mem 0xd7010000-0xd701007f irq 11 at device 14.0 on pci0 miibus0: on xl0 xlphy0: <3Com internal media interface> on miibus0 xlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto xl0: Ethernet address: 00:04:76:95:18:8e hcfmdm0: port 0xec00-0xec07 mem 0xd7000000-0xd700ffff irq 12 at device 16.0 on pci0 WITNESS: spin lock hcfmdm_state_mtx not in order list pmtimer0 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0 at port 0x3f8-0x3ff irq 4 flags 0x90 on isa0 sio0: type 16550A sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 unknown: can't assign resources (port) unknown: can't assign resources (memory) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) Timecounter "TSC" frequency 233865531 Hz quality 800 Timecounters tick every 1.000 msec ad0: 4126MB at ata0-master UDMA33 Trying to mount root from ufs:/dev/ad0s1a IP Filter: v4.1.13 initialized. Default = pass all, Logging = enabled >How-To-Repeat: Try to disable ipfilter with ipf -D >Fix: >Release-Note: >Audit-Trail: From: Oleg Bulyzhin To: bug-followup@freebsd.org Cc: Subject: kern/103569: [ipfilter] ipf -D cause kernel panic Date: Sat, 30 Sep 2006 14:39:16 +0400 --FL5UXtIhxfXey3p5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Could you please test attached patch? -- Oleg. --FL5UXtIhxfXey3p5 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="ipfilter_lock_fix.diff" Index: sys/contrib/ipfilter/netinet/ip_fil_freebsd.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/ipfilter/netinet/ip_fil_freebsd.c,v retrieving revision 1.4 diff -u -r1.4 ip_fil_freebsd.c --- sys/contrib/ipfilter/netinet/ip_fil_freebsd.c 16 Aug 2006 12:06:35 -0000 1.4 +++ sys/contrib/ipfilter/netinet/ip_fil_freebsd.c 27 Sep 2006 17:46:39 -0000 @@ -481,7 +481,8 @@ } SPL_NET(s); - READ_ENTER(&ipf_global); + if (fr_running > 0) + READ_ENTER(&ipf_global); error = fr_ioctlswitch(unit, data, cmd, mode); if (error != -1) { @@ -514,7 +515,10 @@ else (void) ipldetach(); } else { - error = ipldetach(); + if (fr_running <= 0) + error = 0; + else + error = ipldetach(); if (error == 0) fr_running = -1; } @@ -627,7 +631,9 @@ break; } - RWLOCK_EXIT(&ipf_global); + if (fr_running > 0) + if (mtx_owned(&(&ipf_global)->ipf_lk)) + RWLOCK_EXIT(&ipf_global); SPL_X(s); return error; --FL5UXtIhxfXey3p5-- From: Alexey Illarionov To: bug-followup@FreeBSD.org, littlesavage@rambler.ru, Oleg Bulyzhin Cc: Subject: Re: kern/103569: [ipfilter] ipf -D cause kernel panic Date: Sun, 01 Oct 2006 12:48:37 +0400 Yes, it works. No panics any more. Thanks. Responsible-Changed-From-To: freebsd-bugs->oleg Responsible-Changed-By: oleg Responsible-Changed-When: Wed Oct 4 15:40:46 UTC 2006 Responsible-Changed-Why: take over. http://www.freebsd.org/cgi/query-pr.cgi?pr=103569 From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/103569: commit references a PR Date: Thu, 5 Oct 2006 09:48:41 +0000 (UTC) oleg 2006-10-05 09:48:25 UTC FreeBSD src repository Modified files: sys/contrib/ipfilter/netinet ip_fil_freebsd.c Log: Workaround bad locking design: do not try to lock/unlock destroyed/non-existsing mutex. PR: kern/103569 Reviewed by: guido Approved by: glebius (mentor) Silence from: darrenr MFC: 2 week Revision Changes Path 1.5 +9 -3 src/sys/contrib/ipfilter/netinet/ip_fil_freebsd.c _______________________________________________ cvs-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/cvs-all To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org" State-Changed-From-To: open->patched State-Changed-By: oleg State-Changed-When: Thu Oct 5 11:12:13 UTC 2006 State-Changed-Why: patched in -CURRENT. MFC in 2 weeks. http://www.freebsd.org/cgi/query-pr.cgi?pr=103569 From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/103569: commit references a PR Date: Thu, 26 Oct 2006 11:22:24 +0000 (UTC) oleg 2006-10-26 11:22:04 UTC FreeBSD src repository Modified files: (Branch: RELENG_6) sys/contrib/ipfilter/netinet ip_fil_freebsd.c Log: MFC rev. 1.5 Workaround bad locking design: do not try to lock/unlock destroyed/non-existing mutex. PR: kern/103569 Reviewed by: guido Silence from: darrenr Approved by: re (hrs) Revision Changes Path 1.1.1.1.2.2 +10 -4 src/sys/contrib/ipfilter/netinet/ip_fil_freebsd.c _______________________________________________ cvs-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/cvs-all To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org" State-Changed-From-To: patched->closed State-Changed-By: oleg State-Changed-When: Thu Oct 26 12:01:44 UTC 2006 State-Changed-Why: Merged to RELENG_6. http://www.freebsd.org/cgi/query-pr.cgi?pr=103569 >Unformatted: