From andrey.zverev@electro-com.ru Sat Dec 9 17:37:43 2006 Return-Path: Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6982116A416 for ; Sat, 9 Dec 2006 17:37:43 +0000 (UTC) (envelope-from andrey.zverev@electro-com.ru) Received: from mail.electro-com.ru (mail.electro-com.ru [86.110.161.242]) by mx1.FreeBSD.org (Postfix) with ESMTP id D52AF43CA1 for ; Sat, 9 Dec 2006 17:36:37 +0000 (GMT) (envelope-from andrey.zverev@electro-com.ru) Received: from az by mail.electro-com.ru with local (Exim 4.63 (FreeBSD)) (envelope-from ) id 1Gt68r-000Efb-I6 for FreeBSD-gnats-submit@freebsd.org; Sat, 09 Dec 2006 20:37:41 +0300 Message-Id: Date: Sat, 09 Dec 2006 20:37:41 +0300 From: Andrej Zverev Sender: Andrej Zverev Reply-To: Andrej Zverev To: FreeBSD-gnats-submit@freebsd.org Cc: Subject: [panic] ipfw + dummynet X-Send-Pr-Version: 3.113 X-GNATS-Notify: >Number: 106534 >Category: kern >Synopsis: [ipfw] [panic] ipfw + dummynet >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-ipfw >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Dec 09 17:40:11 GMT 2006 >Closed-Date: Sat Jul 12 11:06:47 UTC 2008 >Last-Modified: Sat Jul 12 11:06:47 UTC 2008 >Originator: Andrej Zverev >Release: >Organization: >Environment: >Description: Using dumment for traffic shaping with about 900 queues or pipes and bandwith > 30Mbit/s provide panic on 6.1 and 6.2PRERELEASE Before panic (kernel build with INVARIANTS) on console i can show Memory modified after free 0xc4f55800(2048) val=c75a43d4 @ 0xc4f55880 Memory modified after free 0xc4e02800(2048) val=488e26e3 @ 0xc4e028c0 dummynet: OUCH! pipe should have been idle! Memory modified after free 0xc4e05800(2048) val=f4f21018 @ 0xc4e05880 Memory modified after free 0xc4e64000(2048) val=413c203e @ 0xc4e64080 Memory modified after free 0xc4b7d800(2048) val=98d450d7 @ 0xc4b7d880 Memory modified after free 0xc520d000(2048) val=36a81ffb @ 0xc520d080 Memory modified after free 0xc4f96000(2048) val=66407a4b @ 0xc4f961c0 Memory modified after free 0xc84c1000(2048) val=2037322e @ 0xc84c1080 Memory modified after free 0xc4f8a000(2048) val=7b38df64 @ 0xc4f8a0c0 Time to get panic about 5-15 minutes. ctm# ifconfig ste0: flags=8943 mtu 1500 options=48 ether 00:11:95:cb:66:6e media: Ethernet 100baseTX status: active ste1: flags=8943 mtu 1500 options=48 ether 00:11:95:cb:66:7a media: Ethernet 100baseTX status: active vr0: flags=8843 mtu 1500 inet x.y.z.a netmask 0xfffffffc broadcast x.y.z.a ether 00:11:95:fc:81:85 media: Ethernet 100baseTX status: active lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000 bridge0: flags=8843 mtu 1500 ether fa:96:da:98:10:ca priority 32768 hellotime 2 fwddelay 15 maxage 20 member: ste1 flags=3 member: ste0 flags=3 >How-To-Repeat: >Fix: --- 1.txt begins here --- ctm# uname -v FreeBSD 6.2-PRERELEASE #1: Fri Dec 8 14:56:55 MSK 2006 root@y.x.la-com.int:/usr/obj/usr/src/sys/CTM_DEBUG GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode fault virtual address = 0xc fault code = supervisor read, page not present instruction pointer = 0x20:0xc065ded3 stack pointer = 0x28:0xe339ab5c frame pointer = 0x28:0xe339ab80 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 11 (swi4: clock sio) trap number = 12 panic: page fault Uptime: 10m42s Dumping 1007 MB (2 chunks) chunk 0: 1MB (159 pages) ... ok chunk 1: 1007MB (257776 pages) 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 #0 doadump () at pcpu.h:165 in pcpu.h (kgdb) f 0 #0 doadump () at pcpu.h:165 165 in pcpu.h (kgdb) f 1 #1 0xc062813a in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 409 doadump(); (kgdb) f 2 #2 0xc06283d0 in panic (fmt=0xc083002b "%s") at /usr/src/sys/kern/kern_shutdown.c:565 565 boot(bootopt); (kgdb) f 3 #3 0xc07eb314 in trap_fatal (frame=0xe339ab1c, eva=12) at /usr/src/sys/i386/i386/trap.c:837 837 panic("%s", trap_msg[type]); (kgdb) f 4 #4 0xc07eb07b in trap_pfault (frame=0xe339ab1c, usermode=0, eva=12) at /usr/src/sys/i386/i386/trap.c:745 745 trap_fatal(frame, eva); (kgdb) f 5 #5 0xc07eacd9 in trap (frame= {tf_fs = 8, tf_es = 40, tf_ds = 40, tf_edi = -988715436, tf_esi = 387, tf_ebp = -482759808, tf_isp = -482759864, tf_ebx = -988715520, tf_edx = 0, tf_ecx = -985141232, tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -1067065645, tf_cs = 32, tf_eflags = 66050, tf_esp = 0, tf_ss = -482759804}) at /usr/src/sys/i386/i386/trap.c:435 435 (void) trap_pfault(&frame, FALSE, eva); (kgdb) f 6 #6 0xc07d9cba in calltrap () at /usr/src/sys/i386/i386/exception.s:139 139 call trap Current language: auto; currently asm (kgdb) f 7 #7 0xc065ded3 in m_copym (m=0x0, off0=1500, len=1480, wait=1) at /usr/src/sys/kern/uipc_mbuf.c:400 400 if (off < m->m_len) Current language: auto; currently c (kgdb) f 8 #8 0xc06d5784 in ip_fragment (ip=0xc547f010, m_frag=0xe339ac3c, mtu=-988715520, if_hwassist_flags=0, sw_csum=1) at /usr/src/sys/netinet/ip_output.c:975 975 m->m_next = m_copy(m0, off, len); (kgdb) f 9 #9 0xc06d542b in ip_output (m=0xc5511700, opt=0xc4b6a800, ro=0xe339ac08, flags=1, imo=0x0, inp=0x0) at /usr/src/sys/netinet/ip_output.c:804 804 error = ip_fragment(ip, &m, ifp->if_mtu, ifp->if_hwassist, sw_csum); (kgdb) f 10 #10 0xc06c8069 in dummynet_send (m=0xc5511700) at /usr/src/sys/netinet/ip_dummynet.c:771 771 ip_output(m, NULL, NULL, IP_FORWARDING, NULL, NULL); (kgdb) f 11 #11 0xc06c7ffc in dummynet (unused=0x0) at /usr/src/sys/netinet/ip_dummynet.c:753 753 dummynet_send(head); (kgdb) f 12 #12 0xc0634543 in softclock (dummy=0x0) at /usr/src/sys/kern/kern_timeout.c:290 290 c_func(c_arg); (kgdb) f 13 #13 0xc0612549 in ithread_execute_handlers (p=0xc4a51a78, ie=0xc4a9e300) at /usr/src/sys/kern/kern_intr.c:682 682 ih->ih_handler(ih->ih_argument); (kgdb) f 14 #14 0xc0612654 in ithread_loop (arg=0xc4a19720) at /usr/src/sys/kern/kern_intr.c:765 765 ithread_execute_handlers(p, ie); (kgdb) f 15 #15 0xc06114d0 in fork_exit (callout=0xc0612600 , arg=0xc4a19720, frame=0xe339ad38) at /usr/src/sys/kern/kern_fork.c:821 821 callout(arg, frame); (kgdb) f 16 #16 0xc07d9d1c in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:208 208 call fork_exit Current language: auto; currently asm (kgdb) quit --- 1.txt ends here --- >Release-Note: >Audit-Trail: Responsible-Changed-From-To: freebsd-bugs->freebsd-ipfw Responsible-Changed-By: linimon Responsible-Changed-When: Sun Dec 10 13:25:58 UTC 2006 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=106534 From: Andrej Zverev To: bug-followup@FreeBSD.org, az@freebsd.org, wpaul@freebsd.org Cc: Subject: Re: kern/106534: [ipfw] [panic] ipfw + dummynet Date: Mon, 11 Dec 2006 19:08:25 +0300 Little notice about it. Chaging network card from ste(4) to em(4) helps, so it's might me problem in ste(4) driver. Maybe wpaul@ can look at this situation ? WBR, Andrej Zverev From: wpaul@FreeBSD.ORG (Bill Paul) To: andrey.zverev@electro-com.ru (Andrej Zverev) Cc: bug-followup@FreeBSD.org, az@freebsd.org Subject: Re: kern/106534: [ipfw] [panic] ipfw + dummynet Date: Mon, 11 Dec 2006 21:53:22 +0000 (GMT) > Little notice about it. > > Chaging network card from ste(4) to em(4) helps, so it's might me > problem in ste(4) driver. > > Maybe wpaul@ can look at this situation ? Absolutely not. The manual for this chip is here: http://www.freebsd.org/~wpaul/Sundance/st201.pdf Feel free to investigate the problem and fix it yourself. -Bill From: Olexandr Davydenko To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/106534: [ipfw] [panic] ipfw + dummynet Date: Mon, 15 Jan 2007 14:20:22 +0200 Similar problem with ipfw + dummynet and 132 pipes for traffic shaping: sometimes panic when trafshow run and put interface in promiscuous mode or in any another time. FreeBSD 6.1-RELEASE-p10 #1: Wed Oct 4 12:46:05 EEST 2006 root@xxx:/server/OBJ/server/SRC/RELENG_6_1/sys/xxx # ifconfig fxp0: flags=8843 mtu 1500 options=48 ether 00:90:27:10:33:b4 media: Ethernet autoselect (100baseTX ) status: active fxp1: flags=8843 mtu 1500 options=48 ether 00:30:48:22:58:79 media: Ethernet 100baseTX status: active fxp2: flags=8843 mtu 1500 options=48 ether 00:30:48:22:58:7a media: Ethernet autoselect (100baseTX ) status: active lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000 kgdb output: GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode fault virtual address = 0xc fault code = supervisor read, page not present instruction pointer = 0x20:0xc0516ffb stack pointer = 0x28:0xcbd36b60 frame pointer = 0x28:0xcbd36b84 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi4: clock sio) trap number = 12 panic: page fault Uptime: 29d17h3m30s Dumping 254 MB (2 chunks) chunk 0: 1MB (159 pages) ... ok chunk 1: 254MB (65024 pages) 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 #0 doadump () at pcpu.h:165 in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc04e1b65 in boot (howto=260) at /server/SRC/RELENG_6_1/sys/kern/kern_shutdown.c:402 #2 0xc04e1dfc in panic (fmt=0xc063e3b0 "%s") at /server/SRC/RELENG_6_1/sys/kern/kern_shutdown.c:558 #3 0xc06254fc in trap_fatal (frame=0xcbd36b20, eva=12) at /server/SRC/RELENG_6_1/sys/i386/i386/trap.c:836 #4 0xc0625263 in trap_pfault (frame=0xcbd36b20, usermode=0, eva=12) at /server/SRC/RELENG_6_1/sys/i386/i386/trap.c:744 #5 0xc0624ec1 in trap (frame= {tf_fs = 8, tf_es = 40, tf_ds = 40, tf_edi = -1014294700, tf_esi = 320, tf_ebp = -875336828, tf_isp = -875336884, tf_ebx = -1014294784, tf_edx = 0, tf_ecx = -1026586592, tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -1068404741, tf_cs = 32, tf_eflags = 590338, tf_esp = 0, tf_ss = -875336824}) at /server/SRC/RELENG_6_1/sys/i386/i386/trap.c:434 #6 0xc06153ba in calltrap () at /server/SRC/RELENG_6_1/sys/i386/i386/exception.s:139 #7 0xc0516ffb in m_copym (m=0x0, off0=1500, len=1480, wait=1) at /server/SRC/RELENG_6_1/sys/kern/uipc_mbuf.c:400 #8 0xc056bff8 in ip_fragment (ip=0xc2cf8820, m_frag=0xcbd36c3c, mtu=-1014294784, if_hwassist_flags=0, sw_csum=1) at /server/SRC/RELENG_6_1/sys/netinet/ip_output.c:975 #9 0xc056bc9e in ip_output (m=0xc33a1c00, opt=0xc1d8e000, ro=0xcbd36c08, flags=1, imo=0x0, inp=0x0) at /server/SRC/RELENG_6_1/sys/netinet/ip_output.c:804 #10 0xc055ef71 in dummynet_send (m=0xc33a1c00) at /server/SRC/RELENG_6_1/sys/netinet/ip_dummynet.c:771 #11 0xc055ef04 in dummynet (unused=0x0) at /server/SRC/RELENG_6_1/sys/netinet/ip_dummynet.c:753 #12 0xc04edc97 in softclock (dummy=0x0) at /server/SRC/RELENG_6_1/sys/kern/kern_timeout.c:290 #13 0xc04cc391 in ithread_execute_handlers (p=0xc1d97830, ie=0xc1d95600) at /server/SRC/RELENG_6_1/sys/kern/kern_intr.c:684 #14 0xc04cc4a8 in ithread_loop (arg=0xc1d83780) at /server/SRC/RELENG_6_1/sys/kern/kern_intr.c:767 #15 0xc04cb300 in fork_exit (callout=0xc04cc454 , arg=0xc1d83780, frame=0xcbd36d38) at /server/SRC/RELENG_6_1/sys/kern/kern_fork.c:805 #16 0xc061541c in fork_trampoline () at /server/SRC/RELENG_6_1/sys/i386/i386/exception.s:208 (kgdb) up 7 #7 0xc0516ffb in m_copym (m=0x0, off0=1500, len=1480, wait=1) at /server/SRC/RELENG_6_1/sys/kern/uipc_mbuf.c:400 400 if (off < m->m_len) (kgdb) list 395 MBUF_CHECKSLEEP(wait); 396 if (off == 0 && m->m_flags & M_PKTHDR) 397 copyhdr = 1; 398 while (off > 0) { 399 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); 400 if (off < m->m_len) 401 break; 402 off -= m->m_len; 403 m = m->m_next; 404 } (kgdb) quit -- WBR, Davidenko Alexandr From: Oleg Bulyzhin To: Olexandr Davydenko Cc: freebsd-ipfw@freebsd.org Subject: Re: kern/106534: [ipfw] [panic] ipfw + dummynet Date: Wed, 17 Jan 2007 02:00:39 +0300 On Mon, Jan 15, 2007 at 12:40:23PM +0000, Olexandr Davydenko wrote: > The following reply was made to PR kern/106534; it has been noted by GNATS. > > From: Olexandr Davydenko > To: bug-followup@FreeBSD.org > Cc: > Subject: Re: kern/106534: [ipfw] [panic] ipfw + dummynet > Date: Mon, 15 Jan 2007 14:20:22 +0200 > > Similar problem with ipfw + dummynet and 132 pipes for traffic shaping: > sometimes panic when trafshow run and put interface in promiscuous > mode or in any another time. > > FreeBSD 6.1-RELEASE-p10 #1: Wed Oct 4 12:46:05 EEST 2006 > root@xxx:/server/OBJ/server/SRC/RELENG_6_1/sys/xxx > > # ifconfig > fxp0: flags=8843 mtu 1500 > options=48 > ether 00:90:27:10:33:b4 > media: Ethernet autoselect (100baseTX ) > status: active > fxp1: flags=8843 mtu 1500 > options=48 > ether 00:30:48:22:58:79 > media: Ethernet 100baseTX > status: active > fxp2: flags=8843 mtu 1500 > options=48 > ether 00:30:48:22:58:7a > media: Ethernet autoselect (100baseTX ) > status: active > lo0: flags=8049 mtu 16384 > inet 127.0.0.1 netmask 0xff000000 > > kgdb output: > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "i386-marcel-freebsd". > > Unread portion of the kernel message buffer: > > > Fatal trap 12: page fault while in kernel mode > fault virtual address = 0xc > fault code = supervisor read, page not present > instruction pointer = 0x20:0xc0516ffb > stack pointer = 0x28:0xcbd36b60 > frame pointer = 0x28:0xcbd36b84 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 12 (swi4: clock sio) > trap number = 12 > panic: page fault > Uptime: 29d17h3m30s > Dumping 254 MB (2 chunks) > chunk 0: 1MB (159 pages) ... ok > chunk 1: 254MB (65024 pages) 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 > > #0 doadump () at pcpu.h:165 > in pcpu.h > (kgdb) bt > #0 doadump () at pcpu.h:165 > #1 0xc04e1b65 in boot (howto=260) at /server/SRC/RELENG_6_1/sys/kern/kern_shutdown.c:402 > #2 0xc04e1dfc in panic (fmt=0xc063e3b0 "%s") at /server/SRC/RELENG_6_1/sys/kern/kern_shutdown.c:558 > #3 0xc06254fc in trap_fatal (frame=0xcbd36b20, eva=12) at /server/SRC/RELENG_6_1/sys/i386/i386/trap.c:836 > #4 0xc0625263 in trap_pfault (frame=0xcbd36b20, usermode=0, eva=12) > at /server/SRC/RELENG_6_1/sys/i386/i386/trap.c:744 > #5 0xc0624ec1 in trap (frame= > {tf_fs = 8, tf_es = 40, tf_ds = 40, tf_edi = -1014294700, tf_esi = 320, tf_ebp = -875336828, tf_isp = -875336884, tf_ebx = -1014294784, tf_edx = 0, tf_ecx = -1026586592, tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -1068404741, tf_cs = 32, tf_eflags = 590338, tf_esp = 0, tf_ss = -875336824}) > at /server/SRC/RELENG_6_1/sys/i386/i386/trap.c:434 > #6 0xc06153ba in calltrap () at /server/SRC/RELENG_6_1/sys/i386/i386/exception.s:139 > #7 0xc0516ffb in m_copym (m=0x0, off0=1500, len=1480, wait=1) > at /server/SRC/RELENG_6_1/sys/kern/uipc_mbuf.c:400 > #8 0xc056bff8 in ip_fragment (ip=0xc2cf8820, m_frag=0xcbd36c3c, mtu=-1014294784, if_hwassist_flags=0, > sw_csum=1) at /server/SRC/RELENG_6_1/sys/netinet/ip_output.c:975 > #9 0xc056bc9e in ip_output (m=0xc33a1c00, opt=0xc1d8e000, ro=0xcbd36c08, flags=1, imo=0x0, inp=0x0) > at /server/SRC/RELENG_6_1/sys/netinet/ip_output.c:804 > #10 0xc055ef71 in dummynet_send (m=0xc33a1c00) at /server/SRC/RELENG_6_1/sys/netinet/ip_dummynet.c:771 > #11 0xc055ef04 in dummynet (unused=0x0) at /server/SRC/RELENG_6_1/sys/netinet/ip_dummynet.c:753 > #12 0xc04edc97 in softclock (dummy=0x0) at /server/SRC/RELENG_6_1/sys/kern/kern_timeout.c:290 > #13 0xc04cc391 in ithread_execute_handlers (p=0xc1d97830, ie=0xc1d95600) > at /server/SRC/RELENG_6_1/sys/kern/kern_intr.c:684 > #14 0xc04cc4a8 in ithread_loop (arg=0xc1d83780) at /server/SRC/RELENG_6_1/sys/kern/kern_intr.c:767 > #15 0xc04cb300 in fork_exit (callout=0xc04cc454 , arg=0xc1d83780, frame=0xcbd36d38) > at /server/SRC/RELENG_6_1/sys/kern/kern_fork.c:805 > #16 0xc061541c in fork_trampoline () at /server/SRC/RELENG_6_1/sys/i386/i386/exception.s:208 > (kgdb) up 7 > #7 0xc0516ffb in m_copym (m=0x0, off0=1500, len=1480, wait=1) > at /server/SRC/RELENG_6_1/sys/kern/uipc_mbuf.c:400 > 400 if (off < m->m_len) > (kgdb) list > 395 MBUF_CHECKSLEEP(wait); > 396 if (off == 0 && m->m_flags & M_PKTHDR) > 397 copyhdr = 1; > 398 while (off > 0) { > 399 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); > 400 if (off < m->m_len) > 401 break; > 402 off -= m->m_len; > 403 m = m->m_next; > 404 } > (kgdb) quit > > > > -- > WBR, > Davidenko Alexandr > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" As i can see kernel dies trying to fit packet into interface with negative mtu. Would be fine to dump ro->ro_rt->rt_ifp structure. (Do some 'up' commands until you are in ip_output, then print ro->ro_rt->rt_ifp). -- Oleg. ================================================================ === Oleg Bulyzhin -- OBUL-RIPN -- OBUL-RIPE -- oleg@rinet.ru === ================================================================ _______________________________________________ freebsd-ipfw@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" State-Changed-From-To: open->closed State-Changed-By: az State-Changed-When: Sat Jul 12 11:06:46 UTC 2008 State-Changed-Why: No way to repeat such panic http://www.freebsd.org/cgi/query-pr.cgi?pr=106534 >Unformatted: