From nobody@FreeBSD.org Sat Sep 3 23:36:28 2011 Return-Path: Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 72C551065672 for ; Sat, 3 Sep 2011 23:36:28 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22]) by mx1.freebsd.org (Postfix) with ESMTP id 587368FC0A for ; Sat, 3 Sep 2011 23:36:28 +0000 (UTC) Received: from red.freebsd.org (localhost [127.0.0.1]) by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p83NaSfM015232 for ; Sat, 3 Sep 2011 23:36:28 GMT (envelope-from nobody@red.freebsd.org) Received: (from nobody@localhost) by red.freebsd.org (8.14.4/8.14.4/Submit) id p83NaSdG015231; Sat, 3 Sep 2011 23:36:28 GMT (envelope-from nobody) Message-Id: <201109032336.p83NaSdG015231@red.freebsd.org> Date: Sat, 3 Sep 2011 23:36:28 GMT From: Ian To: freebsd-gnats-submit@FreeBSD.org Subject: Packets transmitted on vlan(4) interfaces with a parent vge(4) vanish. X-Send-Pr-Version: www-3.1 X-GNATS-Notify: >Number: 160442 >Category: kern >Synopsis: [vlan] Packets transmitted on vlan(4) interfaces with a parent vge(4) vanish. >Confidential: no >Severity: non-critical >Priority: low >Responsible: yongari >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Sep 03 23:40:00 UTC 2011 >Closed-Date: Thu Sep 15 17:33:01 UTC 2011 >Last-Modified: Thu Sep 15 17:33:01 UTC 2011 >Originator: Ian >Release: 8.2-RELEASE-p2 >Organization: - >Environment: FreeBSD 8.2-RELEASE-p2 i386 >Description: When using a vge(4) physical interface for an 802.1Q trunk, packets sent via the vlan(4) interfaces tied to it seem to disappear. When debugging this, I came across a few curious things. The physical interface receives the packet, as reported by tcpdump. Then, the tagging is stripped off, and it's handed to the vlan interface, where tcpdump again sees the packet, unencapsulated, as one would suspect. Replies sent back out can be seen on the vlan interface, but tcpdump never sees any packets being sent back out the parent interface, and they never are! To illustrate, here's a DHCP request, as it appears on the physical interface: 07:57:13.147576 00:11:43:f6:00:28 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 410: vlan 3, p 0, ethertype IPv4, (tos 0x0, ttl 128, id 35779, offset 0, flags [none], proto UDP (17), length 306) 0.0.0.0.68 > 255.255.255.255.67: [no cksum] BOOTP/DHCP, Request from 00:11:43:f6:00:28, length 278, xid 0x52525230, secs 2868, Flags [Broadcast] (0x8000) Client-Ethernet-Address 00:11:43:f6:00:28 [|bootp] This is then received on the vlan interface, and a reply is sent back out on the vlan interface: 07:58:25.142617 00:11:43:f6:00:28 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 406: (tos 0x0, ttl 128, id 13346, offset 0, flags [none], proto UDP (17), length 306) 0.0.0.0.68 > 255.255.255.255.67: [no cksum] BOOTP/DHCP, Request from 00:11:43:f6:00:28, length 278, xid 0x52525230, secs 2940, Flags [Broadcast] (0x8000) Client-Ethernet-Address 00:11:43:f6:00:28 [|bootp] 07:58:25.146633 00:40:63:e6:8d:a5 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 192.168.0.221.67 > 255.255.255.255.68: BOOTP/DHCP, Reply, length 300, xid 0x52525230, secs 2940, Flags [Broadcast] (0x8000) Your-IP 192.168.0.193 Client-Ethernet-Address 00:11:43:f6:00:28 [|bootp] ..but no reply is ever seen on the physical interface dump, and nothing hits the wire. This happens with all packets sent, including ARP replies. Without any vlans configured, the interface itself works well. I have not yet tested configuring vlans on it and then attempting to just use the native vlan on the interface itself, but I'm willing to if it would help. Strangely, if I disable HW VLAN tagging, the incoming packet can still be seen with tcpdump on the vge and vlan interfaces. However, I don't then see a reply on either interface, vlan or vge. Also, even though polling is being used in my case, disabling/enabling it seems to have no effect on the situation. >How-To-Repeat: Configure a vge interface with one or more vlan like so: vge0: flags=8843 metric 0 mtu 1500 options=38db ether 00:40:63:e6:8d:a5 media: Ethernet autoselect (1000baseT ) status: active vlan3: flags=8843 metric 0 mtu 1500 options=3 ether 00:40:63:e6:8d:a5 inet 192.168.0.222 netmask 0xffffffe0 broadcast 192.168.0.223 inet 192.168.0.221 netmask 0xffffffff broadcast 192.168.0.221 [...] media: Ethernet autoselect (1000baseT ) status: active vlan: 3 parent interface: vge0 Attempt to ARP any of the addresses on the vlan interface, and you will see no reply. Check for the odd behavior with tcpdump in the full description of the problem. >Fix: None known. >Release-Note: >Audit-Trail: Responsible-Changed-From-To: freebsd-bugs->freebsd-net Responsible-Changed-By: linimon Responsible-Changed-When: Sun Sep 4 06:27:20 UTC 2011 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=160442 State-Changed-From-To: open->feedback State-Changed-By: yongari State-Changed-When: Tue Sep 6 00:43:02 UTC 2011 State-Changed-Why: It seems parent interface vge(4) thinks it does not have established link. vge(4) actively keeps track of current link state and it does not try to send packets when it lost link. Unlike other drivers, vge(4) relies on its PHY hardware generates an interrupt on link status change. If the PHY failed to generate interrupt for link establishment, parent device may not see the update link status. To experiment this, try unplug the UTP cable and replug it and see whether that makes any difference for you. I'd also like to know your PHY hardware and controller revision so post dmesg and "pciconf -lcbv" output. I failed to reproduce the issue on my VIA VT6130 PCIe controller though. Responsible-Changed-From-To: freebsd-net->yongari Responsible-Changed-By: yongari Responsible-Changed-When: Tue Sep 6 00:43:02 UTC 2011 Responsible-Changed-Why: Grab. http://www.freebsd.org/cgi/query-pr.cgi?pr=160442 From: Ian To: bug-followup@FreeBSD.org, yoitsmeremember@gmail.com Cc: Subject: Re: kern/160442: [vlan] Packets transmitted on vlan(4) interfaces with a parent vge(4) vanish. Date: Mon, 05 Sep 2011 22:21:44 -0500 Relevant lines from dmesg: vge0: port 0xf800-0xf8ff mem 0xfdffe000-0xfdffe0ff irq 11 at device 14.0 on pci0 miibus0: on vge0 ciphy0: PHY 1 on miibus0 ciphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto vge0: Ethernet address: 00:40:63:e6:8d:a5 vge0: [ITHREAD] If I missed something, or you need the entire thing, I can provide that as well. Full output of pciconf -lcbv: hostb0@pci0:0:0:0: class=0x060000 card=0xaa081106 chip=0x03141106 rev=0x00 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'CN700/VN800/P4M800CE/Pro Host Bridge' class = bridge subclass = HOST-PCI bar [10] = type Prefetchable Memory, range 32, base 0xf8000000, size 33554432, enabled cap 02[80] = AGP v3 8x 4x SBA disabled cap 01[50] = powerspec 2 supports D0 D3 current D0 hostb1@pci0:0:0:1: class=0x060000 card=0x00000000 chip=0x13141106 rev=0x00 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'CN700/VN800/P4M800CE/Pro Standard Host Bridge' class = bridge subclass = HOST-PCI hostb2@pci0:0:0:2: class=0x060000 card=0x00000000 chip=0x23141106 rev=0x00 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'CN700/VN800/P4M800CE/Pro Standard Host Bridge' class = bridge subclass = HOST-PCI hostb3@pci0:0:0:3: class=0x060000 card=0x00000000 chip=0x32081106 rev=0x00 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'CPU to PCI Bridge (PT890)' class = bridge subclass = HOST-PCI hostb4@pci0:0:0:4: class=0x060000 card=0x00000000 chip=0x43141106 rev=0x00 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'CN700/VN800/P4M800CE/Pro Host Bridge' class = bridge subclass = HOST-PCI hostb5@pci0:0:0:7: class=0x060000 card=0x00000000 chip=0x73141106 rev=0x00 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'CN700/VN800/P4M800CE/Pro Host Bridge' class = bridge subclass = HOST-PCI pcib1@pci0:0:1:0: class=0x060400 card=0x00000000 chip=0xb1981106 rev=0x00 hdr=0x01 vendor = 'VIA Technologies, Inc.' device = 'ProSavageDDR P4X600,Apollo KT400/A/600 CPU to AGP Bridge' class = bridge subclass = PCI-PCI cap 01[70] = powerspec 2 supports D0 D1 D3 current D0 fwohci0@pci0:0:13:0: class=0x0c0010 card=0x30441106 chip=0x30441106 rev=0x80 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'VT6306 VIA Fire II IEEE-1394 OHCI Link Layer Controller' class = serial bus subclass = FireWire bar [10] = type Memory, range 32, base 0xfdfff000, size 2048, enabled bar [14] = type I/O Port, range 32, base 0xfc00, size 128, enabled cap 01[50] = powerspec 2 supports D0 D2 D3 current D0 vge0@pci0:0:14:0: class=0x020000 card=0x01101106 chip=0x31191106 rev=0x11 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = ''Velocity' Gigabit Ethernet Controllers (VT6120/VT6121/VT6122)' class = network subclass = ethernet bar [10] = type I/O Port, range 32, base 0xf800, size 256, enabled bar [14] = type Memory, range 32, base 0xfdffe000, size 256, enabled cap 01[50] = powerspec 2 supports D0 D1 D2 D3 current D0 atapci0@pci0:0:15:0: class=0x01018a card=0xaa081106 chip=0x05711106 rev=0x06 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'Bus Master IDE Controller (VT82C686B/VT823x/A/C)' class = mass storage subclass = ATA bar [20] = type I/O Port, range 32, base 0xf400, size 16, enabled cap 01[c0] = powerspec 2 supports D0 D3 current D0 uhci0@pci0:0:16:0: class=0x0c0300 card=0xaa081106 chip=0x30381106 rev=0x81 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'VT82xxxxx UHCI USB 1.1 Controller (All VIA Chipsets)' class = serial bus subclass = USB bar [20] = type I/O Port, range 32, base 0xf000, size 32, enabled cap 01[80] = powerspec 2 supports D0 D1 D2 D3 current D0 uhci1@pci0:0:16:1: class=0x0c0300 card=0xaa081106 chip=0x30381106 rev=0x81 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'VT82xxxxx UHCI USB 1.1 Controller (All VIA Chipsets)' class = serial bus subclass = USB bar [20] = type I/O Port, range 32, base 0xec00, size 32, enabled cap 01[80] = powerspec 2 supports D0 D1 D2 D3 current D0 uhci2@pci0:0:16:2: class=0x0c0300 card=0xaa081106 chip=0x30381106 rev=0x81 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'VT82xxxxx UHCI USB 1.1 Controller (All VIA Chipsets)' class = serial bus subclass = USB bar [20] = type I/O Port, range 32, base 0xe800, size 32, enabled cap 01[80] = powerspec 2 supports D0 D1 D2 D3 current D0 ehci0@pci0:0:16:4: class=0x0c0320 card=0xaa081106 chip=0x31041106 rev=0x86 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'VT6202/12 USB 2.0 Enhanced Host Controller' class = serial bus subclass = USB bar [10] = type Memory, range 32, base 0xfdffd000, size 256, enabled cap 01[80] = powerspec 2 supports D0 D1 D2 D3 current D0 isab0@pci0:0:17:0: class=0x060100 card=0xaa081106 chip=0x32271106 rev=0x00 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'PCI-to-ISA Bridge (VT8237)' class = bridge subclass = PCI-ISA cap 01[c0] = powerspec 2 supports D0 D3 current D0 vgapci0@pci0:1:0:0: class=0x030000 card=0x33441106 chip=0x33441106 rev=0x01 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'VIA/S3G UniChrome Pro IGP (VT3314)' class = display subclass = VGA bar [10] = type Prefetchable Memory, range 32, base 0xf4000000, size 67108864, enabled bar [14] = type Memory, range 32, base 0xfb000000, size 16777216, enabled cap 01[60] = powerspec 2 supports D0 D1 D2 D3 current D0 cap 02[70] = AGP v3 8x 4x SBA disabled Interesting that this is interrupt related. I first tried your suggestion, unplugging and replugging the cable. Ultimately, no change in the bug, but ifconfig at least reported "no carrier". Remembering that polling essentially ignores interrupts, I changed my rc.conf from ifconfig_vge0="up polling" to simply ifconfig_vge0="up", so polling would not be present during boot, and restarted the machine. Things were working again! I could even enable polling later (now that presumably the state was correctly internally determined to be "up and connected" before polling was ever enabled) without issue. So, is polling's indifference to IRQs what is ultimately causing the issue? I missed this being the issue earlier as I enabled polling at the same time I set up trunking, and thought the issue was strictly vlan related. Many thanks for the assistance, and let me know if you need anything else from me to hunt this down. I'm happy to help. - Ian From: YongHyeon PYUN To: Ian Cc: yongari@freebsd.org, bug-followup@FreeBSD.org Subject: Re: kern/160442: [vlan] Packets transmitted on vlan(4) interfaces with a parent vge(4) vanish. Date: Tue, 6 Sep 2011 10:51:17 -0700 On Tue, Sep 06, 2011 at 03:50:03AM +0000, Ian wrote: [...] > Relevant lines from dmesg: > [...] Thanks for the information. > Interesting that this is interrupt related. I first tried your > suggestion, unplugging and replugging the cable. Ultimately, no change > in the bug, but ifconfig at least reported "no carrier". Remembering > that polling essentially ignores interrupts, I changed my rc.conf from > ifconfig_vge0="up polling" to simply ifconfig_vge0="up", so polling > would not be present during boot, and restarted the machine. Things > were working again! I could even enable polling later (now that > presumably the state was correctly internally determined to be "up and > connected" before polling was ever enabled) without issue. > Oh, I completely forgot you're POLLING user. Sorry I should have noticed that. > So, is polling's indifference to IRQs what is ultimately causing the > issue? I missed this being the issue earlier as I enabled polling at Yes. As I said, vge(4) needs interrupt to detect link state changes. This means vge(4) does not correctly detect link state with DEVICE_POLLING. For instance, you should have seen "vge0: link state changed to DOWN" for lost link on console and you should see "vge0: link state changed to UP" when PHY successfully established a link with link partner. Upper stack also relies on driver's link state to decide which route to use so it's important to correctly keep track of current link state. > the same time I set up trunking, and thought the issue was strictly vlan > related. Many thanks for the assistance, and let me know if you need > anything else from me to hunt this down. I'm happy to help. I made a workaround patch to handle link state interrupt with DEVICE_POLLING. This patch allows generating link state interrupt even with DEVICE_POLLING but it may help to address the issue. Could you try the patch at the following URL? http://people.freebsd.org/~yongari/vge/vge.linkintr.diff BTW, I don't know why you have to use DEVICE_POLLING on vge(4). vge(4) supports various interrupt moderation mechanism such that it does not generate excessive interrupts under high network load. Using DEVICE_POLLING almost always adds more latency and consumes CPU cycles. I admit there could a couple of specific usage for DEVICE_POLLING but most cases it wouldn't be needed on most ethernet controllers that have interrupt moderation mechanism. If the interrupt line is shared with other slow devices like USB on legacy PCI controllers that lack MSI capability, DEVICE_POLLING may yield slightly better performance under high network load though. From: Ian To: pyunyh@gmail.com Cc: yongari@freebsd.org, bug-followup@FreeBSD.org Subject: Re: kern/160442: [vlan] Packets transmitted on vlan(4) interfaces with a parent vge(4) vanish. Date: Tue, 06 Sep 2011 17:01:02 -0500 > I made a workaround patch to handle link state interrupt with > DEVICE_POLLING. This patch allows generating link state interrupt > even with DEVICE_POLLING but it may help to address the issue. > Could you try the patch at the following URL? > http://people.freebsd.org/~yongari/vge/vge.linkintr.diff This solves the problem. I tested with the exact same setup I had before (ifconfig_vge0="up polling"), with no issues. I also tested booting with the cable unplugged, and plugging it in later, and again it worked flawlessly. > BTW, I don't know why you have to use DEVICE_POLLING on vge(4). > vge(4) supports various interrupt moderation mechanism such that it > does not generate excessive interrupts under high network load. > Using DEVICE_POLLING almost always adds more latency and consumes > CPU cycles. I admit there could a couple of specific usage for > DEVICE_POLLING but most cases it wouldn't be needed on most > ethernet controllers that have interrupt moderation mechanism. > If the interrupt line is shared with other slow devices like USB on > legacy PCI controllers that lack MSI capability, DEVICE_POLLING may > yield slightly better performance under high network load though. Ah, thanks for the information. The main reason I have it in my kernel is to have it as an option, and to only enable it if it is necessary or shows performance improvements. In the past I've had interfaces that have crippled a system with IRQ spam, but I think they were some junk re(4) or rl(4) interfaces. Although I probably won't be using polling with vge(4), if polling is supported, it should work, and I'm glad it does with the patch. If you need to make any changes to the patch and require additional testing before committing, just let me know. State-Changed-From-To: feedback->patched State-Changed-By: yongari State-Changed-When: Wed Sep 7 16:58:41 UTC 2011 State-Changed-Why: Fixed in HEAD(r225440). Will MFC after a week. Thanks a lot for testing and reporting! http://www.freebsd.org/cgi/query-pr.cgi?pr=160442 From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/160442: commit references a PR Date: Wed, 7 Sep 2011 16:57:58 +0000 (UTC) Author: yongari Date: Wed Sep 7 16:57:43 2011 New Revision: 225440 URL: http://svn.freebsd.org/changeset/base/225440 Log: vge(4) hardwares poll media status and generates an interrupt whenever the link state is changed. Using software based polling for media status tracking is known to cause MII access failure under certain conditions once link is established so vge(4) used to rely on link status change interrupt. However DEVICE_POLLING completely disables generation of all kind of interrupts on vge(4) such that this resulted in not detecting link state change event. This means vge(4) does not correctly detect established/lost link with DEVICE_POLLING. Losing the interrupt made vge(4) not to send any packets to peer since vge(4) does not try to send any packets when there is no established link. Work around the issue by generating link state change interrupt with DEVICE_POLLING. PR: kern/160442 Approved by: re (kib) Modified: head/sys/dev/vge/if_vge.c head/sys/dev/vge/if_vgereg.h Modified: head/sys/dev/vge/if_vge.c ============================================================================== --- head/sys/dev/vge/if_vge.c Wed Sep 7 14:37:51 2011 (r225439) +++ head/sys/dev/vge/if_vge.c Wed Sep 7 16:57:43 2011 (r225440) @@ -1752,6 +1752,10 @@ vge_intr(void *arg) #ifdef DEVICE_POLLING if (ifp->if_capenable & IFCAP_POLLING) { + status = CSR_READ_4(sc, VGE_ISR); + CSR_WRITE_4(sc, VGE_ISR, status); + if (status != 0xFFFFFFFF && (status & VGE_ISR_LINKSTS) != 0) + vge_link_statchg(sc); VGE_UNLOCK(sc); return; } @@ -2109,11 +2113,10 @@ vge_init_locked(struct vge_softc *sc) #ifdef DEVICE_POLLING /* - * Disable interrupts if we are polling. + * Disable interrupts except link state change if we are polling. */ if (ifp->if_capenable & IFCAP_POLLING) { - CSR_WRITE_4(sc, VGE_IMR, 0); - CSR_WRITE_1(sc, VGE_CRC3, VGE_CR3_INT_GMSK); + CSR_WRITE_4(sc, VGE_IMR, VGE_INTRS_POLLING); } else /* otherwise ... */ #endif { @@ -2121,9 +2124,9 @@ vge_init_locked(struct vge_softc *sc) * Enable interrupts. */ CSR_WRITE_4(sc, VGE_IMR, VGE_INTRS); - CSR_WRITE_4(sc, VGE_ISR, 0xFFFFFFFF); - CSR_WRITE_1(sc, VGE_CRS3, VGE_CR3_INT_GMSK); } + CSR_WRITE_4(sc, VGE_ISR, 0xFFFFFFFF); + CSR_WRITE_1(sc, VGE_CRS3, VGE_CR3_INT_GMSK); sc->vge_flags &= ~VGE_FLAG_LINK; mii_mediachg(mii); @@ -2280,8 +2283,9 @@ vge_ioctl(struct ifnet *ifp, u_long comm return (error); VGE_LOCK(sc); /* Disable interrupts */ - CSR_WRITE_4(sc, VGE_IMR, 0); - CSR_WRITE_1(sc, VGE_CRC3, VGE_CR3_INT_GMSK); + CSR_WRITE_4(sc, VGE_IMR, VGE_INTRS_POLLING); + CSR_WRITE_4(sc, VGE_ISR, 0xFFFFFFFF); + CSR_WRITE_1(sc, VGE_CRS3, VGE_CR3_INT_GMSK); ifp->if_capenable |= IFCAP_POLLING; VGE_UNLOCK(sc); } else { Modified: head/sys/dev/vge/if_vgereg.h ============================================================================== --- head/sys/dev/vge/if_vgereg.h Wed Sep 7 14:37:51 2011 (r225439) +++ head/sys/dev/vge/if_vgereg.h Wed Sep 7 16:57:43 2011 (r225440) @@ -302,6 +302,8 @@ VGE_ISR_LINKSTS|VGE_ISR_RXNODESC| \ VGE_ISR_RXDMA_STALL|VGE_ISR_TXDMA_STALL) +#define VGE_INTRS_POLLING (VGE_ISR_PHYINT|VGE_ISR_LINKSTS) + /* Interrupt mask register */ #define VGE_IMR_RXOK_HIPRIO 0x00000001 /* hi prio RX int */ _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/160442: commit references a PR Date: Thu, 15 Sep 2011 17:20:34 +0000 (UTC) Author: yongari Date: Thu Sep 15 17:20:20 2011 New Revision: 225594 URL: http://svn.freebsd.org/changeset/base/225594 Log: MFC r225440: vge(4) hardwares poll media status and generates an interrupt whenever the link state is changed. Using software based polling for media status tracking is known to cause MII access failure under certain conditions once link is established so vge(4) used to rely on link status change interrupt. However DEVICE_POLLING completely disables generation of all kind of interrupts on vge(4) such that this resulted in not detecting link state change event. This means vge(4) does not correctly detect established/lost link with DEVICE_POLLING. Losing the interrupt made vge(4) not to send any packets to peer since vge(4) does not try to send any packets when there is no established link. Work around the issue by generating link state change interrupt with DEVICE_POLLING. PR: kern/160442 Modified: stable/8/sys/dev/vge/if_vge.c stable/8/sys/dev/vge/if_vgereg.h Directory Properties: stable/8/sys/ (props changed) stable/8/sys/amd64/include/xen/ (props changed) stable/8/sys/cddl/contrib/opensolaris/ (props changed) stable/8/sys/contrib/dev/acpica/ (props changed) stable/8/sys/contrib/pf/ (props changed) Modified: stable/8/sys/dev/vge/if_vge.c ============================================================================== --- stable/8/sys/dev/vge/if_vge.c Thu Sep 15 17:11:03 2011 (r225593) +++ stable/8/sys/dev/vge/if_vge.c Thu Sep 15 17:20:20 2011 (r225594) @@ -1752,6 +1752,10 @@ vge_intr(void *arg) #ifdef DEVICE_POLLING if (ifp->if_capenable & IFCAP_POLLING) { + status = CSR_READ_4(sc, VGE_ISR); + CSR_WRITE_4(sc, VGE_ISR, status); + if (status != 0xFFFFFFFF && (status & VGE_ISR_LINKSTS) != 0) + vge_link_statchg(sc); VGE_UNLOCK(sc); return; } @@ -2109,11 +2113,10 @@ vge_init_locked(struct vge_softc *sc) #ifdef DEVICE_POLLING /* - * Disable interrupts if we are polling. + * Disable interrupts except link state change if we are polling. */ if (ifp->if_capenable & IFCAP_POLLING) { - CSR_WRITE_4(sc, VGE_IMR, 0); - CSR_WRITE_1(sc, VGE_CRC3, VGE_CR3_INT_GMSK); + CSR_WRITE_4(sc, VGE_IMR, VGE_INTRS_POLLING); } else /* otherwise ... */ #endif { @@ -2121,9 +2124,9 @@ vge_init_locked(struct vge_softc *sc) * Enable interrupts. */ CSR_WRITE_4(sc, VGE_IMR, VGE_INTRS); - CSR_WRITE_4(sc, VGE_ISR, 0xFFFFFFFF); - CSR_WRITE_1(sc, VGE_CRS3, VGE_CR3_INT_GMSK); } + CSR_WRITE_4(sc, VGE_ISR, 0xFFFFFFFF); + CSR_WRITE_1(sc, VGE_CRS3, VGE_CR3_INT_GMSK); sc->vge_flags &= ~VGE_FLAG_LINK; mii_mediachg(mii); @@ -2280,8 +2283,9 @@ vge_ioctl(struct ifnet *ifp, u_long comm return (error); VGE_LOCK(sc); /* Disable interrupts */ - CSR_WRITE_4(sc, VGE_IMR, 0); - CSR_WRITE_1(sc, VGE_CRC3, VGE_CR3_INT_GMSK); + CSR_WRITE_4(sc, VGE_IMR, VGE_INTRS_POLLING); + CSR_WRITE_4(sc, VGE_ISR, 0xFFFFFFFF); + CSR_WRITE_1(sc, VGE_CRS3, VGE_CR3_INT_GMSK); ifp->if_capenable |= IFCAP_POLLING; VGE_UNLOCK(sc); } else { Modified: stable/8/sys/dev/vge/if_vgereg.h ============================================================================== --- stable/8/sys/dev/vge/if_vgereg.h Thu Sep 15 17:11:03 2011 (r225593) +++ stable/8/sys/dev/vge/if_vgereg.h Thu Sep 15 17:20:20 2011 (r225594) @@ -302,6 +302,8 @@ VGE_ISR_LINKSTS|VGE_ISR_RXNODESC| \ VGE_ISR_RXDMA_STALL|VGE_ISR_TXDMA_STALL) +#define VGE_INTRS_POLLING (VGE_ISR_PHYINT|VGE_ISR_LINKSTS) + /* Interrupt mask register */ #define VGE_IMR_RXOK_HIPRIO 0x00000001 /* hi prio RX int */ _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/160442: commit references a PR Date: Thu, 15 Sep 2011 17:22:52 +0000 (UTC) Author: yongari Date: Thu Sep 15 17:22:38 2011 New Revision: 225595 URL: http://svn.freebsd.org/changeset/base/225595 Log: MFC r225440: vge(4) hardwares poll media status and generates an interrupt whenever the link state is changed. Using software based polling for media status tracking is known to cause MII access failure under certain conditions once link is established so vge(4) used to rely on link status change interrupt. However DEVICE_POLLING completely disables generation of all kind of interrupts on vge(4) such that this resulted in not detecting link state change event. This means vge(4) does not correctly detect established/lost link with DEVICE_POLLING. Losing the interrupt made vge(4) not to send any packets to peer since vge(4) does not try to send any packets when there is no established link. Work around the issue by generating link state change interrupt with DEVICE_POLLING. PR: kern/160442 Modified: stable/7/sys/dev/vge/if_vge.c stable/7/sys/dev/vge/if_vgereg.h Directory Properties: stable/7/sys/ (props changed) stable/7/sys/cddl/contrib/opensolaris/ (props changed) stable/7/sys/contrib/dev/acpica/ (props changed) stable/7/sys/contrib/pf/ (props changed) Modified: stable/7/sys/dev/vge/if_vge.c ============================================================================== --- stable/7/sys/dev/vge/if_vge.c Thu Sep 15 17:20:20 2011 (r225594) +++ stable/7/sys/dev/vge/if_vge.c Thu Sep 15 17:22:38 2011 (r225595) @@ -1753,6 +1753,10 @@ vge_intr(void *arg) #ifdef DEVICE_POLLING if (ifp->if_capenable & IFCAP_POLLING) { + status = CSR_READ_4(sc, VGE_ISR); + CSR_WRITE_4(sc, VGE_ISR, status); + if (status != 0xFFFFFFFF && (status & VGE_ISR_LINKSTS) != 0) + vge_link_statchg(sc); VGE_UNLOCK(sc); return; } @@ -2110,11 +2114,10 @@ vge_init_locked(struct vge_softc *sc) #ifdef DEVICE_POLLING /* - * Disable interrupts if we are polling. + * Disable interrupts except link state change if we are polling. */ if (ifp->if_capenable & IFCAP_POLLING) { - CSR_WRITE_4(sc, VGE_IMR, 0); - CSR_WRITE_1(sc, VGE_CRC3, VGE_CR3_INT_GMSK); + CSR_WRITE_4(sc, VGE_IMR, VGE_INTRS_POLLING); } else /* otherwise ... */ #endif { @@ -2122,9 +2125,9 @@ vge_init_locked(struct vge_softc *sc) * Enable interrupts. */ CSR_WRITE_4(sc, VGE_IMR, VGE_INTRS); - CSR_WRITE_4(sc, VGE_ISR, 0xFFFFFFFF); - CSR_WRITE_1(sc, VGE_CRS3, VGE_CR3_INT_GMSK); } + CSR_WRITE_4(sc, VGE_ISR, 0xFFFFFFFF); + CSR_WRITE_1(sc, VGE_CRS3, VGE_CR3_INT_GMSK); sc->vge_flags &= ~VGE_FLAG_LINK; mii_mediachg(mii); @@ -2281,8 +2284,9 @@ vge_ioctl(struct ifnet *ifp, u_long comm return (error); VGE_LOCK(sc); /* Disable interrupts */ - CSR_WRITE_4(sc, VGE_IMR, 0); - CSR_WRITE_1(sc, VGE_CRC3, VGE_CR3_INT_GMSK); + CSR_WRITE_4(sc, VGE_IMR, VGE_INTRS_POLLING); + CSR_WRITE_4(sc, VGE_ISR, 0xFFFFFFFF); + CSR_WRITE_1(sc, VGE_CRS3, VGE_CR3_INT_GMSK); ifp->if_capenable |= IFCAP_POLLING; VGE_UNLOCK(sc); } else { Modified: stable/7/sys/dev/vge/if_vgereg.h ============================================================================== --- stable/7/sys/dev/vge/if_vgereg.h Thu Sep 15 17:20:20 2011 (r225594) +++ stable/7/sys/dev/vge/if_vgereg.h Thu Sep 15 17:22:38 2011 (r225595) @@ -302,6 +302,8 @@ VGE_ISR_LINKSTS|VGE_ISR_RXNODESC| \ VGE_ISR_RXDMA_STALL|VGE_ISR_TXDMA_STALL) +#define VGE_INTRS_POLLING (VGE_ISR_PHYINT|VGE_ISR_LINKSTS) + /* Interrupt mask register */ #define VGE_IMR_RXOK_HIPRIO 0x00000001 /* hi prio RX int */ _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" State-Changed-From-To: patched->closed State-Changed-By: yongari State-Changed-When: Thu Sep 15 17:31:53 UTC 2011 State-Changed-Why: MFC to stable/8 and stable/7 done. Thanks a lot! http://www.freebsd.org/cgi/query-pr.cgi?pr=160442 >Unformatted: