From nobody@FreeBSD.org Wed Nov 28 18:25:58 2007 Return-Path: Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB6E916A46C for ; Wed, 28 Nov 2007 18:25:58 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id D35F613C457 for ; Wed, 28 Nov 2007 18:25:58 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.2/8.14.2) with ESMTP id lASIPoES078744 for ; Wed, 28 Nov 2007 18:25:50 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.2/8.14.1/Submit) id lASIPo50078743; Wed, 28 Nov 2007 18:25:50 GMT (envelope-from nobody) Message-Id: <200711281825.lASIPo50078743@www.freebsd.org> Date: Wed, 28 Nov 2007 18:25:50 GMT From: Kostik Belousov To: freebsd-gnats-submit@FreeBSD.org Subject: ath/snd_hda on the same irq line cause solid hang X-Send-Pr-Version: www-3.1 X-GNATS-Notify: >Number: 118316 >Category: kern >Synopsis: [ath] ath/snd_hda on the same irq line cause solid hang >Confidential: no >Severity: serious >Priority: medium >Responsible: sam >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Nov 28 18:30:01 UTC 2007 >Closed-Date: Tue Dec 30 17:45:49 UTC 2008 >Last-Modified: Tue Dec 30 17:45:49 UTC 2008 >Originator: Kostik Belousov >Release: 7.0-BETA3 >Organization: - >Environment: FreeBSD alf 7.0-BETA3 FreeBSD 7.0-BETA3 #46: Tue Nov 27 21:47:09 EET 2007 root@alf:/usr/bsd/obj/usr/bsd/src/sys/ALF i386 >Description: I have a laptop that shares interrupt between ath0 and snd_hda0 devices. Machine worked normally under the RELENG_6. Now, on the RELENG_7 (and CURRENT), attempt to ifconfig ath0 up usualy hangs the machine in the next 2-5 minutes. Hang is hard, I even cannot switch the console to look at the kernel messages or enter the ddb. Also, laptop does not have a serial port or firewire. Not setting the ath0 up, or, unloading snd_hda.ko before activating the interface, allows machine to work. Wireless works without problem, interrupt rate is as expected. hostb0@pci0:0:0:0: class=0x060000 card=0x00901025 chip=0x27a08086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = '955XM/945GM/PM/GMS/940GML Express Processor to DRAM Controller' class = bridge subclass = HOST-PCI pcm0@pci0:0:27:0: class=0x040300 card=0x00901025 chip=0x27d88086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801G (ICH7 Family) High Definition Audio' class = multimedia ath0@pci0:6:2:0: class=0x020000 card=0x04181468 chip=0x001a168c rev=0x01 hdr=0x00 vendor = 'Atheros Communications Inc.' device = 'Atheros AR5005G Atheros AR5005G 802.11abg NIC Chipset / TP-Link (TL-WN551G)' class = network subclass = ethernet wlan: <802.11 Link Layer> wlan: <802.11 MAC ACL support> wlan: mac acl policy registered wlan_amrr: ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) ath_rate: version 1.2 pci0: driver added found-> vendor=0x8086, dev=0x27da, revid=0x02 domain=0, bus=0, slot=31, func=3 class=0c-05-00, hdrtype=0x00, mfdev=0 cmdreg=0x0101, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=19 pci0:0:31:3: reprobing on driver added pci2: driver added pci3: driver added pci4: driver added pci5: driver added pci6: driver added found-> vendor=0x168c, dev=0x001a, revid=0x01 domain=0, bus=6, slot=2, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0002, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x0a (2500 ns), maxlat=0x1c (7000 ns) intpin=a, irq=22 powerspec 2 supports D0 D3 current D0 pci0:6:2:0: reprobing on driver added ath0: mem 0xd0000000-0xd000ffff irq 22 at device 2.0 on pci6 ath0: Reserved 0x10000 bytes for rid 0x10 type 3 at 0xd0000000 ath0: [MPSAFE] ath0: [ITHREAD] ath0: hal channel 2412/a0 -> 1 ath0: hal channel 2412/c0 -> 1 ath0: hal channel 2417/a0 -> 2 ath0: hal channel 2417/c0 -> 2 ath0: hal channel 2422/a0 -> 3 ath0: hal channel 2422/c0 -> 3 ath0: hal channel 2427/a0 -> 4 ath0: hal channel 2427/c0 -> 4 ath0: hal channel 2432/a0 -> 5 ath0: hal channel 2432/c0 -> 5 ath0: hal channel 2437/a0 -> 6 ath0: hal channel 2437/c0 -> 6 ath0: hal channel 2442/a0 -> 7 ath0: hal channel 2442/c0 -> 7 ath0: hal channel 2447/a0 -> 8 ath0: hal channel 2447/c0 -> 8 ath0: hal channel 2452/a0 -> 9 ath0: hal channel 2452/c0 -> 9 ath0: hal channel 2457/a0 -> 10 ath0: hal channel 2457/c0 -> 10 ath0: hal channel 2462/a0 -> 11 ath0: hal channel 2462/c0 -> 11 ath0: hal channel 2467/2a0 -> 12 ath0: hal channel 2467/2c0 -> 12 ath0: hal channel 2472/2a0 -> 13 ath0: hal channel 2472/2c0 -> 13 ath0: using obsoleted if_watchdog interface ath0: bpf attached ath0: Ethernet address: 00:19:7d:74:58:c9 ath0: bpf attached ath0: bpf attached ath0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps ath0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps ath0: mac 7.8 phy 4.5 radio 5.6 ath0: Use hw queue 1 for WME_AC_BE traffic ath0: Use hw queue 0 for WME_AC_BK traffic ath0: Use hw queue 2 for WME_AC_VI traffic ath0: Use hw queue 3 for WME_AC_VO traffic ath0: Use hw queue 8 for CAB traffic ath0: Use hw queue 9 for beacons snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_buffersize=16 384 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 pci0: driver added found-> vendor=0x8086, dev=0x27d8, revid=0x02 domain=0, bus=0, slot=27, func=0 class=04-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0106, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=22 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit pci0:0:27:0: reprobing on driver added pcm0: mem 0xd0240000-0xd0243fff irq 22 at device 27.0 on pci0 pcm0: TCSEL: 0x07 -> 0x00 pcm0: DMA Coherency: Uncacheable / vendor=0x8086 pcm0: Reserved 0x4000 bytes for rid 0x10 type 3 at 0xd0240000 ioapic0: routing intpin 22 (PCI IRQ 22) to vector 57 pcm0: [MPSAFE] pcm0: [ITHREAD] Everything wlan/ath/snd is loaded as modules. alf% uname -a FreeBSD alf 7.0-BETA3 FreeBSD 7.0-BETA3 #46: Tue Nov 27 21:47:09 EET 2007 root@alf:/usr/bsd/obj/usr/bsd/src/sys/ALF i386 Kernel config: # $Id: ALF.kernconf,v 1.7 2007/11/25 11:25:07 root Exp root $ machine i386 cpu I686_CPU device apic options ADAPTIVE_GIANT options PREEMPTION options HWPMC_HOOKS options INTR_FILTER ident ALF makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols options KDB options KDB_TRACE options KDB_UNATTENDED options DDB options DDB_NUMSYM options INET #InterNETworking options INET6 #IPv6 communications protocols options IPSTEALTH #support for stealth forwarding options TCPDEBUG options IPSEC options IPSEC_FILTERTUNNEL options SCTP options ALTQ options ALTQ_CBQ # Class Bases Queueing options ALTQ_RED # Random Early Detection options ALTQ_RIO # RED In/Out options ALTQ_HFSC # Hierarchical Packet Scheduler options ALTQ_CDNR # Traffic conditioner options ALTQ_PRIQ options SCHED_ULE options FFS options SOFTUPDATES options UFS_EXTATTR options UFS_EXTATTR_AUTOSTART options UFS_ACL options UFS_DIRHASH options QUOTA options UFS_GJOURNAL options SUIDDIR options COMPAT_43TTY #Compatible with BSD 4.3 [KEEP THIS!] options COMPAT_FREEBSD4 options COMPAT_FREEBSD5 options COMPAT_FREEBSD6 options SCSI_DELAY=15000 #Delay (in ms) before probing SCSI options KTRACE #ktrace(1) support options _KPOSIX_PRIORITY_SCHEDULING options KBD_INSTALL_CDEV # install a CDEV entry in /dev options MAC options AUDIT device isa device pci device atkbdc device atkbd device psm device vga # syscons is the default console driver, resembling an SCO console device sc options SC_NO_CUTPASTE #options SC_NO_SYSMOUSE options SC_DISABLE_REBOOT device npx device pmtimer device loop device ether device mem device io device pty device crypto device bpf device enc >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: Responsible-Changed-From-To: freebsd-bugs->sam Responsible-Changed-By: kib Responsible-Changed-When: Wed Nov 28 18:32:43 UTC 2007 Responsible-Changed-Why: Reassign to Sam as was discussed on the channel. http://www.freebsd.org/cgi/query-pr.cgi?pr=118316 State-Changed-From-To: open->feedback State-Changed-By: sam State-Changed-When: Thu Nov 29 19:08:04 UTC 2007 State-Changed-Why: please try w/o INTR_FILTER in your kernel config http://www.freebsd.org/cgi/query-pr.cgi?pr=118316 State-Changed-From-To: feedback->open State-Changed-By: kib State-Changed-When: Fri Nov 30 05:10:28 UTC 2007 State-Changed-Why: I removed the INTR_FILTER from the kernel config, as suggested. Machine did survived the usual crash interval (now it is running for ~20 minutes with ath0 being associated, while previously it takes at most 2 minutes to hang). Meantime, I had to reboot my workstation that serves as AP. During the AP downtime, laptop becomes almost totally unusable (I thought it hung again). When AP turned back, laptop restored the normal operation too. The top indicated 99% interrupt time for several seconds while laptop comes to life again. http://www.freebsd.org/cgi/query-pr.cgi?pr=118316 From: Konstantin Belousov To: bug-followup@FreeBSD.org, kib@FreeBSD.org Cc: Subject: Re: kern/118316: [ath] ath/snd_hda on the same irq line cause solid hang Date: Wed, 5 Dec 2007 12:36:36 +0200 It seems I am still able to reproduce the hang, but this might be different issue. As was said, I removed the INTR_FILTER from kernel config. At home, I assocciate the laptop with my workstation that works as AP using the script: ifconfig ath0 inet 192.168.103.111 netmask 255.255.255.0 ssid pooma up sleep 1 /usr/sbin/wpa_supplicant -B -i ath0 -c /usr/local/etc/pooma_supplicant.conf AP does the WPA-PSK. Note the sleep 1. Removing it causes the same solid hang as reported before. Doing ifconfig/running supplicant with small pause between allows to proceed normally. State-Changed-From-To: open->patched State-Changed-By: sam State-Changed-When: Mon Nov 24 17:48:15 UTC 2008 State-Changed-Why: I think we resolved this as caused by ath MIB interrupts firing due to high noise but because the irq was shared and powerd was running they were not serviced quickly enough correct the state of the card. The correct fix is to MFC the latest hal from HEAD but this cannot happen during the 7.1 freeze so set the state to reflect this. We can re-open the PR if the issue is unresolved with the new hal (and related driver mods). http://www.freebsd.org/cgi/query-pr.cgi?pr=118316 State-Changed-From-To: patched->closed State-Changed-By: sam State-Changed-When: Tue Dec 30 17:45:26 UTC 2008 State-Changed-Why: believe this is resolved http://www.freebsd.org/cgi/query-pr.cgi?pr=118316 >Unformatted: