From owner-freebsd-atm Tue Jan 16 7:37: 7 2001 Delivered-To: freebsd-atm@freebsd.org Received: from smtp1.sentex.ca (smtp1.sentex.ca [199.212.134.4]) by hub.freebsd.org (Postfix) with ESMTP id 80D1537B69B for ; Tue, 16 Jan 2001 07:36:43 -0800 (PST) Received: from simoeon.sentex.net (simeon.sentex.ca [209.112.4.47]) by smtp1.sentex.ca (8.11.1/8.11.1) with ESMTP id f0GFagg95785 for ; Tue, 16 Jan 2001 10:36:42 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <5.0.1.4.0.20010116102645.02568220@marble.sentex.ca> X-Sender: mdtpop@marble.sentex.ca X-Mailer: QUALCOMM Windows Eudora Version 5.0.1 Date: Tue, 16 Jan 2001 10:30:27 -0500 To: freebsd-atm@freebsd.org From: Mike Tancsa Subject: ATM under 4.x STABLE ? Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I am about to try CLIP using the Efficient OC-3 card under 4.x. Are there any issues I should be aware of ? Is anyone still using ATM under FreeBSD ? The other end is a Cisco using what I am told, routed 1483 PVCs and I was told I just setup via Classical IP over ATM. The other that I will be connecting to shows interface ATM1/0/0.14 multipoint description - Sentex ATM ip address 192.168.0.1 255.255.255.252 no ip directed-broadcast no ip mroute-cache map-group atm atm pvc 15 0 130 aal5snap 23000 23000 inarp Can the FreeBSD ATM stack connect to this ? Should I be using the hea or en drivers ? Thanks for any tips, ---Mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Tue Jan 16 8:46:31 2001 Delivered-To: freebsd-atm@freebsd.org Received: from mail.matriplex.com (ns1.matriplex.com [208.131.42.8]) by hub.freebsd.org (Postfix) with ESMTP id 38ABF37B402 for ; Tue, 16 Jan 2001 08:46:14 -0800 (PST) Received: from mail.matriplex.com (mail.matriplex.com [208.131.42.9]) by mail.matriplex.com (8.9.2/8.9.2) with ESMTP id IAA00723; Tue, 16 Jan 2001 08:46:11 -0800 (PST) (envelope-from rh@matriplex.com) Date: Tue, 16 Jan 2001 08:46:11 -0800 (PST) From: Richard Hodges To: Mike Tancsa Cc: freebsd-atm@FreeBSD.ORG Subject: Re: ATM under 4.x STABLE ? In-Reply-To: <5.0.1.4.0.20010116102645.02568220@marble.sentex.ca> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 16 Jan 2001, Mike Tancsa wrote: > I am about to try CLIP using the Efficient OC-3 card under 4.x. Are there > any issues I should be aware of ? Well, the mbuf (network buffer) size was changed going from 4.0 to 4.1 and the Fore PCA200 driver broke (patch available). My recommendation is to give the ENI driver a try, and if it also has a problem I or someone else can take a look at it. > Is anyone still using ATM under FreeBSD? You bet! > The other end is a Cisco using what I am told, routed 1483 PVCs and I > was told I just setup via Classical IP over ATM. That sounds fine. > Can the FreeBSD ATM stack connect to this ? I don't have ATM on a Cisco router, but it looks ok to me... > Should I be using the hea or en drivers ? The "en" driver is a lot simpler, since it is intended for one card family (ENI, Adaptec, SMC), and may be more straightforward if you only want to use a small number of PVCs. The HARP stack is really the way to go if you want to use other cards or SVCs. -Richard ------------------------------------------- Richard Hodges | Matriplex, inc. | 769 Basque Way rh@matriplex.com | Carson City, NV 89706 775-886-6477 | www.matriplex.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Tue Jan 16 9: 1:50 2001 Delivered-To: freebsd-atm@freebsd.org Received: from smtp1.sentex.ca (smtp1.sentex.ca [199.212.134.4]) by hub.freebsd.org (Postfix) with ESMTP id 24DC237B400 for <freebsd-atm@FreeBSD.ORG>; Tue, 16 Jan 2001 09:01:33 -0800 (PST) Received: from simoeon.sentex.net (simeon.sentex.ca [209.112.4.47]) by smtp1.sentex.ca (8.11.1/8.11.1) with ESMTP id f0GH1EM22113; Tue, 16 Jan 2001 12:01:14 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <5.0.1.4.0.20010116114713.033bd7c0@marble.sentex.ca> X-Sender: mdtpop@marble.sentex.ca X-Mailer: QUALCOMM Windows Eudora Version 5.0.1 Date: Tue, 16 Jan 2001 11:54:59 -0500 To: Richard Hodges <rh@matriplex.com> From: Mike Tancsa <mike@sentex.net> Subject: Re: ATM under 4.x STABLE ? Cc: freebsd-atm@FreeBSD.ORG In-Reply-To: <Pine.BSF.4.10.10101160835310.676-100000@mail.matriplex.com > References: <5.0.1.4.0.20010116102645.02568220@marble.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Thanks for the quick response! I have the card installed, but it seems to be complaining about broken DMA. This is on an Intel 810e motherboard. I ran the DOS diag util and according to it, all tests including the DMA tests passed. en0: <Efficient Networks ENI-155p> mem 0xfe000000-0xfe3fffff irq 11 at device 8.0 on pci1 en0: unexpected timeout in rx DMA test en0: WARNING: DMA test detects a broken PCI chipset! trying to work around the problem... but if this doesn't work for you, you'd better switch to a newer motherboard. en0: ATM midway v0, board IDs 6.0, Utopia (pipelined), 512KB on-board RAM en0: maximum DMA burst length = 64 bytes (must align) en0: 7 32KB receive buffers, 8 32KB transmit buffers allocated en0: End Station Identifier (mac address) 00:20:ea:0c:1f:ba en0: driver is using old-style compatability shims At 08:46 AM 1/16/01 -0800, Richard Hodges wrote: >On Tue, 16 Jan 2001, Mike Tancsa wrote: > > > I am about to try CLIP using the Efficient OC-3 card under 4.x. Are there > > any issues I should be aware of ? > >Well, the mbuf (network buffer) size was changed going from 4.0 to 4.1 and >the Fore PCA200 driver broke (patch available). Super. I will have a look through the archives for the patch. > > Should I be using the hea or en drivers ? > >The "en" driver is a lot simpler, since it is intended for one card >family (ENI, Adaptec, SMC), and may be more straightforward if you >only want to use a small number of PVCs. The HARP stack is really >the way to go if you want to use other cards or SVCs. This box will only have one function, to connect to my upstream on OC-3. I ordered an IMC SM to MM media convertor that I will try out. The box will only push about 30Mb/s through it as well as do BGP with the upstream peer. Currently I am doing via ethernet on a slower box and it handles the load just fine. Hopefully, the ATM card will do the trick as well. ---Mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Tue Jan 16 9:19: 3 2001 Delivered-To: freebsd-atm@freebsd.org Received: from mail.matriplex.com (ns1.matriplex.com [208.131.42.8]) by hub.freebsd.org (Postfix) with ESMTP id 2951637B401 for <freebsd-atm@FreeBSD.ORG>; Tue, 16 Jan 2001 09:18:46 -0800 (PST) Received: from mail.matriplex.com (mail.matriplex.com [208.131.42.9]) by mail.matriplex.com (8.9.2/8.9.2) with ESMTP id JAA00836; Tue, 16 Jan 2001 09:18:43 -0800 (PST) (envelope-from rh@matriplex.com) Date: Tue, 16 Jan 2001 09:18:43 -0800 (PST) From: Richard Hodges <rh@matriplex.com> To: Mike Tancsa <mike@sentex.net> Cc: freebsd-atm@FreeBSD.ORG Subject: Re: ATM under 4.x STABLE ? In-Reply-To: <5.0.1.4.0.20010116114713.033bd7c0@marble.sentex.ca> Message-ID: <Pine.BSF.4.10.10101160907280.676-100000@mail.matriplex.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 16 Jan 2001, Mike Tancsa wrote: > > Thanks for the quick response! I have the card installed, but it seems to > be complaining about broken DMA. This is on an Intel 810e motherboard. I > ran the DOS diag util and according to it, all tests including the DMA > tests passed. > > en0: <Efficient Networks ENI-155p> mem 0xfe000000-0xfe3fffff irq 11 at > device 8.0 on pci1 > en0: unexpected timeout in rx DMA test > en0: WARNING: DMA test detects a broken PCI chipset! > trying to work around the problem... but if this doesn't > work for you, you'd better switch to a newer motherboard. Ouch! Looking at /usr/src/sys/dev/en/midway.c I see some very interesting notes about broken DMA in the Efficient card (but not Adaptec). The driver is supposed to work around that, though. Your DMA message above seems to refer to the chipset not being able to handle "misaligned 64 byte DMA", which might not be a major problem, but just might reduce the performance. Of course, the real authority is Chuck Cranor himself... -Richard ------------------------------------------- Richard Hodges | Matriplex, inc. <title> | 769 Basque Way rh@matriplex.com | Carson City, NV 89706 775-886-6477 | www.matriplex.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Tue Jan 16 9:55:53 2001 Delivered-To: freebsd-atm@freebsd.org Received: from smtp1.sentex.ca (smtp1.sentex.ca [199.212.134.4]) by hub.freebsd.org (Postfix) with ESMTP id 0017A37B402 for <freebsd-atm@FreeBSD.ORG>; Tue, 16 Jan 2001 09:55:35 -0800 (PST) Received: from simoeon.sentex.net (simeon.sentex.ca [209.112.4.47]) by smtp1.sentex.ca (8.11.1/8.11.1) with ESMTP id f0GHtXM41260; Tue, 16 Jan 2001 12:55:33 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <5.0.1.4.0.20010116121839.033a2a70@marble.sentex.ca> X-Sender: mdtpop@marble.sentex.ca X-Mailer: QUALCOMM Windows Eudora Version 5.0.1 Date: Tue, 16 Jan 2001 12:49:17 -0500 To: Richard Hodges <rh@matriplex.com> From: Mike Tancsa <mike@sentex.net> Subject: Re: ATM under 4.x STABLE (DMA and en driver ?) Cc: freebsd-atm@FreeBSD.ORG In-Reply-To: <Pine.BSF.4.10.10101160907280.676-100000@mail.matriplex.com > References: <5.0.1.4.0.20010116114713.033bd7c0@marble.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 09:18 AM 1/16/01 -0800, Richard Hodges wrote: >On Tue, 16 Jan 2001, Mike Tancsa wrote: > > > > en0: <Efficient Networks ENI-155p> mem 0xfe000000-0xfe3fffff irq 11 at > > device 8.0 on pci1 > > en0: unexpected timeout in rx DMA test > > en0: WARNING: DMA test detects a broken PCI chipset! > > trying to work around the problem... but if this doesn't > > work for you, you'd better switch to a newer motherboard. > >Ouch! Looking at /usr/src/sys/dev/en/midway.c I see some very interesting >notes about broken DMA in the Efficient card (but not Adaptec). The >driver is supposed to work around that, though. Your DMA message above >seems to refer to the chipset not being able to handle "misaligned 64 >byte DMA", which might not be a major problem, but just might reduce >the performance. > >Of course, the real authority is Chuck Cranor himself... Hmmm... Any guesses as to how much of a performance drop ? If its a hardware issue, I wonder if the LINUX crowd has run into this. Time to do some more archive searching! BTW, for my simple protocol requirements, should I try and pick up a couple of used Fore cards instead ? I am hoping the box will be able to forward 30-50Mb/s over its life time. ---Mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Tue Jan 16 10:29:14 2001 Delivered-To: freebsd-atm@freebsd.org Received: from mail.matriplex.com (ns1.matriplex.com [208.131.42.8]) by hub.freebsd.org (Postfix) with ESMTP id 6723637B404 for <freebsd-atm@FreeBSD.ORG>; Tue, 16 Jan 2001 10:28:57 -0800 (PST) Received: from mail.matriplex.com (mail.matriplex.com [208.131.42.9]) by mail.matriplex.com (8.9.2/8.9.2) with ESMTP id KAA00993; Tue, 16 Jan 2001 10:28:53 -0800 (PST) (envelope-from rh@matriplex.com) Date: Tue, 16 Jan 2001 10:28:52 -0800 (PST) From: Richard Hodges <rh@matriplex.com> To: Mike Tancsa <mike@sentex.net> Cc: freebsd-atm@FreeBSD.ORG Subject: Re: ATM under 4.x STABLE (DMA and en driver ?) In-Reply-To: <5.0.1.4.0.20010116121839.033a2a70@marble.sentex.ca> Message-ID: <Pine.BSF.4.10.10101161004200.676-100000@mail.matriplex.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 16 Jan 2001, Mike Tancsa wrote: > At 09:18 AM 1/16/01 -0800, Richard Hodges wrote: > >On Tue, 16 Jan 2001, Mike Tancsa wrote: > > > > > > en0: <Efficient Networks ENI-155p> mem 0xfe000000-0xfe3fffff irq 11 at > > > device 8.0 on pci1 > > > en0: unexpected timeout in rx DMA test > > > en0: WARNING: DMA test detects a broken PCI chipset! > > > trying to work around the problem... but if this doesn't > > > work for you, you'd better switch to a newer motherboard. > > Your DMA message above seems to refer to the chipset not being able > > to handle "misaligned 64 byte DMA", which might not be a major > > problem, but just might reduce the performance. > Hmmm... Any guesses as to how much of a performance drop ? Gueses? Sure :-) I think this is referring to the longest burst of data that the card can push through the PCI bus at a time. Since you need one PCI cycle for setup, your max PCI throughput would be roughly 133MB/sec / (4 + burstlen). Obviously, 64-byte bursts will be more productive than 4-byte bursts (50% wasted) for example. > BTW, for my simple protocol requirements, should I try and pick up a couple > of used Fore cards instead ? I am hoping the box will be able to forward > 30-50Mb/s over its life time. That should be no problem at all. I rarely see better than 80mb/sec on an FTP file transfer, but for raw throughput you should be able to reach 120+ Unless the Efficient card is *really* broken, I assume you should see comparable results. -Richard ------------------------------------------- Richard Hodges | Matriplex, inc. <title> | 769 Basque Way rh@matriplex.com | Carson City, NV 89706 775-886-6477 | www.matriplex.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Tue Jan 16 17:55:17 2001 Delivered-To: freebsd-atm@freebsd.org Received: from mailhub.fokus.gmd.de (mailhub.fokus.gmd.de [193.174.154.14]) by hub.freebsd.org (Postfix) with ESMTP id 2102237B404 for <freebsd-atm@FreeBSD.ORG>; Tue, 16 Jan 2001 17:54:55 -0800 (PST) Received: from beagle (beagle [193.175.132.100]) by mailhub.fokus.gmd.de (8.8.8/8.8.8) with ESMTP id CAA27048; Wed, 17 Jan 2001 02:54:47 +0100 (MET) Date: Wed, 17 Jan 2001 02:54:47 +0100 (CET) From: Harti Brandt <brandt@fokus.gmd.de> To: Mike Tancsa <mike@sentex.net> Cc: Richard Hodges <rh@matriplex.com>, freebsd-atm@FreeBSD.ORG Subject: Re: ATM under 4.x STABLE (DMA and en driver ?) In-Reply-To: <5.0.1.4.0.20010116121839.033a2a70@marble.sentex.ca> Message-ID: <Pine.BSF.4.21.0101170250590.4794-100000@beagle.fokus.gmd.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 16 Jan 2001, Mike Tancsa wrote: > BTW, for my simple protocol requirements, should I try and pick up a couple > of used Fore cards instead ? I am hoping the box will be able to forward > 30-50Mb/s over its life time. I have just finished a driver for the FORE PCA200E which uses the same NATM stack as the en driver. On a 900MHz Athlon it can send 135MBit/sec with plain AAL5 (this is near to the theoretical maximum) and around 15kbyte/sec over TCP. The TCP sending code seems to be the limiting factor in this case. And well, its -current. Drop me a note if you want to give a try... harti -- harti brandt, http://www.fokus.gmd.de/research/cc/cats/employees/hartmut.brandt/private brandt@fokus.gmd.de, harti@begemot.org, lhbrandt@mail.ru To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Wed Jan 17 12:12:50 2001 Delivered-To: freebsd-atm@freebsd.org Received: from smtp1.sentex.ca (smtp1.sentex.ca [199.212.134.4]) by hub.freebsd.org (Postfix) with ESMTP id D718C37B698 for <freebsd-atm@freebsd.org>; Wed, 17 Jan 2001 12:12:31 -0800 (PST) Received: from simoeon.sentex.net (simeon.sentex.ca [209.112.4.47]) by smtp1.sentex.ca (8.11.1/8.11.1) with ESMTP id f0HKCQW28058 for <freebsd-atm@freebsd.org>; Wed, 17 Jan 2001 15:12:26 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <5.0.1.4.0.20010117134944.01f3b890@marble.sentex.ca> X-Sender: mdtpop@marble.sentex.ca X-Mailer: QUALCOMM Windows Eudora Version 5.0.1 Date: Wed, 17 Jan 2001 15:06:03 -0500 To: freebsd-atm@freebsd.org From: Mike Tancsa <mike@sentex.net> Subject: en performance/configuration.. WOW! Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Well, so far so good. I setup two en0 cards back to back and was able to configure a simple link. I tried using netperf to measure throughput and again, so far so good! On simple tcp stream tests, I was seeing /usr/local/netperf/netperf -l 60 -H 192.168.1.2 -t TCP_STREAM -i 10,2 -I 99,5 -- -m 4096 -s 57344 -S 57344 TCP STREAM TEST to 192.168.1.2 : +/-2.5% @ 99% conf. : histogram Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 57344 57344 4096 60.01 134.58 This is on a Intel 810e PIII 800e with 512MB of RAM. The box is going to be a border router running bgp. Now, the task is to do the conversion. On the other end, my upstream has interface ATM1/0/0.14 multipoint description - Sentex ATM ip address 192.168.1.1 255.255.255.252 no ip directed-broadcast no ip mroute-cache map-group atm atm pvc 15 0 130 aal5snap 35000 35000 inarp And the local Cisco 4700 has Interface ATM0 description Sentex ATM ip address 192.168.1.2 255.255.255.252 atm pvc 1 0 130 aal5snap 35000 35000 inarp map-group atm . . map-list ip 192.168.1.2 atm-vc 1 ip 192.168.1.1 atm-vc 1 I need to replace the Cisco 4700 with the FreeBSD box (I have a SM to MM m/c to do the fibre conversion). However, being an ATM novice, I am not sure how to translate the above config into a FreeBSD config. On the Cisco, I have atm pvc vcd vpi 130 rates rates ... and then inarp... If they are using inverse arp, how does this affect my configuration. Do I need to use the harp stack instead ? I am not sure how the inarp figures into this. On the two machines I have here, I did as the man pages suggested to bring the back to back config up, and that worked... But I am not sure if the above config only effects arp timeouts or works with an arp server somehow ? Any suggestions / tips would be much appreciated! ---Mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Wed Jan 17 13:57:36 2001 Delivered-To: freebsd-atm@freebsd.org Received: from cc762335-a.ebnsk1.nj.home.com (cc762335-a.ebnsk1.nj.home.com [24.3.219.36]) by hub.freebsd.org (Postfix) with SMTP id C2D6537B712 for <freebsd-atm@FreeBSD.ORG>; Wed, 17 Jan 2001 13:57:17 -0800 (PST) Received: (qmail 73879 invoked from network); 17 Jan 2001 21:56:06 -0000 Received: from athena.faerun.com (HELO athena) (192.168.0.2) by cc762335-a.ebnsk1.nj.home.com with SMTP; 17 Jan 2001 21:56:06 -0000 Message-Id: <4.2.2.20010117164809.00c78ec0@netmail.home.com> X-Sender: damascus@netmail.home.com X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.2 Date: Wed, 17 Jan 2001 16:57:07 -0500 To: Mike Tancsa <mike@sentex.net> From: Carroll Kong <damascus@home.com> Subject: Re: en performance/configuration.. WOW! Cc: freebsd-atm@FreeBSD.ORG In-Reply-To: <5.0.1.4.0.20010117134944.01f3b890@marble.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >On the other end, my upstream has > >interface ATM1/0/0.14 multipoint > description - Sentex ATM > ip address 192.168.1.1 255.255.255.252 > no ip directed-broadcast > no ip mroute-cache > map-group atm > atm pvc 15 0 130 aal5snap 35000 35000 inarp > >And the local Cisco 4700 has > >Interface ATM0 > description Sentex ATM > ip address 192.168.1.2 255.255.255.252 > atm pvc 1 0 130 aal5snap 35000 35000 inarp > map-group atm >. >. >map-list > ip 192.168.1.2 atm-vc 1 > ip 192.168.1.1 atm-vc 1 > > >I need to replace the Cisco 4700 with the FreeBSD box (I have a SM to MM >m/c to do the fibre conversion). However, being an ATM novice, I am not >sure how to translate the above config into a FreeBSD config. On the >Cisco, I have >atm pvc vcd vpi 130 rates rates ... and then inarp... If they are using >inverse arp, how does this affect my configuration. Do I need to use the >harp stack instead ? I am not sure how the inarp figures into this. >On the two machines I have here, I did as the man pages suggested to bring >the back to back config up, and that worked... But I am not sure if the >above config only effects arp timeouts or works with an arp server somehow ? > >Any suggestions / tips would be much appreciated! > > > ---Mike I am somewhat an amateur in ATM as well but I know a few things and I have worked on some Cisco boxes. Looks like the Cisco 4700 is calling a maplist, on VP 0, VC 130. Using aal5snap, with some projected bitrate. Inverse ARP seems like an automated way to map IP addresses to PVCs. See, normally in ethernet that is done for us with ARP since ethernet has a MAC address that maps back to an IP. With ATM, there is no such animal (short of other methods, but for simplicity, I will not mention them). So ATM needs a way to map back, and it is usually done with some static map listing IPs -> PVCs or in Cisco's case, Inverse ARP protocol. You only need an ARP Server if you are using SVCs. (IIRC... and/or LANE). I think the Inverse ARP Protocol helps alleviate people who normally have to do some kind of static mapping between IPs and PVCs. Could you send me privately your Cisco 4700's config? Please omit any logins or enable passwds sequences. Just curious on your overall setup and I might be able to figure some more out. (so omit enable lines, and passwd lines on your console or other lines). I am fairly certain that syntax is deprecated and replaced with newer lines if you have a more recent version of Cisco IOS. As to how to configure the FreeBSD box to emulate this. I am not quite sure. I suppose you can statically define the IP to PVC mapping. And ask the other end to do so as well since it will no longer have Inverse ARP protocol to rely on. But I get a feeling they already have done the static map, and Inverse Arp is being redundant now. -Carroll Kong To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Wed Jan 17 16:52:48 2001 Delivered-To: freebsd-atm@freebsd.org Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.30.102]) by hub.freebsd.org (Postfix) with ESMTP id 5C6DB37B401 for <freebsd-atm@FreeBSD.ORG>; Wed, 17 Jan 2001 16:52:31 -0800 (PST) Received: from alliance.research.att.com (alliance.research.att.com [135.207.26.26]) by mail-blue.research.att.com (Postfix) with ESMTP id AE2114CE7A; Wed, 17 Jan 2001 19:52:26 -0500 (EST) Received: from chips-ha.research.att.com (chips-ha.research.att.com [135.207.27.139]) by alliance.research.att.com (8.8.7/8.8.7) with ESMTP id TAA02849; Wed, 17 Jan 2001 19:52:18 -0500 (EST) Received: (from chuck@localhost) by chips-ha.research.att.com (SGI-8.9.3/8.8.5) id TAA62786; Wed, 17 Jan 2001 19:52:17 -0500 (EST) Date: Wed, 17 Jan 2001 19:52:17 -0500 From: Chuck Cranor <chuck@research.att.com> To: Richard Hodges <rh@matriplex.com> Cc: Mike Tancsa <mike@sentex.net>, freebsd-atm@FreeBSD.ORG Subject: Re: ATM under 4.x STABLE ? Message-ID: <20010117195217.A2037890@chips.research.att.com> References: <5.0.1.4.0.20010116114713.033bd7c0@marble.sentex.ca> <Pine.BSF.4.10.10101160907280.676-100000@mail.matriplex.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <Pine.BSF.4.10.10101160907280.676-100000@mail.matriplex.com>; from rh@matriplex.com on Tue, Jan 16, 2001 at 09:18:43AM -0800 Organization: AT&T Labs-Research Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, Jan 16, 2001 at 09:18:43AM -0800, Richard Hodges wrote: > On Tue, 16 Jan 2001, Mike Tancsa wrote: > > en0: <Efficient Networks ENI-155p> mem 0xfe000000-0xfe3fffff irq 11 at > > device 8.0 on pci1 > > en0: unexpected timeout in rx DMA test > > en0: WARNING: DMA test detects a broken PCI chipset! > > trying to work around the problem... but if this doesn't > > work for you, you'd better switch to a newer motherboard. en0: maximum DMA burst length = 64 bytes (must align) > Ouch! Looking at /usr/src/sys/dev/en/midway.c I see some very interesting > notes about broken DMA in the Efficient card (but not Adaptec). The > driver is supposed to work around that, though. Your DMA message above > seems to refer to the chipset not being able to handle "misaligned 64 > byte DMA", which might not be a major problem, but just might reduce > the performance. > Of course, the real authority is Chuck Cranor himself... hi- Adaptec totally redid the DMA interface to this card and it works great. the Efficient version is more painful to program and gets some things wrong. the card init code does some extensive testing to find the best mode that will work for the card. Mike's system supports large DMA bursts (64 byte), but the starting address of the buffer must be aligned to a 64 byte boundary (that's why it says "(must align)" and printed the warning). i don't think this will be a major problem (if you've got a large DMA, you can break it up into chunks to get the needed 64 byte alignment... the driver will do this for you). chuck -- Chuck Cranor <chuck@research.att.com> Senior Technical Staff Member, AT&T Labs-Research Room B135, 180 Park Ave, Florham Park NJ, 07932 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Wed Jan 17 16:57: 4 2001 Delivered-To: freebsd-atm@freebsd.org Received: from cage.simianscience.com (cage.simianscience.com [64.7.134.1]) by hub.freebsd.org (Postfix) with ESMTP id B01DA37B400 for <freebsd-atm@freebsd.org>; Wed, 17 Jan 2001 16:56:45 -0800 (PST) Received: (from root@localhost) by cage.simianscience.com (8.11.1/8.11.1) id f0I0uiS37387; Wed, 17 Jan 2001 19:56:44 -0500 (EST) (envelope-from mike@sentex.net) Received: from chimp (fcage [192.168.0.2]) by cage.simianscience.com (8.11.1/8.11.1av) with ESMTP id f0I0udd37379; Wed, 17 Jan 2001 19:56:39 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <4.2.2.20010117194902.01edcf68@marble.sentex.net> X-Sender: mdtancsa@marble.sentex.net X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.2 Date: Wed, 17 Jan 2001 19:56:38 -0500 To: Carroll Kong <damascus@home.com> From: Mike Tancsa <mike@sentex.net> Subject: Re: en performance/configuration.. WOW! Cc: freebsd-atm@freebsd.org In-Reply-To: <4.2.2.20010117164809.00c78ec0@netmail.home.com> References: <5.0.1.4.0.20010117134944.01f3b890@marble.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 04:57 PM 1/17/2001 -0500, Carroll Kong wrote: >I am somewhat an amateur in ATM as well but I know a few things and I have >worked on some Cisco boxes. Looks like the Cisco 4700 is calling a >maplist, on VP 0, VC 130. Using aal5snap, with some projected >bitrate. Inverse ARP seems like an automated way to map IP addresses to >PVCs. See, normally in ethernet that is done for us with ARP since >ethernet has a MAC address that maps back to an IP. With ATM, there is no >such animal (short of other methods, but for simplicity, I will not >mention them). So ATM needs a way to map back, and it is usually done >with some static map listing IPs -> PVCs or in Cisco's case, Inverse ARP >protocol. Thanks for the explanation! What I dont understand is the purpose of the map-list. Does this not take care of the 'arp' issue ? i.e telling the two devices that these IP address are associated with these PVCs ? >You only need an ARP Server if you are using SVCs. (IIRC... and/or >LANE). I think the Inverse ARP Protocol helps alleviate people who >normally have to do some kind of static mapping between IPs and PVCs. > >Could you send me privately your Cisco 4700's config? I would if I could. The 4700 that sits here on prem, is on the other side of the demark point. I interface to it through FastE and do my BGP session with the 4700. I believe that my upstream redistributes everything via iBGP to the 4700. >As to how to configure the FreeBSD box to emulate this. I am not quite >sure. I suppose you can statically define the IP to PVC mapping. And ask >the other end to do so as well since it will no longer have Inverse ARP >protocol to rely on. But I get a feeling they already have done the >static map, and Inverse Arp is being redundant now. In my little test setup, I got everything working just using what was in the MAN pages. It was quite simple ifconfig en0 192.168.1.1 netmask 0xffffff00 up route add -iface 192.168.1.2 -link en0:3.0.0.c9 ifconfig en0 192.168.1.2 netmask 0xffffff00 up route add -iface 192.168.1.1 -link en0:3.0.0.c9 And that was that. -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Network Administration, mike@sentex.net Sentex Communications www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Wed Jan 17 17: 4:21 2001 Delivered-To: freebsd-atm@freebsd.org Received: from cage.simianscience.com (cage.simianscience.com [64.7.134.1]) by hub.freebsd.org (Postfix) with ESMTP id CA74937B69F for <freebsd-atm@freebsd.org>; Wed, 17 Jan 2001 17:04:03 -0800 (PST) Received: (from root@localhost) by cage.simianscience.com (8.11.1/8.11.1) id f0I142g37422; Wed, 17 Jan 2001 20:04:02 -0500 (EST) (envelope-from mike@sentex.net) Received: from chimp (fcage [192.168.0.2]) by cage.simianscience.com (8.11.1/8.11.1av) with ESMTP id f0I13vd37414; Wed, 17 Jan 2001 20:03:57 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <4.2.2.20010117195924.01bde6f8@marble.sentex.net> X-Sender: mdtancsa@marble.sentex.net X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.2 Date: Wed, 17 Jan 2001 20:03:56 -0500 To: Chuck Cranor <chuck@research.att.com> From: Mike Tancsa <mike@sentex.net> Subject: Re: ATM under 4.x STABLE ? Cc: freebsd-atm@freebsd.org In-Reply-To: <20010117195217.A2037890@chips.research.att.com> References: <Pine.BSF.4.10.10101160907280.676-100000@mail.matriplex.com> <5.0.1.4.0.20010116114713.033bd7c0@marble.sentex.ca> <Pine.BSF.4.10.10101160907280.676-100000@mail.matriplex.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 07:52 PM 1/17/2001 -0500, Chuck Cranor wrote: > Mike's system supports large DMA bursts (64 byte), but the starting >address of the buffer must be aligned to a 64 byte boundary (that's >why it says "(must align)" and printed the warning). i don't think >this will be a major problem (if you've got a large DMA, you can break >it up into chunks to get the needed 64 byte alignment... the driver >will do this for you). Thanks for the follow up/clarification. From my initial netperf tests, I am quite happy with the throughput of the card. The box only needs to push 30Mb/s, and it seems it can handle that in tests no problem. Not sure how it will do in the real world, but I figure its worth a try since if this works, there are a few other applications I can use this type of setup for! ---Mike ---Mike -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Network Administration, mike@sentex.net Sentex Communications www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Thu Jan 18 14:13:46 2001 Delivered-To: freebsd-atm@freebsd.org Received: from smtp1.sentex.ca (smtp1.sentex.ca [199.212.134.4]) by hub.freebsd.org (Postfix) with ESMTP id 4A9AD37B401 for <freebsd-atm@freebsd.org>; Thu, 18 Jan 2001 14:13:28 -0800 (PST) Received: from simoeon.sentex.net (simeon.sentex.ca [209.112.4.47]) by smtp1.sentex.ca (8.11.1/8.11.1) with ESMTP id f0IMDR152994 for <freebsd-atm@freebsd.org>; Thu, 18 Jan 2001 17:13:27 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <5.0.1.4.0.20010118170127.02188060@marble.sentex.ca> X-Sender: mdtpop@marble.sentex.ca X-Mailer: QUALCOMM Windows Eudora Version 5.0.1 Date: Thu, 18 Jan 2001 17:06:58 -0500 To: freebsd-atm@freebsd.org From: Mike Tancsa <mike@sentex.net> Subject: ATM problems with HARP Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I have been playing with the HARP stack and run into a small problem. When running netperf, I seem to loose the PVC on my hea cards. hespler2# uname -a FreeBSD hespler2.sentex.ca 4.2-STABLE FreeBSD 4.2-STABLE #0: Thu Jan 18 07:45:03 EST 2001 mdtancsa@hespler2.sentex.ca:/usr/src/sys/compile/hesp.harp i386 hespler2# ./snapshot_script 192.168.1.2 > /tmp/snap.out Netperf snapshot script started at Thu Jan 18 16:32:12 EST 2001 Starting 56x4 TCP_STREAM tests at Thu Jan 18 16:32:45 EST 2001 Starting 32x4 TCP_STREAM tests at Thu Jan 18 16:35:45 EST 2001 Starting 1,1 TCP_RR tests at Thu Jan 18 16:38:45 EST 2001 Starting 1,1 UDP_RR tests at Thu Jan 18 16:41:51 EST 2001 Starting 512,4 UDP_RR tests at Thu Jan 18 16:44:57 EST 2001 Starting 32x4 UDP_STREAM tests at Thu Jan 18 16:48:03 EST 2001 Starting 32x1 UDP_STREAM tests at Thu Jan 18 16:50:03 EST 2001 establish_control: control socket connect failed: Operation timed out Are you sure there is a netserver running on 192.168.1.2 at port 12865? Tests completed at Thu Jan 18 16:51:18 EST 2001 hespler2# atm show stats vcc Input Input Input Output Output Output Interface VPI VCI PDUs Bytes Errs PDUs Bytes Errs hea0 0 130 3308599 144044200 0 3692025 -1804270564 0 hespler2# atm show vcc Interface VPI VCI AAL Type Dir State Encaps Owner hea0 0 130 AAL5 PVC InOut ACTIVE LLC/SNAP IP hespler2# atm show ipvcc Net Intf VPI VCI State Flags IP Address atm0 0 130 ACTIVE PLM ruby2atm (192.168.1.2) hespler2# atm show config Intf Vendor Model Media Bus Serial No hea0 ENI ENI-155p OC-3c PCI 794554 MAC address = 00:20:ea:0c:1f:ba Hardware version = 0/4/0 Firmware version = hespler2# hespler2# ifconfig atm0 atm0: flags=43<UP,BROADCAST,RUNNING> mtu 9180 inet 192.168.1.1 netmask 0xfffffffc broadcast 192.168.1.3 ether 00:20:ea:0c:1f:ba I try and delete the PVC and recreate it, but no dice. It seems only a reboot takes care of the issue. The config is pretty simple, atm_enable="YES" # Configure ATM interfaces (or NO). atm_netif_hea0="atm 1" # Network interfaces for physical interface. atm_sigmgr_hea0="sigpvc" # Signalling manager for physical interface. atm_pvcs="test" # Set to PVC list (or leave empty). atm_pvc_test="hea0 0 130 aal5 snap ip atm0 192.168.1.2" # Any thoughts on what it might be ? ---Mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Thu Jan 18 16:59: 1 2001 Delivered-To: freebsd-atm@freebsd.org Received: from cage.simianscience.com (cage.simianscience.com [64.7.134.1]) by hub.freebsd.org (Postfix) with ESMTP id 455C637B6A2 for <freebsd-atm@freebsd.org>; Thu, 18 Jan 2001 16:58:43 -0800 (PST) Received: (from root@localhost) by cage.simianscience.com (8.11.1/8.11.1) id f0J0wgt39583 for freebsd-atm@freebsd.org; Thu, 18 Jan 2001 19:58:42 -0500 (EST) (envelope-from mike@sentex.net) Received: from chimp (fcage [192.168.0.2]) by cage.simianscience.com (8.11.1/8.11.1av) with ESMTP id f0J0wbd39575 for <freebsd-atm@FreeBSD.ORG>; Thu, 18 Jan 2001 19:58:37 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <4.2.2.20010118194243.0387b7e8@marble.sentex.net> X-Sender: mdtancsa@marble.sentex.net X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.2 Date: Thu, 18 Jan 2001 19:58:36 -0500 To: freebsd-atm@freebsd.org From: Mike Tancsa <mike@sentex.net> Subject: HARP (lockups) vs NATM (Efficient OC-3) In-Reply-To: <5.0.1.4.0.20010118170127.02188060@marble.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org OK, I ran the same tests using the Chuck Cranor driver and the tests completed as expected. Any ideas how I would go debugging this ? ---Mike At 05:06 PM 1/18/2001 -0500, Mike Tancsa wrote: >I have been playing with the HARP stack and run into a small >problem. When running netperf, I seem to loose the PVC on my hea cards. > >hespler2# uname -a >FreeBSD hespler2.sentex.ca 4.2-STABLE FreeBSD 4.2-STABLE #0: Thu Jan 18 >07:45:03 EST >2001 mdtancsa@hespler2.sentex.ca:/usr/src/sys/compile/hesp.harp i386 >hespler2# > > >./snapshot_script 192.168.1.2 > /tmp/snap.out >Netperf snapshot script started at Thu Jan 18 16:32:12 EST 2001 >Starting 56x4 TCP_STREAM tests at Thu Jan 18 16:32:45 EST 2001 >Starting 32x4 TCP_STREAM tests at Thu Jan 18 16:35:45 EST 2001 >Starting 1,1 TCP_RR tests at Thu Jan 18 16:38:45 EST 2001 >Starting 1,1 UDP_RR tests at Thu Jan 18 16:41:51 EST 2001 >Starting 512,4 UDP_RR tests at Thu Jan 18 16:44:57 EST 2001 >Starting 32x4 UDP_STREAM tests at Thu Jan 18 16:48:03 EST 2001 >Starting 32x1 UDP_STREAM tests at Thu Jan 18 16:50:03 EST 2001 >establish_control: control socket connect failed: Operation timed out >Are you sure there is a netserver running on 192.168.1.2 at port 12865? >Tests completed at Thu Jan 18 16:51:18 EST 2001 -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Network Administration, mike@sentex.net Sentex Communications www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Thu Jan 18 17:13:51 2001 Delivered-To: freebsd-atm@freebsd.org Received: from mail.matriplex.com (ns1.matriplex.com [208.131.42.8]) by hub.freebsd.org (Postfix) with ESMTP id DC57037B6A6 for <freebsd-atm@FreeBSD.ORG>; Thu, 18 Jan 2001 17:13:33 -0800 (PST) Received: from mail.matriplex.com (mail.matriplex.com [208.131.42.9]) by mail.matriplex.com (8.9.2/8.9.2) with ESMTP id RAA07917; Thu, 18 Jan 2001 17:13:13 -0800 (PST) (envelope-from rh@matriplex.com) Date: Thu, 18 Jan 2001 17:13:13 -0800 (PST) From: Richard Hodges <rh@matriplex.com> To: Mike Tancsa <mike@sentex.net> Cc: freebsd-atm@FreeBSD.ORG Subject: Re: HARP (lockups) vs NATM (Efficient OC-3) In-Reply-To: <4.2.2.20010118194243.0387b7e8@marble.sentex.net> Message-ID: <Pine.BSF.4.10.10101181707270.2085-100000@mail.matriplex.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 18 Jan 2001, Mike Tancsa wrote: > OK, I ran the same tests using the Chuck Cranor driver and the tests > completed as expected. Any ideas how I would go debugging this ? You could compile the driver with "DO_LOG" and see if anything interesting shows up in the system log. I don't see any obvious problem with private data vs. mbuf header (like in the Fore driver), but I still wonder if it is possible that this driver also needs to be updated. Is this problem associated with receiving data? Are you getting UDP checksum errors? ("netstat -p udp") Sorry, no good ideas pop to mind... -Richard > At 05:06 PM 1/18/2001 -0500, Mike Tancsa wrote: > > >I have been playing with the HARP stack and run into a small > >problem. When running netperf, I seem to loose the PVC on my hea cards. > > > >hespler2# uname -a > >FreeBSD hespler2.sentex.ca 4.2-STABLE FreeBSD 4.2-STABLE #0: Thu Jan 18 > >07:45:03 EST > >2001 mdtancsa@hespler2.sentex.ca:/usr/src/sys/compile/hesp.harp i386 > >hespler2# > > > > > >./snapshot_script 192.168.1.2 > /tmp/snap.out > >Netperf snapshot script started at Thu Jan 18 16:32:12 EST 2001 > >Starting 56x4 TCP_STREAM tests at Thu Jan 18 16:32:45 EST 2001 > >Starting 32x4 TCP_STREAM tests at Thu Jan 18 16:35:45 EST 2001 > >Starting 1,1 TCP_RR tests at Thu Jan 18 16:38:45 EST 2001 > >Starting 1,1 UDP_RR tests at Thu Jan 18 16:41:51 EST 2001 > >Starting 512,4 UDP_RR tests at Thu Jan 18 16:44:57 EST 2001 > >Starting 32x4 UDP_STREAM tests at Thu Jan 18 16:48:03 EST 2001 > >Starting 32x1 UDP_STREAM tests at Thu Jan 18 16:50:03 EST 2001 > >establish_control: control socket connect failed: Operation timed out > >Are you sure there is a netserver running on 192.168.1.2 at port 12865? > >Tests completed at Thu Jan 18 16:51:18 EST 2001 ------------------------------------------- Richard Hodges | Matriplex, inc. <title> | 769 Basque Way rh@matriplex.com | Carson City, NV 89706 775-886-6477 | www.matriplex.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Thu Jan 18 20:36: 2 2001 Delivered-To: freebsd-atm@freebsd.org Received: from cage.simianscience.com (cage.simianscience.com [64.7.134.1]) by hub.freebsd.org (Postfix) with ESMTP id 2070D37B402 for <freebsd-atm@freebsd.org>; Thu, 18 Jan 2001 20:35:42 -0800 (PST) Received: (from root@localhost) by cage.simianscience.com (8.11.1/8.11.1) id f0J4Zf939952; Thu, 18 Jan 2001 23:35:41 -0500 (EST) (envelope-from mike@sentex.net) Received: from chimp (fcage [192.168.0.2]) by cage.simianscience.com (8.11.1/8.11.1av) with ESMTP id f0J4ZYd39944; Thu, 18 Jan 2001 23:35:35 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <4.2.2.20010118205333.01e8dcd8@marble.sentex.net> X-Sender: mdtancsa@marble.sentex.net X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.2 Date: Thu, 18 Jan 2001 23:35:33 -0500 To: Richard Hodges <rh@matriplex.com> From: Mike Tancsa <mike@sentex.net> Subject: Re: HARP (lockups) with hea Cc: freebsd-atm@freebsd.org In-Reply-To: <Pine.BSF.4.10.10101181707270.2085-100000@mail.matriplex.co m> References: <4.2.2.20010118194243.0387b7e8@marble.sentex.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 05:13 PM 1/18/2001 -0800, Richard Hodges wrote: >On Thu, 18 Jan 2001, Mike Tancsa wrote: > > > OK, I ran the same tests using the Chuck Cranor driver and the tests > > completed as expected. Any ideas how I would go debugging this ? > >You could compile the driver with "DO_LOG" and see if anything interesting >shows up in the system log. Sounds good. I will try that. Another thing I noticed was that doing ping -q -s 8000 -f <other side> on the HARP stack, gives me a max of 20Mb/s. Using the en, I get close to 90Mb/s. Quite a difference in throughput. >I don't see any obvious problem with private data vs. mbuf header >(like in the Fore driver), but I still wonder if it is possible >that this driver also needs to be updated. Is this problem associated >with receiving data? Are you getting UDP checksum errors? >("netstat -p udp") Hmm. I didnt check that, but a quick retest of it now shows with the en (after the netperf test and the big long ping flood) hespler2# ping -q -f -s 8000 192.168.1.2 PING 192.168.1.2 (192.168.1.2): 8000 data bytes ^C --- 192.168.1.2 ping statistics --- 4409936 packets transmitted, 4409561 packets received, 0% packet loss round-trip min/avg/max/stddev = 2.305/5.083/7.002/0.409 ms and netstat -s looks just fine. Also, doing the same tests with one machine using HARP and the other using the en driver, they also complete without any errors. going from HARP to en0, also work. Jan 18 22:08:11 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:08:11 ruby2 last message repeated 501 times Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:08:11 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:08:11 ruby2 last message repeated 112 times Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 last message repeated 23 times Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 /kernel: eni_output: not enoug>Jan 18 22:08:11 ruby2 /kernel: eni_outputm in bu ffer Jan 18 22:08:11 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:08:11 ruby2 last message repeated 60 times Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 last message repeated 22 times Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 /kernel: room in buffer Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 last message repeated 22 times Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 /kernel: room in buffer Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:08:11 ruby2 /kernel: Jan 18 22:08:11 ruby2 /kernel: eni_output: not enough room in buffer ruby2# netstat -p udp udp: 1652231 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 24 dropped due to no socket 0 broadcast/multicast datagrams dropped due to no socket 0 dropped due to full socket buffers 0 not for hashed pcb 1652207 delivered 13057700 datagrams output Doesnt really show too much. But kern.* is full of Jan 18 22:29:10 ruby2 last message repeated 23 times Jan 18 22:29:10 ruby2 /kernel: eni_outt: not enough room in buffer Jan 18 22:29:10 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:29:10 ruby2 last message repeated 23 times Jan 18 22:29:10 ruby2 /kernel: t: not enough room in buffer Jan 18 22:29:10 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:29:10 ruby2 last message repeated 23 times Jan 18 22:29:10 ruby2 /kernel: eni_outt: not enough room in buffer Jan 18 22:29:10 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:29:10 ruby2 last message repeated 23 times Jan 18 22:29:10 ruby2 /kernel: t: not enough room in buffer Jan 18 22:29:10 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:29:10 ruby2 last message repeated 23 times Jan 18 22:29:10 ruby2 /kernel: eni_outt: not enough room in buffer Jan 18 22:29:10 ruby2 /kernel: eni_output: not enough room in buffer Jan 18 22:29:10 ruby2 last message repeated 797 times Jan 18 22:32:35 ruby2 /kernel: eni_output: not enough room in buffer There is nothing on the other end of the test, even with debugging enabled. ruby2# atm show stats vcc Input Input Input Output Output Output Interface VPI VCI PDUs Bytes Errs PDUs Bytes Errs hea0 0 130 3675302 161664468 0 4421957 -94882376 0 Is there any other way to reset the adaptor short of a reboot ? I tried delete the pvc on both ends, downing the interface, but nothing short of a reboot seems to work. Syslog shows atm0: not multicast capable, IPv6 not enabled eni_output: not enough room in buffer eni_output: not enough room in buffer atm0: not multicast capable, IPv6 not enabled eni_output: not enough room in buffer eni_output: not enough room in buffer eni_output: not enough room in buffer Reading through the archives, Chuck Cranor pointed out that - the HARP driver does not test the DMA engine at boot time to verify proper operation. - the HARP driver does not dynamically determine what DMA burst sizes are valid on a specific system and does not handle the strict DMA alignment required on some systems (e.g. the "alburst" restriction common on sparcs). Instead the HARP driver hardwares the max DMA burst to 8 words and doesn't make use of 16 word DMA bursts. I guess I will need to stick with the en driver for now. Thanks for any suggestions and tips! ---Mike -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Network Administration, mike@sentex.net Sentex Communications www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Fri Jan 19 9:13:14 2001 Delivered-To: freebsd-atm@freebsd.org Received: from marcos.networkcs.com (marcos.networkcs.com [137.66.16.1]) by hub.freebsd.org (Postfix) with ESMTP id 1B00537B6A6 for <freebsd-atm@FreeBSD.ORG>; Fri, 19 Jan 2001 09:12:53 -0800 (PST) Received: from us.networkcs.com (us.networkcs.com [137.66.11.15]) by marcos.networkcs.com (8.9.3/8.9.3) with ESMTP id LAA71346; Fri, 19 Jan 2001 11:12:52 -0600 (CST) (envelope-from jpt@us.networkcs.com) Received: (from jpt@localhost) by us.networkcs.com (8.9.2/8.9.2) id LAA08022; Fri, 19 Jan 2001 11:12:51 -0600 (CST) (envelope-from jpt) From: Joseph Thomas <jpt@networkcs.com> Message-Id: <200101191712.LAA08022@us.networkcs.com> Subject: Re: HARP (lockups) with hea To: mike@sentex.net (Mike Tancsa) Date: Fri, 19 Jan 2001 11:12:51 -0600 (CST) Cc: rh@matriplex.com (Richard Hodges), freebsd-atm@FreeBSD.ORG In-Reply-To: <4.2.2.20010118205333.01e8dcd8@marble.sentex.net> from "Mike Tancsa" at Jan 18, 2001 11:35:33 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Couple of things: - First, a great big THANKS to all those that have stepped up and done some real detective work on HARP (especially with regards to the MBUF issue) and those that have offered encouraging advice and support! - Second, an appology for us (the developers) for being as responsive as we'd like/should be. For some background, there were four of us on the original project which was developed under DARPA sponsorship. Two of those are no longer here, the two of us that stayed just completed/are still on sabbatical, and there is no funding or support (from DARPA, others, or current emplyers) to continue this work. In addition, the equipment used (switches, adapters, fiber, etc) is government owned and no more funding, we get into some sticky issues dealing with appropriate use, etc. Without some sponsorship/approval to work on HARP, I suspect the best we can offer is looking at code and offering suggestions/design decisions without actually being able to test anything and given all our current job duties, I can't say that we'll be timely in our responses. History tends to support that we won't be... - HARP and Chuck's code were never trying to solve the same problem. Chuck and done some excellent work, has provided valuable advice and insight to us, and has even modified his IDT driver to support the HARP stack. There will be numerous differences and as seen, some of very signifcant magnitude. If Chuck's code works better for an application, please use it! Onto below: The ENI device uses buffer space on the adapter for sending and receiving PDUs. Alignment, as most driver developers well know, is always a big issue. To begin with, the ENI memory must be divided up to provide receive space for any open VCs and the adapter places limits on what size these must be and how they are aligned. Within the buffers, the PDUs themselves need alginmet -- you can't simply start the next PDU where the last one ended. In short, you never can have 100% memory utilization by real data. This differs dignifcantly from the FORE device where all PDU data is stored in system memory, not adapter memory. What's happening below is that HARP does not use any throttle mechanism or secondary queueing for transmitting PDUs. When a PDU comes down through the stack, if there are no buffers for transmit, the PDU (mbuf chain) is discarded. I believe that Chuck actually maintains a seperate queue to store these on and will attempt to drain it when current tranmit operations complete. Because UDP packets can be generated faster than TCP, and netperf defaults to UDP (at least it used to), yes, you will see really bad netperf numbers. Rolled into this, and I never examined Chuck's code to see how susceptable his stack is, is the classic LFN -- large fat network, problem. FreeBSD has never been optimized out of the box (at least upto where we were still developing and testing) for large fast networks. The send and receive space for TCP and UDP used to be 8K. [BTW - this is not a FreeBSD problem, it exists in all network implementations.] In testing various systems, I used ttcp and varied PDU and socket (send/receive) space. Best performance was obviously with larger packets but seemed like the knee for performance seemed to be around a 60-80KB send/receive space. I have measured ttcp throughputs of 120-125Kb/s on an OC3c interface. [As an aside, one of the early systems we worked with was the SGI Challenge/Onyx line with multiple CPUs, etc. We wanted to move data at OC12 rates before OC12 adapters were available by using multiple OC3c adapters. In my tests on the SGI and with early FreeBSD (2.X, very early 3.X line), no matter how many adapters I had running, even on seperate I/O buses when in the SGI, the systems all seemed to peak at the same point. Adding adapters dropped the per adapter performance at roughly a 1/n'th share, but the overall throughput didn't vary. It also appeared that this limitation was more on the receiving side vs. the sending side and appeared to be in the kernel code vs. HARP code as I could reproduce that same results with 100Mbit enet. It would be interesting to redo these tests today since alot of good things have happened since then...] Bottom line -- HARP is throwing away one PDU for each "not enough room" message whereas I believe Chuck's code will queue and attempt to send. > > At 05:13 PM 1/18/2001 -0800, Richard Hodges wrote: > >On Thu, 18 Jan 2001, Mike Tancsa wrote: > > > > > OK, I ran the same tests using the Chuck Cranor driver and the tests > > > completed as expected. Any ideas how I would go debugging this ? > > > >You could compile the driver with "DO_LOG" and see if anything interesting > >shows up in the system log. > [snip] > Doesnt really show too much. > But kern.* is full of > > Jan 18 22:29:10 ruby2 last message repeated 23 times > Jan 18 22:29:10 ruby2 /kernel: eni_outt: not enough room in buffer > Jan 18 22:29:10 ruby2 /kernel: eni_output: not enough room in buffer > Jan 18 22:29:10 ruby2 last message repeated 23 times > Jan 18 22:29:10 ruby2 /kernel: t: not enough room in buffer > Jan 18 22:29:10 ruby2 /kernel: eni_output: not enough room in buffer > Jan 18 22:29:10 ruby2 last message repeated 23 times > Jan 18 22:29:10 ruby2 /kernel: eni_outt: not enough room in buffer > Jan 18 22:29:10 ruby2 /kernel: eni_output: not enough room in buffer > Jan 18 22:29:10 ruby2 last message repeated 23 times > Jan 18 22:29:10 ruby2 /kernel: t: not enough room in buffer > Jan 18 22:29:10 ruby2 /kernel: eni_output: not enough room in buffer > Jan 18 22:29:10 ruby2 last message repeated 23 times > Jan 18 22:29:10 ruby2 /kernel: eni_outt: not enough room in buffer > Jan 18 22:29:10 ruby2 /kernel: eni_output: not enough room in buffer > Jan 18 22:29:10 ruby2 last message repeated 797 times > Jan 18 22:32:35 ruby2 /kernel: eni_output: not enough room in buffer > [snip] Yes, HARP does not attempt to do anything with determinging DMA burst sizes supported. We found numerous early systems which wouldn't even support anything over 1WORD DMA. Fortunately, most of these problems have long since been fixed but the original HARP was defaulted to only use 1WORD DMA sizes. Our READMEs talked about the problem and suggested that a knowledgeable user could change the DMA size and find the break point where things work/don't work. Because we are in effect a commercial development organization, we had lot's of issues to deal with as far use of other code/works and distribution issues. Being limited in time and money, we couldn't go off and develop/redevelop all the things that we would have liked HARP to include. ENI provided us with code which determines supported DMA sizes as well as Chuck's code, but for the above reasons, we couldn't use anything we had access to and couldn't justify the time/effort to develop/document a clean version of this. If someone wanted to do this, yes, it'd be a great addition. As another aside, this same process is what prevented us from distributing a microcode object file for the FORE adapters even though FORE more or less gave us permission to do so. I'm sure there are very good legal reasons for doing so and have great respect and trust for our legal staff here. As a developer, especially one developing in the free (or atleast open) source arena, yes, it causes some headaches that I'd rather just "do it" than ask about it. Oh well... > - the HARP driver does not test the DMA engine at boot time to verify > proper operation. > - the HARP driver does not dynamically determine what DMA burst > sizes are valid on a specific system and does not handle the > strict DMA alignment required on some systems (e.g. the > "alburst" restriction common on sparcs). Instead the HARP > driver hardwares the max DMA burst to 8 words and doesn't make > use of 16 word DMA bursts. > Somethings to try: - use Chuck's code; - look at adding DMA_USE16WORD support to eni_transmit.c. One would of course need to ensure that 16WORD DMA works on your machine; - check to see if netperf can insert a inter-packet delay. Causing a small delay between when packets are sent might allow the driver to free enough resources to reduce the number of drops; - add a delayed transmit queue which could hold packets which would otherwise have been dropped. Check it when sending before looking for new additions to the interface xmit queue. -- Joseph Thomas E/Mail: jpt@networkcs.com Network Computing Services, Inc. jpt@magic.net 1200 Washington Ave So. Tel: +1 612 337 3558 Minneapolis, MN 55415-1227 FAX: +1 612 337 3400 An elephant is a mouse with an operating system. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Fri Jan 19 18:32:30 2001 Delivered-To: freebsd-atm@freebsd.org Received: from workhorse.iMach.com (workhorse.iMach.com [206.127.77.89]) by hub.freebsd.org (Postfix) with ESMTP id 6CBC937B401 for <freebsd-atm@freebsd.org>; Fri, 19 Jan 2001 18:32:13 -0800 (PST) Received: from localhost (forrestc@localhost) by workhorse.iMach.com (8.9.3/8.9.3) with ESMTP id TAA11434 for <freebsd-atm@freebsd.org>; Fri, 19 Jan 2001 19:32:33 -0700 (MST) Date: Fri, 19 Jan 2001 19:32:33 -0700 (MST) From: "Forrest W. Christian" <forrestc@imach.com> To: freebsd-atm@freebsd.org Subject: Two OC3's, 100mb/s ethernet and BGP Message-ID: <Pine.BSF.4.21.0101191925090.11347-100000@workhorse.iMach.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I am considering using FreeBSD as a router for connectivity to my two upstream providers. These will most likely be coming in on an ATM OC3 or similar. I was wondering if anyone on the list has real experience with running multiple ATM interfaces in a box, perhaps with BGP routing for an application like this. I would like to find out about what type of stability and throughput I can expect. (I am already convinced of the stability of FreeBSD. It just makes me nervous to use it as a core routing device for some irrational reason (maybe cisco brainwashing)) - Forrest W. Christian (forrestc@imach.com) AC7DE ---------------------------------------------------------------------- iMach, Ltd., P.O. Box 5749, Helena, MT 59604 http://www.imach.com Solutions for your high-tech problems. (406)-442-6648 ---------------------------------------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message From owner-freebsd-atm Fri Jan 19 18:48:52 2001 Delivered-To: freebsd-atm@freebsd.org Received: from cage.simianscience.com (cage.simianscience.com [64.7.134.1]) by hub.freebsd.org (Postfix) with ESMTP id 1E99837B404 for <freebsd-atm@freebsd.org>; Fri, 19 Jan 2001 18:48:34 -0800 (PST) Received: (from root@localhost) by cage.simianscience.com (8.11.1/8.11.1) id f0K2mXt41932; Fri, 19 Jan 2001 21:48:33 -0500 (EST) (envelope-from mike@sentex.net) Received: from chimp (fcage [192.168.0.2]) by cage.simianscience.com (8.11.1/8.11.1av) with ESMTP id f0K2mSd41924; Fri, 19 Jan 2001 21:48:28 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <4.2.2.20010119214107.019dac30@marble.sentex.net> X-Sender: mdtancsa@marble.sentex.net X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.2 Date: Fri, 19 Jan 2001 21:48:27 -0500 To: Joseph Thomas <jpt@networkcs.com> From: Mike Tancsa <mike@sentex.net> Subject: Re: HARP (lockups) with hea Cc: freebsd-atm@freebsd.org In-Reply-To: <200101191712.LAA08022@us.networkcs.com> References: <4.2.2.20010118205333.01e8dcd8@marble.sentex.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-atm@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org At 11:12 AM 1/19/2001 -0600, Joseph Thomas wrote: >Couple of things: First off, thanks very much for the great background and suggestions. It is very much appreciated! [snip] > Yes, HARP does not attempt to do anything with determinging DMA Playing around a little more with it, it seems the problem is only when sending a lot of data. I installed LINUX on the other box and did the same tests with it against the other FreeBSD box and no lockups on the LINUX box. Only the FreeBSD box when sending. I would have thought the lockup would be in either direction. Strange. > - use Chuck's code; I am going to give that a try for sure. The other strange thing I ran into, is that I cant make the en driver talk to the LINUX box. With HARP its no problem. Anyone out there using the en driver to speak to LINUX ATM 0.78 ? The box's purpose is pretty simple: to peer with my upstream and run gated for now, zebra in perhaps a month. If C.C's en driver works, I will run with it see what happens. With FreeBSD to FreeBSD, I tried a range of network stress tests and didnt see a single problem, and got great pps. The current 4700 is down to very little RAM remaining and will have to go sooner than later :-( ---Mike -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Network Administration, mike@sentex.net Sentex Communications www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-atm" in the body of the message