From nobody@FreeBSD.org Tue Sep 21 21:35:31 2010 Return-Path: Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCF011065674 for ; Tue, 21 Sep 2010 21:35:30 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id C02AB8FC1A for ; Tue, 21 Sep 2010 21:35:30 +0000 (UTC) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o8LLZU6G027242 for ; Tue, 21 Sep 2010 21:35:30 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.3/8.14.3/Submit) id o8LLZUGu027241; Tue, 21 Sep 2010 21:35:30 GMT (envelope-from nobody) Message-Id: <201009212135.o8LLZUGu027241@www.freebsd.org> Date: Tue, 21 Sep 2010 21:35:30 GMT From: Andrey Voitenkov To: freebsd-gnats-submit@FreeBSD.org Subject: ipfw2 fwd rule matches packets but does not do the job in fact. X-Send-Pr-Version: www-3.1 X-GNATS-Notify: >Number: 150798 >Category: kern >Synopsis: [ipfw] ipfw2 fwd rule matches packets but does not do the job in fact. >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-ipfw >State: closed >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Sep 21 21:40:02 UTC 2010 >Closed-Date: Wed Jul 06 06:59:37 UTC 2011 >Last-Modified: Wed Jul 6 07:00:34 UTC 2011 >Originator: Andrey Voitenkov >Release: tag=RELENG_8_1 date=2010.08.08.00.00.00 >Organization: >Environment: FreeBSD thin.XXX.ua 8.1-RELEASE FreeBSD 8.1-RELEASE #1: Tue Aug 24 18:08:01 EEST 2010 root@thin.XXX.ua:/usr/obj/usr/src/sys/THIN amd64 >Description: Faced a strange issue - fwd rule matches packets but does not do the job in fact. The test host has 2 active interfaces: fxp0 with 10.0.1.115/24 and em0 with 10.0.0.66/24. Default route is set to 10.0.1.1. There is also router 10.0.0.1 available via em0. The ruleset: # ipfw -d show 00050 0 0 allow ip from any to any via lo0 00055 0 0 allow ip6 from any to any via lo0 00200 4 220 skipto 700 log logamount 100 tag 1 ip from any to any dst-ip 10.0.0.66 in 00210 8 440 skipto 700 log logamount 100 tag 1 ip from any to any src-ip 10.0.0.66 out 00300 0 0 allow icmp from any to any 00600 19 1540 allow tcp from any to me dst-port 22 in 00610 17 1932 allow tcp from me 22 to any out 00700 0 0 check-state 00710 5 240 deny log logamount 100 tcp from any to any established 00720 2 120 skipto 2000 log logamount 100 ip from any to any tagged 1 00800 0 0 allow tcp from me to any out setup keep-state 00810 0 0 allow udp from me to any out keep-state 00820 0 0 allow tcp from any to 10.0.1.115 dst-port 80 in setup keep-state 00830 0 0 allow tcp from any to 10.0.1.115 dst-port 21 in setup keep-state 01000 11 528 skipto 65000 log logamount 100 ip from any to any 02000 0 0 skipto 60000 log logamount 100 tcp from me to any out setup keep-state 02010 0 0 skipto 60000 log logamount 100 udp from me to any out keep-state 02020 0 0 skipto 60000 log logamount 100 icmp from any to any keep-state 02030 4 240 skipto 60000 log logamount 100 tcp from any to me dst-port 25,465,587 in setup keep-state 02040 3 180 skipto 60000 log logamount 100 tcp from any to me dst-port 143,993 in setup keep-state 03000 0 0 skipto 65000 log logamount 100 ip from any to any 60000 7 420 count ip from any to any 60010 3 180 allow log logamount 100 ip from any to any in tagged 1 60020 4 240 fwd 10.0.0.1 log logamount 100 ip from 10.0.0.66 to any out 65000 11 528 deny log logamount 100 ip from any to any 65001 0 0 deny ip6 from any to any 65535 348792682 269452781083 allow ip from any to any ## Dynamic rules (2): 02030 3 180 (298s) STATE tcp 192.168.0.86 39598 <-> 10.0.0.66 465 02040 2 120 (300s) STATE tcp 91.215.8.2 55670 <-> 10.0.0.66 143 ----------------------------------------------------------------- Trying to connect to 10.0.0.66 port 465 (exim is up and running) from host 192.168.0.86, ipfw.log looks exactly as I expected: Sep 21 23:46:31 thin kernel: ipfw: 200 SkipTo 700 TCP 192.168.0.86:39598 10.0.0.66:465 in via em0 Sep 21 23:46:31 thin kernel: ipfw: 720 SkipTo 2000 TCP 192.168.0.86:39598 10.0.0.66:465 in via em0 Sep 21 23:46:31 thin kernel: ipfw: 2030 SkipTo 60000 TCP 192.168.0.86:39598 10.0.0.66:465 in via em0 Sep 21 23:46:31 thin kernel: ipfw: 60010 Accept TCP 192.168.0.86:39598 10.0.0.66:465 in via em0 Sep 21 23:46:31 thin kernel: ipfw: 210 SkipTo 700 TCP 10.0.0.66:465 192.168.0.86:39598 out via fxp0 Sep 21 23:46:31 thin kernel: ipfw: 2030 SkipTo 60000 TCP 10.0.0.66:465 192.168.0.86:39598 out via fxp0 Sep 21 23:46:31 thin kernel: ipfw: 60020 Forward to 10.0.0.1 TCP 10.0.0.66:465 192.168.0.86:39598 out via fxp0 Sep 21 23:46:34 thin kernel: ipfw: 200 SkipTo 700 TCP 192.168.0.86:39598 10.0.0.66:465 in via em0 Sep 21 23:46:34 thin kernel: ipfw: 2030 SkipTo 60000 TCP 192.168.0.86:39598 10.0.0.66:465 in via em0 Sep 21 23:46:34 thin kernel: ipfw: 60010 Accept TCP 192.168.0.86:39598 10.0.0.66:465 in via em0 Sep 21 23:46:34 thin kernel: ipfw: 210 SkipTo 700 TCP 10.0.0.66:465 192.168.0.86:39598 out via fxp0 Sep 21 23:46:34 thin kernel: ipfw: 2030 SkipTo 60000 TCP 10.0.0.66:465 192.168.0.86:39598 out via fxp0 Sep 21 23:46:34 thin kernel: ipfw: 60020 Forward to 10.0.0.1 TCP 10.0.0.66:465 192.168.0.86:39598 out via fxp0 Sep 21 23:46:37 thin kernel: ipfw: 200 SkipTo 700 TCP 192.168.0.86:39598 10.0.0.66:465 in via em0 Sep 21 23:46:37 thin kernel: ipfw: 2030 SkipTo 60000 TCP 192.168.0.86:39598 10.0.0.66:465 in via em0 Sep 21 23:46:37 thin kernel: ipfw: 60010 Accept TCP 192.168.0.86:39598 10.0.0.66:465 in via em0 Sep 21 23:46:37 thin kernel: ipfw: 210 SkipTo 700 TCP 10.0.0.66:465 192.168.0.86:39598 out via fxp0 Sep 21 23:46:37 thin kernel: ipfw: 2030 SkipTo 60000 TCP 10.0.0.66:465 192.168.0.86:39598 out via fxp0 Sep 21 23:46:37 thin kernel: ipfw: 60020 Forward to 10.0.0.1 TCP 10.0.0.66:465 192.168.0.86:39598 out via fxp0 ----------------------------------------------------------------- But client does not get response from the test host. In spite of the fact the the rule 60020 matches the ack's, I still see them on fxp0 going out to the default gw: # tcpdump -n -i em0 src 192.168.0.86 or dst 192.168.0.86 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on em0, link-type EN10MB (Ethernet), capture size 96 bytes 23:46:31.188517 IP 192.168.0.86.39598 > 10.0.0.66.465: Flags [S], seq 579129109, win 65535, options [mss 1460,nop,wscale 3,sackOK,TS val 2443437844 ecr 0], length 0 23:46:34.188457 IP 192.168.0.86.39598 > 10.0.0.66.465: Flags [S], seq 579129109, win 65535, options [mss 1460,nop,wscale 3,sackOK,TS val 2443440844 ecr 0], length 0 23:46:37.388808 IP 192.168.0.86.39598 > 10.0.0.66.465: Flags [S], seq 579129109, win 65535, options [mss 1460,nop,wscale 3,sackOK,TS val 2443444044 ecr 0], length 0 # tcpdump -n -i fxp0 src 192.168.0.86 or dst 192.168.0.86 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on fxp0, link-type EN10MB (Ethernet), capture size 96 bytes 23:46:31.188609 IP 10.0.0.66.465 > 192.168.0.86.39598: Flags [S.], seq 1231665894, ack 579129110, win 65535, options [mss 1460,nop,wscale 3,sackOK,TS val 3020614121 ecr 2443437844], length 0 23:46:34.188526 IP 10.0.0.66.465 > 192.168.0.86.39598: Flags [S.], seq 1231665894, ack 579129110, win 65535, options [mss 1460,nop,wscale 3,sackOK,TS val 3020614121 ecr 2443440844], length 0 23:46:37.188384 IP 10.0.0.66.465 > 192.168.0.86.39598: Flags [S.], seq 1231665894, ack 579129110, win 65535, options [mss 1460,nop,wscale 3,sackOK,TS val 3020614121 ecr 2443440844], length 0 ----------------------------------------------------------------- Tried a couple of variations of rule 60020: fwd 10.0.0.1 log logamount 100 src-ip 10.0.0.66 out fwd 10.0.0.1 log logamount 100 tagged 1 out result is exactly the same. At the same time this 60020 rule works fine for outgoing connections from the test host: % telnet -s 10.0.0.66 192.168.0.86 25 Trying 192.168.0.86... Connected to 192.168.0.86. Escape character is '^]'. ^C Outgoing packets are matched and forwarded as expected. ----------------------------------------------------------------- fwd works ok with a very simple ruleset in the same test case: 00050 490 363540 allow ip from any to any via lo0 00055 0 0 allow ip6 from any to any via lo0 00100 596 240883 fwd 10.0.0.1 ip from 10.0.0.66 to any out xmit fxp0 65535 348794572 269453265568 allow ip from any to any >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: Responsible-Changed-From-To: freebsd-bugs->freebsd-ipfw Responsible-Changed-By: linimon Responsible-Changed-When: Thu Sep 23 05:17:52 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=150798 From: "Andrey V. Elsukov" To: bug-followup@FreeBSD.org, av@holymail.biz Cc: Subject: Re: kern/150798: [ipfw] ipfw2 fwd rule matches packets but does not do the job in fact. Date: Mon, 30 May 2011 15:37:16 +0400 Hi, It seems your problem is the same as described in kern/147720. Can you test the following patch? http://people.freebsd.org/~ae/ipfw_fwd.diff -- WBR, Andrey V. Elsukov From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/150798: commit references a PR Date: Wed, 1 Jun 2011 19:45:03 +0000 (UTC) Author: ae Date: Wed Jun 1 19:44:52 2011 New Revision: 222582 URL: http://svn.freebsd.org/changeset/base/222582 Log: O_FORWARD_IP is only action which depends from the result of lookup of dynamic rules. We are doing forwarding in the following cases: o For the simple ipfw fwd rule, e.g. fwd 10.0.0.1 ip from any to any out xmit em0 fwd 127.0.0.1,3128 tcp from any to any 80 in recv em1 o For the dynamic fwd rule, e.g. fwd 192.168.0.1 tcp from any to 10.0.0.3 3333 setup keep-state When this rule triggers it creates a dynamic rule, but this dynamic rule should forward packets only in forward direction. o And the last case that does not work before - simple fwd rule which triggers when some dynamic rule is already executed. PR: kern/147720, kern/150798 MFC after: 1 month Modified: head/sys/netinet/ipfw/ip_fw2.c Modified: head/sys/netinet/ipfw/ip_fw2.c ============================================================================== --- head/sys/netinet/ipfw/ip_fw2.c Wed Jun 1 18:42:44 2011 (r222581) +++ head/sys/netinet/ipfw/ip_fw2.c Wed Jun 1 19:44:52 2011 (r222582) @@ -2118,7 +2118,8 @@ do { \ case O_FORWARD_IP: if (args->eh) /* not valid on layer2 pkts */ break; - if (!q || dyn_dir == MATCH_FORWARD) { + if (q == NULL || q->rule != f || + dyn_dir == MATCH_FORWARD) { struct sockaddr_in *sa; sa = &(((ipfw_insn_sa *)cmd)->sa); if (sa->sin_addr.s_addr == INADDR_ANY) { _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" State-Changed-From-To: open->patched State-Changed-By: ae State-Changed-When: Mon Jun 6 07:17:50 UTC 2011 State-Changed-Why: Patched in head/. http://www.freebsd.org/cgi/query-pr.cgi?pr=150798 From: ten To: bug-followup@FreeBSD.org, av@holymail.biz Cc: Subject: Re: kern/150798: [ipfw] ipfw2 fwd rule matches packets but does not do the job in fact. Date: Mon, 6 Jun 2011 21:35:14 +0700 --000e0cd22f68f6d1b604a50c02bc Content-Type: text/plain; charset=ISO-8859-1 It seems I have too old version, and patch not applicable to me. 7.3-STABLE FreeBSD 7.3-STABLE #2 amd64 __FBSDID("$FreeBSD: src/sys/netinet/ip_fw_nat.c,v 1.2.2.2 2008/06/23 14:15:53 mav Exp $"); --000e0cd22f68f6d1b604a50c02bc Content-Type: text/html; charset=KOI8-R Content-Transfer-Encoding: base64 PHNwYW4gaWQ9InJlc3VsdF9ib3giIGNsYXNzPSIiIGxhbmc9ImVuIj48c3BhbiB0aXRsZT0i7sHW zcnUxSwg3tTPwtkg1dfJxMXU2CDBzNjUxdLOwdTJ187ZyiDQxdLF18/EIiBjbGFzcz0iaHBzIj5J dCBzZWVtczwvc3Bhbj4gPHNwYW4gdGl0bGU9Iu7B1s3J1MUsIN7Uz8LZINXXycTF1NggwczY1MXS zsHUydfO2cog0MXSxdfPxCIgY2xhc3M9ImhwcyI+SSBoYXZlPC9zcGFuPiA8c3BhbiB0aXRsZT0i 7sHWzcnUxSwg3tTPwtkg1dfJxMXU2CDBzNjUxdLOwdTJ187ZyiDQxdLF18/EIiBjbGFzcz0iaHBz Ij50b288L3NwYW4+IDxzcGFuIHRpdGxlPSLuwdbNydTFLCDe1M/C2SDV18nExdTYIMHM2NTF0s7B 1MnXztnKINDF0sXXz8QiIGNsYXNzPSJocHMiPm9sZDwvc3Bhbj4gPHNwYW4gdGl0bGU9Iu7B1s3J 1MUsIN7Uz8LZINXXycTF1NggwczY1MXSzsHUydfO2cog0MXSxdfPxCIgY2xhc3M9ImhwcyI+dmVy c2lvbjwvc3Bhbj48c3BhbiBjbGFzcz0iIiB0aXRsZT0i7sHWzcnUxSwg3tTPwtkg1dfJxMXU2CDB zNjUxdLOwdTJ187ZyiDQxdLF18/EIj4sPC9zcGFuPiBhbmQgPHNwYW4gdGl0bGU9Iu7B1s3J1MUs IN7Uz8LZINXXycTF1NggwczY1MXSzsHUydfO2cog0MXSxdfPxCIgY2xhc3M9ImhwcyI+cGF0Y2gg PC9zcGFuPjwvc3Bhbj48c3BhbiBpZD0icmVzdWx0X2JveCIgY2xhc3M9InNob3J0X3RleHQiIGxh bmc9ImVuIj48c3BhbiB0aXRsZT0i7sHWzcnUxSwg3tTPwtkg1dfJxMXU2CDBzNjUxdLOwdTJ187Z yiDQxdLF18/EIiBjbGFzcz0iaHBzIj5ub3Q8L3NwYW4+IDxzcGFuIHRpdGxlPSLuwdbNydTFLCDe 1M/C2SDV18nExdTYIMHM2NTF0s7B1MnXztnKINDF0sXXz8QiIGNsYXNzPSJocHMiPmFwcGxpY2Fi bGU8L3NwYW4+PC9zcGFuPjxzcGFuIGlkPSJyZXN1bHRfYm94IiBjbGFzcz0iIiBsYW5nPSJlbiI+ PHNwYW4gdGl0bGU9Iu7B1s3J1MUsIN7Uz8LZINXXycTF1NggwczY1MXSzsHUydfO2cog0MXSxdfP xCIgY2xhc3M9ImhwcyI+IHRvIG1lPC9zcGFuPjxzcGFuIHRpdGxlPSLuwdbNydTFLCDe1M/C2SDV 18nExdTYIMHM2NTF0s7B1MnXztnKINDF0sXXz8QiIGNsYXNzPSJocHMiPi48YnI+CjcuMy1TVEFC TEUgRnJlZUJTRCA3LjMtU1RBQkxFICMyIGFtZDY0PGJyPjwvc3Bhbj48L3NwYW4+X19GQlNESUQo JnF1b3Q7JEZyZWVCU0Q6IHNyYy9zeXMvbmV0aW5ldC9pcF9md19uYXQuYyx2IDEuMi4yLjIgMjAw OC8wNi8yMyAxNDoxNTo1MyBtYXYgRXhwICQmcXVvdDspOzxicj4K --000e0cd22f68f6d1b604a50c02bc-- From: Andrey Voitenkov To: "Andrey V. Elsukov" Cc: bug-followup@freebsd.org Subject: Re: kern/150798: [ipfw] ipfw2 fwd rule matches packets but does not do the job in fact. Date: Thu, 9 Jun 2011 12:32:56 +0300 Hi I don't have access to that host where the problem occured. I've tested it on i386 8.2-release host with a very similar ruleset - looks like the patch fixes the problem. Thanks. 2011/5/30 Andrey V. Elsukov : > Hi, > > It seems your problem is the same as described in kern/147720. > Can you test the following patch? > http://people.freebsd.org/~ae/ipfw_fwd.diff > > -- > WBR, Andrey V. Elsukov > -- Andrey Voitenkov State-Changed-From-To: patched->closed State-Changed-By: ae State-Changed-When: Wed Jul 6 06:59:20 UTC 2011 State-Changed-Why: Merged to stable/7 and stable/8. Thanks! http://www.freebsd.org/cgi/query-pr.cgi?pr=150798 From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/150798: commit references a PR Date: Wed, 6 Jul 2011 06:56:41 +0000 (UTC) Author: ae Date: Wed Jul 6 06:56:31 2011 New Revision: 223819 URL: http://svn.freebsd.org/changeset/base/223819 Log: MFC r222582: O_FORWARD_IP is only action which depends from the result of lookup of dynamic rules. We are doing forwarding in the following cases: o For the simple ipfw fwd rule, e.g. fwd 10.0.0.1 ip from any to any out xmit em0 fwd 127.0.0.1,3128 tcp from any to any 80 in recv em1 o For the dynamic fwd rule, e.g. fwd 192.168.0.1 tcp from any to 10.0.0.3 3333 setup keep-state When this rule triggers it creates a dynamic rule, but this dynamic rule should forward packets only in forward direction. o And the last case that does not work before - simple fwd rule which triggers when some dynamic rule is already executed. PR: kern/136695, kern/147720, kern/150798 Modified: stable/8/sys/netinet/ipfw/ip_fw2.c Directory Properties: stable/8/sys/ (props changed) stable/8/sys/amd64/include/xen/ (props changed) stable/8/sys/cddl/contrib/opensolaris/ (props changed) stable/8/sys/contrib/dev/acpica/ (props changed) stable/8/sys/contrib/pf/ (props changed) Modified: stable/8/sys/netinet/ipfw/ip_fw2.c ============================================================================== --- stable/8/sys/netinet/ipfw/ip_fw2.c Wed Jul 6 06:34:08 2011 (r223818) +++ stable/8/sys/netinet/ipfw/ip_fw2.c Wed Jul 6 06:56:31 2011 (r223819) @@ -2070,7 +2070,8 @@ do { \ case O_FORWARD_IP: if (args->eh) /* not valid on layer2 pkts */ break; - if (!q || dyn_dir == MATCH_FORWARD) { + if (q == NULL || q->rule != f || + dyn_dir == MATCH_FORWARD) { struct sockaddr_in *sa; sa = &(((ipfw_insn_sa *)cmd)->sa); if (sa->sin_addr.s_addr == INADDR_ANY) { _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From: dfilter@FreeBSD.ORG (dfilter service) To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/150798: commit references a PR Date: Wed, 6 Jul 2011 06:57:17 +0000 (UTC) Author: ae Date: Wed Jul 6 06:57:07 2011 New Revision: 223820 URL: http://svn.freebsd.org/changeset/base/223820 Log: MFC r222582: O_FORWARD_IP is only action which depends from the result of lookup of dynamic rules. We are doing forwarding in the following cases: o For the simple ipfw fwd rule, e.g. fwd 10.0.0.1 ip from any to any out xmit em0 fwd 127.0.0.1,3128 tcp from any to any 80 in recv em1 o For the dynamic fwd rule, e.g. fwd 192.168.0.1 tcp from any to 10.0.0.3 3333 setup keep-state When this rule triggers it creates a dynamic rule, but this dynamic rule should forward packets only in forward direction. o And the last case that does not work before - simple fwd rule which triggers when some dynamic rule is already executed. PR: kern/136695, kern/147720, kern/150798 Modified: stable/7/sys/netinet/ip_fw2.c Directory Properties: stable/7/sys/ (props changed) stable/7/sys/cddl/contrib/opensolaris/ (props changed) stable/7/sys/contrib/dev/acpica/ (props changed) stable/7/sys/contrib/pf/ (props changed) Modified: stable/7/sys/netinet/ip_fw2.c ============================================================================== --- stable/7/sys/netinet/ip_fw2.c Wed Jul 6 06:56:31 2011 (r223819) +++ stable/7/sys/netinet/ip_fw2.c Wed Jul 6 06:57:07 2011 (r223820) @@ -3284,7 +3284,8 @@ check_body: sa = &(((ipfw_insn_sa *)cmd)->sa); if (args->eh) /* not valid on layer2 pkts */ break; - if (!q || dyn_dir == MATCH_FORWARD) { + if (q == NULL || q->rule != f || + dyn_dir == MATCH_FORWARD) { if (sa->sin_addr.s_addr == INADDR_ANY) { bcopy(sa, &args->hopstore, sizeof(*sa)); _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" >Unformatted: