From rand@meridian-enviro.com Thu Sep 14 19:51:27 2006 Return-Path: Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B2F8F16A416 for ; Thu, 14 Sep 2006 19:51:27 +0000 (UTC) (envelope-from rand@meridian-enviro.com) Received: from newman.meridian-enviro.com (newman.meridian-enviro.com [207.109.235.166]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7CAE243D72 for ; Thu, 14 Sep 2006 19:51:20 +0000 (GMT) (envelope-from rand@meridian-enviro.com) Received: from delta.meridian-enviro.com (delta.meridian-enviro.com [10.10.10.43]) by newman.meridian-enviro.com (8.13.1/8.13.1) with ESMTP id k8EJpJAj036169 for ; Thu, 14 Sep 2006 14:51:20 -0500 (CDT) (envelope-from rand@meridian-enviro.com) Received: (from rand@localhost) by delta.meridian-enviro.com (8.13.6/8.13.6/Submit) id k8EJpJOk027618; Thu, 14 Sep 2006 14:51:19 -0500 (CDT) (envelope-from rand) Message-Id: <200609141951.k8EJpJOk027618@delta.meridian-enviro.com> Date: Thu, 14 Sep 2006 14:51:19 -0500 (CDT) From: "Douglas K. Rand" Reply-To: "Douglas K. Rand" To: FreeBSD-gnats-submit@freebsd.org Cc: Subject: pfsync fails to sucessfully transfer some sessions X-Send-Pr-Version: 3.113 X-GNATS-Notify: >Number: 103283 >Category: kern >Synopsis: pfsync fails to sucessfully transfer some sessions >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-pf >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Thu Sep 14 20:00:34 GMT 2006 >Closed-Date: >Last-Modified: Wed Jan 14 22:26:53 UTC 2009 >Originator: Douglas K. Rand >Release: FreeBSD 6.2-PRERELEASE i386 >Organization: Meridian Environmental Technology, Inc. >Environment: System: FreeBSD luna-0.meridian-enviro.com 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE #12: Thu Sep 14 00:03:32 CDT 2006 rand@luna-0.meridian-enviro.com:/usr/obj/usr/src/sys/LUNA i386 >Description: With a pair of pfsync and carp firewalls some times, perhaps about 5% to 10% of the time, some TCP sessions will not transfer during a transition of the carp master. The error, if pf loud debugging is enabled, is a number of kernel messages like: pf: State failure on: 1 | pf: BAD state: TCP 67.134.74.10:53846 67.134.74.10:53846 207.109.234.245:80 [lo=3086765454 high=3086830989 win=65535 modulator=0] [lo=310734435 high=310734435 win=65535 modulator=0] 4:4 A seq=310747575 ack=3086765454 len=1460 ackskew=0 pkts=82511:161080 dir=in,rev Some more detail can be found at http://lists.freebsd.org/pipermail/freebsd-pf/2006-July/002317.html The symptom is that TCP sessions will simply hang if they are not transferred correctly. The chance of a TCP stream not transferring seems to be directly related to the speed of the stream. We have not experienced the problem with essentially idle, or interactive streams. But transfers of large files from fast sites fairly often experience the problem. >How-To-Repeat: Setup a pair of pfsync/carp firewalls, do some high-speed transfers across the firewalls, and repeatedly reboot the master. >Fix: While not a fix, this RC script in /usr/local/etc/rc.d seems to work around the problem most of the time. But we still experience stuck TCP sessions even with this script. #!/bin/sh # # PROVIDE: carp # REQUIRE: NETWORKING # KEYWORD: nojail shutdown . /etc/rc.subr . /etc/network.subr name="carp" start_cmd="carp_start" stop_cmd="carp_stop" _interfaces="`ifconfig -l | tr ' ' '\n' | grep '^carp[0-9]'`" startup_delay=90 advskew_up=0 advskew_down=100 carp_start() { echo "Changing advskew on carp interfaces to ${advskew_up} in ${startup_delay} seconds." ( sleep ${startup_delay} echo -n "Changing advskew on carp interfaces to ${advskew_up}:" > /dev/console for ifn in ${_interfaces}; do ifconfig ${ifn} advskew $advskew_up echo -n " ${ifn}" > /dev/console done echo "." > /dev/console ) & } carp_stop() { echo -n "Changing advskew on carp interfaces to ${advskew_down}:" for ifn in ${_interfaces}; do ifconfig ${ifn} advskew $advskew_down echo -n " ${ifn}" done echo '.' } load_rc_config $name run_rc_command $* >Release-Note: >Audit-Trail: Responsible-Changed-From-To: freebsd-bugs->freebsd-pf Responsible-Changed-By: vwe Responsible-Changed-When: Wed Jan 14 22:26:42 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=103283 >Unformatted: