What is it ?

IPFW2+IPv6+dummynet is patch originally produced by Luigi Rizzo and later maintained by Mariano Tortoriello. (tortomari AT email.it) This patch enables dummynet support for IPv6 via IPFW2. IPFW2 implements superset of old IPv6 firewall (ip6fw) functionality.

I needed the functionality of the patch primarily in RELENG_4_10 so I have fixed several small bugs in patch from Mariano, added IPFW2 support to IPv6 forwarding code, added IPv6 packet logging code (from ip6fw) code and made it all compile.

Note: I am not the original author of the patch, I've just added bits of code and tweaked it for my purposes. The credits belong to L.Rizzo and his students.

Download

• patch against FreeBSD-5.4 RELEASE (tagged RELENG_5_4_0_RELEASE in FreeBSD CVS tags)
  This is preliminary test patch against FreeBSD 5.4-release. It does not compile yet.
• patch against FreeBSD-4.11 RELEASE
  The only file changed from 4.10-RELEASE was ipfw2.c. Moreover, only usage part of this file was changed, so this is rather cosmetic patch.
• patch against FreeBSD-4.10 patch branch
  this patch applies cleanly also on RELENG_4_9.
• patch against FreeBSD-4.8 patch branch
  this patch applies and compiles cleanly, but is untested

• sample config script for FreeBSD-based IPv6 router
• supfile for getting 4.10-RELEASE sources
• relevant part of kernel config to enable IPFW2+IPv6+DUMMYNET

Changelog

• 2005/06/03 - added patch against FreeBSD 5.4-release.
• 2005/06/02 - fixed next bug in parsing ipv4+ipv6 rulesets:
  It exhibited following behavior:

    # ipfw add 120 allow tcp from 195.1.2.152/29 to any 22
    ipfw: bad source address 195.1.2.152/29
    

  This is fixed in latest version of patch against 4.11-releng.
• 2005/02/11 - fixed bug in mixed ipv4+ipv6 rulesets
  (fixed only in patch against 4.11-RELEASE)
• 2005/02/01 - made patch against FreeBSD-4.11-RELEASE

How to install

Installation process comprises of getting/patching the sources, compiling/installing new kernel and sbin/ipfw binary:

• get sources and patch em' :
  

  cvsup supfile
  cd FreeBSD-src/FreeBSD-4-releng
  patch -p0 < src-ipfw2-ipv6-dummynet.patch
  

• kernel
  Addition to your kernel config should look like this:

  # IPFW2
  options         IPFW2
  options         IPFIREWALL_VERBOSE      #enable logging to syslogd(8)
  options         IPFIREWALL_FORWARD      #enable transparent proxy support
  options         IPFIREWALL_VERBOSE_LIMIT=100    #limit verbosity
  options         IPFIREWALL_DEFAULT_TO_ACCEPT    #allow everything by default
  
  # used on majority BSD MW routers
  # narrows monotonic interval to cca 2ms for 64Kbit/s pipe
  option          HZ=1000
  
  # dummynet
  options         DUMMYNET
  

• sbin/ipfw

    cd /<PATH-TO-PATCHED-SRC>/sbin/ipfw
    # install patched includes
    cp sys/netinet/ip_fw2.h /usr/include/netinet
    cp sys/netinet/ip_dummynet /usr/include/netinet
    make -DIPFW2
  

How to make release based on one of the patches

The patches above contain both userland (/sbin/ipfw) and kernel modifications. For easier install, you can make a release from them.

You will need to set couple of variables first. For building release based on FreeBSD-5.4-release, you can use the following:

bname=5.4-RELEASE-i2i6d
reltag=RELENG_5_4_0_RELEASE
TMP_DIR=/tmp/FreeBSD--release
CVS_ROOT=:pserver:anoncvs@anoncvs.at.FreeBSD.org:/home/ncvs
patch=src-ipfw2-ipv6-dummynet-5.4.patch
patchscript=ipfw2-script.sh

The CVS server can be arbitrary public CVS from the list of FreeBSD CVS servers.

The script ipfw2-script.sh set in patchscript should contain the following:

#!/bin/sh
# for building release with IPFW2

# presume we are in CHROOTDIR for release building now
#   (according to /usr/src/release/Makefile)
#   this script is meant to be run as LOCAL_SCRIPT

echo "IPFW2=1" >> etc/make.conf

After setting the variables and populating /usr/obj according to FreeBSD release building document (by executing make world in /usr/src) you can execute following script:

cd /usr/src/release
if ! make release "BUILDNAME=$bname" "RELEASETAG=$reltag" \
    CHROOTDIR=$TMP_DIR CVSROOT=$CVS_ROOT NODOC=YES NOPORTS=YES \
    "LOCAL_PATCHES=$patch" "LOCAL_SCRIPT=$patchscript" > $TMP_DIR.log 2>&1; then
  echo "Release making failed, see $TMP_DIR.log" >&2
  exit 1
fi

When the script completes its work, you will find release files in TMP_DIR/R/ftp.

Known bugs/TODO

• modules do not compile with IPFW2 define (they actually compile, but with old firewall, not IPFW2)
• parsing bugs in sbin/ipfw
• add IPv6 forwarding initiated directly by IPFW2 ('fwd' keyword) - PACKET_TAG_IPFORWARD in ip6_output.c is unimplemented. (this is other than regular IPv6 forwarding, which works with dummynet)
• cleanup. e.g. (hlen == 0) semantics in ip_fw2.c

Function call flow

IPv6 forwarding with dummynet

Unlike IPv4 forwarding function call flow, KAME IPv6 forwarding call flow passes packet from ip6_input() routine to ip6_forward(), which in turn passes the mbuf chain to nd6_output(), not ip6_output().
In IPv4, the packet goes from ip_input() through ip6_forward() to ip_output(). The pfil hooks are only in ip_input(), ip_output().

• ip_input() calls ip_forward() in case the packet needs to be forwarded
• ip_forward() calls ip_output() after various conditions are checked and correct route is found
• ip_output() calls interface-specific processing funtion *ifp->if_output ( )

The difference is that ip6_forward() does not call ip6_output() routine , but uses nd6.c:nd6_output() function which first check various ND related conditions and after it calls interface processing function ((*ifp->if_output)(ifp, m, (struct sockaddr *)dst, rt))

summary: pfil hooks need to be in ip6_input(), ip6_output() and also in ip6_forward() function.

Consider following scenario:

                                                                                
                                                                                
  fec0:2::2/32             fec0:2::1/32    fec0:1::1/32         fec0:1::2/32    
  |-----------|                   |----------|                  |---------|     
  |  notebook |--- cross cable -- |  router  |------ hub -------|   pc    |     
  |-----------|    100baseTX  fxp1|----------|fxp0  10baseT/UTP |---------|     
                  full-duplex                                                   

$ipfw=/sbin/ipfw

## outgoing queue
$ipfw pipe 1400 config bw 64Kbit/s queue 16KBytes
## incoming queue
$ipfw pipe 1401 config bw 64Kbit/s queue 16KBytes

# ND
$ipfw add 01000 allow ipv6 from any to ff02::/32 # via fxp1
$ipfw add 01001 allow ipv6 from fe80::/32 to fe80::/32 # via fxp1

ipv6addr="fec0:2::2/128"

# queue only on fxp1
$ipfw add 01500 pipe 1400 log ipv6 from any to $ipv6addr out
$ipfw add 01501 allow log ipv6 from any to $ipv6addr out
$ipfw add 01600 pipe 1401 log ipv6 from $ipv6addr to any in
$ipfw add 01601 allow log ipv6 from $ipv6addr to any in

# default deny on fxp1
$ipfw add 65420 deny log all from any to any via fxp1

# allow the rest
$ipfw add 65421 allow log ip from any to any

Now, when host (pc) behind one interface sends packet to another host (notebook) behind second interface of the router, the function flow for packet sent from pc to notebook goes like this:

         kernel
       |---------------------------------------------------------------------
       |						   		    |
       |         	                1/HZ 	     /<--- ip_dn_io_ptr()   |
   -------------                           /--  <---.           ^           |
<--| interface |<----- ip6_output()--<----/          \          |           |
   |  logic    |       ip_fw_chk_ptr()   |  dummynet  |	  ip6_forward()     |
   -------------                         \   sched.   |         ^           |
       |	                          \          /          |           |
       |	                           '________/   ip_fw_chk_ptr()     |
       |	                         	           ip6_input()  |-------|
       |	                                                ^       |  ifc  |
       |						  (ip6intr())<--| logic |<--
       |							        ---------
       |								    |
       ---------------------------------------------------------------------|

1. packet is received on interface fxp0 via ip6intr() ip6intr() just passes the packet to ip6_input(), it does not do much more than this.
2. ip6_input() checks firewall via call to ip_fw_chk_ptr(). it matches the rule 65421.
   ip6_input() realizes that packet needs to be forwarded and calls ip6_forward().
   no queueing is done.
3. ip6_forward() checks firewall via ip_fw_chk_ptr()
   The packet is matched by rules 01500 and 01501.
   ip6_forward() enqueues the packet to dummynet via dip_dn_io_ptr() which is pointer to dummynet_io() in this case. the packet is scheduled for outgoing processing via

     error = ip_dn_io_ptr(m, i & 0xffff, DN_TO_IP6_OUT,
                                   &args); 

4. now "input" delay occurs.
   every 1/HZ in a second, dummynet scheduler is called, which can result in calling transmit_event() which will call ip6_output().
5. transmit_event() calls ip6_output() via
   

       (void)ip6_output((struct mbuf *)pkt, NULL, NULL, 0,
                           NULL, NULL, NULL); 

   (called from netinet/ip_dummynet.c:transmit_event())
   ip6_output() parses mbuf chain which represents the packet and finds that is was already processed by dummynet - the mbuf chain has PACKET_TAG_DUMMYNET tag.
   The firewall is checked via ip_fw_chk_ptr(), which returns 0, which leads to "common case":

                     if (off == 0 && dst == old)             /* common case */
                           goto pass6; 

   The packet is now sent directly to the network (to notebook host) via

   		error = nd6_output(ifp, origifp, m, dst, ro->ro_rt); 

The reply packet coming from notebook to pc has similar function call flow. (rules 01600 and 01601 are matched this time)
Both packets can be clearly seen on following log snippet: