For the longest time I’ve used an AirPort Extreme as my gateway router. When I moved to the US, I purchased an Airport Extreme 5th Generation and it worked well. It tied in with a Mac Mini Server I purchased at the time and the Mac OS X Server application integrated nicely with the device to handle port forwarding. I also set up Cacti in a Linux VM to monitor this device over SNMP and that worked reasonably well. Then Apple redesigned the AirPort Utility to become a much simpler interface. A lot of the settings and control that used to be there disappeared with the version 6 release. AirPort Utility 5.6 provided extra details like logs and a list of wireless clients with their relative connection strength. For a while I continued on using Airport Utility 5.6 with various levels of hacks to keep it working. Somewhere along the line I also ended up picking up a 6th Generation Airport Extreme to play with the 802.11ac support and found that it lacked many of the features like SNMP that the 5th generation supported. In a recent move I ended up coming across the 6th generation first, unpacked it and set it up as my main gateway but was disappointed by the lack of introspection into the device which put me on the path of looking at pfSense.

I first came across pfSense when I was in China and looking for ways to build a more solid and secure gateway out of the country. China’s internet is protected by the Great Firewall of China which generally drops a bunch of external traffic on the floor, the New York Times being one. pfSense came up as one of the better recommended systems out there and I dug into it a little more. I was looking online at various places to get pfSense supported devices from Chinese manufacturers however I ended up looking at the pfSense store and seeing their supported devices.

I left China and my in-laws came to visit. The last time they visited my AirPort Extreme started requiring a weekly reboot to work properly. This time it started happening again and I decided that it was time to get a device that could let me see what was happening with my network. The AirPort Extreme didn’t give me any insight into what was going on and I started looking at building a device to packet sniff. After a few iterations of network design I decided it’d be easier to setup a pfSense box and use that. While I could have potentially taken the time to build my own box, make sure the hardware was correct and then install pfSense on it, I really just wanted to buy something to get up and running quickly. I ended up heading to the pfSense store and checking out their options.

After looking at the range, I ended up settling on a SG-2440. This device comes with four network ports: one for WAN, one preconfigured for LAN and two extra ports. I grabbed the WiFi option and while I ended up keeping my two AirPort Extreme’s as my primary wireless devices, I want to have the option to have another network to play with being able to lock down access. I also gave it some extra storage to be able to use the device for packet capture without worrying about running out of space.

The device arrived and I set about getting it configured. I plugged in the WAN cable to my modem and plugged the LAN cable into my Mac Mini. This enabled me to reconfigure the device to land on my normal network without breaking everything. The next step was to reconfigure my AirPort Extreme to move to bridged mode and once I did that I was able to get online. From here I tried to connect my Mac Mini on the OPT1 network interface and wasn’t able to get it up and running properly. I played with a bunch of settings and trying to get the bridge to work but it wasn’t able to communicate over ethernet properly. After some work it seemed to be working ok, I could communicate with it properly on the local network and I was more or less happy.

A couple of days later I logged into the box and noticed that it couldn’t get to the internet. I did some traceroute tests and sure enough it wasn’t getting online. Here’s where I hit a roadblock with pfSense: the online documentation is disjointed. It looks and feels more comprehensive than it actually ends up being. I started trawling around the doc.pfsense.org site for information only to realise that the depth of information I needed wasn’t there. It dawned on me why there was such a heavy push for pfSense Gold (it’s right there in the UI) and why there is a separate paid user guide that likely covered the information I needed to know. Bundled with the SG-2440 was access to an online version of the pfSense book but it’s written as a book instead of quick recipes for how to do things. This meant that I ended up spending more time on searching for an answer instead of trawling through the book. It seems like I’m not the only one with various forum posts ending with “I didn’t realise…” and connecting the various dots together. With the knowledge I have now, I can see those nuggets of gold in the pfSense book but they’re generally only written in passing.

My problem ended up being that I’d not set up the firewall rules on the OPT1 interface and that caused it to drop packets going externally. However because of the way the bridge was set up that meant that I was able to talk to everything else on the network perfectly fine. Even the pfSense box could talk to it some how because it was port forwarding to it. I eventually found a guide on the web for setting up a bridge and this pointed me to the need to add a firewall rule to the interface. This also turned out to have been my problem setting up the WiFi interface on the box because the DHCP requests were being dropped by the pfSense device and my wireless devices would never properly join the network. Once I set up the firewall rules, everything seemed to more or less connect fine. That said my iPhone is still having periodic issues with the WiFi so perhaps not completely solved.

I’d also made the mistake of setting an IP address on the OPT1 interface as well when I was trying to get everything up. The bridging guide and some other forum posts made it clear that this was not the right thing to do however undoing it broke my network in it’s own right. The DHCP server had picked up the OPT1 IP address and handed it out as a gateway. When I removed the IP address from the interface, it broke connectivity for a bunch of devices so I ended up putting it back. Eventually I removed it and restarted the pfSense box which seemed to flush everything out. I also needed to reboot the device after reconfiguring the wireless interface as well to get it to work better. I ended up needing to reconnect all of the devices and once that happened everything seemed to be working properly.

The last part of the adventure was getting back IPv6 support. The AirPort Extreme in this regard just worked and had configured IPv6 support automatically with all of my devices getting an IPv6 address perfectly fine. With the pfSense box it required some configuration to get working. I fortunately found a guide on getting IPv6 with Comcast and pfSense to play nicely and it worked remarkably easy.

At this point I’ve go my pfSense installation in a mostly stable state. There is some tech debt I still need to address in terms of reconfiguring the settings from the LAN interface onto the bridge interface as well as changing the IP addresses around from the temporary one I used to avoid conflicting back to the .1 address on my network to ensure I don’t break any other settings I have.

I’ve now also reset my two AirPort Extreme’s up with the 6th generation connected to the pfSense box LAN port and the 5th generation extending the 6th over another ethernet connection. Since I’ve done this the network seems much more stable though I’m not quite sure which change did it: adding the pfSense box or adding the second AirPort Extreme back. Given previously I’d had the two AirPort’s running in tandem, I’m inclined to say the pfSense box running as the gateway helps reduce some of the strain on the AirPort Extreme while having two devices helps split up some of the wireless traffic.

The pfSense box also has it’s own DNS server and DHCP server which I’ve used to set up reservations for some of my static devices (like my Transporter’s) and add DNS addresses so that they show up properly. I could have done this earlier but I’d need to configure the DHCP mapping on the AirPort Extreme and then use my Mac Mini Server’s DNS to map the IP address to a name. With pfSense it’s still two distinct areas but it’s encapsulated in a single device.

All in all I’m happy with the configuration right now and I’m slowly going through the pfSense book to pick up information that I might have missed. Unfortunately only the HTML version is available with my purchase, the PDF copy seems only accessible with a pfSense Gold subscription. My next step will be working out how the VPN integration works for my next trip to China.