Pliny and my main web server share the same subnet, and each device has only one NIC. My router forwards all relevant ports down to the main server, and the main server proxies requests to Pliny. This setup allows me to avoid messing with my gateway router (which serves my whole home, see below) whenever I need to change the services running in my internet-facing vLan. Instead, I can avoid accidentally taking out all of my home’s internet by isolating changes to those made on the main server.
XXXXXXXXX XXX XXXXX XXX The XX X Intertubes XX XXXX XXX XXXXXXXXXXX ^ +-----------------> To VLan2 | | ("Home" network) | +-----------+ +-+---------+ +---> DSL +----+ Router | | Modem | +-+---------+ +-----------+ |10.0.0.1 | | VLan1 +------------+----------+ |10.0.0.2 |10.0.0.3 +------+------+ +-----+------+ | Main Server | | Pliny | | (NetBSD) | | (NetBSD) | +-------------+ +------------+ A simple sketch of my network
With HTTP(S) traffic, proxying is an easy affair – set up Apache as a reverse proxy to the other server’s IP and httpd port and you’re done. But Pliny needed to have its ircd connected out to the web, not just its httpd. This was more difficult, and so I decided to try using NetBSD’s pf packet filter in order to do some TCP/UDP port forwarding.
However, the one special caveat with pf is that you cannot “reflect” a packet out of the same interface in which it came; that is to say – with a single NIC, you cannot redirect traffic onto the same subnet as the server is on. Iptables on Linux can do some pretty amazing things, including reflecting, so this limitation struck me quite hard. In fact, the following simple 3-liner will redirect all incoming TCP traffic on port 6667 of the host to 10.0.0.3 port 6667.
iptables -t nat -A PREROUTING -p tcp --dport 6667 -j DNAT --to 10.0.0.3:6667 iptables -A FORWARD -d 10.0.0.3 -p tcp --dport 6667 -j ACCEPT iptables -t nat -A POSTROUTING -j MASQUERADE
This applies even if the Linux host was on the same subnet (in this case, 10.0.0.2) and with only one NIC. I spent a good two days researching if there were ways in BSD to create a 2nd virtual interface, pass traffic into that, and then redirect that back through the NIC, but with no success. If you’re curious, here is NetBSD’s final word on the issue from the pf.conf(5) manpage:
Redirections cannot reflect packets back through the interface they arrive on, they can only be redirected to hosts connected to different interfaces or to the firewall itself
The solution then? OpenBSD has an entire page on circumventing this redirection limit by creating a TCP proxy with netcat / nc . Netcat listens on localhost and on the other server on the network, and pf then bounces requests incoming to the desired port to the lo interface, and from there they are relayed by netcat. On my system, all I needed to do for this was add one line each to pf.conf and inetd.conf:
# In pf.conf: rdr pass $if_eth proto tcp from any to any port 6667 -> \ 127.0.0.1 port 6667 # In inetd.conf: 127.0.0.1:6667 stream tcp nowait nobody /usr/pkg/bin/nc nc -w \ 180 10.0.0.3 6667
The only caveat with this setup is the need for nc to have the -w (wait) flag set to prevent unlimited, zombie connections. For this, the wait has to be matched to the expected timeout of a connection. In this case, Pliny’s IRCD has a 1m30sec ping, so doubling that (180 seconds) and setting that as the wait prevents early EOF’d connections.
What an adventure! Hopefully that will be the last bit of firewalling I need to toy with for the next while.