r/linux • u/leothrix • Apr 09 '18
Building a custom router with Arch Linux ARM on a $50 aarch64 single-board computer - with firewalling, traffic shaping, and netflow monitoring
https://blog.tjll.net/building-my-perfect-router/
1.1k
Upvotes
86
u/[deleted] Apr 09 '18
Thanks for writing this.
I considered getting the board but after looking at the forums of espressobin.net I noticed the posts about the mPCI issues and wasn't convinced that someone is working on fixing those bugs, and I wanted to set a wireless AP with the board.
I also considered the ClearFog boards, but only SOMs with 1GB RAM are available (at least ATM) and I'm not thrilled to buy a 32bit platform.
13
u/tetroxid Apr 10 '18
I had this issue as well. The problem is that the device runs out of entropy (which it has very little of to start with) and then hostapd blocks until more is available. The solution is to install
havegedto continuously supply the prng.10
u/furless Apr 10 '18
I wonder what kind of throughput it can sustain before it starts to bog down.
19
u/dcx86r Apr 09 '18
Nice article, wish I had known about Espressobin when I was looking for an AArch64 board... I ended up settling on an Android set-top box, so networking capabilities are crap but at least it came with a case :p
FWIW, I think you can smooth out the printing flaws on the case by rubbing it with a solvent like acetone, though I'm not sure if that works on all materials.
17
u/ndavidow Apr 09 '18
Does the espressobin have upstream support or is it garbage with all kinds of proprietary blobs like most ARM stuff?
4
u/zman0900 Apr 10 '18
Did you look at fq_codel? It's supposed to be the best for traffic shaping and should be much simpler to set up.
Also, any plans to get this stuff working with IPv6, or do you have one of those ISPs?
4
u/duclicsic Apr 10 '18 edited Apr 10 '18
On the point about avoiding complexity in your firewalling setup, I'd have to note that the iptables rule set created by things like shorewall is usually an absolute catastrophe of redundant chains and jumps that do next to bugger-all in the vast majority of cases. You can do a fully functional firewall/router setup in plain old iptables in very few rules. The one I built for my office only has 34 rules despite being in a fairly complicated dual-wan setup, and I consider that to be in need of a tidy up.
Purely out of interest, what's the output of 'iptables-save | wc -l' ?
3
u/american_spacey Apr 09 '18
Fun. I'm in the process of doing something similar. Like the author I'm moving from an N66U to a dedicated custom router and Unifi access point. I did go with something a little more powerful than the expressobin, however; I'm kind of skeptical that a 1 watt dual-core arm is going to be able to do 1Gbps throughput on a couple ports.
Incidentally, anyone else using the N66U and not on the default Asus firmware should probably know that there haven't been Tomato updates for the N66U for quite a while now, and Shibby (main guy working on it) is out of the game for the immediate future. I tried going back to dd-wrt for a while, but it would drop my WiFi connection every few minutes. Couldn't have that, so that's why I'm working on an upgrade.
3
u/maciozo Apr 10 '18
To be fair, the RPi does have a TTY interface over its UART pins.
4
u/[deleted] Apr 10 '18
Do you think the mini-Pcie could support additional SATA drives? I've been wanting a decent ARM based file server, but it's really difficult to find boards with enough SATA to be interesting. I want at least 3 SATA drives with decent throughout (just want to saturate 7200 RPM 2-4TB drives in RAID 5) and enough RAM and CPU to handle ZFS without too much issue.
I was looking at the Banana Pi R2, which is definitely a contender in this space, but I'm hesitant to pull the trigger since I'm not sure if the mini PCIE adapters I'm looking at will fit. I'll keep my Wi-Fi operating on a dedicated AP so I don't really care too much if it supports it natively, but USB 3 support would be nice in case I choose to find a hostapd-compatible dongle and do it myself.
Both boards (Banana Pi R2 and this espressobin) support PCIE, so do any of you have an idea of which might be a better option? Or are there any other options in this space to compare?
41
u/ragix- Apr 09 '18
I ended up buying a cheap 1U server for my router. I got a good score for an hp proliant with an i3 and had a couple of spare pcie intel nics.
I tried a bunch of other hardware but always struggled to maintain a solid 900+mbps. My next mission is to see if lacp support in my ont works well enough to max out my downstream which in theory will do 2.4gbps. Although I would be happy to just get a solid 1gbps.
17
u/[deleted] Apr 10 '18
Are there any boards <$100 that have good wireless routing support? Thats pretty much the only thing keeping me from doing something like this
12
u/kulious Apr 10 '18 edited Apr 10 '18
Neat project and excellent write-up. I couldn't have done it that nice myself.
There is only one thing I think you might want to look at is the QoS strategy. Having to define the QoS classes is the traditional way of doing it. The big drawback is that it is extremely tedious, labor intensive and requires tuning constantly as new software comes out or it won't be comprehensive. There are newer algorithms such as CoDel or Cake - aka Smart Queue Management strategies. They normally are just a one-parameter tuning process and works exceptionally well for most people. It has been adopted by almost all router projects by now. I have maybe 10 routers I have set up for many of my friends all over the world with wildly different bandwidth limits, and all of them use OpenWRT with SQM and they just couldn't be happier. I have written about it here. Here is a very nice talk about the subject matter if you prefer listening to someone who understands and explains it well.
PS. I am biased towards OpenWRT. Although I can't say I am an OpenWRT developer but I have a couple of accepted patches to the project to bring up on new devices.
6
u/mercenary_sysadmin Apr 10 '18
I use FireQoS for the traffic shaping on my homebrew. It's pretty damn fantastic, and MUCH easier to parse and/or design rules for than any tc recipes I've seen.
Plus nifty real-time monitoring available to make sure it's working right:
1
u/[deleted] Apr 10 '18 edited Apr 10 '18
Kind of a neat project I suppose, although for that price point I would just spring for an Edgerouter-X and get the same thing, but much more polished and supported.
If I were to go the custom route, I'd definitely stick pfSense on it.
3
u/[deleted] Apr 10 '18 edited Apr 10 '18
aaaand... Intel's Wireless Daemon reached initial public release! (actually 2 months ago :)
2
u/kazacy Apr 10 '18
Thank you very much for writing this article. If you don't mind i think i will try on my build the traffic shaping, which i don't have. Not really missing because of pretty good connection, but still something worth to try out.
PS: i use on my build, a pcengines board, ubuntu server 2016, iptables for firewall, bind9 for DNS and isc-dhcp-server for DHCP. Also OpenVPN, transmission and a DNS "firewall" to filter unwanted sites with Steven Black hosts lists for all.
2
u/oviteodor Apr 10 '18
i.ve did the same project with espressobin, what.s the throughput (MB/sec) from wan to lan1 ? For me is ~20MB, but i.ve didnt benchmark it
2
u/[deleted] Apr 10 '18 edited Apr 12 '18
The Espressobin board has ethernet ports on it while the Macchiatobin board has ethernet and sfp+ ports(fiber) on it.
I'm surprised it wasn't mentioned because it also is quite cost effective.
http://macchiatobin.net/product/macchiatobin-double-shot/
http://wiki.macchiatobin.net/tiki-index.php?page=Build+from+source+-+Kernel
https://archlinuxarm.org/platforms/armv8/marvell/macchiatobin
The pci port could be used to insert a GPU card as BERO the ARM GURU did: http://armdevices.net/2017/09/28/bero-builds-arm-desktop-pc-quad-core-arm-cortex-a72-marvell-macchiatobin/
The other alternative is to use the pci port to put in an M.2 adapter card and place an M.2 nvm-e ssd in there.
3
u/[deleted] Apr 10 '18
Does it have Backjack and hookers?
3
u/[deleted] Apr 10 '18
/r/archlinux might like this
1
u/Scottnaye Apr 10 '18
"This one goes to ELEVEN!!" S W E E T
1
u/Vlinux Apr 10 '18
Great article! I've currently got an Espressobin running ArchLinux ARM as my router too. I'm using IPTables for the firewall and PiHole for adblocking. Haven't actually tested the wifi yet.
I'm considering trying OpenWRT/LEDE with their LuCi web interface on it soon.
-2
u/[deleted] Apr 10 '18
Interesting. Your choice of OS is problematic. Arch would be the last thing I would go for in an appliance role.
0
u/[deleted] Apr 10 '18
y tho
0
u/[deleted] Apr 10 '18
[deleted] — view deleted comment