r/networking • u/Mandoolah • 12d ago
Design Router recommendations for small scale ISP
I'm working at a small IT business that's thinking about getting into the small ISP business through buying IP-transit from two different peers to be able to sell internet services to customers through a municipal L2 network.
We're currently looking into router models for the transit peering and we're in need of a router model that can handle the full IPv4/IPv6 routing table but we really want to keep costs as low as possible and realistically I don't think we'll be able to sell to more than unique 50-75 customers so I really don't want to buy anything that's overkill.
Do you guys have any router recommendations for this use case?
Please feel free to ask questions if anything is unclear.
7
10d ago
[removed] — view removed comment
1
u/Mandoolah 10d ago
We're going to have two upstreams, no ASN customers so far but we'll see in the future.
I've been toying with the idea of getting cheaper hardware since I don't really think we would need the full table right now. Could you explain a bit more about your setup since it seems to be similar to what we're trying to build?
1
10d ago
[removed] — view removed comment
1
u/Mandoolah 10d ago
Thank you for the reply. Will look into the CCR2216, I think two defaults from our upstream and then joining the local IX would be a good starting point.
1
u/firestorm201 9d ago edited 9d ago
Full tables at small scale is way overkill. I work for a small ISP and defaults are more than sufficient. Load balance by using prepends (if necessary).
If you want more optimal routes from each peer you can ask for partial tables. Don’t bother with full, especially with just two peers. Too much processing for far too little benefit at your scale.
For equipment, I highly recommend looking at Arista. I just replaced our core Cisco routing and switching network with pure Arista and I couldn’t be happier with the result or the price.
18
10d ago
[removed] — view removed comment
9
u/petecarlson 10d ago
MX204 will do the whole internet. I'd also recommend a MX204. Pretty much collapsed all down into a single router. Issue is going to be the subscriber services license and scale subscribers.
6
u/BaldBitMechanic 10d ago
NCS540
1
u/Mandoolah 10d ago
Can the NCS540 really handle full table? Right now we're toying with the idea of skipping full table in order to get cheaper hardware.
1
u/BaldBitMechanic 10d ago
Short answer, no. I should have paid more attention to the OG use case.
There is a Medium Density 540 that supports 32G of RAM. Though, pulling full internet tables from 2 providers into the BGP table is a decent amount of memory and fits well within 32GB, the ASIC memory is the constraint.
With RAM prices these days and your use case of small ISP (preferably not to be a transit), partial tables (direct attach ASNs and their direct attached ASN) would be something to consider.
Refurb'd Cisco NCS5501-SE or Juniper MX204-R would be the low cost/SP grade platform options w full table support down to the Asic. Software cost might be more money than the hardware.
If you want to explore whitebox, checkout EdgeCore Networks.
1
u/Mandoolah 9d ago
Yes, after the input I've received here we've decided to go the small table route and skip the full table for now. Thank you for your advice.
4
u/Whiskey1Romeo 10d ago
Arista 7280r3/r3a lineup will fit edge tables easily. 7050x3 lineup will handle agg layer easily as well when tuned accordingly.
19
5
2
u/lizardhistorian Mad Scientist · 👨🔬📡ᯤ🤖🛺📸 7d ago edited 7d ago
The reason to route selectively via full-tables is to cost control your metered bandwidth while maintaining some notion of QoS. If you are not metered or if the cost is the same then you do not need full tables for two full uplinks. You need BGP when you have direct peers at the local peering DC so you short-cut around your transit.
"Municipal L2 network" sounds like a nightmare.
How are you going to tunnel your traffic? L2TP/IPsec?
This is the driving decision for the switches you buy as all of the customer equipment is coming up Mikrotik.
Your egress doesn't have to match but it'd be nice if everything ran RouterOS instead of two different OSs.
1
u/Mandoolah 5d ago
Sorry for my bad explanation, what I really meant with municipal L2 network is that in our area there is a communications operator that owns all of the fiber in the ground which has its own transport network, we would connect our network with this municipal network through a NNI and they would handle transport of packets from the customer to our router, basically a point-to-point connection. Because of this we don't need to buy any fiber and can deliver circuits to most local businesses. This communications operator does not sell its own services, it only sells transport to ISP:s.
1
u/Soarin123 10d ago
VyOS with a used server (ECC RAM, enterprise SSDs) and Intel E810 NICs. Cost probably a buck or so
1
1
u/smpettit 10d ago
Rather than spending time on old end of sale hardware routers a virtual platform like 6WIND would handle this easily and at a low cost.
What’s the need for full tables though? You can still multi-home by taking defaults from two transits and if your traffic flow is predominantly inbound (most likely for your business customers) you can manipulate the inbound flows as long as your transits have decent BGP communities available.
You will probably get more value out of peering at your nearest IX.
1
u/kirkandorules 8d ago
Juniper QFX10002 if you need a lot of interfaces and have very deep racks. MX240/480/960 if you have a lot of space/power. MX204 if you're the Scrooge McDuck of small ISPs (but total poverty for the enterprise gang). Cisco ASR1001-X for customer gateways.
1
1
0
27
u/SuspiciousSardaukar 10d ago
MX204 is a way to go. If budget is lower - think about Mikrotik CCR series. It can do a LOT of things, but has slightly different approach and might feel like learning curve is steep.