How to be an ISP!
The NBN arrived at my address recently!
Well, it half did. My unit has the NBN available since July 2019. But the other 5 units in our complex had to wait until October. Such are the whims of large, government run projects with significant political interferance (not that I’m knocking it, I’m greatful for any improvement on 4Mbps ADSL).
Apparently, the physical cable for my unit runs from the telecommunications pit on the street, while all other units are run from a separate pit within the unit complex. And “something” made it difficult to do the run from the street to our common property. (I never got a straight story about what “something” actually was).
Anyway, I jumped on the bandwagon 30 seconds after I found the connection was available (because 4Mbps ADSL sucks). My neighbour (suffering even slower ADSL speeds), rang me the next day to complain that NBN was at my unit but not his!
So, I offered to share my connection.
Allow Internet connections to up to five additional units in my complex via WiFi or PPPoE, so that they can use their existing residential routers.
It’s ~60m from the street to the furthest unit in our complex. All are single storey. Four share common walls, and the two others are physically separate (with a driveway running between them, but sharing one common wall).
My plan was:
- Use a guest WiFi network for the closest unit, as coverage is pretty good through one wall.
- Run an ethernet cable through the ceiling cavity for other units.
- If more than one neighbour is interested, I’ll need an old switch to split my one available ethernet port / cable.
As it turns out, the two neighbours who signed up were well beyond my WiFi range, so I never used that option. Instead we ran ethernet (in cheap conduit) from my unit, out through my roof, via the rain gutter to the closest neighbour. He had an old 100Mbps switch which he used to split from my router to his own, and another cable run to the furthest unit, while keeping clear of the driveway.
Most ISPs I’ve worked with in Australia require your router to connect via PPPoE. This allows them to share or abstract telecommunications infrustructure, and present the same logical way of connecting. It’s more and more common for all Internet connections to come via a physical ethernet connection from a modem of some kind, but the login to your ISP requires a PPPoE login.
First step was to configure my HeX router to allow my neighbours to connect with PPPoE.
Goto PPP > Profile, and create a new Profile. As with a L2TP VPN, this profile will let us share configuration between all logins. I’ve configured DNS to use the router and Quad9, disabled compression and UPnP.
Next, PPP > Secret, and add logins for each user. I created a login for myself and plus all other units. Then assigned static IP addresses and IPv6 prefixes from a new, isolated subnet. And selected the profile I just created.
Finally, goto PPPoE > PPPoE Servers, and create a new PPPoE Service. Select the profile created above, and bind it to the interface which will be connected to your customers… err… neighbours.
I only had one spare port, so I bound directly to the ethernet port. If you’re planning to do multiple cable runs to several ports on your router, you’ll need to create a bridge and bind the PPPoE server to the bridge.
/interface pppoe-server server
Finally, add a PPPoE Server Binding interface for each user. This was an optional step with a VPN, but a named interface is a must when each user will need a queue and firewall rules.
That’s it! All the PPPoE stuff is done!
Mikrotik reference for PPPoE.
- New IP Address
- Firewall rules (make sure you drop any traffic going to your LAN)
- Queues (so that no one customer can use all the bandwidth)
Unlike most new networks, this does not require a DHCP server, because I’ve chosen to statically allocate IP addresses for each user. Even if you were serving 100 units, I’d still stick with a static allocations for each one - it makes troubleshooting and management easier, and it’s not like the units come and go like real customers do.
For queue configuration, I found that 44/16Mbps works well.
I’ve got three households sharing a 100/40Mbps connection (which is closer to 85/35Mbps in real life) and allocating around half the available bandwidth each seems to work.
And yes, that limit applies to me as well my neighbours!
Just remember to use an
sfq queue; the out-of-the-box
fifo queues can cause latency issues due to buffer bloat.
The router is all configured. All that remains is to configure my neighbour’s routers.
The assumption is, your neighbour already has a residential router with an ethernet port for WAN access.
If, like me, you consider most residential routers total crap and avoid them like plague, may I recommend Mikrotik’s hAP ac² device. It has 2.4 and 5Ghz WiFi, 5 ethernet ports, 4 cores, runs RouterOS and costs AU$110. That is, it’s a pretty good device for home users to get started with Mikrotik, and has similar headline specs to “high end” residential routers, which cost twice as much.
But it turns out PPPoE isn’t as standard as I thought (or some providers lock their devices to their own networks). Both neighbours ran into trouble with routers. In one case, the router had a terrible web interface and simply didn’t offer the right options. Another router said it supported username & password logins and PPPoE, but it would never connect. Both routers had an ethernet WAN port, and one was even “NBN certified” and was in use with an old ISP.
In the end, I got them both going using newer routers.
All routers are different, but here are two way to try and make them work:
Go through whatever configuration wizard the router offers.
Say that you need a login, enter the username and password from PPP Secrets (eg:
Click next / save / OK, and hope for the best.
If it doesn’t work, try the next option:
I prefer to know what’s actually going on, but that’s just me. Find the router’s “advanced settings” or “expert mode”, and you should find something like this:
|Address||DHCP||Might also be labelled automatic|
|DNS||Automatic||Or ISP assigned|
Higher end routers may have several options for your WAN connection (eg: ethernet or cable, ADSL, 4G / LTE). You’ll want to choose ethernet or however it’s labelled.
If you’re able to configure a Mikrotik device, entered a few basic settings is straight forward. Assuming the router supports such settings and labels them intelligently (and that’s a big assumption).
If all else fails, you could see if the manufacturer provides any documentation or a forum where you can ask a question.
Once configured, residential routers should Just Work™, and think that the “Internet” comes from the upstream Mikrotik device. You should see the assigned IP address in the status page.
And the graphs / aggregate traffic statistics.
If my ISP ever comes and reads this, I want to be very clear than I’m cost sharing. That is, I’m not attempting to setup a rival ISP to cheat them out of a few extra customers. I’m just trying to help out poor neighbours who were on 4Mbps ADSL.
I’ve subscribed to the highest cost plan for Internode, and added a “power pack” (to gain a static IP address). Partly so I can be sure there’s enough bandwidth to go around, and partly to tell my ISP “I’m not cheating you”.
I just split the cost equally between each neighbour and myself. They have my bank details, and deposit the amount each month.
As at writing, I am serving Internet to two other units in our complex. Including my own unit, that’s three of six sharing a single 100/40Mbps connection. With no adverse effects or obvious slow downs, even during the evening peak when half the city is watching Netflix.
With some long ethernet cables, a Mikrotik router running as a PPPoE server makes it quite easy to pretend I am an ISP. And the hEX S device has more than enough CPU to push 100Mbps to ~30 devices.
The only down side is that I’m now “ISP Murray” in my neighbours’ address book, aka tech support. Fortunately, Mikrotik is reliable enough that I haven’t had to field any 2am support calls. Yet.