wifi
Wifi replacement project
Discussed on reddit. The plan is to have three hotspots in the house, one for the whole second floor and two in the first floor.
Note that my wifi tuning procedures should probably be followed when setting up the new APs.
Requirements
- small footprint
- ceiling mount, ideally with a nice socket for easier installation
- 2.4GHz and 5GHz hard requirement, WiFi 6 bonus
- no need for USB ports
- only one Gbit port necessary, ideally 2.5gbps+ (wiring is cat6a)
- power over ethernet is a must, no injector
- must run OpenWRT and still be supported
- can be bought in Canada, ideally
Possible models
Ubiquiti Access Point U6 Lite: 100$, 132$CAD at .ca store, backorder, 150$CAD at convertbit.ca, 144$ at deploydepot.ca, WiFi 6, 1gbps, no SNMP, has some Ubiquiti bracket mounting option that actually sounds interesting and easier to install, worried about a bright blue LED and Ubiquiti's proprietary stuff... Ubiquiti's default shop only ships to the US by default, but there's a .ca store with a smaller product selection. Ubiquiti also makes the PoE EdgeSwitches XP like the 10XP, 160$CAD at deploydepot, back-order, 153$CAD at tnt.ca, back-order, not available in Canada. There we have Switch Lite 8 PoE (desk-mount, 52W, 145$CAD, 266$ for 16 ports, 45W), or a rack-mount 16-port PoE switch (400$, 42W). They also have a neat toolless rack but only 6U (400$CAD) and a blank 24-port keystone patch panel for 25$. Be warned that SNMP support in Ubiquiti is spotty at best...
TP-Link EAP245v3, openwrt page, 100$ at Canada Computers, 105$CAD at Staples, 102$ at BestBuy with injector, 94$CAD at CDW (BO), Wifi 6, 2 gbit ports, PoE, ceiling mount bracket, TP-Link particularly interesting because widely available, cheap, known to produce devices working with OpenWRT, and even has some switches that can be flashed e.g. the SG2210P, although PoE doesn't work in "auto" according to the commit which sounds bad, 170$ at Canada Computers, 177$CAD at CDW, watch out that this switch will fill up quick as each EAP245 takes its own 12.3W of power and the switch can only provide 60W, so only 5 of the 8 ports are actually usable in that sense. Update: ordered from staples, shipped the wrong device, returned.
Zyxel NWA50AX was recommended on reddit, supported, with antenna radiation patterns, mounting bracket, Wifi 6, NO SNMP, one gigabit port, 16W PoE 802.3at, full specs, okay docs, only from a third party at newegg.com, 120$USD, same at amazon.ca, 180$CAD+, really promising especially since they have switches that run OpenWRT like the GS1900 (which can give pretty awesome per-port power usage reports but availability is a problem... they have a Europe store but it doesn't ship to Canada, so the only alternative is basically Amazon.com, e.g. GS1900-8HP at 144$USD, NWA50AX at USD70$+20$ shipping. WARNING: Zyxel routers recently had a serious security issue that led to thousands of routers joining a botnet.
Update: got the tplink, bigger than expected but otherwise nice. Ships with an injector and seems like it can feed PoE for the next device as well but this must be enabled in the GUI. Support for that in OpenWRT undetermined. No bridge capability seems built in the stock firmware, so needs to be flashed with OpenWRT.
Update: The TP-Link was returned! I ordered it from Staples, on sale at 90$. WARNING: it was not an actual v3, Staples shipped me a v4. Filed a negative review and asked for refund. Canada Computers price matched and are now out of stock, BestBuy also B/O as of 2023-08-31. So, instead, I got the cute little Ubiquiti Access Point U6 Lite that was pretty much fire-and-forget. Device was setup as svetlana and has been humming on the ceiling reliably ever since.
Discarded
- TP-Link EAP690 also interesting (10gbit port!) but not in sale locally and not supported (6GHz), EPA620 also interesting and sold at staples but possibly not supported either, AP comparison
- Engenius is also listed as supported and I found an interesting model but it's discontinued and what's available on staples is too expensive (150$+), furthermore none of the three indoor models are listed as supported by OpenWRT
- gl-inet b1300: seems interesting, i have heard good things about their travel router that comes with openwrt, wireguard and Tor pre-installed (!), but their PoE setup is with an "optional module" and was confirmed unavailable
- Turrix MOX: 146$, a little gimmicky, 5GHz only, PoE adapter an extra 45$... i already have one of those so i guess i could just buy the PoE adapter... but i find their thing a little confusing
Other racks and hardware
- deploydepot.ca has racks and shelves, this 10" shelf could hold routers (44lbs, 48$), and this 2U shelf could hold marcos (100$, 150lbs, back-ordered), and they have this 2-post 12U rack (195$)
- LexTec has racks and shelves but their shelves section is the most interesting, e.g. this 3u vented shelf might work for marcos (20" depth is just 0.9" short of the machine's depth (20.9"), 93.92$, and this 7" short shelf could hold router and other small equipment. Bewarned that their 16-port "keystone" patch panel (15.25$) is not compatible with standard keystone modules at all. I asked for a refund.
- Monoprice has racks but not a great selection. be warned that their Cat6a 24-port patch panel is crap as their plastic sockets are too small for a standard 110 punch down tool. I got credit from them for this.
- fs.com has a pretty good blank keystone panel but I found it too late, it does have racks but the larger ones are back ordered at the time of writing (2023-11-13), this tilted 8U rack is interesting and this shelf as well
- Canada Computers also has racks and shelves, this 2U one might do it for marcos (102$) but barely, this cheap shelf (45$) could hold the routers, but is back-ordered, so maybe this deeper shelf (16" 44lbs) could do it, and this 18U rack is quite interesting, 385$
- recyborg sometimes has racks and gigabit switches
Some other home lab had the following recommendations:
- 1U power switch: basically a rack-mounted power bar
- 1U cyberpower UPS: but i've also heard bad things about those, that they just crash when the battery runs out, even when plugged in?
- 10" rack shelf
Why OpenWRT
The point of running OpenWRT on the APs is to get monitoring about traffic on each node, which can be done with the Prometheus node exporter that can be installed in OpenWRT.
But we could also use the existing implementation, with something like the snmp_exporter (complete with dashboard and alerts!) to monitor APs with SNMP, as long as they support that which is not the case for the Zyxel NWA50AX and doesn't seem to be the case for the Ubiquiti U6 Lite either...
The point is to not be locked into any one system and being able to fallback to OpenWRT if we don't like the basic system. It also allows us to run a heterogeneous environment and not be forced to use a single solution for all networking hardware.
Current inventory
- atwood: Turris MOX 5GHz, location and configuration unknown (bridge?)
- plastik: bridge, in office storage, TP-LINK TL-WR1043ND v1.x
- rosa: bridge, in office storage, TP-Link AC1750 v5
- octavia: router, in home storage, Turris Omnia
Underground cross-connect
I need to cross connect a small office outside with the main patch panel. This is about 50' of distance, outdoors, preferably underground. We'll dig a trench for power and hop in for networking as well.
Copper or optical?
According to this post it's definitely worth passing fiber through, and I tend to agree: while I used Cat6a wiring inside the house because, right now, almost devices use copper connections, it's worth having an optical link for a longer distance like this, particularly outdoors.
There's multiple warnings about installing surge protectors on both ends of a copper circuit, and it seems like a nice idea to just avoid that problem altogether. There's also a risk of interference with the power run that will follow the network link right along the same trench.
Part of the problem is direct burial cable is new cable I need to buy, and it's typically 1000' spools, e.g. 340$ at prime cables, 250$ and B/O at monoprice, meanwhile a 60' run is about 60$ at cdw.ca (multimode, see other prices at cdw.ca) or 20$ at lextec (single mode!) So while that is more expensive, it's not that much more expensive, and it just gives me what I need instead of a giant spool I don't need. Also, I can buy it by weight at recyborg.
Heck, fs.com has 60' for 15$, a 1000' run is actually much cheaper than copper, at 115$, although that's for a plain PVC jacket, not a burial cable. That would still be more expensive, with fs.com quoting 630$ for an "industrial cable" that's suitable for outdoors use.
What's interesting with fiber is that, fundamentally, you do not need to bury the darn thing in the first place. It is perfectly fine through rain, snow and freeze, as it's not metal! I've seen outdoors fiber runs in the middle of the wood, just laid down on the ground, often buried under the last leaf fall...
Direct burial or pipe?
Direct burial seems ill-advised. Multiple reports of water eventually getting into wires, and the wires are much more expensive. There are fiber cables designed for burial, but those are more expensive as well.
PVC pipes are cheap, let's lay down two lines in the trench, this post suggests a ¾" PVC pipe, and "Most PVC conduit at home depot is rated for this" although this post suggests 1" pipes and "schedule 80 PVC".
- ¾" schedule 40 pipe: 10.75$ for 10' at HD, 2.91$ for elbow, 12.18$ for gang box
- 1": 16.97$ for 10' at HD, 4.42$ for elbow
I'll also need some glue not sure how that works.
Safety warning
Many suggest laying the pipes (or wires) at 16 to 18" deep and lay down some other layer, 6" deep, to warn future diggers. Something like this tape seem designed for this but maybe this narrower less explicit tape might also cut it. Some suggest even layout a layer of wood paneling or insulation to make a more robust warning. Possible to do both.
Optical modules and modes
Problem with fiber is we're not in the simple, "standard" RJ-45 jack (although there are of course many standards, from Cat5 to Cat8, they are generally compatible). There's many different fiber connectors, different cables, and different modules at the end. It's kind of a mess.
First off, there are different modules. The Turris Omnia has a "plain" SFP connector, rated at 2.5Gbit, which is probably the standard right before SFP+, so that's our target. According to WP, those are mostly on LC connectors.
Which brings us to connectors. It seems like I need a dual (or "duplex", one connector per direction) LC connector, so one needs to be careful because there are SC connectors out there as well, for example cdw has a couple of cheap LC/SC ones that show up on top. So LC/LC it is, I think. I guess it depends on the other end, but then again, I guess I'll just use the same SFP module on both ends, so LC/LC. FS.com has a good explainer on connectors.
Then we "just" need to pick the actual fiber. And that is confusing as heck. There is multi-mode fiber and single-mode fiber. According to this post:
The cost of multimode fiber suitable for high bandwidth so greatly exceeds the cost of singlemode equipment at the ends of the fiber that it makes no sense.
It turns out the multimode fiber has the promise of higher bandwidth and capacity, but that, presumably, means newer generations of cabling incompatible with previous ones. Unclear. This post from fs.com debunks some of that by showing how much more expensive SMF gets with higher speeds. That said, at the bandwidth target I'm going with (10Gbit), the cost difference for modules is marginal (7$)
So I guess I'll just go with whatever I find first. The point of having PVC pipes, after all, is that the wiring can be replaced, even though it's kind of hellish, from what I have heard.
One thing that is interesting is that at least some of the fs.com generic SFP modules are marked as compatible with the Omnia. That device is a tricky one though, because it needs a matching (but reversed!) transceiver on the other end! SMF.
Other SFP module options:
Note: I'm not getting into WDM here, but let's just say it's a way for either SMF or MMF to get higher bandwidth, for example this is how the 10GBASE-LX4 and 100GBASE-LR4 physical layers work.
Tentative build
- 20m SMF LC/LC cable: 2×15$ at fs.com (30$)
- alternative, armored: 40$ at fs.com
- two 60' 1" PVC runs: 12×16.97$ at HD (203.64$)
- four 1" PVC elbows: 4×4.42$ at HD (17.68$)
- safety tape, 200'×3": 11.98$ at HD
- SFP LC SMF module: 14$ at FS and 19$ at FS (33$)
- wifi router
- total: 296.30 + about 400$ for a new router
copper build
The SFP/fiber requirement complicates significantly that setup so we might just pull our existing wiring through the pipe and see if we can reuse existing hardware instead of doing anything too complicated for now. We have a TP-Link AC1750 in a drawer (rosa) we could just reuse here, if we stay with copper. That way, the project becomes:
- two 60' 1" PVC runs: 12×16.97$ at HD (203.64$)
- four 1" PVC elbows: 4×4.42$ at HD (17.68$)
- safety tape, 200'×3": 11.98$ at HD
- ~120 feet of Cat6a: already available
- cheap TP-Link router: already available (rosa)
- total: 233.30$
We just have to make sure to keep it possible to pull new wires through.
Also let's not forget the elephant in the room here, which is that excavation is much more expensive than any of this, in the order of thousands of dollars.
Mixed build
In this scenario, we do both: we pull a copper link and two fiber links for the future. The copper link is used at first, to avoid the hassle of figuring out a new router and just getting things going, but the fiber is used for future-proofing.
- 20m SMF LC/LC cable: 2×15$ at fs.com (30$)
- alternative, armored: 40$ at fs.com
- two 60' 1" PVC runs: 12×16.97$ at HD (203.64$)
- four 1" PVC elbows: 4×4.42$ at HD (17.68$)
- safety tape, 200'×3": 11.98$ at HD
- SFP LC SMF module: 14$ at FS and 19$ at FS (33$)
- ~120 feet of Cat6a: already available
- wifi router: new turris omnia, optional, for now already available (rosa)
- total: 296.30 + about 400$ for a new router
That's obviously the same price as the "tentative build" above.
core router replacement
Thinking outside the box: I need a biggest switch. The Omnia is full and I want to connect more ports. I ideally, I'd get rid of that PoE injector as well to be able to power more devices remotely. This goes back to the switch research I touched on in the wifi replacement project.
A OpenWRT forum participant suggested to just get a switch and a core router, which is a problem I completely forgot about but, now that I think of it, actually makes a lot of sense. GS1900-24HP would actually fit the bill perfectly: 24 gbit / PoE ports with 2 SFP ports. Then a 4-port (393$) or 2-port (273$) Protectli core router (with optional wifi, although that might be better served by a separate AP) could do the core router.
So, BOM:
- 2-port Protectli 4GB RAM 32GB SSD (273$, no wifi)
- GS1900-24HP: 544.16$CAD at ithardwarehub.ca, 400$USD
(533$CAD?) at ZyxelGuard, 220$USD (300$CAD!) at B&H
(special order)
- alternative: GS1900-48 without PoE (we keep the injector), 210$USD at B&H
- alternative: GS1900-24 without PoE: 170$ at B&H
U6 Lite AP: 132$ (optional? maybe reuse the Mox? atwoodusing the Omnia- simplex bidirectional SFP LC SMF module: 14$ at FS and 19$ at FS (33$)
- SMF LC/LC cable LC UPC to LC UPC Simplex OS2 Single Mode 30m 98ft (8.50$) ou 20m 66ft (6.70$)
- Total: 573$, 508$ for 24 port without PoE, 563$ for 48 ports without PoE, 705$ with extra AP
Notes:
- Omnia goes in the office, connected over SFP or copper, to the protectli or the switch, depending on arrivals
- rosa can serve as a replacement for the omnia if we don't want to get another U6
- this article previously had comparisons between various routers, this is now in margaret
Another build could be done with the Turris Mox:
Trick with the MOX is the bandwidth between modules is limited to 2.5Gbps, so the traffic between the switch ports can get saturated more quickly than a normal switch. It's also on the expensive side for a switch, compared to a normal one. Still, it's an interesting project, with close-to-mainline support!
AP public (désuet)
J'ai depuis longtemps un point d'accès ouvert mais maintenant contrôlé pour donner accès publiquement à internet.
Cherchez le point d'accès acces.reseaulibre.ca
.
C'est du wifi régulier, sur la fréquence 2.4 GHz, canal 1.
Ports ouverts: 22 (SSH), 53 (DNS), 80 (HTTP), 443 (HTTPS), 8080 (icecast), 123 (NTP), 587 (Mail submission agent), 993 (IMAPS), 5222, 8010, 7777, 5000 (XMPP), 1194 (OpenVPN), 11371 (HKP), ICMP
(ping)
Update: le point d'accès public est fermé.
Multicasting
Je roule un multicast d'une radio interne (voir radio). Ceci a tendance à polluer le wifi catastrophiquement: contrairement à un réseau fillaire, où le multicast peut être diffusé de façon efficace (chaque paquet envoyé une seule fois pour tout le monde), un réseau sans fil a des problèmes avec le multicast (chaque paquet doit être envoyé séparément pour chaque client). Des explications des problèmes avec le multicast et le wifi sont bien décrits ici:
- Why do some WiFi routers block multicast packets going from wired to wireless?
- Why are my listener-less multicast packets affecting Wifi performance?
- Multicast Wifi Problem Statement
En bref, ceci est particulièrement un problème avec l'encryption parce que chaque client doit recevoir un paquet séparément encrypté même si le contenu est le même. Mais ceci est un problème même sans encryption car le AP doit envoyer les paquets à la vitesse la plus basse supportée par tous les clients, ce qui peut ralentir considérablement le transfert et donc prendre paradoxalement plus de temps d'antenne.
Il semblerait y avoir des solutions théoriques avec un truc appelé DTIM, mais je ne vois pas clairement comment ceci peut régler les problèmes de fond mentionnés plus haut. Pour moi, la question du multicast performant sur wifi est non-résolue. Je me suis pour l'instant concentré à désactiver la diffusion multicast sur le wifi.
Pour résoudre ce problème, j'ai configuré mes routeurs OpenWRT pour ne
pas diffuser le traffic IGMP à moins que ceci soit demandé par les
clients, en utilisant une fonctionalité nommée igmp_snooping
(dans
OpenWRT) ou multicast_snooping
(dans le kernel). Ceci peut être
activé avec:
echo "1" > /sys/devices/virtual/net/br-lan/bridge/multicast_snooping
ou en ajoutant option igmp_snooping 1
à la config du bridge dans
/etc/config/network
. (Source)
Malheureusement, je vois toujours du traffic multicast sur les interfaces bridges et pire, wifi, avec cette configuration. Le problème est peut-être dû au fait qu'il n'y a pas seulement l'audio qui passe par le multicast: le protocole "Bonjour" ou "Avahi" utilise également le multicast pour annoncer des services. Les machines Apple, en particulier, vont envoyer des annonces qui vont activer le multicast sur le wifi et ainsi recevoir du traffic...
J'ai fini par ignorer les questions de igmpproxy
parce que ceci me
semble plus conçu pour contourner des problèmes de NAT et forcer la
diffusion des flux multicast. Je veux faire l'inverse. Voir la
documentation de igmpproxy
pour des infos à ce sujet.
À noter aussi que IPv6 utilise un autre protocole pour découvrir les hosts voulant du multicast, nommé MLD.
Bref: pas réussi à désactiver le multicast sur le wifi - il faudrait changer de pare-feu pour faire ça à ce niveau, et splitter les VLANs. Yerk. Voir aussi cette discussion.
The Freifunk community have done a bunch of patches to Enable bridge multicast snooping - that we may want to review.
Documentation about multicast settings on the bridge