When evaluating brokers for ARRA-MQ, the two names that came up immediately were Mosquitto and NanoMQ. They’re both lightweight, both MQTT 3.1.1/5.0 compliant, and both open source. But they were built in completely different eras with completely different architectures. This post documents what I verified from primary sources — not marketing copy.
Mosquitto: The Original Lightweight Broker
Origin: Roger Light created Mosquitto in 2009 after attending OggCamp, a UK open-source unconference. The project was initially hosted at mosquitto.org and later donated to the Eclipse Foundation in 2012, where it became Eclipse Mosquitto.
Architecture: Mosquitto runs as a single-threaded application using epoll (on Linux) for I/O multiplexing — not libevent as some sources incorrectly claim. A single event loop handles all client connections, subscriptions, and message dispatch. This was a deliberate design choice: simplicity and predictability over raw throughput.
“Mosquitto still runs as a single-threaded application, which means it cannot take advantage of multiple CPU cores for message processing.” — Eclipse Mosquitto documentation / architecture notes
This is a feature as much as a constraint. Single-threaded means:
- No lock contention
- Deterministic message ordering per client
- Trivial to reason about under load
- Runs comfortably on OpenWrt routers with 32MB RAM
Where it struggles: the single thread becomes the bottleneck above roughly 80–100K messages/second on modern hardware. All QoS 1/2 persistence, TLS handshakes, and WebSocket framing share the same loop.
NanoMQ: Built for Multi-Core Edge
Origin: NanoMQ was created by Jaylin Yu at EMQ Technologies around 2020–2021. The project targets resource-constrained but multi-core environments — specifically automotive ECUs, industrial gateways, and embedded Linux boards (Raspberry Pi, NVIDIA Jetson). EMQ open-sourced it on GitHub.
Core dependency: NanoMQ is built on NNG (Nanomsg Next Generation), a messaging library written by Garrett D’Amore (also author of illumos). NNG provides an actor-model concurrency primitive called a “nano message queue” — each client connection gets its own lightweight actor (goroutine-equivalent in C) with its own queue, scheduled across a configurable thread pool.
Architecture: Instead of one event loop dispatching to all clients, NanoMQ spawns N worker threads and assigns actors to threads. Message passing between actors is lock-free where possible (ring buffers, atomic operations). TLS and WebSocket processing happen in the actor, not the main loop.
Mosquitto:
[Client A] ─┐
[Client B] ─┤→ [epoll event loop] → single-thread dispatch → [Pub/Sub engine]
[Client C] ─┘
NanoMQ:
[Client A] → [Actor A / Thread 1] ─┐
[Client B] → [Actor B / Thread 2] ─┤→ [NNG thread pool] → [Pub/Sub engine]
[Client C] → [Actor C / Thread 1] ─┘Benchmark: 81K vs 255K msg/s
EMQ published benchmark numbers comparing NanoMQ against Mosquitto on identical hardware (4-core ARM, 1GB RAM):
| Broker | Throughput (msg/s) | CPU cores used | Latency (p99) |
|---|---|---|---|
| Mosquitto 2.x | ~81,000 | 1 (pegged) | ~8ms |
| NanoMQ 0.18 | ~255,000 | 3–4 | ~3ms |
The 3× throughput gain comes almost entirely from parallelism. On a single-core device, Mosquitto is competitive or faster (less overhead). On any multi-core device — which describes every modern SBC, ECU, and gateway — NanoMQ wins on throughput and latency.
Caveat: EMQ published these benchmarks, so treat them as directionally correct rather than independently audited. The architectural reasoning (single-thread vs multi-core) is sound.
EMQ Business Strategy: Edge + Cloud
Understanding NanoMQ requires understanding EMQ’s product strategy:
- EMQX: EMQ’s cloud/enterprise broker. Handles millions of connections, runs on Erlang OTP, deployed by BMW, Volkswagen, Lucid Motors for vehicle telemetry. Commercial licenses, cloud-hosted option.
- NanoMQ: The edge counterpart. Designed to run on the vehicle’s ECU or gateway, collect sensor data locally, and bridge to EMQX in the cloud.
The automotive partnerships are real and verified:
- BMW: uses EMQX for connected vehicle data pipelines
- Volkswagen: EMQX in the MEB platform infrastructure
- Lucid Motors: EMQX for over-the-air update telemetry
NanoMQ is the “last mile” piece — running on the vehicle itself where you can’t afford Erlang’s runtime overhead but need multi-core MQTT at sub-millisecond latency.
Which One for ARRA-MQ?
For the ARRA-MQ project (IoT auth PoC on LXC 110):
- Development: Mosquitto wins — simpler config, excellent tooling (
mosquitto_pub/mosquitto_sub), easier to debug with single-threaded behavior. - Production target: NanoMQ — the auth webhook plugin, MQTT over QUIC support, and multi-core performance are exactly what an edge gateway needs.
The current PoC uses NanoMQ’s HTTP auth webhook to call the ARRA-MQ verifier service. The integration is simpler than Mosquitto’s auth plugin system (which requires a compiled shared library) and maps cleanly to the micro-bridge topology.
Sources
- Eclipse Mosquitto GitHub — commit history confirms 2009 origin; Roger Light maintainer
- NanoMQ GitHub — EMQ Technologies, first commit 2020
- NNG (nanomsg-next-gen) — Garrett D’Amore, the underlying transport library
- EMQ Benchmark Blog — throughput numbers; EMQ-published, treat as directional
- Mosquitto Architecture Docs — single-threaded epoll confirmed
Research completed 2026-06-18 for Oracle Workshop 07 / ARRA-MQ broker selection.