The Brief
A centralised data concentrator for a network of industrial air quality monitors deployed across a site. Each monitor reports continuously to the concentrator, which aggregates the data, persists it to a database, and exposes it via a REST API for integration with the client's web application. The service needed to handle concurrent data streams from multiple devices reliably, provide low-latency query access to historical and live readings, and support remote alarm management, all within a single containerised deployment per installation.
System Architecture
The concentrator is a Go service running inside a Docker container on a Linux host, one instance per site installation. Go was chosen for its lightweight goroutine model, which maps cleanly onto the problem of handling concurrent inbound UDP streams from multiple monitor units without the overhead of a thread-per-connection approach. The containerised deployment keeps the installation footprint small and consistent across sites, with no runtime dependencies beyond Docker on the host.
UDP Data Ingestion
Each air quality monitor on the network transmits its readings over UDP to the concentrator. The service maintains a listener that demultiplexes incoming datagrams by device identity, parses the sensor payload, and dispatches each reading to the persistence pipeline. UDP's low overhead suits the continuous, high-frequency nature of the monitoring data. The concentrator tolerates packet loss, and the simplicity of the protocol reduces the firmware burden on the monitor devices themselves.
REST API
The concentrator exposes a REST API consumed by the client's web application. The API provides endpoints to query live and historical readings from individual units or across the full site network, allowing the client to display current air quality status and trend data. Alarm management is also handled via the API: operators can acknowledge and clear active alarms on specific units remotely, with the concentrator forwarding the command to the relevant device and confirming the state change.
Database Integration
All sensor readings are persisted to a Microsoft SQL Server database, providing a reliable store for historical data and alarm records. The Go service writes incoming readings asynchronously to avoid introducing latency into the ingestion pipeline. A write buffer decouples the UDP listener from the database layer, ensuring data continues to be collected cleanly even under brief periods of database load. The schema supports efficient range queries across device and time dimensions, matching the access patterns of the client-facing API.
Working on something similar?
If you need to aggregate, store, and expose data from a network of connected devices, we'd like to hear about it.
Start a conversation →