Product

Product Features How to Compare Docs Screenshots Why

Start here

Getting started Download Guides Frequently asked questions

More

Security Support Roadmap About
  1. Home
  2. How it works

Architecture

Every node holds everything. Nothing sits in the middle.

There is no controller to install, no database server to keep up, and nothing whose loss stops you managing the rest. That is the whole design in one sentence, and the rest of this page is why it was built that way.

Your browser TCP 7443 nginx server 1 · active Manager container uid 10001, no privileges, read only unix socket Host agent root, fixed verb list, path allow list writes and reloads nginx, built from source serves TCP 80 and 443, plus UDP 443 for HTTP/3 nginx server 2 · standby Manager container same image, same config unix socket Host agent root, fixed verb list, path allow list writes and reloads nginx, built from source serves TCP 80 and 443, plus UDP 443 for HTTP/3 mTLS + HMAC TCP 7444 Your application servers · nothing installed here app-01 10.0.0.11:8080 app-02 10.0.0.12:8080 app-03 10.0.0.13:8080 They never know the manager exists. They just answer requests.
live traffic and signed peer messages standby and background work
One manager container per nginx server, sitting above a small root agent. There is no central controller to install and nothing extra to lose when a box dies.

Manager container

Runs on every nginx server. The GUI, the database, the health checks, and talking to peers. Unprivileged.

Host agent

Runs on every nginx server. The only root part. Writes config, tests it, reloads nginx. Nothing else.

nginx

Runs on every nginx server. Serves the traffic. Built from source so the modules are there.

Your backends

Wherever they already are. Nothing is installed on them and they never know this exists.

The manager has to be on the same box as its nginx

Because it talks to the root agent over a unix socket on that machine, which is a file rather than a TCP or UDP port, so nothing on your network can reach it at all. The manager does not go on your application servers. Those are just servers in a pool.

The privilege boundary

One small program has root. Everything else asks it nicely.

The web GUI is the part most likely to be attacked, so it is the part with the fewest privileges. It runs as uid 10001, drops every Linux capability, and has a read only filesystem. It cannot write /etc/nginx and it cannot restart anything.

The host agent runs as root, because writing nginx config and reloading a service needs root. It is deliberately small enough to read start to finish in one sitting, and it is the part worth auditing if you audit one thing.

  • A fixed list of verbs over a unix socket. There is no "run this command" verb and there never will be.
  • Every path is resolved, symlinks and all, and checked against an allow list before anything is touched.
  • A systemd unit that gives it root and takes nearly everything else away.
The whole security model

What the agent will do

Stage config
Write a complete configuration beside the live one, without touching it.
Test config
Run nginx -t against the staged copy with the real binary.
Commit
Swap the staged copy in and reload nginx gracefully.
Roll back
Put the previous configuration back.
Write certificates
Place certificate and key files, key at 0600 owned by root.
Run the build
Drive the nginx installer and report progress.
Install data files
The country database and the WAF rule set, to fixed paths.

That is the shape of the list. It does not include anything that takes a command, a URL or an arbitrary path, because either of those would be a way to do anything as root.

Changing things

Two phases, and nothing changes until every node agrees

This is the mechanism the whole product is built around. Everything you save is written down and none of it is live. Applying is what makes it real.

Phase one · ask before doing Save and apply on the active node Build the whole config from the database, not from whatever is on disk lb-01 stages it nginx -t lb-02 stages it nginx -t passes passes Phase two · only now does anything change Commit to every node at once Swap in, reload nginx old workers finish what they were doing, nobody is cut off Version 178 live on every node, or on none If one node had said no both staged copies are thrown away and nothing goes live A node that was switched off during the change does not block it. The change goes into that node's queue and is replayed when it comes back, so it catches up rather than quietly drifting out of step.
A config that would stop nginx starting cannot get halfway across your fleet. Either the change lands everywhere or it lands nowhere.

Why every node tests separately

Because the binaries can differ. A directive belonging to a module compiled into one build and not another is valid on one node and fatal on the next. Testing centrally would catch none of that, and the failure would land after the change had already gone out.

Why the whole config is rebuilt

Not patched, built. Every apply renders the complete configuration from the database, so the files on disk can never slowly diverge from what the screen thinks is there. It also means a node that has been offline gets the same complete answer as everybody else rather than a chain of patches.

What happens to a node that was switched off

The change goes into its queue and is replayed when it comes back. It catches up rather than quietly drifting, which is the difference between a fleet and two servers that used to agree.

apply log
A good apply, then a bad one 0:00 / 0:00

The second half shows what happens when one node refuses the config.

Certificates

Worked out fresh at every renewal

Only the node that currently owns the public address can answer an http-01 challenge, and which node that is changes when you fail over. So it is never assumed.

Renewal due shop.example.net 30 days left Look the name up against public DNS, fresh, every time Which node has that address today? it moves when you fail over Token pushed to every node first lb-02 runs certbot webroot, http-01, TCP 80 or dns-01 if you set it up Copied to every node in the fleet key stays 0600 root The certificate authority picks which address it connects to and you do not get a say, so the challenge token is on every node before validation starts. That is the part people get wrong by hand. The certbot timer is deliberately switched off, because it fires on every box and fails on the ones that do not currently own the address, filling a mailbox with noise that hides the real problem.
Certificates in a cluster are the fiddly part. Only the node that currently owns the public address can answer, and which node that is changes when you fail over.
renewal log
A renewal, start to finish 0:00 / 0:00

What a renewal prints when nobody is watching, which is most of the time.

The three things that go wrong by hand

  1. The timer fires everywhere. On a node that does not own the address the challenge fails every time, and the mailbox of failures buries the real problem. So the certbot timer is switched off during installation.
  2. The token is only on one node. The certificate authority picks which address it connects to. Push the token everywhere first and it does not matter which one it picks.
  3. The result stays on one node. Then the site works until the next failover, at which point it does not, and the two events are far enough apart that nobody connects them.
Certificates in detail

Failover

Two layers, because one of them cannot help

Cluster failover moves the ability to make changes. It cannot move traffic that was already heading for a machine that has gone, because that decision was made before the connection was attempted.

Both healthy Visitor needs an address The fleet answers www.gslb.example.net TTL 30 seconds lb-01 · healthy 203.0.113.10 lb-02 · healthy 203.0.113.11 Both addresses handed out. Visitors spread across the two. Nothing to configure per visitor. lb-01 has gone Visitor needs an address lb-02 answers alone it already knew lb-01 had stopped answering lb-01 · down left out of the answer lb-02 · healthy 203.0.113.11 Dead node out of the answer in about thirty seconds. A visitor whose computer already asked moves once the old answer expires, so about a minute in total. Lookups arrive on UDP port 53 and fall back to TCP port 53 when the answer is too big for one packet, so open both. Nothing can make this instant. DNS answers are remembered, and how long for is a request rather than a rule.
Measured on a real pair of nodes: the failed one was out of the answer in about thirty seconds, and back in about five seconds after it recovered.

Cluster failover

The standby notices the heartbeats on TCP port 7444 have stopped and promotes itself. A node will not do that unless it can see more than half the fleet, so a network split cannot leave two nodes both taking writes.

Fixes: who is in charge. Does not fix: visitors already pointed at a dead machine.

DNS failover

Your nodes answer DNS for a zone you delegate to them, on UDP port 53 with TCP port 53 as the fallback for large answers, and leave out any node that is not healthy. Measured on a real pair: out of the answer in about thirty seconds, back in about five.

The part that answers runs without root, so it really listens on port 5353, again on both UDP and TCP, and a firewall rule the software manages sends port 53 to it.

Fixes: where new visitors are sent. Cannot fix: a resolver that already has the old answer and will keep it until it expires.

Reaching backends

When the backend has no public address at all

Your office or private rack app server, no public address 10.77.0.4:8080 on the tunnel Firewall nothing inbound is allowed, and nothing needs to be dials out UDP 51820 Your load balancers lb-01 10.77.0.1 lb-02 10.77.0.2 The workload is an ordinary pool member health checks, weights and everything else work exactly as they do for a public backend It is not a VPN. The machine gets a route to these load balancers and to nothing else, and its own traffic does not change.
One command on the machine you want to reach. It connects to every load balancer, so losing one does not lose the backend.
How the tunnel works

Design decisions

Five choices, and why

SQLite rather than Postgres

Every node keeps its own full copy and the nodes sync with each other. Adding a database server would mean one more thing that has to be up before you can manage your load balancers, which is exactly backwards. When something is broken at three in the morning you still need to get in and fix nginx.

nginx from source rather than from apt

The distribution build is missing modules several of these features need. Building it means you own the upgrade cycle, which is a real cost, and it is what makes post quantum key exchange, HTTP/3, the WAF and country blocking possible at all.

Server rendered pages, no build step

The GUI is HTML from the server with a small amount of JavaScript. There is no bundler, no node modules, and nothing to rebuild. A management tool you cannot fix without a toolchain is a management tool that fails at the wrong moment.

Only offer what this nginx can do

The manager reads nginx -V from the real binary on each node and works out which modules are compiled in. Settings for a module you do not have are simply not shown, which is why you cannot save something that would stop nginx starting.

Config generated, but hand written config kept

The generated files are rewritten from scratch on every apply, so editing them is pointless. Anything you type in the raw config box on a site is stored in the database and survives, which is why that box exists.

Reloads rather than a runtime API

Everything here ends in a graceful reload: old workers finish what they have, new ones take the new connections, nobody is cut off. The honest limit is that if your backends change many times a minute, the reload rate becomes the ceiling.

Two fresh servers is all it takes

Ubuntu 22.04 or newer, root access, and about twenty minutes. The installer does the rest and it is safe to run twice.

Getting started Search the how to