How the software is hardened

The container is not trusted

The manager runs as uid 10001, drops every Linux capability, and has a read only filesystem. It cannot write /etc/nginx and it cannot restart anything. Whatever an attacker does inside it, they are still on the wrong side of the boundary.

The host agent is the boundary

The agent runs as root, because writing nginx config and reloading a service needs root. It is deliberately small enough to read start to finish in one sitting.

• A fixed list of verbs over a unix socket. There is no "run this command" verb and there never will be.
• Every path is resolved, symlinks and all, and checked against an allow list before anything is touched.
• A systemd unit that gives it root and takes nearly everything else away.

This is the part worth auditing if you audit one thing. Everything else in the system is a program that asks the agent to do something, and the agent is the program that decides whether it is allowed.

The peer port is not open

Three separate things have to line up on every peer call:

1. A client certificate signed by this cluster's own certificate authority and nothing else.
2. A source address that is on the node roster.
3. An HMAC signature made with a key only that node holds.

The timestamp and a nonce are inside the signed part, so a captured request cannot be replayed. A browser hitting that port gets nowhere at all.

The GUI has its own allow list

NFM_ADMIN_ALLOWLIST is checked before the login page even renders. A stolen password from the wrong network still gets nothing.

It lives in the environment file rather than the database on purpose, because it has to be in force before anything in the database can be read.

Secrets at rest

Authenticator seeds, peer keys, DNS provider credentials, paid authority credentials and backend trust tokens are encrypted with a key in each node's own environment file. That is why every node needs its own file and why copying one between boxes is a mistake the installer now refuses.

Joining is a two hour secret

The join token works once and expires in two hours. Tokens are stored hashed, so a copy of the database does not hand anybody a working one. There is no long lived cluster password to leak.

A new node makes its own key pair and the private half never leaves that box.

Being honest about scope

Common questions