Installing

For the walk-through — install, first person, rules, first machine — see Getting started. This page is the reference: the package, and the same by hand on a system without it.

One static binary, one directory of files, one directory of state. No database daemon: state is SQLite inside the state directory.

Debian and derivatives — the APT repository, added on this machine (it is not there because an agent's machine has it):

curl -fsSL https://repo.lrob.net/apt/shieldlist/shieldlist.gpg -o /usr/share/keyrings/shieldlist.gpg
echo "deb [signed-by=/usr/share/keyrings/shieldlist.gpg] https://repo.lrob.net/apt/shieldlist stable main" > /etc/apt/sources.list.d/shieldlist.list
apt update && apt install shieldlist-controller

Key fingerprint B528 0C88 7016 8A94 40A9 05C4 3BB5 8113 CFAB 6DC5; dev in place of stable for our development builds; or the package from the release page, one per architecture, apt install ./shieldlist-controller_<version>_amd64.deb. The package creates the shieldlist user, a starter tree under /etc/shieldlist/controller (local listeners, no rules yet), installs the unit and starts the service; then, as root, shieldlist-controller api-token -name you for the first operator token. Upgrading restarts it; purging removes the tree and the store. An agent may live on the same machine (its own package, its own files).

By hand, elsewhere:

# 1. the binary, from the release page (SHA256SUMS beside it)
install -m 0755 shieldlist-controller_linux_amd64 /usr/local/sbin/shieldlist-controller

# 2. a user, and the configuration tree it may edit
useradd --system --home /var/lib/shieldlist/controller --shell /usr/sbin/nologin shieldlist
mkdir -p /etc/shieldlist/controller/{agents,parsers,rules,policies,reports,exemptions.d}
chown -R shieldlist:shieldlist /etc/shieldlist/controller

# 3. the service
install -m 0644 shieldlist-controller.service /etc/systemd/system/
systemctl daemon-reload && systemctl enable --now shieldlist-controller

Put the fleet's rule language in the tree — the same files an agent reads: parsers/, rules/, policies/, reports/, exemptions.yaml — and validate it: shieldlist-controller -t -config /etc/shieldlist/controller reads the tree with the agent's own loader and shows what every declared machine would get. Copying a working machine's /etc/shieldlist (minus agent.yaml and secrets.yaml) is a fine start.

Where it listens#

By default the agent transport answers every address on 17453 with a self-signed certificate the controller mints under its state directory (agents pin its fingerprint, carried by the join token), and the API and the Web UI answer 127.0.0.1:17454 over plain HTTP. To reach the Web UI from elsewhere, put a TLS-terminating proxy on the same machine in front of 17454 — the controller believes X-Forwarded-For and X-Forwarded-Proto only from a loopback peer. With nginx and Let's Encrypt:

nginx and certbot's nginx plugin first, if they are not there — apt install nginx certbot python3-certbot-nginx on Debian and Ubuntu, dnf install nginx certbot python3-certbot-nginx on Red Hat and its family. Without the plugin, --nginx is not a flag certbot knows.

Write it in /etc/nginx/sites-available/shield.example.net.conf (Debian and Ubuntu) or /etc/nginx/conf.d/shield.example.net.conf (Red Hat and its family, where there is no symlink to make):

server {
    server_name shield.example.net;
    listen 80;
    listen [::]:80;
    location / {
        proxy_pass http://127.0.0.1:17454;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 300s;
    }
}

Then enable it, check it and reload — on Debian and Ubuntu the symlink is what enables it:

ln -s /etc/nginx/sites-available/shield.example.net.conf /etc/nginx/sites-enabled/
nginx -t && systemctl reload nginx

certbot --nginx -d shield.example.net --redirect does the rest: it adds the certificate, the listen 443 ssl lines and the redirection from 80, in place, and renews on its own. The name must already resolve to this machine — certbot proves you own it over the port 80 you just opened.

proxy_read_timeout matters: the Web UI holds long polls, and nginx's default of 60 seconds cuts them. Through a firewall: the agents' port must be reachable from every protected machine to this controller — outbound from the agent, inbound here. Nothing goes the other way: the controller never opens a connection to an agent, so an agent behind NAT with no inbound rule at all works. The console's port (443 through your proxy) is for people, not agents.

Agents keep talking to 17453 directly; a machine whose egress allows only 443 can have listen.agents bound there, and tls.cert/tls.key name the operator's own certificate when the controller sits on a public name.

Why the X-Forwarded-For line matters (do not skip it)#

The public unblock page treats the visitor as their address: a person unblocks only the IP their request came from, and the page's abuse limits are counted per address. The controller learns that address from X-Forwarded-For, and — because the header is trivially forgeable — it believes it only from a loopback peer and takes the last element of the list (the one the proxy in front appended).

So the proxy in front must set that last element to the real client and never let a client-supplied value be the last one. proxy_set_header X-Forwarded-For $remote_addr; (above) does exactly that: it replaces the header with the connecting peer, discarding anything the client sent. $proxy_add_x_forwarded_for is also safe (it appends the real peer at the end). What is not safe:

• passing the client's header through unchanged (e.g. proxy_set_header X-Forwarded-For $http_x_forwarded_for;), or omitting the line while another upstream already trusts a client header — then the last element is attacker-chosen;
• a second proxy or CDN hop in front of nginx: whatever that hop puts in X-Forwarded-For becomes the trailing element nginx forwards. If you run behind a CDN, terminate its forwarded-IP header explicitly and re-set X-Forwarded-For to the real client at the nginx that talks to the controller.

Get this wrong and a visitor can set X-Forwarded-For: <someone-else> to lift another address's ban and to spend someone else's rate budget. Get it right — one trusted nginx on loopback that overwrites the header — and a client cannot choose its own address. The same rule governs X-Forwarded-Proto (used to know the request arrived over TLS).

The first person, the first machine#

shieldlist-controller account -name you      # asks for a password; -totp enrols a one-time code
shieldlist-controller api-token -name deploy  # a token for scripts and for shield
shieldlist-controller token                   # a join token for one machine, valid a day

Twelve characters at least, and there is a reason it is strict: an account is never locked out, however many times its password is refused. Locking accounts would hand an attacker a way to lock you out; the refusals are logged instead, and a rule of your own can ban whoever collects them. A short password with no lockout behind it is the one combination that does not work.

These commands open the store directly, hence the service user. Then on the machine to enrol — running an agent that speaks the protocol (0.0.4 and newer):

# /etc/shieldlist/agent.yaml
mode: enrolled
controller: https://shield.example.net:17453

systemctl restart shieldlist-agent
shield enrol <join-token>

Within seconds the machine appears in the Web UI with the revision it holds; the file agents/<hostname>.yaml is optional — a machine without one runs the defaults.

Reloading#

Edit any file, then systemctl reload shieldlist-controller (SIGHUP), shield reload, or the Web UI's Reload and push: the tree is loaded whole or refused whole with every error, and a new revision reaches every connected machine at once. Edits made through the Web UI, the API or shield are edits of the same files, so a git diff in the tree shows them like anyone else's.