Shieldlist Agent 0.0.9

What it is

The agent is what actually protects a machine: it tails its logs, scores what it reads against your rules, and blocks the offenders in the kernel — by itself, with no database and no daemon beside it. It works alone with local files, and it can be enrolled with a controller to be driven with the rest of a fleet.

What you gain over 0.0.3

Mostly speed and steadiness: a large ban set no longer costs you the loop, a flood no longer stalls detection while the firewall is being written, and a restart's restore takes seconds instead of minutes.

Then what came with the fleet: enrolled mode, a simpler configuration format, rules that count by site or by origin, lists that block whole networks, and a test mode that decides like production and bans nobody.

Everything that changed

Faster, and steadier under attack

• Single addresses live in plain firewall sets and only ranges in interval sets: listing an interval set dumps all of it before any change, so with a hundred thousand bans every call cost the best part of a second, stalled each round and turned a restore into minutes.
• A refused batch is retried without the element the kernel names, then bisected — never one process per element — and the reason it was refused is logged.
• Orders and lifts from a controller are applied as one pass: one transaction, one firewall batch, one log line. Thousands applied one at a time used to freeze evaluation for minutes.
• The sync sends what changed instead of scanning the store, and the machine's own counters are published rather than recomputed.
• A query that walked the whole sanctions table on every sync was split into two indexed halves merged in order.
• The bans in force come from the published aggregates, not a table scan on every sync.
• WatchKeep in two passes: count every group, build lines and progress only for the survivors.
• The control loop's questions are answered the moment they arrive, not on the next snapshot.
• A permanent ban's timeout is written in days — the kernel refuses nine digits of seconds, and long bans were silently refused.
• Lifting keeps a kernel element another live sanction still needs; every value entering the store is canonical, so two rows can never fight over one element.
• The unit asks for a CPU weight instead of a hard quota: a quota rationed the restore and the enforcement exactly under attack.
• The restore says where its time went — store, kernel, waiting for the loop.
• One named drop counter read by reset, instead of listings that dump the sets.
• Aho-Corasick prefilters for the rules whose patterns are long lists of literals.

Detection

• Hits survive a restart: the rules' windows are snapshotted to disk and come back, and they cross an in-place reload.
• {target} in a log input's path names the site every line of that file is about — a field, a filter and a facet, without allocating; a fixed file can declare its target.
• Rules count by origin: countries, asns and their exceptions, with the geo databases pulled from the controller or named locally.
• targets / except_targets: count only what was aimed at a site, or everything but.
• A severer rule's conviction extends and renames an existing sanction instead of stacking a second one.
• tighten: a returning offender needs less evidence per past episode.
• Evidence lines keep what the parser captured, by name — the username tried, the path asked for, the site aimed at.
• Lines two web servers write for one request are counted once.
• A hit is a line some rule scores against the offender: a successful login weighed against a ban is not counted as one.
• Several thresholds per rule, and a verdict restarts every threshold's counter for that offender.
• The machine is never an offender: loopback and its own interface addresses are exempt in the core, and a non-routable address is never reported.

Enrolled mode

• shield enrol <token> attaches the machine: the bundle is validated before it is swapped in and reloaded in place — the pipeline is rebuilt between rounds, the store and the kernel untouched.
• A received configuration that no longer loads is set aside at start; the machine runs its own files until a good one arrives.
• The sync carries what it decided, who it is counting and how close they are, its running view (memory, lines and hits per minute, pending verdicts, shed, errors) and its clock.
• It is nudged the moment a round bans or hits, so the console shows this second's decisions, and falls back to its interval on a quiet machine.
• Suspects travel as a delta, with their lines; a suspect new to the list sends its last lines whole, so the controller always holds lines for what it shows.
• Bans the operator orders are applied like any sanction — same checks, same enforcement, recorded as manual with the reason as evidence, reported when a preset is named.
• Blocked lists arrive as data and load into their own firewall sets off the main loop, carved around the exemptions; shield why and shield status say when an address is covered by one.
• The controller's log inputs replace the local ones; exemptions by country or AS are assembled for the machine.
• Detect-only from the controller is remembered across a restart, and arming says when a restart is needed.
• Nothing executable ever comes down that link: a controller sends configuration, never code.

The configuration made simple

• One rule, one file, five blocks; parsers named or written inline; thresholds as 5/10m, as a list, or scaled by level (instant, strict, standard, lenient).
• Packs group rules; -convert rewrites a 0.0.x tree once.
• state: test on a rule or a machine: decides like production, bans nothing.
• share: no keeps a rule's bans off the fleet.
• An old-format file is named once and dropped, not half-validated.

Reporting

• The comment ends with the episode's minute, first report and renewal alike, so a repeat offence is actually stored by the provider.
• Sends may run in parallel; an unreachable provider is no longer read as a spent daily allowance; reporting can be switched off without losing the key.
• The rule is unchanged: one chance at the moment of the decision, and shield report -last <duration> is the manual catch-up.

Commands and diagnostics

• shield status <key> prints one value alone; new_today counts from midnight in the machine's own zone.
• shield tail follows decisions and their evidence live; shield why explains one address; shield ranges shows the busiest buckets.
• SHIELDLIST_DEBUG=1 writes a debug log with each sync's cost; SHIELDLIST_PPROF=<addr> opens Go's profiler on a loopback address.
• Per-parser and per-rule cost sampling, published in health and shown by shield stats — a rule at zero hits is visible.

Operations

• Debian packages for amd64, arm64, armhf and riscv64 from our APT repository; a hardened unit; logrotate; man pages; third-party licence notices.
• The unit never wants nftables.service: Debian's own unit flushes the whole ruleset when it starts, taking every other firewall's rules with it.

Security

• A hardening pass on the geolocation database reader against crafted files.
• https:// required for a controller address.
• The packaged unit no longer blocks shield exempt.

Early days

Formats can still move between releases and an upgrade can ask for a re-enrolment. Start on test servers, keep what protects you today, and tell us what breaks.

Credits

Geolocation: DB-IP Lite (CC BY 4.0). Go modules: THIRD_PARTY_LICENSES. Detection rules translated from public sources keep their attribution in the rule files.

Shieldlist agent v0.0.3 — the reporting release. The agent can now tell the world about the offenders it bans, watch itself do it, and post its decisions to your own endpoints, while several sharp edges found by running it armed on a production host were filed off.

Abuse reporting (AbuseIPDB)

• The agent submits the offenders it bans to AbuseIPDB under your API key. Off unless you arm it: reporting.provider in agent.yaml is the only switch, the key and your daily_limit live in secrets.yaml (0600), and what each report says is a preset file you own — reports/*.yaml, referenced by a rule's report_as:. An agent in detect-only never reports anybody.
• Redaction is default-deny: a preset's include: list is the only way a parsed field can leave the machine, the comment is composed the moment the ban is decided and stored with it (shield why shows the exact text), and the validator refuses a preset naming a field its rule never captures. A comment describes the offender's behaviour — one sample per included field with a distinct count — and never your thresholds, your server's configuration, or anyone else's usernames and mailboxes.
• A report gets one chance, at the moment of the detection, and it leaves on the ban's heels: the decision loop nudges the sender as soon as a round's bans are durable. Whatever stops a report — allowance spent, provider down, key refused — the reason is one log line and the sanction stays on record, still owed. Nothing is retried on a cadence and no backlog drains by itself.
• shield report -last 6h sends the past on purpose: it counts what is owed in the window you name, shows the estimate, asks, then submits freshest first at full speed, stopping cleanly when the provider says the allowance is spent. The allowance is watched from both ends — your daily_limit binds absolutely, the provider's rate-limit headers are read on every response, the reset is midnight UTC, and a bounded probe notices an upgraded plan or another machine freeing room.
• Addresses that are not publicly routable are never submitted, and the machine's own addresses are never offenders in the first place (see below).

Outbound hooks

• hooks: in agent.yaml POSTs one JSON document per enforced ban to any URL you name — a chat relay, a SIEM, your own dashboard — carrying the ban's facts and the same default-deny comment a report would, never a raw log line. Hooks are local configuration by invariant: a controller can never push one. One delivery attempt each, right behind the ban, outcome logged, nothing in detect-only.

Watching it: shield tail, and a store that counts returns

• shield tail follows decisions live, each with the log lines that convicted it, plus again lines when a sanctioned offender keeps coming back, reported lines when the report leaves (with the provider's abuse score), and the provider's own words when it refuses. One terminal shows the whole story.
• Sanctions now carry hits and last seen (schema 11, migrated automatically): every absorbed return is counted, evidenced (bounded per sanction), and said — an address that keeps coming back is visible coming back. shield bans grows HITS, LAST and REPORTED columns; shield why tells each sanction's full story including its report's fate; last seen is the log's clock, not the verdict's.
• shield status answers more without costing more: reporting counters (owed, sent, duplicate, refused) and the exemption count in the default output, published by the agent's own aggregates pass — plus every loaded rule listed, the zero-ban ones included, because a rule that never fires should be visibly at zero, not absent. Reports piling past ten thousand owed is one edge-triggered WARN.

The machine itself is never an offender

• Loopback and the machine's own interface addresses cannot be sanctioned by any rule, built into the core rather than left to an exemption file — no configuration of this program can firewall the machine it runs on. Loopback events are dropped at evaluation, before any state exists: a local service authenticating against another (a webmail against its IMAP server) is plumbing, not an attack, and the real client is in the web application's own log.

Faster list parsers

• prefilter_any: — a parser may now gate on a list of tokens instead of a single literal, answered by a multi-literal automaton in under a microsecond. This is what makes big list parsers affordable: a 683-entry User-Agent list that cost ~920µs per line ungated now costs ~1µs on the lines it ignores, a measured ~3300× improvement. Exactly one gate per parser, and the gate only ever admits lines the pattern then judges.

Also

• Log lines about reports read like the decision lines they follow — rule, action, bare copyable address — and every egress outcome is one line with the provider's exact words where they matter.
• The systemd-less quick start, examples and wiki are updated; docs/CONFIG.md gains the full reporting, hooks and prefilter_any reference.

Upgrading from v0.0.2: replace the binaries and restart; the store migrates itself (schema 9 → 11). The reporting block, presets and hooks are opt-in — nothing changes until you write them.

Three fixes from putting v0.0.1 on a real, busy production machine — a hosting node with a database, twenty PHP pools and twenty-three sites. Every one of them was invisible on a quiet test box.

The agent's memory ceiling no longer follows the machine

The agent used to subtract a reserve from the machine's currently available memory, so that it would "give room back" as the protected service grew. On a machine that genuinely uses its memory it did the opposite of its intent: a nightly job pushed available memory under the reserve, and the agent's budget collapsed from 929 446 buffered events to 16 384 — while the process was holding 16 MB. It shed a capacity it had never taken, because of what something else was doing at midnight.

limits.memory is now a fixed ceiling and the whole of the policy. The agent reads MemTotal — enough to resolve 25% — and nothing else. It does not watch free memory and does not shrink when something else grows.

A program whose behaviour depends on the rest of the box is one whose behaviour cannot be reproduced, and one that competes with every other program for the same slack. Tell this one what it may have; dividing the rest is the administrator's job, and MemoryMax in the unit is the tool for it.

Exemptions were being silently discarded

loadList decoded straight into the destination slice, and yaml.v3 replaces a slice rather than appending to it. So the second exemption file read discarded the first: the moment shield exempt created exemptions-cli.yaml, every exemption written by hand stopped existing — an operator's own infrastructure included — and nothing said so. The count in shieldlist-agent -t was the only trace, and only if you knew what it should have been.

If you have ever run shield exempt, check the count:

shieldlist-agent -t

exemptions.d/

The useful exemption lists are generated: the ranges Google and Bing publish run to a couple of thousand entries and are refreshed from a source. Pasted into exemptions.yaml they bury the dozen a person maintains, and the next refresh has to preserve them.

/etc/shieldlist/
  exemptions.yaml         yours
  exemptions.d/*.yaml     one file per generated source
  exemptions-cli.yaml     what `shield exempt` wrote

All of them load, and each entry remembers which file it came from.

The agent writes its own log

Under systemd the log goes to the journal, which is right — but /var/log is the first place anyone looks after starting a security daemon, and finding nothing there reads as "it did not start".

log_file: /var/log/shieldlist/agent.log

Set in the shipped configuration, with the same lines as journalctl -u shieldlist-agent. The lines written before the configuration was read are replayed into it, so it holds the whole run. No logrotate snippet ships with it: the file is identified by (device, inode) like every other file the agent follows, so a rename is noticed and the file reopened. The unit gains LogsDirectory=shieldlist, without which ProtectSystem=strict leaves /var/log unwritable.

Upgrading

install.sh again — it replaces the binaries and touches no configuration. Which means an existing install needs log_file added by hand if you want the file.

Still pre-1.0 and still unsigned: the checksums prove the download was not corrupted and nothing more.

The first real release of the standalone agent.

It tails the log inputs you declare, evaluates your rules, applies sanctions to the machine's own nftables, and keeps every decision in a local store that survives restarts. One binary, plain YAML, no database daemon, no external service on the machine it protects. shield ships alongside for status, bans, why, unban, exempt and replay.

Install

curl -fsSLO https://git.lrob.net/Shieldlist/shieldlist-agent/releases/download/v0.0.1/install.sh
less install.sh          # one file of sh — read it before running it
sh install.sh

It enforces nothing until you say so

A fresh install writes a starter configuration in detect-only: every rule records what it would have done, and nftables is never touched. Run it beside fail2ban, CrowdSec or whatever already guards the machine, read a day of it —

shield status               # what it would be holding
shield bans                 # which addresses, by which rule
shield why 203.0.113.9      # the log lines that convicted one of them

— then set state: on in /etc/shieldlist/agent.yaml and restart.

Exempt yourself first, in /etc/shieldlist/exemptions.yaml.

What is in it

• Detection: literal-prefiltered RE2 parsers, rules that score weighted evidence across parsers within a window, negative weights so good traffic can exculpate, distinct counting, group_by ip and range:/n.
• Sanctions: duration ladders that climb only when an offender returns after a ban ran out, port scopes per rung, decay measured from a ban's end, exemptions (with until:), trust weights on origins.
• Enforcement: one inet table, prerouting, ct direction original so machine-initiated replies are spared; persist-first, so losing the agent never unbans; a restart rebuilds the data plane from the store in the background while the agent already watches.
• Reading the past: shieldlist-agent -dry-run 24h [-rule NAME] judges by window and ban duration — a crossing counts only if the sanction it earns would still be in force now.
• Bounds: a memory ceiling the agent holds itself to (half the machine by default), measured per-rule and per-parser cost, ordered shedding, and figures for all of it in shield stats.

Proven on a 2-core VPS: ~68k lines/s at one core, and 5.1M distinct addresses banned without losing one. What nothing can bound — nftables set memory is kernel memory — is written down in docs/RESOURCES.md.

It checks the kernel before trusting it

At startup the agent builds its own table under a throwaway name, with no hook — so nothing in it is ever evaluated against a packet — puts elements in it with timeouts and concatenated address·port keys, and removes it again. What it proves is what the agent will actually ask for, because the probe renders the same script.

A kernel that refuses gc-interval gets a warning and a working agent; an expired element stops matching traffic whether or not it has been collected. A kernel that refuses anything else stops the agent at startup with one sentence naming what it refused, rather than a parse error out of a child process in the middle of an attack. shieldlist-agent -t runs the probe too — that is where an operator asks whether a machine can enforce, and the only place an agent in detect-only would ever find out.

Verified on Debian 12 (kernel 6.1, nft 1.0.6) and Debian 13 (kernel 6.12, nft 1.1.3).

Its log is where you will look for it

Under systemd the log goes to the journal, which is right — but /var/log is the first place anyone looks after starting a security daemon, and fail2ban and CrowdSec both write a file there. So does this now: log_file: /var/log/shieldlist/agent.log, set in the shipped configuration, with the same lines as journalctl -u shieldlist-agent.

No logrotate snippet ships with it. The agent identifies the file by (device, inode) like every other file it follows, notices when logrotate has renamed it away, and reopens.

Pre-1.0, and honest about it

• Releases are not signed. The checksums are served from the same host as the binaries: they prove the download was not corrupted, and nothing more. Signing is an open decision (SPEC §11).
• The configuration format may still change.
• Shipped rules cover sshd only. Rules for web, mail and panels are being written; docs/CONFIG.md and the wiki's Writing rules page are how to write your own meanwhile.
• The central controller does not exist yet. Standalone is a mode, not a fork: the same files carry over when it does.

Requires Linux with nftables. Builds for amd64, arm64, armv7 and riscv64; no cgo.

Shieldlist Controller 0.0.9

What it is

The controller is the brain of a Shieldlist fleet: your servers report what they see, it keeps the history, decides what each machine runs, and gives you one console to run all of it. You host it — there is no cloud service in the middle, and every protected machine keeps defending itself if the controller is down.

What you gain over 0.0.1

One console for the whole fleet instead of one shell per server: what each machine decides, what it is counting right now, and why an address is blocked.

A rule language you can read and edit from a screen, with a diff before anything is pushed, and a test mode that decides exactly like production and bans nobody.

Lists: block an autonomous system, a country, a set of prefixes or a public blocklist as one firewall set, carved around your exemptions.

Abuse reporting done for you, repeat offenders included, and a public page that takes unblock requests off your inbox.

A store that answers in milliseconds on a year of history, and a controller that starts serving in about a second.

Everything that changed

The fleet

• Enrolment with a join token; the tree is assembled per machine and per agent version, only the rules that differ are rewritten, and the result is validated by the agent's own loader before it travels.
• Packs are the unit of assignment: a pack is a list of rules, a server names its packs, nothing else reaches it.
• Sharing by pack: machines running a common pack accept each other's decisions; a ban lifted at its origin takes its copies down; a machine joining the sharing is served the still-active backlog.
• A ban by hand on the fleet is ONE order — every machine applies it at its next sync, every machine enrolled later at its first, with what is left of the duration; a fleet lift cancels it everywhere.
• A ban given from a detection carries that detection's log lines, so the manual sanction holds its evidence on every machine and reports with those facts.
• Re-ask a machine for its history (POST /v1/agents/{host}/resync); a re-enrolled machine's fresh numbering no longer overwrites its past.
• Per-server display names; a declared but not yet enrolled server is marked as such.
• The controller reports on its agents' behalf, or leaves it to each machine — per fleet and per server.
• Clock skew measured on every sync, with warnings per agent.
• shieldlist-controller status, token, agents, why, unban, api-token, account.

The rule language, and editing it

• The configuration made simple: one rule, one file, five blocks — what it recognises, from how much, what it bans with, who it notifies, how it is reported.
• Levels scale the numbers: instant, strict, standard, lenient.
• Editors in the console for rules, patterns, policies, report presets and packs, with a pattern tester, a diff preview, and "edit as file" for the raw YAML.
• Every edit is a file edit, validated whole and reverted if it does not load.
• Rules can count by origin (countries, asns, and their exceptions) and by what was aimed at (targets, except_targets).
• A {target} segment in a log path names the site every line of that file is about; a fixed file can declare its target.
• Servers → Logs shows a machine's declared log inputs.
• Prod and test are two axes: a rule or a server in test decides like production and bans nothing; the console says «would ban» where it means it.

The feed and the tickets

• Sanctions and hits as tickets, not tables: a live feed filterable by state, server, rule, country, network, target, username and dates, refreshed by the second, pausable, with the requests behind each decision on demand.
• Hits — the offenders being counted but not convicted — with their progress toward each threshold, their log lines, and the ones that came to nothing kept an hour and shown greyed.
• One kind at a time or all at once: All, Bans, Hits, Detection-only (test), Expired, Exempted.
• Every ban says how long: in force with what is left, ticking to the second; over, with its length; lifted; a detection with what it would have been.
• Cards laid out by areas — flag and address left, the moment on top, the facts beside — with the network name, the target, the username, and the AbuseIPDB link.
• A returning offender says so; a new sanction flashes as an overlay that fades over the ticket's own colours.
• Copy buttons on the address and on the log lines; log boxes keep where you scrolled and open at their end.
• Evidence lines carry what the parser captured, by name — statistics on any field, a filter on any value.

The map

• A world map of the sanctions under the current filters, drawn in-house from Natural Earth outlines.
• Attacks ripple out of their country, then out of their city when a city database is loaded; your own servers sit on it at their own place.
• Rays from the attacker's city to the server that was hit, carrying the volume — one head per departure, in file, landing on the server's point.
• A dot for a country too small to see; countries with no outline get a spot.
• Bans or hits, your choice; arcs can be turned off; full screen; the height follows the chart's.

The chart

• Bans, detections and hits stacked over the window you choose, from one-second steps to a year.
• The line of bans in force — distinct addresses banned at each step — over the whole history.
• Sixty round bars per window, a soft log scale with a ruler, hover counts, and a click that narrows the whole page to that moment, seconds included.
• An incremental tail: the live edge is merged in rather than reloading the series.

The side panel

• Servers, countries, networks, targets, rules and usernames tallied under the current filters, sanctions and hits side by side.
• Pins keep what you watch first; an eye sets an entry aside as a lens, not a setting; multi-select with ctrl/⌘ and with checkboxes on a phone.
• Sortable by bans or hits, compact thousands, and the panel slides over the page on narrow screens.

Windows and dates

• The window is a wheel — from the last minute to everything on record — turned by mouse, wheel or finger, and it drives the feed, the map, the chart and the panel at once.
• Custom dates with times and presets (today, 7 days, this month) that keep running to now.
• Saved views per account: a set of filters under a name, following you between browsers.

Lists, exemptions, published blocklists

• Lists are named sets of addresses — an AS, a country, prefixes, or a published URL — with a role (blocked or exempted) and a scope (the fleet, a pack's servers, or named servers).
• A resolved list travels as data and becomes one nftables set on each covered machine, with its own counter, carved around the exemptions — no ban orders, no history buried under thousands of rows.
• Published blocklists can be followed by URL, fetched and bounded, with a catalogue to start from (Romain Marcoux's malicious-ip list, MIT, credited).
• Exemption sources for the crawlers everyone keeps: Google, Bing, Apple, DuckDuckGo, Cloudflare, Fastly, Ahrefs, Stripe, UptimeRobot, Pingdom, OpenAI, Jetpack, and the private ranges — each on or off, with its own scope and refresh.
• Networks and countries picked by name or number from the ASN database; any origin dialog can block or exempt what it shows.
• Blocked and Exempted as two tabs, with exact figures, sortable, dated to the minute; "Banned by hand" grouped one row per address.
• Check addresses: paste up to a thousand and see, for each, the sanctions, every list that blocks or exempts it with the entry that covers it, and its network — the answer shield why gives on a machine, for a batch.
• The address dialog says which list covers an address, so an address inside a blocked prefix is no longer reported as "nothing on record".

The network view

• What each prefix of a network did over a window — bans and hits apart, share in addresses — and the prefix an address belongs to.
• Ban the address, its prefix, or the whole network, from the rows that say which one attacks.

Abuse reporting and reputation

• The controller runs the same one-chance sender on its agents' behalf, eight in flight, resuming behind its last report after a restart.
• A report's comment ends with the episode's minute: without it, a provider answers 200 and stores nothing for a repeat, and every recidivist was lost.
• A return after a report is reported again once the renew interval has passed, with the latest hit as its time.
• Settings → AbuseIPDB: the key, the daily plan, the allowance left, who reports for whom, and a switch that stops reporting without losing the key.
• An address's reputation on demand; your servers' own addresses checked daily.
• A manual ban is reported with the facts of the detection it was given from.

The public unblock page

• An optional page where a blocked visitor lifts their own address, asks for another, or asks to be whitelisted; requests are approved or refused from a Requests menu with a badge.
• The visitor's address found on both families through your own resolvers, their reverse name, network and country, and — if you allow it — the lines that earned the block.
• Branding, contact, language (en/fr), built-in captcha or Turnstile; refusals logged so a rule can act on abuse of the page.
• Mail: one SMTP relay configured once, each account choosing what it is notified about, with a test mail.

Where addresses are from

• An in-house MaxMind-DB reader and the free DB-IP Lite databases (CC BY 4.0): country, AS, and optionally city, fetched monthly or pointed at your own files.
• The databases are served to the agents, so rules and exemptions by country or AS run on the machines themselves.
• Origins resolved at ingest and backfilled; they drive the filters, the facets, the map and the search.

Statistics, history, maintenance

• A statistics page: countries, networks, rules, servers and targets over 7, 30, 90 or 365 days or any range, exportable as CSV, with a clicked bar filtering the feed to that window.
• Below-threshold suspects kept as episodes with their raw lines, so a past window shows who was being counted then — not who is being counted now.
• Retention set per kind from the console (evidence, samples, hits, hit lines, sanctions), swept daily; Maintenance says what the store holds and can flush or compact it.
• Counters maintained by database triggers at ingestion rather than scanned on demand.

Accounts and the panel itself

• Sign in with a name and a password, a one-time code if you want one, or an API token; sessions slide with activity and their length is a setting.
• Rename yourself, change your password and e-mail, turn two-factor on or off from the account dialog.
• Theme toggle in the topbar (auto, dark, light) remembered with the account, alongside the time zone and the display choices.
• The ban colour means one thing everywhere — the rail, the counters, the map, the chart, the headers — and hits keep their own.
• A ✕ empties every search and filter field; ⓘ explains a setting where it is; hints wrap and stay inside the box.
• The console notices when the controller under it was updated and offers to reload; a daily check tells you when a newer release exists.
• It works on a phone: stacked map and chart, menus as centred sheets, dialogs inside the screen, a film over the page while the panel is open.

Operations

• Debian packages for amd64, arm64, armhf and riscv64, from our APT repository; a service user and a starter tree created on install; a hardened systemd unit; its own log directory and logrotate; man pages; third-party licence notices.
• GET /v1/vitals, Settings → Vitals and shieldlist-controller status read the same pulse: CPU, memory, sync cost, queue, errors — and the agents' own errors are one click away.
• SHIELDLIST_DEBUG=1 writes a debug log with each sync's cost; SHIELDLIST_PPROF=<addr> opens Go's profiler on a loopback address.
• Boot no longer waits on the heavy loads: the geo databases and the counters load in the background, each saying when it started, how long it took, or what is degraded if it failed.

Performance

• Write transactions queued in Go, WAL with synchronous=NORMAL, page cache and mmap sized from the machine — a large batch no longer loses the lock to short writers until it gives up and is resent every second.
• Queries pinned to their indexes on every periodic path: the sender's cursor, the changed-rows scan, facets grouped so the start index is kept, field facets behind a covering index.
• Bulk work in one transaction, never row by row on the big table.
• What the console polls is memoised; one round at a time; heavy views at most every fifteen seconds.

Security

• Two read-only audit passes: constant-time sign-in, configuration files written 0600, a bounded join token, the SSRF guard extended to carrier-NAT space, a captcha whose digits are not in the markup, documented guidance for running behind a reverse proxy.
• The console renders text attackers wrote: everything escaped, a strict CSP, cookies HttpOnly and SameSite=Strict, and a write needs a header a cross-site form cannot send.
• Accounts are never locked out by failed sign-ins — a lockout is a way to lock you out. Failures are logged so a rule can ban whoever is trying.

Early days

Formats can still move between releases, an upgrade can ask for a re-enrolment, and one controller serves one fleet on SQLite. Start on test servers, keep what protects you today, and tell us what breaks. Support is best-effort; known issues are on the forge.

Credits

Geolocation: DB-IP Lite (CC BY 4.0). Map outlines: Natural Earth. Blocklist catalogue: Romain Marcoux's malicious-ip list (MIT). Exemption sources: the ranges published by each operator. Go modules: THIRD_PARTY_LICENSES. The full list is under Settings → About.

First prerelease of the controller: the control plane for a fleet of Shieldlist agents. Much of it will move; the protocol is versioned and additive from here on.

What it does. Assembles each machine's configuration from a shared tree of parsers, rules, policies, presets and exemptions plus one declaration per machine (level, state, who may lift, who reports, fleet switches, per-rule overrides), validates it with the agent's own loader, and pushes it to enrolled agents the moment it is loaded — over a long-lived, agent-initiated connection, with polling as the fallback. Records every sanction the agents decide, redistributes them across the fleet according to each machine's switches, and carries the operator's lifts back to the machines within seconds.

Operator's faces. A JSON API on its own listener with scoped tokens; a console served by the same binary — fleet, machine settings and overrides, bans with lift and exempt from any row, rules, address lookup, exemptions and published sources, audit, people and tokens, configuration; local accounts with a password and an optional one-time code; and shield as a client of the API (-controller URL -token T) with the same verbs on the whole fleet. Every write through the API or the console is a write to the configuration files, reloaded whole or reverted with the loader's errors as the answer.

Published exemption sources. Googlebot, Google's special crawlers, Bingbot, Applebot and Cloudflare, fetched on command or on a schedule when named in controller.yaml, materialised as generated files under exemptions.d, refused when a list looks like nothing a crawler publishes.

Transport. HTTP with protobuf bodies (proto/shieldlist/v1/agent.proto), TLS self-signed by default with the fingerprint carried by the join token and pinned by the agent, or the operator's certificate, or plain HTTP behind a proxy that terminates TLS. Both sides exchange versions on every request; a refusal names both and says which side is behind. Requires an agent that speaks the protocol — the next agent release; agents 0.0.3 and older cannot enrol.

Not yet. Log ingestion and the event store, controller-placed rules and fleet replay, controller-side abuse reporting, owner scoping, packages. State is SQLite; the schema is versioned and migrated in place.

Assets: static binaries for linux amd64, arm64, armv7 and riscv64, the systemd unit, SHA256SUMS. Docs: docs/SPEC.md, docs/CONFIG.md ("The controller"), packaging/README.md.