Shieldlist

Configuration file reference

Cette page n'existe qu'en anglais pour le moment.

YAML, one format only (SPEC §5.9). Hand-written config is the primary interface: flat keys, a default for everything that has a sensible one, compact scalar forms. The dry-run validator is the ground truth for this document: shieldlist-agent -t -config <dir> reports every problem at once, each with its file and line.

Layout

/etc/shieldlist/
  agent.yaml            # this machine (optional; absent = standalone defaults)
  parsers/**.yaml       # one parser per file — sub-directories by source (nginx/, sshd/…) for people
  rules/**.yaml         # one rule per file
  policies/**.yaml      # one ban policy per file
  reports/**.yaml       # one report preset per file (abuse reporting)
  packs/*.yaml          # optional: a pack = a named list of rules (what a controller gives a machine)
  trust.yaml            # optional list
  exemptions.yaml       # optional list, yours
  exemptions.d/*.yaml   # optional lists, one file per source
  exemptions-cli.yaml   # optional list, written by `shield exempt`
  secrets.yaml          # credentials — 0600, never deployed with the rest

The four language directories are read flat and one level deep: a sub-directory is an ordering for people (a parser's source, a rule's category) and means nothing to the loader — one namespace, a name is defined once wherever its file sits. On a machine of its own every rule in the tree runs; packs/ is documentation there and the unit of assignment on a controller.

A file in the format of the 0.0.x releases (match:, pattern:, category:, thresholds: per level, decay:, log_inputs:) is named by the loader and refused with one sentence: run shieldlist-agent -config DIR -convert NEWDIR (or the controller's -convert) — it rewrites the tree, comments kept, says what it changed, and the new tree is moved into place when it loads.

exemptions-cli.yaml is shieldlist's own file, beside yours so that neither rewrites the other's. It is ordinary YAML in the same format: read it, edit it, delete entries from it with whatever you normally use.

Canonical style

The remote control (SPEC §5.9) writes files in one style, and hand-written files are expected to follow it: two-space indentation, list entries as - blocks separated by one blank line so each entry reads as a unit, comments attached to the key they describe. Names are lower-case letters, digits and dashes.

agent.yaml

mode: local             # local (default) | enrolled
controller: ""          # base URL, required when enrolled
state: on               # on (default) | test
level: standard         # scales every rule's threshold here: instant | strict | standard | lenient (see Rule)
log_file: /var/log/shieldlist/agent.log   # as well as standard error

logs:
  - path: /var/log/nginx/access.log   # a file…
    log: http                          # …of this kind of line
    ports: [80, 443]                   # the service's ports here — what `ports: service` in a policy bans
  - path: /var/www/vhosts/system/*/logs/proxy_access_ssl_log   # …or many
    log: http
  - unit: ssh.service                 # …or a journald unit — exactly one
    log: ssh
    ports: [22]

limits:
  memory: 50%           # the ceiling the agent holds itself to: a share of
                        # the machine (the default is 50%) or a size
                        # ("512MB", "2GB"); see the agent's docs/RESOURCES.md

log_file is where the agent writes its own log, as well as to standard error — which under systemd is the journal, and has the same lines. Unset means the journal alone.

It exists because /var/log is the first place anyone looks after starting a security daemon, and finding nothing there reads as "it did not start". The agent reopens the file by itself when logrotate takes it away — it identifies it by (device, inode), like every other file it follows — so no logrotate configuration ships with it. The shipped systemd unit uses LogsDirectory=shieldlist-controller, which is what makes /var/log/shieldlist-controller writable under ProtectSystem=strict (its own directory, not the agent's /var/log/shieldlist: two units with two users cannot share one — the agent's unit took it back at every start).

log is what ties an input to its parsers: every parser reading that kind of line (log: http in the parser) runs over the input's lines. One source per kind per machine — reading the same lines from a file and a journald unit would count everything twice, and declaring the same input twice is refused for the same reason. Two patterns matching the same file is not: it is read once. ports are the service's ports on this machine — an sshd on 2222, a site on 8443 — the one place they are true; a policy with ports: service bans exactly those, and bans every port when none are declared (said once at start).

A path may be a pattern (*, ?, […]), which is how a machine hosting sites is watched at all — a panel gives every site its own log directory, and sites are added by people who are not thinking about this agent. The pattern is re-read every fifteen seconds: a site created this afternoon is watched this afternoon, and its log is read whole, as any file appearing after the agent started is. A pattern matching nothing is said at startup rather than left to look like silence.

Enrolled: taking configuration from a controller

mode: enrolled
controller: https://controller.example.net:17453

Then, once, on the machine — with a join token minted on the controller (shieldlist-controller token):

shield enrol <join-token>

The agent keeps the credential it receives in its state store, pinned to the controller's certificate when the token carried its fingerprint, and from then on holds a connection to the controller: it presents the configuration revision it holds and receives a new one the moment the controller loads it, reports every sanction it decides, applies the fleet's sanctions and the lifts decided centrally, and sends its enforcement summary. It dials out only; nothing listens on the machine. Where a long-lived connection cannot be held, it asks every thirty seconds instead — same messages.

What comes from the controller: parsers, rules, policies, report presets, trust and the fleet's exemptions — the tree the controller assembled for this machine (its level and overrides applied), written under /var/lib/shieldlist/received/ so that shieldlist-agent -t and a person read the same files the agent runs. What stays local: agent.yaml, secrets.yaml, hooks, and your own exemptions*.yaml, which add to the fleet's. A new revision is applied in place — no restart; the rules' windows start over, sanctions and the kernel's sets are untouched. Until the first revision arrives, an enrolled agent runs on its local files. When the controller is away, it keeps enforcing on the last configuration received and says so once.

shield status shows both versions and where the conversation stands (controller_version, controller_revision, controller_sync, and controller_error when there is one); the controller's agent list shows the same from its side.

Running it beside what you already have

state: test puts every rule in test and keeps the agent off the data plane entirely: no nftables table, no restore, nothing added or removed. It decides, records and reports exactly as it would otherwise — shield status, shield why, the ban log — with every sanction marked unenforced.

That is how this is meant to be adopted: run it for a day next to fail2ban, CrowdSec or whatever guards the machine, read what it would have done, then set state: on. Doing the same thing by editing every rule is how one gets missed.

shield status and shield stats answer enforcement DETECT-ONLY first, before any count, because the one dangerous misreading is a screen full of bans on a machine that is not defending itself. While the agent is in that mode the CLI counts the records it made rather than the bans it holds — otherwise every figure would read zero next to a log full of decisions — and shield bans says so above its table.

The agent still checks that this machine could enforce: run shieldlist-agent -t and it builds the table it would use under a throwaway name, with no hook, and removes it. That is where a kernel that will not take some part of it says so, since an agent in test never finds out on its own.

Reading a log's past is off unless asked for:

replay_on_start: 6h             # read this much of the past at startup,
                                # enforce what would still be in force

A live agent joins the present. An operator who turns up after an attack began wants it to catch up, and restarting it is the tidiest way to ask. Only crossings whose sanction would still be running are acted on — window and ban duration. Files need their parser to declare a time_field; journald records its own time.

Parser

parser: ssh-auth-fail
log: ssh                        # the kind of line it reads — the word the machine's logs declare
prefilter: "Failed password"    # mandatory literal gate before the regex
regex: 'Failed password for (?:invalid user )?(?P<user>\S+) from (?P<ip>\S+)'

Named groups in regex become the parsed fields; (?P<ip>…) is what a rule bans. RE2 only. Ports are not a parser's business (see logs: in agent.yaml).

A parser may also say where the line carries its own timestamp:

time_field: time                # a capture group of the regex
time_format: nginx              # nginx, apache, syslog, rfc3339, unix,
                                # or a Go layout for anything else

Both halves or neither. A live agent never uses them — it stamps a line with the moment it read it, which is the same instant and costs nothing. They are what makes reading a log's past possible: replaying a week-old file where every line claimed to have happened now would have a windowed rule ban whoever appears in it.

syslog prints no year. The reader supplies its own, and a line dated ahead of the reader is taken as last year's — so a December log is still read correctly in January.

Point time_field at a timestamp the server writes, never at anything the client controls: a replay trusts the line's own clock, and an attacker who can write the timestamp can date their attack out of the window.

A parser needs exactly one gate: prefilter (a single literal) or prefilter_any (a list of tokens — the line must contain at least one, case-insensitively, before the regex runs). prefilter_any is what makes a list parser affordable: a 683-entry User-Agent list has no literal common to all entries, and ungated it cost ~920µs per line; the multi-literal gate answers in under a microsecond. The gate is a superset of the regex — it can only admit lines the regex then judges, never hide one from it.

The machine itself is never an offender — built in, not configured. Loopback and the machine's own interface addresses cannot be sanctioned by any rule, and addresses that are not publicly routable are never submitted to an abuse provider. No configuration of this program can firewall the machine it runs on.

Rule

One rule, one file, five blocks in the order things happen — what it reads, what it recognises, from how much, what it does, what it says:

rule: wp-login
description: Brute force on WordPress logins
category: web                    # where it is filed — web, mail, ssh, database, panel… (for people)
parsers: [wp-login-fail, wp-login-fail-apache]   # what it recognises (their `log:` says what it reads)
threshold: 5/10m                 # from how much: five hits in ten minutes
ban: human-ladder                # what it does: a ban policy (see Policy)
report: wp-bruteforce            # what it says: a report preset (default when absent)

Parsers. A bare name, a list of names, or entries with a weight — every hit adds its parser's weight (1 unless said otherwise; negative for traffic that argues against a ban), so one rule can add up varied behaviour:

parsers:
  - parser: wp-login-fail
    weight: 10                   # three of these alone would fire
  - parser: http-404
    weight: 3                    # or ten of these — or any mix
threshold: 30/10m

A regex may be written in the rule instead of naming a parser file — a rule with no reason to share its regex keeps regex and rule together:

rule: wp-xmlrpc-flood
log: http                        # the kind of line those regexes read
parsers:
  - regex: '^(?P<ip>\S+) .* "POST /xmlrpc\.php'
    prefilter: xmlrpc            # the literal that gates the regex, as in a parser
    weight: 2
  - http-404                     # shared parsers mix in freely
threshold: 10/1m
ban: web

The loader makes a parser of each such entry, named <rule>#<n> (wp-xmlrpc-flood#1) — the name the evidence, shield why and the console show. log, and time_field/time_format when the lines carry their own time, sit on the rule and apply to every regex written in it; an entry names a parser or writes a regex, never both.

Threshold. One crossing — 5/10m, five points in ten minutes — is the rule; several, [5/1m, 20/1h], make it adaptive: the first crossed fires, one sanction either way (a second crossing while the ban runs is absorbed into it). The ban log line names the threshold that fired. threshold: 1/10m bans on the first hit.

Levels. A rule writes its numbers once; the machine's level scales them, the same way for every rule, and a person can predict it: instant — the first hit convicts (one point, the shortest window); strict — half the hits (rounded up, never below one); standard — as written (the default); lenient — twice the hits. level: in agent.yaml, or the machine's declaration on the controller (there, per pack and per rule too). Enrolled, the controller's level is the one that counts: the numbers arrive already scaled.

Ban, notify, report. ban names a policy — how long, on which ports, how the machine remembers (see Policy). notify: info | warning | critical raises a notification besides the ban, or instead of it (a rule with neither only records — a rule with notify and no ban is how "someone logged in as root" reaches you without banning anyone). report names a report preset (see Abuse reporting); every rule reports as one — default, built in, when the key is absent: the rule's name and a generic sentence, no field, one generic category. Whether reports leave at all is the machine's reporting:.

Everything else is optional, flat, and defaulted:

Key Default Values
count address what hits are counted per: address, network (a /24). asn, country — accepted, not evaluated yet: the rule loads and is reported inactive
state on on, test (detect-only in older files — read as the same, never written back), off
share yes no: the rule's bans stay on the machine, never offered to the fleet
distinct count distinct values of this field instead of hits (never combined with a negative weight)
countries / except_countries count only hits from these countries (ISO codes, FR, MA or a list) — or from all but these; needs the geo databases on the machine (below), else the rule waits and says so
asns / except_asns the same by AS number (12322 or AS12322): an operator's whole network
targets / except_targets count only hits aimed at these targets — or at all but these: the site or domain the hit was about (a {target} segment in the machine's logs: path names it from the file; a user=jo@example.org names its domain), as a name or a pattern with * (cloud.*, *.example.org); a hit with no target counts for except_targets, not for targets — the whitelist by site
when accepted, not evaluated yet: the rule loads and is reported inactive
description free text

In test a rule records what it would have done — a sanction row marked unenforced, evidence included, shown by shield why — and never touches the data plane.

Policy

What a ban IS, named once, used by any rule:

policy: human-ladder
durations: [10m, 1h, 4h, 24h]   # first offence, second, third, fourth — the last repeats; or one: 24h
ports: all                      # all (default) | service (the machine's ports for the rule's log) | [80, 443]
memory: 30d                     # how long the machine remembers an address after its ban ENDED
tighten: 2x                     # optional: each return within memory divides the hits needed by this
reputation: {above: 90, duration: 30d}   # optional: the controller lengthens the ban when AbuseIPDB agrees

Repeat. An address banned here (by any rule) less than memory after its previous ban ended is a repeat: the next rung of the ladder, and with tighten fewer hits to convict. Measured from the ban's END, never its start — a long ban must not consume its own memory. No memory: the machine never forgets.

Reputation. The controller, not the agent: after a ban lands, it asks the provider (a day's cache, the account's budget) and, when the address's confidence score is at or above above, orders the machine the longer ban — after the fact, never between a hit and a sanction (SPEC principle 5). Needs an AbuseIPDB key in the controller's secrets.yaml.

duration_start is application by default; evidence starts the clock at the attack time instead (late evidence can then arrive already expired).

trust.yaml

A list of graduated weights on origins — where traffic comes from, never what it targets. An origin is auto-detected: a country code (FR), an ASN (AS3215), or a range (198.51.100.0/24).

- origin: FR
  require: 3x           # 3× the evidence before any rule fires
  range_ban: never      # never ban a whole range inside this origin

- origin: AS64500
  require: 0.5x         # below 1x demands less: distrust

exemptions.yaml

The scope is the key itself; exactly one per entry. The traffic is still logged — an exemption removes the sanction, never the observation.

- ip: 203.0.113.7
  note: office

- range: 198.51.100.0/24
  only_paths: ["/api/*"]    # conditional: exempt only on these paths

- ip: 198.51.100.9
  until: 2026-09-01T00:00:00Z   # ends on its own — "let me back in while
  note: fixing the backup job   # I fix this", without a hole left open

until is an RFC3339 instant; without it the exemption is permanent. shield exempt -for 2h writes exactly this form into exemptions-cli.yaml.

Abuse reporting

The agent can submit the offenders it bans to an abuse database (AbuseIPDB today; SPEC §5.5). Three pieces, three files, and it takes all three before a single report leaves:

# agent.yaml — the switch. provider is the ONLY thing that arms it.
reporting:
  provider: abuseipdb
  # endpoint: https://…     # test/proxy override; unset = the real one
  # interval: 30s           # how often the sender looks for new bans

# secrets.yaml — the credential, 0600, never in version control.
abuseipdb:
  key: your-api-key
  daily_limit: 1000         # your plan — or deliberately less, to share
                            # one account between machines; "auto" learns
                            # the plan from the provider's first answer
  # enabled: false          # keep the key, send nothing — the console's
                            # Settings → Reporting switch writes this;
                            # absent means on (the key is the decision)

# reports/ssh-intrusion.yaml — what a report SAYS. Data you own.
report: ssh-intrusion
to: abuseipdb                       # the destination (the only one today; default)
when: always                        # always (default) | repeat-only: returning offenders only
categories: [brute-force, ssh]      # the provider's vocabulary, by name
says: "Unauthorised SSH login attempt on a key-only host"
include: []                         # parsed fields allowed to appear

A rule names its preset — report: ssh-intrusion — or reports as default, built in: the rule's name and a generic sentence, one generic category, no field (write reports/default.yaml to change it). A key placed before the switch is thrown is not a mistake: it is how a machine is prepared in test and armed later, each its own decision. Detect-only never reports: what was not enforced is not submitted.

include is a whitelist, and an empty one is normal. Only the fields it names can appear in the comment; everything else the parser captured — usernames, mailbox names, paths, the raw line — stays on the machine. The validator refuses a preset naming a field its rule never captures, because default-deny is only a property when the names are real. The comment is composed when the ban is decided, stored with it, and inspectable before it leaves (shield why).

A report gets one chance, at the moment of the detection. Whatever stops it — allowance spent, provider down, key refused — the outcome is logged on its own line and the sanction stays on record, still owed. Nothing is retried on a cadence and no backlog drains by itself: that would be the agent deciding to describe last week as if it were happening now. Sending the past is the operator's act:

shield report -last 6h        # counts, shows the estimate, asks
shield report -last 2d -n 500 -y
shield status reports         # owed / sent / duplicate, from the store

The allowance is watched from both ends: the agent stops at daily_limit on its own count, and reads the provider's rate-limit headers on every response — the same account may report from several machines, so only the provider knows the truth. When the provider says the day is spent, one probe every five minutes notices an upgraded plan or another machine freeing room; the daily reset is midnight UTC.

Category names, from the provider's list: dns-compromise, dns-poisoning, fraud-orders, ddos, ftp-brute-force, ping-of-death, phishing, fraud-voip, open-proxy, web-spam, email-spam, blog-spam, vpn-ip, port-scan, hacking, sql-injection, spoofing, brute-force, bad-web-bot, exploited-host, web-app-attack, ssh, iot-targeted. ssh and exploited-host are meant to be combined with a more specific one, which is why categories is a list.

Hooks

The operator's own endpoints, each POSTed one JSON document per enforced ban — a Discord relay, an inventory, a SIEM. Local configuration by invariant (SPEC §5.5): a controller can never push a hook, or whoever controlled the controller would choose where every machine's decisions are sent.

# agent.yaml
hooks:
  - url: https://ops.example.net/shieldlist?token=…
    # timeout: 5s        # per delivery; nothing waits on a hook
{"time":"2026-08-15T12:33:31Z","action":"ban","rule":"srv00-mail",
 "address":"203.0.113.9/32","until":"2026-09-09T12:33:31Z",
 "duration":"25d0h","comment":"…","categories":"11,18"}

ports appears when the ban is service-scoped; comment is the composed default-deny report text (empty for a rule with no report) — a hook never receives a raw log line. Deliveries get one chance each, on the ban's heels, with the outcome logged; a hook that was down missed that ban and that is all. Detect-only produces no egress of any kind, hooks included. Credentials belong in the URL, which never leaves the machine and is logged as its host only.

The controller

The controller's tree is an agent tree — the library — plus what makes it a fleet's (SPEC §5.9):

/etc/shieldlist/controller/
  controller.yaml         the service itself
  parsers/<source>/ rules/ policies/ reports/   the LIBRARY: every parser, rule, policy, preset, once
  packs/<name>.yaml       a PACK: a named list of rules — what a machine is given
  agents/<hostname>.yaml  one machine: its packs, the rules it adds, what it changes, its logs
  trust.yaml exemptions.yaml exemptions.d/      the fleet's, to every machine
  secrets.yaml            the controller's own provider credentials

A machine runs the rules of its packs, plus the rules its declaration names — nothing else. A pack is a list (packs/plesk.yaml: pack: plesk, description, rules: [wp-login, wp-xmlrpc, …]); a rule is never copied, only listed, so a rule in two packs runs once on a machine given both. A machine's bundle carries its rules, the parsers they score, the policies and presets they name, the fleet's exemptions — what it needs and no more. A machine with no packs: and no rules: runs nothing. The console (Rules → Packs, a grid of packs × servers; Servers → a server → Rules) and PATCH /v1/agents/{host}/declaration {packs: [...]} write the same key.

shieldlist-controller -t -config /etc/shieldlist/controller validates all of it, every agent's assembly included, and prints what each machine gets. The defaults are read by the agent's own loader, so shieldlist-agent -t -config /etc/shieldlist/controller says the same thing about them.

# controller.yaml — everything optional
listen:
  agents: 0.0.0.0:17453     # the agent transport
  api: 127.0.0.1:17454      # the operator API and interface
state: /var/lib/shieldlist/controller
tls:                        # absent: a self-signed certificate under state/tls,
  cert: /path/fullchain.pem #   pinned by agents through the join token
  key: /path/privkey.pem    # both: the operator's own certificate
  off: false                # true: plain HTTP behind a proxy that terminates TLS
level: standard             # the fleet's default level
reporting: agent            # who reports a machine's sanctions: agent | controller | off (see below)
lift: owners                # who may lift a machine's sanctions: owners | operator
sources: [googlebot, bingbot, cloudflare]   # published exemption lists to fetch (opt-in)
sources_refresh: 24h        # on this schedule; unset = on command only

Published exemption sources. Search engines and CDNs publish their address ranges; naming a source exempts them without pasting a range. The catalogue is compiled in — googlebot, google-special-crawlers, bingbot, applebot, cloudflare, jetpack (Automattic's Jetpack callers, for shared hosting), ahrefs (AhrefsBot, the SEO crawler) — and nothing is fetched unless sources: names it: a fetch is outbound traffic (SPEC principle 5). A refresh writes exemptions.d/source-<name>.yaml whole (generated, never hand-edited), reloads and pushes it to every agent; an empty list, a huge one, or a prefix wider than any crawler list ever is, is refused and the last good file stays. shield sources shows each source's state, shield refresh <name|all> fetches now.

# agents/srv00.yaml — the file is named after the agent (its hostname, as
# the machine reports it: `hostname` there — web1.example.net or web1)
agent: srv00
name: srv00                      # what the console calls it (default: the short hostname when unambiguous)
description: the infrastructure host
level: strict                    # scales every rule here: instant | strict | standard | lenient
packs: [openssh, plesk, srv00]   # the packs it runs — or, per pack, how:
# packs:
#   plesk: {}
#   infra: {state: test, level: strict}   # this pack test here, at its own level
state: on                        # on | test — the agent's own may relax it, never arm it
fleet:                           # the pack is the unit of sharing: another machine's
                                 # ban applies here when its rule belongs to a pack
                                 # both machines run — a rule outside every pack
                                 # never travels, `share: no` on a rule is absolute
  offer: no                      # its sanctions are offered to the machines sharing its packs (default yes)
  enforce: no                    # it applies the sanctions of the machines sharing its packs (default yes)
lift: operator                   # only the operator may lift its bans
reporting: agent                 # agent | controller | off
rules:                           # what this machine changes about a rule — or adds
  ssh-bruteforce: {level: lenient}           # this rule alone at another level here
  http-bad-ua: {threshold: [3/1m, 10/1h]}    # its own numbers here
  wp-scan: {state: off}
  mail-auth: {ban: mail-hardcore}
  root-login:                                # a rule outside its packs, named: it runs here as written
logs:                            # where this machine's logs are (pushed; empty: its own agent.yaml says)
  - {path: /var/log/nginx/*.log, log: http, ports: [80, 443]}

An entry under rules: changes what one machine runs of one rule — its level (instant, strict, standard, lenient: the rule's numbers scaled), its threshold outright, its state, its policy — and, for a rule outside the machine's packs, adds the rule to it. Parsers, policies and presets are shared, never overridden: a machine that needs a different regex needs a different rule. The controller assembles each machine's tree, rewrites only the rules that differ (comments kept), validates the result with the agent's loader before it leaves, and refuses an agent older than 0.1.0 by name — it keeps running what it last received, and the operator is told to upgrade it. shieldlist-controller -t prints every machine's assembly and what differs. A machine enrolled without a file under agents/ runs nothing until it is given packs.

Who reports. reporting: — the fleet's default in controller.yaml, each machine's own in its declaration — says who submits a machine's enforced sanctions to the abuse provider: agent (the machine itself, with the key in ITS secrets.yaml), controller (this program, on the machine's behalf, with the key and daily_limit in the tree's secrets.yaml — one account, one address, for the fleet), or off. A machine told controller pauses its own sender; the controller then reports each of its sanctions once, as it arrives, composed from the rule's report preset and the fields the convicting lines named — the same words the agent would have used — and records the outcome on the sanction (the console says "reported by the controller"; /v1/config counts them). What was not enforced is not submitted; a report the provider refused or the allowance could not cover is not retried; a lift "with withdraw" withdraws the controller's own report where it was the controller that reported. Detect-only machines report nothing whoever would report.

Sources of your owncustom_sources: in controller.yaml, each {name, url|urls, format: lines|cidr-lines|json-prefixes, note}, named in sources: like the catalogue's; the console adds and removes them. Enabled sources are fetched when older than sources_refresh (never at every start), and on command.

Retentionretention: {evidence: 90d, samples: 60d, sanctions: 0} in controller.yaml (the defaults: evidence and what its lines named kept 90 days, the agents' counters 60 days, the sanctions for good). Swept daily; Settings → Maintenance shows what the store holds and flushes on demand (POST /v1/maintenance/prune, …/vacuum).

Statistics on what the lines named — every convicting line carries what its parser captured, by group name (user, path… — at most 8, values cut at 128 bytes; time and ip left out), one row per name in the store (evidence_fields). To count a new thing, name a group in a pattern — (?P<path>\S+) — nothing else; GET /v1/sanctions/facets?facet=user &facet=path counts the values under the filters, field=user:admin filters by one. They live and die with the evidence (retention), and Settings → Maintenance flushes them apart (fields_older_than).

Hits over time — on every sync the agent sends the positive hits its rules scored since the last one, by offender (a flow, bounded to the busiest 2000 offenders, the rest unnamed); the controller resolves the origins and keeps them per minute (hit_samples: country, network) — the pressure the chart draws beside the bans, summed per step, under the server, country and network filters; kept as long as the samples. When the controller holds none of a machine's flow yet (a first enrolment, a fresh store), the agent backfills, once, the hits still inside its rules' windows, minute by minute — as far back as its longest window, no further: hits that fell out of every window are gone. (Before this the table held a sampled level — the hits inside the windows — which read as a flat lie; it is cleared once on upgrade.)

The public unblock pagepublic: {unblock: on} in controller.yaml (or Settings → Unblock page, which writes the same keys) opens /unblock (no account): a visitor sees whether their own address — the one they come from, as the reverse proxy on the loopback says — is banned on the fleet, and lifts it themselves in one click (logged). After whitelist_after self-unblocks within an hour (3 by default) the page offers a whitelist request; anyone may ask "is this address banned" and ask for another address to be unblocked — those wait under Requests in the console (its own menu entry, with the pending count; approve as a lift or as a fleet-wide exemption, or deny). Off by default; the menu entry goes with it. Settings: captcha: off|builtin|turnstile (built-in = a small arithmetic question, no third party; Turnstile needs turnstile_site and turnstile_secret and loads Cloudflare's script), language: auto|en|fr (auto = the visitor's browser), title, footer, footer_url (the page's brand and last line — for embedding it on your own site), contact (shown to a person still blocked), and the address detection, for a dual-stack visitor (usually banned twice, once per family, reaching the page over one of them): direct panel detectionipv4_host and ipv6_host, two names of this console answering over one family each (an A record alone, an AAAA record alone, the same proxy): the page asks each and unblocks there, the request's source is the address; or custom resolversipv4_url and ipv6_url, two endpoints of your own, each reachable over one family only, answering JSON {"ip": "…"} (an optional "rdns" is shown; other fields ignored) with Access-Control-Allow-Origin for the console's origin: the page learns both addresses there, shows the status of the one of the other family (GET /public/me?value=, no log lines) and, as nothing proves that address is the visitor's, offers to ask for its unblock — a request the operator decides; the controller lifts on the visitor's word only the address their request comes from. Hosts are the sound choice (both families unblock themselves) and win over URLs when both are set. Every action is rate-limited per address and a refusal is logged as msg="public page refused" why=rate|captcha ip=…; the shieldlist-console pack's shieldlist-unblock-abuse rule bans an address that hammers it. API: GET /v1/requests?all=1, GET /v1/requests/count, POST /v1/requests/{id}/approve {action: lift|exempt} or /deny, GET|PATCH /v1/public.

Notifications (mail)mail: {smtp: host:port, user, password, from, security} in controller.yaml (or Settings → Notifications, with a test button) is the relay the controller sends through — 587 STARTTLS or 465 TLS by the port, security: none for a trusted local relay, authentication when user is set; never a delivery of its own. Who receives is per feature, by account: public.notify_requests: [alice, bob] names the accounts mailed (at the e-mail each set on their own account) when a request from the public page waits for a decision, public.notify_unblocks on every self-service unblock; an account without an e-mail is skipped. One try and a retry, the failure in the log; a mail never holds a request up. public.show_lines: off hides from a blocked visitor the log lines that earned the block (their own requests, shown by default). API: GET|PATCH /v1/mail, POST /v1/mail/test (to the caller's own address, or ?to=).

Reverse DNSGET /v1/rdns/{address} looks the name up from the controller (two seconds, remembered an hour); the address dialog shows it beside the address, without waiting.

Reputation (AbuseIPDB) — with a secrets.yaml in the controller's tree (abuseipdb: {key: …}, the same shape as an agent's) the console can ask what the provider knows about an address before a ban by hand (GET /v1/reputation/{value} — from the store when checked within a day, ?fresh=1 asks again, ?cached=1 never asks; each question spends one of the account's daily checks, counted apart from reports), and the controller asks daily about each server's own address, shown on its card. Nothing is checked automatically beyond that.

Clocks — every sync carries the agent's clock; the controller keeps the difference (clock_skew on GET /v1/agents, seconds, agent minus controller) and the console warns past thirty seconds ("clock off by 2m"): a machine that disagrees about the time disagrees about every window, every ban's end and its place on the charts. The agent warns on its side too (shield status: controller_clock). Fix NTP; nothing else is adjusted for it.

Updatesupdates: off stops the daily question to git.lrob.net about the latest releases; on, the console says when a newer controller or agent is published (a notice, never a download).

A machine's own exemptionsexemptions: in agents/<host>.yaml ([{ip|range, until?, note?}]), assembled into its bundle as exemptions.d/machine.yaml; the console writes them (Exempt… → only this server) and removes them.

Where addresses are fromgeoip: in controller.yaml. The files the controller holds are served to its agents: every sync carries their content hashes, an agent holding another edition (or none) fetches them under its state directory (/var/lib/shieldlist/geoip/) and loads them — no configuration on the machine — which is what the rules' countries: / asns: and the exemptions by country or AS run on. A standalone agent names files of its own in agent.yaml (geoip: {country: …, asn: …}). Either source: dbip (the free DB-IP lite country and ASN databases, fetched monthly under <state>/geoip/, licence CC BY 4.0 — the console shows "IP Geolocation by DB-IP"), or files of your own: country: and asn: naming MaxMind-DB files (GeoLite2 or any provider's). Off unless configured. Sanctions are enriched as they arrive (country, AS number and name); rows from before are backfilled; the console shows a flag and the network, filters and counts by them.

The console is served at / on listen.api. Home: an address lookup and the sanctions as they land — a live feed (play/pause) filtered by state (banned now, bans, detections, expired, lifted), server, rule and dates, as many rows as asked, the fields of your choosing, the requests behind each sanction a toggle away. Servers: each machine with four tabs — overview, its bans (the same feed, for it), its rules (the fleet's defaults at its level or its overrides, changed or reset per rule) and its settings. Rules and Exemptions are the fleet's; Activity is who did what; Console is the panel itself — people, tokens, the configuration files, this browser's preferences. An address opens as a dialog wherever it appears: banned where, its history, each sanction's convicting lines, and the two actions — lift (the reason is optional) and exempt (the red one: never sanctioned again while it lasts), each able to withdraw what was reported about the address. People sign in with a name and password — shieldlist-controller account -name NAME creates the first account at the console, account -name NAME -totp enrols a one-time code — or by pasting an API token. Sessions last twelve hours; every action is audited under the person's name.

The API answers on listen.api (localhost by default; on the network, put it behind TLS you control), JSON, with API tokens: shieldlist-controller api-token -name NAME mints the first one at the console, POST /v1/tokens the next. shield -controller URL -token T (or SHIELD_CONTROLLER, SHIELD_TOKEN) is its client: agents, agent <host>, set <host> key=value… (level, state, lift, reporting, offer, enforce, description, packs=a,b, rules.<rule>.{level,threshold,state,ban}, rules.<rule>=none), rules, rule <name> key=value… (state, threshold — 5/10m or 3/1m, 20/1h — ban, report), why, unban [-agent] [-withdraw-report], exempt [-lift [-withdraw-report]], reload, validate, audit, token, sources, refresh, version. Every write is a write to the files — agents/<host>.yaml created or edited in place, the rule's own file edited in place, comments kept — reloaded whole or reverted with the loader's errors as the answer. Endpoints: GET /v1/version, GET /v1/agents[/{host}], GET|PATCH /v1/agents/{host}/declaration (level, state, lift, reporting, offer, enforce, description, packs: [{name, state, level}], logs: [{path|unit, log, ports}], rules: {name: {level|threshold, state, ban} | {} to add | null to drop}), GET /v1/agents/{host}/rules (what the machine runs: each rule with the packs that brought it, its threshold there and why, its state, outcome), the rule language — GET /v1/packs, POST /v1/packs {name, description, rules}, PATCH /v1/packs/{name} {description, rules}, DELETE /v1/packs/{name} (refused while a machine is given it); GET /v1/rules (each with its parsers — {parser, weight} or {regex, prefilter, inline} —, threshold, ban, notify, report, count, state, share, the packs listing it, the servers running it), POST /v1/rules {name, dir, category, description, log, parsers, threshold, ban, notify, report, count, distinct, state, share}, PATCH /v1/rules/{name} (any of those; parsers replaces the list; ?preview=1 on either answers {path, before, after} without writing — the diff the console shows), DELETE /v1/rules/{name}; GET /v1/parsers (inline ones marked with their rule), POST /v1/parsers {name, dir, log, prefilter | prefilter_any, regex, time_field, time_format}, PATCH|DELETE /v1/parsers/{name}, POST /v1/parsers/test {regex, prefilter, prefilter_any, lines} (each line: gated, matched, the fields captured); GET /v1/policies, POST /v1/policies {name, durations, ports (all | service | "80, 443"), memory, tighten, duration_start, reputation ("90/30d")}, PATCH|DELETE /v1/policies/{name}; GET /v1/reports, POST /v1/reports {name, to, when, categories, says, include}, PATCH|DELETE /v1/reports/{name} (editing the built-in default writes reports/default.yaml); GET|PUT /v1/files/{path} (one file of the tree whole — the "edit as file" view; ?validate=1 on PUT tries it through the loader and puts it back) — every write a file created, edited in place or removed, reloaded whole or reverted; POST /v1/agents/{host}/resync (the machine sends its whole history again, evidence included — a machine enrolled again does so on its own), GET /v1/sanctions?state= &agent=&rule=&q=&country=&asn=&since=&until=&limit=&offset=&evidence=1 (newest first, bounded; state is active, bans, recorded, expired, lifted or all; rule contains; q is one search — an address or its start, a rule, a server, a country code, an AS by number or name; since/until RFC 3339 on the start; evidence=1 adds each sanction's convicting lines), GET /v1/sanctions/facets?…&n=&hits=1 (same filters: top countries, networks, rules and usernames, n ≤ 300, the offenders being counted tallied in with hits=1 — the console's panel and map), GET /v1/geoip (what is loaded, the attribution), GET /v1/sanctions/series?…&window=24h&step=30m&since=… (counts per step, bans and detections apart — the console's chart; since inside the window answers the tail alone, on the same grid), POST /v1/bans {value, agent?, duration?, reason?, report_as?, lines?} (a ban by hand: on one machine or every enrolled one, duration like 6h, 1y, or permanent; applied by each agent at its next sync through the same checks as any sanction, recorded as rule manual with the reason as its evidence, reported when a preset is named — with the facts of the detection's lines when given, as the rule's own report would), GET /v1/presets (the report presets a manual ban may be reported as), GET /v1/exemptions (every exemption in force with its origin — the fleet's files, the sources, the console's file, a machine's declaration — and whether the console may remove it), DELETE /v1/exemptions {value, agent?}, POST /v1/exemptions {…, agent?} (one machine's own), PATCH /v1/sources {enable, disable, refresh}, POST /v1/sources/custom {name, url, format, note}, DELETE /v1/sources/custom/{name}, GET /v1/updates, GET /v1/watched?agent=&q=&country=&asn=&value=&exempt=1&limit= (the offenders the agents are counting but have not convicted — how far from a ban, as of each agent's last sync; offenders exempt on their machine carry exempt (why) and are listed only with exempt=1), GET /v1/networks/{asn} (an autonomous system's name and prefixes from the ASN database), GET|PATCH /v1/me (who is asking; a person may change their own password and email), GET /v1/sanctions/{id} (one, lines included), GET /v1/offenders/{address} (its whole history, lines included), POST /v1/lifts {value, agent?, reason?, withdraw_report?} (withdraw_report asks the machine to withdraw what it reported to AbuseIPDB about the address — the provider's clear-address, every report of the account about it), POST /v1/exemptions {ip|range, for?, note?, lift?, withdraw_report?} (a write to exemptions-cli.yaml, reloaded whole or reverted; lift also lifts the bans in force wherever they are), GET /v1/me, GET /v1/config, POST /v1/config/{reload,validate}, GET /v1/sources, POST /v1/sources/{name|all}/refresh, GET /v1/audit, POST /v1/join-tokens, GET|POST /v1/tokens, DELETE /v1/tokens/{name}, GET|POST /v1/accounts, POST /v1/accounts/{name}/{password,totp}, DELETE /v1/accounts/{name}[/totp]. Every action is audited under the token's name.

At the console, without the API: shieldlist-controller token [-uses N] [-expires 24h] mints a join token; agents, why <address> and `unban

[-agent HOST]` read and write the store directly (a console lift lands at the machine's next sync, an API lift within the second). `SIGHUP` reloads the tree — whole or not at all — and pushes the new revision to every connected agent.

Several parsers, one jail

parsers: takes a parser name, a list of names, or weighted entries — [ssh-auth-fail, {parser: ssh-preauth-abort, weight: 2}]. Every hit of any of them counts for the rule (one event, one hit: do not list two parsers that both match the same line of the same attempt).

Reporting again while the attack goes on

reporting:
  provider: abuseipdb
  renew: 24h        # an address still attacking while banned is reported again after this long; `never` turns it off

A ban that is extended because the hits keep coming (check 5) reopens its report once the last one is older than renew — the sender picks it up on its next pass, with the latest hit as the report's time. A report that never left is not renewed: its one chance was taken.

Compact scalars

Form Meaning
30s, 10m, 2h, 1h30m, 30d, 4w durations (days and weeks on top of Go units)
5/10m a threshold: points within a window
3x, 0.5x evidence multiplier
FR / AS3215 / 198.51.100.0/24 origin, auto-detected