Configuration file reference
Cette page n'existe qu'en anglais pour le moment.
YAML, one format only (SPEC §5.9). Hand-written config is the primary
interface: flat keys, a default for everything that has a sensible one,
compact scalar forms. The dry-run validator is the ground truth for this
document: shieldlist-agent -t -config <dir> reports every problem at once,
each with its file and line.
Layout
/etc/shieldlist/
agent.yaml # this machine (optional; absent = standalone defaults)
parsers/**.yaml # one parser per file — sub-directories by source (nginx/, sshd/…) for people
rules/**.yaml # one rule per file
policies/**.yaml # one ban policy per file
reports/**.yaml # one report preset per file (abuse reporting)
packs/*.yaml # optional: a pack = a named list of rules (what a controller gives a machine)
trust.yaml # optional list
exemptions.yaml # optional list, yours
exemptions.d/*.yaml # optional lists, one file per source
exemptions-cli.yaml # optional list, written by `shield exempt`
secrets.yaml # credentials — 0600, never deployed with the rest
The four language directories are read flat and one level deep: a
sub-directory is an ordering for people (a parser's source, a rule's
category) and means nothing to the loader — one namespace, a name is
defined once wherever its file sits. On a machine of its own every rule
in the tree runs; packs/ is documentation there and the unit of
assignment on a controller.
A file in the format of the 0.0.x releases (match:, pattern:,
category:, thresholds: per level, decay:, log_inputs:) is named
by the loader and refused with one sentence: run
shieldlist-agent -config DIR -convert NEWDIR (or the controller's
-convert) — it rewrites the tree, comments kept, says what it changed,
and the new tree is moved into place when it loads.
exemptions-cli.yaml is shieldlist's own file, beside yours so that
neither rewrites the other's. It is ordinary YAML in the same format: read
it, edit it, delete entries from it with whatever you normally use.
Canonical style
The remote control (SPEC §5.9) writes files in one style, and hand-written
files are expected to follow it: two-space indentation, list entries as -
blocks separated by one blank line so each entry reads as a unit, comments
attached to the key they describe. Names are lower-case letters, digits and
dashes.
agent.yaml
mode: local # local (default) | enrolled
controller: "" # base URL, required when enrolled
state: on # on (default) | test
level: standard # scales every rule's threshold here: instant | strict | standard | lenient (see Rule)
log_file: /var/log/shieldlist/agent.log # as well as standard error
logs:
- path: /var/log/nginx/access.log # a file…
log: http # …of this kind of line
ports: [80, 443] # the service's ports here — what `ports: service` in a policy bans
- path: /var/www/vhosts/system/*/logs/proxy_access_ssl_log # …or many
log: http
- unit: ssh.service # …or a journald unit — exactly one
log: ssh
ports: [22]
limits:
memory: 50% # the ceiling the agent holds itself to: a share of
# the machine (the default is 50%) or a size
# ("512MB", "2GB"); see the agent's docs/RESOURCES.md
log_file is where the agent writes its own log, as well as to standard error —
which under systemd is the journal, and has the same lines. Unset means
the journal alone.
It exists because /var/log is the first place anyone looks after
starting a security daemon, and finding nothing there reads as "it did
not start". The agent reopens the file by itself when logrotate takes
it away — it identifies it by (device, inode), like every other file it
follows — so no logrotate configuration ships with it. The shipped
systemd unit uses LogsDirectory=shieldlist-controller, which is what
makes /var/log/shieldlist-controller writable under ProtectSystem=strict
(its own directory, not the agent's /var/log/shieldlist: two units with
two users cannot share one — the agent's unit took it back at every
start).
log is what ties an input to its parsers: every parser reading that
kind of line (log: http in the parser) runs over the input's lines.
One source per kind per machine — reading the same lines from a file
and a journald unit would count everything twice, and declaring the
same input twice is refused for the same reason. Two patterns matching
the same file is not: it is read once. ports are the service's ports
on this machine — an sshd on 2222, a site on 8443 — the one place
they are true; a policy with ports: service bans exactly those, and
bans every port when none are declared (said once at start).
A path may be a pattern (*, ?, […]), which is how a machine
hosting sites is watched at all — a panel gives every site its own log
directory, and sites are added by people who are not thinking about this
agent. The pattern is re-read every fifteen seconds: a site created this
afternoon is watched this afternoon, and its log is read whole, as any
file appearing after the agent started is. A pattern matching nothing is
said at startup rather than left to look like silence.
Enrolled: taking configuration from a controller
mode: enrolled
controller: https://controller.example.net:17453
Then, once, on the machine — with a join token minted on the controller
(shieldlist-controller token):
shield enrol <join-token>
The agent keeps the credential it receives in its state store, pinned to the controller's certificate when the token carried its fingerprint, and from then on holds a connection to the controller: it presents the configuration revision it holds and receives a new one the moment the controller loads it, reports every sanction it decides, applies the fleet's sanctions and the lifts decided centrally, and sends its enforcement summary. It dials out only; nothing listens on the machine. Where a long-lived connection cannot be held, it asks every thirty seconds instead — same messages.
What comes from the controller: parsers, rules, policies, report
presets, trust and the fleet's exemptions — the tree the controller
assembled for this machine (its level and overrides applied), written
under /var/lib/shieldlist/received/ so that
shieldlist-agent -t and a person read the same files the agent runs.
What stays local: agent.yaml, secrets.yaml, hooks, and your own
exemptions*.yaml, which add to the fleet's. A new revision is applied
in place — no restart; the rules' windows start over, sanctions and
the kernel's sets are untouched. Until the first revision arrives, an
enrolled agent runs on its local files. When the controller is away,
it keeps enforcing on the last configuration received and says so once.
shield status shows both versions and where the conversation stands
(controller_version, controller_revision, controller_sync, and
controller_error when there is one); the controller's agent list
shows the same from its side.
Running it beside what you already have
state: test puts every rule in test and keeps the
agent off the data plane entirely: no nftables table, no restore,
nothing added or removed. It decides, records and reports exactly as it
would otherwise — shield status, shield why, the ban log — with
every sanction marked unenforced.
That is how this is meant to be adopted: run it for a day next to
fail2ban, CrowdSec or whatever guards the machine, read what it would
have done, then set state: on. Doing the same thing by editing every
rule is how one gets missed.
shield status and shield stats answer enforcement DETECT-ONLY
first, before any count, because the one dangerous misreading is a
screen full of bans on a machine that is not defending itself. While the
agent is in that mode the CLI counts the records it made rather than the
bans it holds — otherwise every figure would read zero next to a log
full of decisions — and shield bans says so above its table.
The agent still checks that this machine could enforce: run
shieldlist-agent -t and it builds the table it would use under a
throwaway name, with no hook, and removes it. That is where a kernel
that will not take some part of it says so, since an agent in
test never finds out on its own.
Reading a log's past is off unless asked for:
replay_on_start: 6h # read this much of the past at startup,
# enforce what would still be in force
A live agent joins the present. An operator who turns up after an attack
began wants it to catch up, and restarting it is the tidiest way to ask.
Only crossings whose sanction would still be running are acted on — window
and ban duration. Files need their parser to declare a time_field;
journald records its own time.
Parser
parser: ssh-auth-fail
log: ssh # the kind of line it reads — the word the machine's logs declare
prefilter: "Failed password" # mandatory literal gate before the regex
regex: 'Failed password for (?:invalid user )?(?P<user>\S+) from (?P<ip>\S+)'
Named groups in regex become the parsed fields; (?P<ip>…) is what a
rule bans. RE2 only. Ports are not a parser's business (see logs: in
agent.yaml).
A parser may also say where the line carries its own timestamp:
time_field: time # a capture group of the regex
time_format: nginx # nginx, apache, syslog, rfc3339, unix,
# or a Go layout for anything else
Both halves or neither. A live agent never uses them — it stamps a line with the moment it read it, which is the same instant and costs nothing. They are what makes reading a log's past possible: replaying a week-old file where every line claimed to have happened now would have a windowed rule ban whoever appears in it.
syslog prints no year. The reader supplies its own, and a line dated
ahead of the reader is taken as last year's — so a December log is still
read correctly in January.
Point time_field at a timestamp the server writes, never at
anything the client controls: a replay trusts the line's own clock, and
an attacker who can write the timestamp can date their attack out of the
window.
A parser needs exactly one gate: prefilter (a single literal) or
prefilter_any (a list of tokens — the line must contain at least one,
case-insensitively, before the regex runs). prefilter_any is what
makes a list parser affordable: a 683-entry User-Agent list has no
literal common to all entries, and ungated it cost ~920µs per line;
the multi-literal gate answers in under a microsecond. The gate is a
superset of the regex — it can only admit lines the regex then judges,
never hide one from it.
The machine itself is never an offender — built in, not configured. Loopback and the machine's own interface addresses cannot be sanctioned by any rule, and addresses that are not publicly routable are never submitted to an abuse provider. No configuration of this program can firewall the machine it runs on.
Rule
One rule, one file, five blocks in the order things happen — what it reads, what it recognises, from how much, what it does, what it says:
rule: wp-login
description: Brute force on WordPress logins
category: web # where it is filed — web, mail, ssh, database, panel… (for people)
parsers: [wp-login-fail, wp-login-fail-apache] # what it recognises (their `log:` says what it reads)
threshold: 5/10m # from how much: five hits in ten minutes
ban: human-ladder # what it does: a ban policy (see Policy)
report: wp-bruteforce # what it says: a report preset (default when absent)
Parsers. A bare name, a list of names, or entries with a weight — every hit adds its parser's weight (1 unless said otherwise; negative for traffic that argues against a ban), so one rule can add up varied behaviour:
parsers:
- parser: wp-login-fail
weight: 10 # three of these alone would fire
- parser: http-404
weight: 3 # or ten of these — or any mix
threshold: 30/10m
A regex may be written in the rule instead of naming a parser file — a rule with no reason to share its regex keeps regex and rule together:
rule: wp-xmlrpc-flood
log: http # the kind of line those regexes read
parsers:
- regex: '^(?P<ip>\S+) .* "POST /xmlrpc\.php'
prefilter: xmlrpc # the literal that gates the regex, as in a parser
weight: 2
- http-404 # shared parsers mix in freely
threshold: 10/1m
ban: web
The loader makes a parser of each such entry, named <rule>#<n>
(wp-xmlrpc-flood#1) — the name the evidence, shield why and the
console show. log, and time_field/time_format when the lines carry
their own time, sit on the rule and apply to every regex written in it;
an entry names a parser or writes a regex, never both.
Threshold. One crossing — 5/10m, five points in ten minutes — is
the rule; several, [5/1m, 20/1h], make it adaptive: the first crossed
fires, one sanction either way (a second crossing while the ban runs is
absorbed into it). The ban log line names the threshold that fired.
threshold: 1/10m bans on the first hit.
Levels. A rule writes its numbers once; the machine's level
scales them, the same way for every rule, and a person can predict it:
instant — the first hit convicts (one point, the shortest window);
strict — half the hits (rounded up, never below one); standard — as
written (the default); lenient — twice the hits. level: in
agent.yaml, or the machine's declaration on the controller (there, per
pack and per rule too). Enrolled, the controller's level is the one that
counts: the numbers arrive already scaled.
Ban, notify, report. ban names a policy — how long, on which
ports, how the machine remembers (see Policy). notify: info | warning | critical raises a notification besides the ban, or instead of it (a
rule with neither only records — a rule with notify and no ban is
how "someone logged in as root" reaches you without banning anyone).
report names a report preset (see Abuse reporting); every rule reports
as one — default, built in, when the key is absent: the rule's name
and a generic sentence, no field, one generic category. Whether reports
leave at all is the machine's reporting:.
Everything else is optional, flat, and defaulted:
| Key | Default | Values |
|---|---|---|
count |
address |
what hits are counted per: address, network (a /24). asn, country — accepted, not evaluated yet: the rule loads and is reported inactive |
state |
on |
on, test (detect-only in older files — read as the same, never written back), off |
share |
yes |
no: the rule's bans stay on the machine, never offered to the fleet |
distinct |
— | count distinct values of this field instead of hits (never combined with a negative weight) |
countries / except_countries |
— | count only hits from these countries (ISO codes, FR, MA or a list) — or from all but these; needs the geo databases on the machine (below), else the rule waits and says so |
asns / except_asns |
— | the same by AS number (12322 or AS12322): an operator's whole network |
targets / except_targets |
— | count only hits aimed at these targets — or at all but these: the site or domain the hit was about (a {target} segment in the machine's logs: path names it from the file; a user=jo@example.org names its domain), as a name or a pattern with * (cloud.*, *.example.org); a hit with no target counts for except_targets, not for targets — the whitelist by site |
when |
— | accepted, not evaluated yet: the rule loads and is reported inactive |
description |
— | free text |
In test a rule records what it would have done — a sanction
row marked unenforced, evidence included, shown by shield why — and
never touches the data plane.
Policy
What a ban IS, named once, used by any rule:
policy: human-ladder
durations: [10m, 1h, 4h, 24h] # first offence, second, third, fourth — the last repeats; or one: 24h
ports: all # all (default) | service (the machine's ports for the rule's log) | [80, 443]
memory: 30d # how long the machine remembers an address after its ban ENDED
tighten: 2x # optional: each return within memory divides the hits needed by this
reputation: {above: 90, duration: 30d} # optional: the controller lengthens the ban when AbuseIPDB agrees
Repeat. An address banned here (by any rule) less than memory
after its previous ban ended is a repeat: the next rung of the ladder,
and with tighten fewer hits to convict. Measured from the ban's END,
never its start — a long ban must not consume its own memory. No
memory: the machine never forgets.
Reputation. The controller, not the agent: after a ban lands, it
asks the provider (a day's cache, the account's budget) and, when the
address's confidence score is at or above above, orders the machine
the longer ban — after the fact, never between a hit and a sanction
(SPEC principle 5). Needs an AbuseIPDB key in the controller's
secrets.yaml.
duration_start is application by default; evidence starts the clock at
the attack time instead (late evidence can then arrive already expired).
trust.yaml
A list of graduated weights on origins — where traffic comes from, never
what it targets. An origin is auto-detected: a country code (FR), an ASN
(AS3215), or a range (198.51.100.0/24).
- origin: FR
require: 3x # 3× the evidence before any rule fires
range_ban: never # never ban a whole range inside this origin
- origin: AS64500
require: 0.5x # below 1x demands less: distrust
exemptions.yaml
The scope is the key itself; exactly one per entry. The traffic is still logged — an exemption removes the sanction, never the observation.
- ip: 203.0.113.7
note: office
- range: 198.51.100.0/24
only_paths: ["/api/*"] # conditional: exempt only on these paths
- ip: 198.51.100.9
until: 2026-09-01T00:00:00Z # ends on its own — "let me back in while
note: fixing the backup job # I fix this", without a hole left open
until is an RFC3339 instant; without it the exemption is permanent.
shield exempt -for 2h writes exactly this form into
exemptions-cli.yaml.
Abuse reporting
The agent can submit the offenders it bans to an abuse database (AbuseIPDB today; SPEC §5.5). Three pieces, three files, and it takes all three before a single report leaves:
# agent.yaml — the switch. provider is the ONLY thing that arms it.
reporting:
provider: abuseipdb
# endpoint: https://… # test/proxy override; unset = the real one
# interval: 30s # how often the sender looks for new bans
# secrets.yaml — the credential, 0600, never in version control.
abuseipdb:
key: your-api-key
daily_limit: 1000 # your plan — or deliberately less, to share
# one account between machines; "auto" learns
# the plan from the provider's first answer
# enabled: false # keep the key, send nothing — the console's
# Settings → Reporting switch writes this;
# absent means on (the key is the decision)
# reports/ssh-intrusion.yaml — what a report SAYS. Data you own.
report: ssh-intrusion
to: abuseipdb # the destination (the only one today; default)
when: always # always (default) | repeat-only: returning offenders only
categories: [brute-force, ssh] # the provider's vocabulary, by name
says: "Unauthorised SSH login attempt on a key-only host"
include: [] # parsed fields allowed to appear
A rule names its preset — report: ssh-intrusion — or reports as
default, built in: the rule's name and a generic sentence, one generic
category, no field (write reports/default.yaml to change it). A key
placed before the switch is thrown is not a mistake: it is how a machine
is prepared in test and armed later, each its own decision.
Detect-only never reports: what was not enforced is not submitted.
include is a whitelist, and an empty one is normal. Only the fields it
names can appear in the comment; everything else the parser captured —
usernames, mailbox names, paths, the raw line — stays on the machine.
The validator refuses a preset naming a field its rule never captures,
because default-deny is only a property when the names are real. The
comment is composed when the ban is decided, stored with it, and
inspectable before it leaves (shield why).
A report gets one chance, at the moment of the detection. Whatever stops it — allowance spent, provider down, key refused — the outcome is logged on its own line and the sanction stays on record, still owed. Nothing is retried on a cadence and no backlog drains by itself: that would be the agent deciding to describe last week as if it were happening now. Sending the past is the operator's act:
shield report -last 6h # counts, shows the estimate, asks
shield report -last 2d -n 500 -y
shield status reports # owed / sent / duplicate, from the store
The allowance is watched from both ends: the agent stops at
daily_limit on its own count, and reads the provider's rate-limit
headers on every response — the same account may report from several
machines, so only the provider knows the truth. When the provider says
the day is spent, one probe every five minutes notices an upgraded plan
or another machine freeing room; the daily reset is midnight UTC.
Category names, from the provider's list: dns-compromise,
dns-poisoning, fraud-orders, ddos, ftp-brute-force,
ping-of-death, phishing, fraud-voip, open-proxy, web-spam,
email-spam, blog-spam, vpn-ip, port-scan, hacking,
sql-injection, spoofing, brute-force, bad-web-bot,
exploited-host, web-app-attack, ssh, iot-targeted. ssh and
exploited-host are meant to be combined with a more specific one,
which is why categories is a list.
Hooks
The operator's own endpoints, each POSTed one JSON document per enforced ban — a Discord relay, an inventory, a SIEM. Local configuration by invariant (SPEC §5.5): a controller can never push a hook, or whoever controlled the controller would choose where every machine's decisions are sent.
# agent.yaml
hooks:
- url: https://ops.example.net/shieldlist?token=…
# timeout: 5s # per delivery; nothing waits on a hook
{"time":"2026-08-15T12:33:31Z","action":"ban","rule":"srv00-mail",
"address":"203.0.113.9/32","until":"2026-09-09T12:33:31Z",
"duration":"25d0h","comment":"…","categories":"11,18"}
ports appears when the ban is service-scoped; comment is the
composed default-deny report text (empty for a rule with no
report) — a hook never receives a raw log line. Deliveries get
one chance each, on the ban's heels, with the outcome logged; a hook
that was down missed that ban and that is all. Detect-only produces no
egress of any kind, hooks included. Credentials belong in the URL,
which never leaves the machine and is logged as its host only.
The controller
The controller's tree is an agent tree — the library — plus what makes it a fleet's (SPEC §5.9):
/etc/shieldlist/controller/
controller.yaml the service itself
parsers/<source>/ rules/ policies/ reports/ the LIBRARY: every parser, rule, policy, preset, once
packs/<name>.yaml a PACK: a named list of rules — what a machine is given
agents/<hostname>.yaml one machine: its packs, the rules it adds, what it changes, its logs
trust.yaml exemptions.yaml exemptions.d/ the fleet's, to every machine
secrets.yaml the controller's own provider credentials
A machine runs the rules of its packs, plus the rules its declaration
names — nothing else. A pack is a list (packs/plesk.yaml: pack: plesk, description, rules: [wp-login, wp-xmlrpc, …]); a rule is
never copied, only listed, so a rule in two packs runs once on a machine
given both. A machine's bundle carries its rules, the parsers they
score, the policies and presets they name, the fleet's exemptions — what
it needs and no more. A machine with no packs: and no rules: runs
nothing. The console (Rules → Packs, a grid of packs × servers; Servers
→ a server → Rules) and PATCH /v1/agents/{host}/declaration {packs: [...]} write the same key.
shieldlist-controller -t -config /etc/shieldlist/controller validates
all of it, every agent's assembly included, and prints what each machine
gets. The defaults are read by the agent's own loader, so
shieldlist-agent -t -config /etc/shieldlist/controller says the same
thing about them.
# controller.yaml — everything optional
listen:
agents: 0.0.0.0:17453 # the agent transport
api: 127.0.0.1:17454 # the operator API and interface
state: /var/lib/shieldlist/controller
tls: # absent: a self-signed certificate under state/tls,
cert: /path/fullchain.pem # pinned by agents through the join token
key: /path/privkey.pem # both: the operator's own certificate
off: false # true: plain HTTP behind a proxy that terminates TLS
level: standard # the fleet's default level
reporting: agent # who reports a machine's sanctions: agent | controller | off (see below)
lift: owners # who may lift a machine's sanctions: owners | operator
sources: [googlebot, bingbot, cloudflare] # published exemption lists to fetch (opt-in)
sources_refresh: 24h # on this schedule; unset = on command only
Published exemption sources. Search engines and CDNs publish their
address ranges; naming a source exempts them without pasting a range.
The catalogue is compiled in — googlebot, google-special-crawlers,
bingbot, applebot, cloudflare, jetpack (Automattic's Jetpack callers, for shared hosting), ahrefs (AhrefsBot, the SEO crawler) — and nothing is fetched unless
sources: names it: a fetch is outbound traffic (SPEC principle 5). A
refresh writes exemptions.d/source-<name>.yaml whole (generated, never
hand-edited), reloads and pushes it to every agent; an empty list, a
huge one, or a prefix wider than any crawler list ever is, is refused
and the last good file stays. shield sources shows each source's
state, shield refresh <name|all> fetches now.
# agents/srv00.yaml — the file is named after the agent (its hostname, as
# the machine reports it: `hostname` there — web1.example.net or web1)
agent: srv00
name: srv00 # what the console calls it (default: the short hostname when unambiguous)
description: the infrastructure host
level: strict # scales every rule here: instant | strict | standard | lenient
packs: [openssh, plesk, srv00] # the packs it runs — or, per pack, how:
# packs:
# plesk: {}
# infra: {state: test, level: strict} # this pack test here, at its own level
state: on # on | test — the agent's own may relax it, never arm it
fleet: # the pack is the unit of sharing: another machine's
# ban applies here when its rule belongs to a pack
# both machines run — a rule outside every pack
# never travels, `share: no` on a rule is absolute
offer: no # its sanctions are offered to the machines sharing its packs (default yes)
enforce: no # it applies the sanctions of the machines sharing its packs (default yes)
lift: operator # only the operator may lift its bans
reporting: agent # agent | controller | off
rules: # what this machine changes about a rule — or adds
ssh-bruteforce: {level: lenient} # this rule alone at another level here
http-bad-ua: {threshold: [3/1m, 10/1h]} # its own numbers here
wp-scan: {state: off}
mail-auth: {ban: mail-hardcore}
root-login: # a rule outside its packs, named: it runs here as written
logs: # where this machine's logs are (pushed; empty: its own agent.yaml says)
- {path: /var/log/nginx/*.log, log: http, ports: [80, 443]}
An entry under rules: changes what one machine runs of one rule — its
level (instant, strict, standard, lenient: the rule's numbers
scaled), its threshold outright, its state, its policy — and, for a rule
outside the machine's packs, adds the rule to it. Parsers, policies and
presets are shared, never overridden: a machine that needs a different
regex needs a different rule. The controller assembles each machine's
tree, rewrites only the rules that differ (comments kept), validates the
result with the agent's loader before it leaves, and refuses an agent
older than 0.1.0 by name — it keeps running what it last received, and
the operator is told to upgrade it. shieldlist-controller -t prints
every machine's assembly and what differs. A machine enrolled without a
file under agents/ runs nothing until it is given packs.
Who reports. reporting: — the fleet's default in controller.yaml,
each machine's own in its declaration — says who submits a machine's
enforced sanctions to the abuse provider: agent (the machine itself,
with the key in ITS secrets.yaml), controller (this program, on the
machine's behalf, with the key and daily_limit in the tree's
secrets.yaml — one account, one address, for the fleet), or off. A
machine told controller pauses its own sender; the controller then
reports each of its sanctions once, as it arrives, composed from the
rule's report preset and the fields the convicting lines named — the
same words the agent would have used — and records the outcome on the
sanction (the console says "reported by the controller"; /v1/config
counts them). What was not enforced is not submitted; a report the
provider refused or the allowance could not cover is not retried; a lift
"with withdraw" withdraws the controller's own report where it was the
controller that reported. Detect-only machines report nothing whoever
would report.
Sources of your own — custom_sources: in controller.yaml, each
{name, url|urls, format: lines|cidr-lines|json-prefixes, note}, named
in sources: like the catalogue's; the console adds and removes them.
Enabled sources are fetched when older than sources_refresh (never at
every start), and on command.
Retention — retention: {evidence: 90d, samples: 60d, sanctions: 0}
in controller.yaml (the defaults: evidence and what its lines named
kept 90 days, the agents' counters 60 days, the sanctions for good).
Swept daily; Settings → Maintenance shows what the store holds and
flushes on demand (POST /v1/maintenance/prune, …/vacuum).
Statistics on what the lines named — every convicting line carries
what its parser captured, by group name (user, path… — at most 8, values
cut at 128 bytes; time and ip left out), one row per name in the store
(evidence_fields). To count a new thing, name a group in a pattern —
(?P<path>\S+) — nothing else; GET /v1/sanctions/facets?facet=user &facet=path counts the values under the filters, field=user:admin
filters by one. They live and die with the evidence (retention), and
Settings → Maintenance flushes them apart (fields_older_than).
Hits over time — on every sync the agent sends the positive hits its
rules scored since the last one, by offender (a flow, bounded to the
busiest 2000 offenders, the rest unnamed); the controller resolves the
origins and keeps them per minute (hit_samples: country, network) —
the pressure the chart draws beside the bans, summed per step, under the
server, country and network filters; kept as long as the samples. When
the controller holds none of a machine's flow yet (a first enrolment, a
fresh store), the agent backfills, once, the hits still inside its rules'
windows, minute by minute — as far back as its longest window, no
further: hits that fell out of every window are gone. (Before this the
table held a sampled level — the hits inside the windows — which read as
a flat lie; it is cleared once on upgrade.)
The public unblock page — public: {unblock: on} in controller.yaml
(or Settings → Unblock page, which writes the same keys) opens
/unblock (no account): a visitor sees whether their own address — the
one they come from, as the reverse proxy on the loopback says — is
banned on the fleet, and lifts it themselves in one click (logged). After
whitelist_after self-unblocks within an hour (3 by default) the page
offers a whitelist request; anyone may ask "is this address banned" and
ask for another address to be unblocked — those wait under Requests in
the console (its own menu entry, with the pending count; approve as a
lift or as a fleet-wide exemption, or deny). Off by default; the menu
entry goes with it. Settings: captcha: off|builtin|turnstile
(built-in = a small arithmetic question, no third party; Turnstile needs
turnstile_site and turnstile_secret and loads Cloudflare's script),
language: auto|en|fr (auto = the visitor's browser), title, footer,
footer_url (the page's brand and last line — for embedding it on your
own site), contact (shown to a person still blocked), and the
address detection, for a dual-stack visitor (usually banned twice, once
per family, reaching the page over one of them): direct panel
detection — ipv4_host and ipv6_host, two names of this console
answering over one family each (an A record alone, an AAAA record alone,
the same proxy): the page asks each and unblocks there, the request's
source is the address; or custom resolvers — ipv4_url and
ipv6_url, two endpoints of your own, each reachable over one family
only, answering JSON {"ip": "…"} (an optional "rdns" is shown; other
fields ignored) with Access-Control-Allow-Origin for the console's
origin: the page learns both addresses there, shows the status of the
one of the other family (GET /public/me?value=, no log lines) and, as
nothing proves that address is the visitor's, offers to ask for its
unblock — a request the operator decides; the controller lifts on the
visitor's word only the address their request comes from. Hosts are the
sound choice (both families unblock themselves) and win over URLs when
both are set. Every action is
rate-limited per address and a refusal is logged as
msg="public page refused" why=rate|captcha ip=…; the shieldlist-console
pack's shieldlist-unblock-abuse rule bans an address that hammers it.
API: GET /v1/requests?all=1, GET /v1/requests/count, POST /v1/requests/{id}/approve {action: lift|exempt} or /deny, GET|PATCH /v1/public.
Notifications (mail) — mail: {smtp: host:port, user, password, from, security} in controller.yaml (or Settings → Notifications,
with a test button) is the relay the controller sends through — 587
STARTTLS or 465 TLS by the port, security: none for a trusted local
relay, authentication when user is set; never a delivery of its own.
Who receives is per feature, by account: public.notify_requests: [alice, bob] names the accounts mailed (at the e-mail each set on their
own account) when a request from the public page waits for a decision,
public.notify_unblocks on every self-service unblock; an account
without an e-mail is skipped. One try and a retry, the failure in the
log; a mail never holds a request up. public.show_lines: off hides
from a blocked visitor the log lines that earned the block (their own
requests, shown by default). API: GET|PATCH /v1/mail, POST /v1/mail/test (to the caller's own address, or ?to=).
Reverse DNS — GET /v1/rdns/{address} looks the name up from the
controller (two seconds, remembered an hour); the address dialog shows it
beside the address, without waiting.
Reputation (AbuseIPDB) — with a secrets.yaml in the controller's
tree (abuseipdb: {key: …}, the same shape as an agent's) the console
can ask what the provider knows about an address before a ban by hand
(GET /v1/reputation/{value} — from the store when checked within a
day, ?fresh=1 asks again, ?cached=1 never asks; each question spends
one of the account's daily checks, counted apart from reports), and the
controller asks daily about each server's own address, shown on its
card. Nothing is checked automatically beyond that.
Clocks — every sync carries the agent's clock; the controller keeps
the difference (clock_skew on GET /v1/agents, seconds, agent minus
controller) and the console warns past thirty seconds ("clock off by
2m"): a machine that disagrees about the time disagrees about every
window, every ban's end and its place on the charts. The agent warns on
its side too (shield status: controller_clock). Fix NTP; nothing else
is adjusted for it.
Updates — updates: off stops the daily question to git.lrob.net
about the latest releases; on, the console says when a newer controller
or agent is published (a notice, never a download).
A machine's own exemptions — exemptions: in agents/<host>.yaml
([{ip|range, until?, note?}]), assembled into its bundle as
exemptions.d/machine.yaml; the console writes them (Exempt… → only
this server) and removes them.
Where addresses are from — geoip: in controller.yaml. The
files the controller holds are served to its agents: every sync
carries their content hashes, an agent holding another edition (or none)
fetches them under its state directory (/var/lib/shieldlist/geoip/) and
loads them — no configuration on the machine — which is what the rules'
countries: / asns: and the exemptions by country or AS run on. A
standalone agent names files of its own in agent.yaml (geoip: {country: …, asn: …}). Either
source: dbip (the free DB-IP lite country and ASN databases, fetched
monthly under <state>/geoip/, licence CC BY 4.0 — the console shows
"IP Geolocation by DB-IP"), or files of your own: country: and asn:
naming MaxMind-DB files (GeoLite2 or any provider's). Off unless
configured. Sanctions are enriched as they arrive (country, AS number
and name); rows from before are backfilled; the console shows a flag and
the network, filters and counts by them.
The console is served at / on listen.api. Home: an address
lookup and the sanctions as they land — a live feed (play/pause) filtered
by state (banned now, bans, detections, expired, lifted), server, rule
and dates, as many rows as asked, the fields of your choosing, the
requests behind each sanction a toggle away. Servers: each machine with
four tabs — overview, its bans (the same feed, for it), its rules (the
fleet's defaults at its level or its overrides, changed or reset per
rule) and its settings. Rules and Exemptions are the fleet's; Activity is who did what;
Console is the panel itself — people, tokens, the configuration files,
this browser's preferences. An address opens as a dialog wherever it
appears: banned where, its history, each sanction's convicting lines,
and the two actions — lift (the reason is optional) and exempt (the red
one: never sanctioned again while it lasts), each able to withdraw what
was reported about the address. People sign in with a name and password —
shieldlist-controller account -name NAME creates the first account at
the console, account -name NAME -totp enrols a one-time code — or by
pasting an API token. Sessions last twelve hours; every action is
audited under the person's name.
The API answers on listen.api (localhost by default; on the
network, put it behind TLS you control), JSON, with API tokens:
shieldlist-controller api-token -name NAME mints the first one at the
console, POST /v1/tokens the next. shield -controller URL -token T
(or SHIELD_CONTROLLER, SHIELD_TOKEN) is its client: agents, agent <host>, set <host> key=value… (level, state, lift, reporting, offer,
enforce, description, packs=a,b, rules.<rule>.{level,threshold,state,ban},
rules.<rule>=none), rules, rule <name> key=value… (state,
threshold — 5/10m or 3/1m, 20/1h — ban, report),
why, unban [-agent] [-withdraw-report], exempt [-lift [-withdraw-report]], reload, validate, audit,
token, sources, refresh, version. Every write is a write to the
files — agents/<host>.yaml created or edited in place, the rule's own
file edited in place, comments kept — reloaded whole or reverted with
the loader's errors as the answer. Endpoints: GET /v1/version,
GET /v1/agents[/{host}], GET|PATCH /v1/agents/{host}/declaration
(level, state, lift, reporting, offer, enforce, description, packs: [{name, state, level}], logs: [{path|unit, log, ports}], rules: {name: {level|threshold, state, ban} | {} to add | null to drop}),
GET /v1/agents/{host}/rules (what the machine runs: each rule with the
packs that brought it, its threshold there and why, its state, outcome),
the rule language — GET /v1/packs, POST /v1/packs {name, description, rules}, PATCH /v1/packs/{name} {description, rules}, DELETE /v1/packs/{name} (refused while a machine is given it); GET /v1/rules
(each with its parsers — {parser, weight} or {regex, prefilter, inline} —, threshold, ban, notify, report, count, state, share, the packs
listing it, the servers running it), POST /v1/rules {name, dir, category, description, log, parsers, threshold, ban, notify, report, count, distinct, state, share}, PATCH /v1/rules/{name} (any of those;
parsers replaces the list; ?preview=1 on either answers {path, before, after} without writing — the diff the console shows), DELETE /v1/rules/{name}; GET /v1/parsers (inline ones marked with their
rule), POST /v1/parsers {name, dir, log, prefilter | prefilter_any, regex, time_field, time_format}, PATCH|DELETE /v1/parsers/{name},
POST /v1/parsers/test {regex, prefilter, prefilter_any, lines} (each
line: gated, matched, the fields captured); GET /v1/policies, POST /v1/policies {name, durations, ports (all | service | "80, 443"), memory, tighten, duration_start, reputation ("90/30d")}, PATCH|DELETE /v1/policies/{name}; GET /v1/reports, POST /v1/reports {name, to, when, categories, says, include}, PATCH|DELETE /v1/reports/{name}
(editing the built-in default writes reports/default.yaml); GET|PUT /v1/files/{path} (one file of the tree whole — the "edit as file" view;
?validate=1 on PUT tries it through the loader and puts it back) —
every write a file created, edited in place or removed, reloaded whole or
reverted; POST /v1/agents/{host}/resync
(the machine sends its whole history again, evidence included — a
machine enrolled again does so on its own), GET /v1/sanctions?state= &agent=&rule=&q=&country=&asn=&since=&until=&limit=&offset=&evidence=1
(newest first, bounded; state is active, bans, recorded, expired,
lifted or all; rule contains; q is one search — an address or its
start, a rule, a server, a country code, an AS by number or name;
since/until RFC 3339 on the start; evidence=1 adds each sanction's
convicting lines), GET /v1/sanctions/facets?…&n=&hits=1 (same filters: top
countries, networks, rules and usernames, n ≤ 300, the offenders being
counted tallied in with hits=1 — the console's panel and map), GET /v1/geoip
(what is loaded, the attribution), GET /v1/sanctions/series?…&window=24h&step=30m&since=… (counts per step,
bans and detections apart — the console's chart; since inside the window
answers the tail alone, on the same grid), POST /v1/bans {value, agent?, duration?, reason?, report_as?, lines?} (a ban by
hand: on one machine or every enrolled one, duration like 6h, 1y,
or permanent; applied by each agent at its next sync through the same
checks as any sanction, recorded as rule manual with the reason as its
evidence, reported when a preset is named — with the facts of the
detection's lines when given, as the rule's own report would), GET /v1/presets (the report
presets a manual ban may be reported as), GET /v1/exemptions (every exemption in force with its origin — the
fleet's files, the sources, the console's file, a machine's declaration —
and whether the console may remove it), DELETE /v1/exemptions {value, agent?}, POST /v1/exemptions {…, agent?} (one machine's own),
PATCH /v1/sources {enable, disable, refresh}, POST /v1/sources/custom {name, url, format, note}, DELETE /v1/sources/custom/{name},
GET /v1/updates, GET /v1/watched?agent=&q=&country=&asn=&value=&exempt=1&limit= (the offenders the agents are counting
but have not convicted — how far from a ban, as of each agent's last
sync; offenders exempt on their machine carry exempt (why) and are
listed only with exempt=1), GET /v1/networks/{asn} (an autonomous
system's name and prefixes from the ASN database), GET|PATCH /v1/me
(who is asking; a person may change their own password and email), GET /v1/sanctions/{id} (one,
lines included), GET /v1/offenders/{address} (its whole history, lines
included), POST /v1/lifts {value, agent?, reason?, withdraw_report?}
(withdraw_report asks the machine to withdraw what it reported to
AbuseIPDB about the address — the provider's clear-address, every report
of the account about it), POST /v1/exemptions {ip|range, for?, note?, lift?, withdraw_report?} (a write to exemptions-cli.yaml, reloaded
whole or reverted; lift also lifts the bans in force wherever they
are), GET /v1/me, GET /v1/config, POST /v1/config/{reload,validate},
GET /v1/sources, POST /v1/sources/{name|all}/refresh, GET /v1/audit,
POST /v1/join-tokens, GET|POST /v1/tokens, DELETE /v1/tokens/{name},
GET|POST /v1/accounts, POST /v1/accounts/{name}/{password,totp},
DELETE /v1/accounts/{name}[/totp].
Every action is audited under the token's name.
At the console, without the API: shieldlist-controller token [-uses N] [-expires 24h] mints a join token; agents, why <address> and `unban
Several parsers, one jail
parsers: takes a parser name, a list of names, or weighted entries —
[ssh-auth-fail, {parser: ssh-preauth-abort, weight: 2}]. Every hit of
any of them counts for the rule (one event, one hit: do not list two
parsers that both match the same line of the same attempt).
Reporting again while the attack goes on
reporting:
provider: abuseipdb
renew: 24h # an address still attacking while banned is reported again after this long; `never` turns it off
A ban that is extended because the hits keep coming (check 5) reopens
its report once the last one is older than renew — the sender picks
it up on its next pass, with the latest hit as the report's time. A
report that never left is not renewed: its one chance was taken.
Compact scalars
| Form | Meaning |
|---|---|
30s, 10m, 2h, 1h30m, 30d, 4w |
durations (days and weeks on top of Go units) |
5/10m |
a threshold: points within a window |
3x, 0.5x |
evidence multiplier |
FR / AS3215 / 198.51.100.0/24 |
origin, auto-detected |