dnsmasq adds IPs to an nft set only on a fresh upstream reply; a cache
hit never refreshes the set. On Archie the :5350 instance also wedged:
TCP child processes first, then the main process, logged
'nftset ... netlink: Error: cache initialization failed: Protocol error'
on every add and stopped filling bbrkn_v4/v6 until restarted.
warm-nftset.sh now counts those errors for the main PID during the run
(journalctl _PID=) and, if any, restarts the service and warms again;
exit 1 if it still fails. The state dir defaults to /var/lib/bbrkn so the
root cron job and the deploy share one lock (LOCK_WAIT lets deploy wait).
deploy-to-gateway.sh runs the warmer right after flushing the sets,
since clients keep old IPs cached and would bypass the tunnel until the
next fresh resolve. A warm failure only warns; it does not roll back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DR7zpSsdX28T4Kn6v4jPSS
Backups were written as <conf>.backup.<ts> next to the live file. Both
dnsmasq instances load every file in their conf dir (the host excludes
only .dpkg-*), so each backup was parsed as live config, resurrecting
removed domains and piling up one stale copy per deploy.
Write them to BACKUP_DIR (/var/backups/bbrkn) instead and keep the
newest BACKUP_KEEP (10) per file.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DR7zpSsdX28T4Kn6v4jPSS
The gateway (Archie) routes bbrkn domains via nft sets bbrkn_v4/bbrkn_v6
(inet filter), not the legacy ipset. bbrkn was still emitting dead
`ipset=/domain/bbrkn` directives (no such ipset exists) plus a 92-resolve
pointing at 8.8.8.8, both overridden by hand-maintained files on the host.
This makes bbrkn the generator of record for the real scheme.
generate-configs.sh:
- emit `nftset=/domain/$NFTSET_SPEC` (default 4#inet#filter#bbrkn_v4,
6#inet#filter#bbrkn_v6) into 90-nftset.conf instead of ipset= into
91-ipset-bbrkn.conf
- DNS_SERVER default 8.8.8.8 -> 127.0.0.1#5350 (host dnsmasq pihole
delegates bbrkn domains to for VPN resolution + nftset capture)
deploy-to-gateway.sh: two targets, two instances
- 90-nftset.conf -> host /etc/dnsmasq.d (:5350), 92-resolve -> pihole
- full restart of both (dnsmasq SIGHUP does NOT re-read nftset=/server=)
- flush nft sets bbrkn_v4/bbrkn_v6 instead of `ipset flush bbrkn`
- add end-to-end nftset-capture health check via :5350
- rollback restores both files and restarts both instances
Makefile/workflow: rename IPSET_CONF->NFTSET_CONF, add NFTSET_TARGET_DIR
and HOST_DNSMASQ_SVC, DNS_SERVER=127.0.0.1#5350 (escaped `\#` in Make,
quoted in YAML), note runner is ephemeral (cold gekata crawl).
Docs: README + new CLAUDE.md describe the two-dnsmasq / nft-set model;
exit-node DPI failover (10.77.1.2/10.77.2.2) documented as external
(wg-ha.service), not owned by bbrkn.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
