goodvin
d81a4ab3b2
The gateway (Archie) routes bbrkn domains via nft sets bbrkn_v4/bbrkn_v6 (inet filter), not the legacy ipset. bbrkn was still emitting dead `ipset=/domain/bbrkn` directives (no such ipset exists) plus a 92-resolve pointing at 8.8.8.8, both overridden by hand-maintained files on the host. This makes bbrkn the generator of record for the real scheme. generate-configs.sh: - emit `nftset=/domain/$NFTSET_SPEC` (default 4#inet#filter#bbrkn_v4, 6#inet#filter#bbrkn_v6) into 90-nftset.conf instead of ipset= into 91-ipset-bbrkn.conf - DNS_SERVER default 8.8.8.8 -> 127.0.0.1#5350 (host dnsmasq pihole delegates bbrkn domains to for VPN resolution + nftset capture) deploy-to-gateway.sh: two targets, two instances - 90-nftset.conf -> host /etc/dnsmasq.d (:5350), 92-resolve -> pihole - full restart of both (dnsmasq SIGHUP does NOT re-read nftset=/server=) - flush nft sets bbrkn_v4/bbrkn_v6 instead of `ipset flush bbrkn` - add end-to-end nftset-capture health check via :5350 - rollback restores both files and restarts both instances Makefile/workflow: rename IPSET_CONF->NFTSET_CONF, add NFTSET_TARGET_DIR and HOST_DNSMASQ_SVC, DNS_SERVER=127.0.0.1#5350 (escaped `\#` in Make, quoted in YAML), note runner is ephemeral (cold gekata crawl). Docs: README + new CLAUDE.md describe the two-dnsmasq / nft-set model; exit-node DPI failover (10.77.1.2/10.77.2.2) documented as external (wg-ha.service), not owned by bbrkn. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
53 lines
6.2 KiB
Markdown
53 lines
6.2 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## What this is
|
|
|
|
Generates **Pi-hole / dnsmasq** configs that route blocked domains through a WireGuard VPN via **nftables sets**. Input is `domains.txt`; output is two dnsmasq files deployed to **two different dnsmasq instances** on the Archie gateway (see Architecture). README.md (Russian) is the authoritative reference — read it for the classification table, env-var descriptions, and report format.
|
|
|
|
> **Routing model (Archie gateway):** client → pihole (docker, `:53`) delegates bbrkn domains via `92-resolve-bbrkn.conf` (`server=/domain/127.0.0.1#5350`) to a **host dnsmasq on `:5350`**, which resolves via VPN and applies `90-nftset.conf` (`nftset=…#bbrkn_v4,…#bbrkn_v6`) to populate nft sets `inet filter bbrkn_v4`/`bbrkn_v6`. An nft mangle chain marks packets to those set IPs (`0x1`/`0x2`) and policy-routing sends them out a WireGuard exit. Dual-exit DPI-bypass failover (`10.77.1.2`/`10.77.2.2`) is handled by `wg-ha.service`, **external to bbrkn** — bbrkn only maintains domain→set membership.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
make all # generate + deploy (full cycle)
|
|
make generate # generate configs into /tmp only
|
|
make deploy # deploy existing configs to Pi-hole
|
|
make check # validate domains.txt syntax (regex per line)
|
|
make clean # remove generated /tmp configs
|
|
make cache-clean # wipe .cache/api/
|
|
|
|
./scripts/generate-configs.sh --dry-run # generate, print report, write nothing
|
|
DEBUG=1 make generate # verbose log to $DEBUG_LOG
|
|
STRICT_MODE=1 make generate # exit 1 if any domain's API query failed
|
|
CACHE_TTL_DAYS=0 make generate # bypass API cache for one run
|
|
```
|
|
|
|
No test suite. Verification = `--dry-run` + reading the DEBUG REPORT (see README) and diffing generated `/tmp` configs.
|
|
|
|
## Architecture
|
|
|
|
Two bash scripts, driven by env vars (defaults in each script's header, overridden by Makefile and CI):
|
|
|
|
**`scripts/generate-configs.sh`** — classifies each domain by querying a Chromium headless API (`$CHROME_SERVER/domains?domain=`), builds the ordered domain list, writes both config files in one batch. Key logic:
|
|
|
|
- **Classification** (drives whether subdomains are added): API JSON with `relatedDomains` → *site* (base + related subdomains); `ERR_NAME_NOT_RESOLVED`/`Timeout` → *dead*, skipped entirely; any other API error → *service* (base only, no subdomains); API fully unreachable → *fallback* (base only, recorded under `API FAILURES`). **Invariant: every domain in `domains.txt` reaches the output unless it's dead.**
|
|
- **State is held in bash associative arrays** (`DOM_ROLE`, `EXPANDED`, `SOURCES`, `SITE_RELATED`, …). The main loop deliberately avoids subshells so counters survive — do not wrap counter-mutating code in `$(...)` or pipes.
|
|
- **Caching**: successful API responses cached to `$CACHE_DIR/<domain>.json`, reused while newer than `CACHE_TTL_DAYS`; corrupt or expired entries refetch. `find -mtime +N` decides freshness.
|
|
- **Ordering pass** emits each base domain, a `# domain — N subdomains` comment, then its not-yet-written related domains; a related domain is attributed to whichever base comes first alphabetically. A final uniqueness check `exit 5`s (config NOT written) if any domain would appear twice — treat that as a real bug in the ordering logic, not a data issue.
|
|
- Output lines: `nftset=/<domain>/$NFTSET_SPEC` (NFTSET_CONF → `90-nftset.conf`; default `NFTSET_SPEC=4#inet#filter#bbrkn_v4,6#inet#filter#bbrkn_v6`) and `server=/<domain>/$DNS_SERVER` (RESOLVE_CONF → `92-resolve-bbrkn.conf`; `DNS_SERVER` default `127.0.0.1#5350`, the host resolver pihole delegates to).
|
|
|
|
**`scripts/deploy-to-gateway.sh`** — deploys to **two targets**: `90-nftset.conf` → host `$NFTSET_TARGET_DIR` (`/etc/dnsmasq.d`, the `:5350` instance) and `92-resolve-bbrkn.conf` → pihole `$RESOLVE_TARGET_DIR` (`/opt/appdata/pihole/etc/dnsmasq.d`). Flow: timestamped-backup both → copy → **`systemctl restart $HOST_DNSMASQ_SVC` + `docker restart $DOCKER_CONTAINER`** (a full restart is required — dnsmasq **SIGHUP does not re-read `nftset=`/`server=` directives**, only hosts files) → wait for `:5350` listening + pihole `running` → `nft flush set` on `bbrkn_v4`/`bbrkn_v6` → DNS health check via pihole `:53` → end-to-end nftset-capture check (resolve a bbrkn domain via `:5350`, assert the set populated). **On any failure it calls `rollback()`**, restoring both backups and restarting both instances. Edits must preserve that rollback path — a broken deploy can break DNS for every client on the gateway.
|
|
|
|
## Deployment / CI
|
|
|
|
`.forgejo/workflows/deploy.yaml` runs on push to `main` (and `workflow_dispatch`) on a **self-hosted `forgejo-runner` on the gateway itself** — no SSH, deploy is local. Note: the runner is **ephemeral** (no persisted `.cache/api` was found on the host despite the older "persists workspace" claim), so a fresh CI run performs a **cold gekata crawl of every domain** (~30-70s each, serialized) — the first run after a cache reset is slow and heavy on gekata. CI runs `make clean && make all` and uploads the generated configs + debug log as artifacts even on failure. CI-only values (`NFTSET_TARGET_DIR`, `TARGET_DIR`, `CHROME_SERVER`, `DOCKER_CONTAINER`, `HOST_DNSMASQ_SVC`, `DNS_SERVER`, `IGNORE_PARTS`) live in the workflow `env:` block, not the scripts.
|
|
|
|
## Gotchas
|
|
|
|
- `domains.txt` is the primary data file; commits are typically domain additions. Run `make check` after editing it.
|
|
- `IGNORE_PARTS` is space-separated substrings; any domain (base or related) containing one is dropped.
|
|
- **`#` gotcha**: `DNS_SERVER=127.0.0.1#5350` — `#` starts a comment in Makefiles (escape as `127.0.0.1\#5350`) and can start one in YAML (quote it). The scripts themselves take the plain value.
|
|
- The nft sets (`bbrkn_v4`/`bbrkn_v6`, `inet filter`) and mangle/routing rules live on the gateway (`/etc/nftables.d/`, Ansible-managed) — bbrkn only fills the sets via dnsmasq, it does not define them.
|
|
- Requires `bash` 4.2+ (associative arrays, `${var,,}`), `curl`, `jq`, `docker`, `nft`, `systemctl`; `dig`/`nslookup` optional (health checks skipped if absent).
|