Scaling plan

Every server owner needs one. This is the capacity model for a 4got instance, worked for the reference deployment (1 vCPU, 1 GB RAM, 300 mbps, 1 TB/month), and the API-key policy that survives growth.

The surprise: bandwidth is the binding constraint, not compute

Go handles thousands of concurrent connections on one core; a cached search serve is sub-millisecond work. The ceilings, in the order you will actually hit them:

1. Monthly traffic cap. 1 TB/month ≈ 33 GB/day ≈ 385 KB/s sustained. A results page is ~100 KB raw, ~30 KB with compression, before proxied images. That prices the box at roughly ~1M page views/month as an absolute ceiling — and proxied images eat it far faster, which is why the traffic guardrails throttle heavy transfers first while search stays fast. Watch the Traffic line in settings; the knobs are traffic-* in config.kdl.
2. Upstream engine tolerance. Every UNCACHED search fans out to dozens–hundreds of engines from one IP. Public engines throttle single IPs long before your hardware blinks. The realistic uncached-fanout budget is hundreds to low thousands per day per engine — the communal cache’s hit rate is therefore the real multiplier of the whole system. Everything that raises hit rate (shared anonymous cache, prefetch, peers) is scaling work.
3. Compute/RAM. Last by a wide margin: ~50–100 simultaneous uncached fanouts before TLS handshake latency degrades on one core; cached serves in the hundreds per second; SQLite’s single writer is nowhere near stressed at these rates.

Net: thousands of daily active users are realistic on the reference box if the cache hit rate is healthy; millions of monthly uniques require more egress — a bigger pipe, and more usefully, peers: federated instances sharing caches are the horizontal-scaling story, and each peer brings its own upstream IP budget.

API keys: the tiered policy

Server-covered keys do not scale — a 25-req/day API handed to the public is exhausted by breakfast and everyone’s annoyed. The policy that scales:

• Trusted groups (deluxe: friends, family, beta testers) — keys covered by the server, limits shown in settings.
• Everyone else — bring your own key. BYO keys can opt into the pool: draws are randomly scrambled across contributed keys, which (a) gives contributors anonymity-by-randomness against the API provider — the provider sees the server’s IP either way, but key-scrambling breaks the which-human-asked-what linkage that identity-tied keys otherwise leak — and (b) absorbs surges, since most people use a fraction of their quota on most days, so a member can burst far beyond their own key some days (their agents too) against the pool’s slack.
• Grace budget (Monero era): pool members may run negative within a configured grace; the deeper negative, the harder the scheduler works to lend their key out. Balances settle when the ledger lands.
• ToS note for owners: some API providers prohibit key pooling. Enabling the pool is the owner’s call per API; the per-key limit fields and group gating already let you keep sensitive keys unpooled.

Changing server defaults without fighting users’ cookies

Revoking a covered key, changing a default, renaming a setting — all must be supported behavior, not a migration crisis. Mechanism (planned): a settings epoch — the server stamps an epoch into served settings; cookies carry the epoch they were written under; on mismatch the server migrates known-renamed keys, drops known-dead ones, and (where a change is user-visible) says so once. Until the epoch machinery lands, the rule of thumb: additive changes are free; renames keep a legacy-read fallback (as ai_mode → ai_mode_single/double does today); revocations of covered keys degrade to “bring your own” messaging rather than errors.