the Solid take-home

solid-takehome is my solution to Solid's backend interview take-home: a distributed remote-bash-execution system in TypeScript with Effect — API servers, a CLI client, an executor daemon, Postgres. The repo is public, and I work at Solid now. It records no score.

The assignment is preserved verbatim in-repo, and it is a resilience spec: commands to 1KB, output to 1MB; survive latency, packet loss, and network, database, or server outages up to 60 seconds; the client exits failed within 120 seconds; no conceptual races — "we may add 100ms sleep statements anywhere." I committed a constitution of fourteen invariants in the first PR, before implementation, and worked a 10-PR roadmap with per-PR specs.

The load-bearing decision: Postgres is the only clock. Every lease and session expiry compares against NOW(); no machine clock is consulted; the schema gives sessions no status column, because connectivity is a predicate over timestamps; servers hold no state that correctness depends on. The alternative, in-memory server state, dies with the server. Timestamps do not.

Signaling splits along the same line. LISTEN/NOTIFY wakes workers as a best-effort hint; dropped notifications are tolerated because a tick-based catch-up fetch guarantees progress. Correctness comes from FOR UPDATE SKIP LOCKED claim transactions and clients replaying output from a cursor. Output is an append-only log with a per-attempt monotonic sequence number, so duplicates and reconnects are harmless by construction. The alternatives were per-chunk NOTIFY, or pure polling.

failed is a client-side verdict only. The 120-second detector lives in the CLI and resets only on genuine progress, and the database never records failed for a command a client gave up on. I built the alternative first, a server-side timeout column, and removed it as out-of-spec, migration and all.

Numbers: 29 commits, 2025-12-29 to 2026-01-04, 22 of them merged PRs; 113 test cases across 15 files, each vitest worker in its own throwaway Postgres schema; 5 runtime dependencies; 5 tables in 100 lines of SQL; 7 chaos scripts covering partition, packet loss, latency, DB outage, server restart, and executor kill; zero setInterval calls in src.

What is wrong with it. The repo's own coverage matrix marks 24 requirements "needs test," including the 1KB and 1MB limits. The constitution still forbids the client NOTIFY channel that PR #20 shipped. Chaos testing is manual scripts against live VMs, and there is no CI. The final commit, four days after the rest, removes pubsub timeouts that had been silencing streams — my own bug outlived the deadline.

The design held under iptables. The documentation did not keep up with it.