Reliability engineering for systems where the output isn't the same twice, where money has to reconcile to the penny, and where one tenant must never see another's data. Working code — run it from the repo.
The browser-based tools that work before a line of code is written. Listed here for the full picture — try them on the main toolkit page.
Paste a user story, get acceptance criteria bucketed into positive, negative, and edge case — each tagged with a test strategy. For when the team goes silent in the three amigos session.
Paste an oversized story, get back vertical slices — each with its own dev scope, QA scope, and a verdict on whether it's independently releasable. Stops teams creating sub-tasks instead of properly splitting work.
Contracts and prototypes that aren't finished products yet — described honestly as such. QA Sentinel's rubric is published; FORGE has two of its eight agents running.
A GitHub Action that scores PRs against a published QA rubric. The RUBRIC.md and rubric.yml ship under MIT licence as a verifiable open-source artifact — a real contract that separates this from generic LLM reviewers. The Action code that implements the rubric is not built yet.
Form: GitHub Action (CI), not a web tool. The rubric is the source of truth the Action will implement.
A multi-agent pipeline that turns a plain-English requirement into executable Playwright + Cucumber BDD. Two of the eight agents run today — the Feature Agent (requirement → Gherkin, happy path + a negative scenario per business rule) and the Step Definition Agent (Gherkin → typed step stubs). The remaining six (POM, Locator/Inspector, Code Review, Healer, Execution, Report) are designed, not built.
Runs offline with a deterministic brain — zero API keys for the demo, with a clean seam to swap in the Claude API. Assembly-line framing inspired by the public TAAL concept; an independent implementation, not affiliated.
Five QA problems an autonomous, multi-tenant, AI-driven platform actually has. Each is working code you run from the repo — not a hosted web tool, and labelled as such.
An AI agent classifies every inbound conversation, so assert actual == expected is useless. This harness uses graded assertions — exact on labels, tolerance on confidence, meaning-overlap on free text — so it fails on real degradation but tolerates harmless rephrasing. A FAIL exits non-zero and blocks the CI gate.
# golden model — reworded summary still PASSes [✓] conv_new_lead_002 → PASS exit 0 # regressed model — real degradation caught [✗] conv_new_lead_002 → FAIL └─ exact:intent FAIL (expected 'new_lead', got 'general_enquiry') └─ actions:required FAIL (missing ['book_callback','create_lead']) SUITE VERDICT: FAIL exit 1 # ← blocks the gate
Reproducible via recorded fixtures + replay. Zero API keys to run the demo.
Proves no tenant can ever see another's data, API responses, or routed events. Every test stands up two tenants and asserts one can never observe the other across all three boundaries. A buggy mode injects the two classic leaks to prove the suite catches them.
pytest for the clean run · BUGGY=1 pytest to watch the leaks get caught.
Reconciles metered usage — tokens, voice minutes, SMS — to the invoice, to the penny. Catches undercounts, wrong unit prices, phantom line items, and tampered totals.
Exact Decimal arithmetic, never float. The total is independently verified, not trusted. Stripe test-clock ready.
The full onboarding→inbox journey written in both frameworks, plus a documented decision rule for when migration pays off — flake, multi-tab, slow CI, hard-to-debug failures.
Fixed cy.wait() becomes auto-retrying web-first assertions. Run both suites one release before retiring the old one.
A blocking GitHub Actions gate — tests, lint, security, migrations, AI eval — plus a metrics engine that turns CI history into flake rate, coverage trend, and p50/p95 feedback time: one health score and a PR comment.
A flake is defined precisely — a test that passes and fails on the same commit — so you don't chase tests that were correctly failing on broken code.