The harness reads your scenario, drives your Spring Boot app through real HTTP and SQL calls, and passes judgment on every step — reported as ordinary JUnit tests.
you@vps:~/app$
This is the entire test. No endpoints, no request bodies, no assertions in code — the harness works out the calls from your OpenAPI spec.
Drop .md files under src/test/resources/scenarios/.
Each ## heading is a step; write it as strictly or as loosely as
you like — fenced requests, Gherkin, or plain intent.
One test-scope dependency, one @Harness annotation, one empty
@Scenario method per scenario — the Markdown is resolved from
the method name.
An LLM actor reads each step, calls your running app over HTTP (or checks the database over SQL) and issues a verdict. Every step shows up as its own JUnit sub-test in your IDE and CI.
@Harness(scenarioDir = "classpath:scenarios/") @SpringBootTest(webEnvironment = RANDOM_PORT) class ScenarioSuiteTest { @Scenario void openAccountAndDeposit() {} @Scenario void overdraftRejected() {} @Scenario void transferBetweenAccounts() {} }
The same test, written three ways — and these three are just the demo's showcase buckets, not a fixed list. Markdown is the only grammar: write the dialect your team already speaks, or invent your own. It runs unchanged.
# Open an account and make a deposit ## Step 1 — Open an account **Intent:** Open a new account for "Alice" in USD. ``` POST /accounts { "owner": "Alice", "currency": "USD" } ``` **Expected response** - Status: 201 Created - Body has a numeric id - balance is 0.00 ## Step 2 — Deposit funds **Intent:** Deposit 500.00 into the account opened in Step 1. ``` POST /accounts/{id}/deposits { "amount": 500.00 } ``` **Expected response** - Status: 201 Created - balance is 500.00
Prescriptive. Fenced requests and bulleted expectations — the closest to a conventional test, and the easiest for a smaller model to follow.
# Open an account and make a deposit ## Open the account - **Given** Alice has no account yet - **When** she opens a new account in USD - **Then** the response is 201 Created - **And** the returned account has a numeric id, a balance of 0.00, and currency USD ## Make a deposit - **Given** Alice's newly opened account - **When** 500.00 is deposited into it - **Then** the response is 201 Created - **And** her balance now reads 500.00
Gherkin, without Cucumber. Given/When/Then your QA team already writes — except there are no step definitions, no glue code, no regex. The prose is the implementation.
# Open an account and make a deposit ## Open an account Open a new bank account for a customer named "Alice", denominated in US dollars. It should be created successfully and start with a zero balance. ## Deposit funds Deposit 500.00 into Alice's newly opened account. The deposit should succeed and her balance should then read 500.00. ## Confirm the balance Look Alice's account up again and confirm its balance is 500.00 and its currency is US dollars.
Pure intent. Names no endpoints, paths, or payloads. With OpenAPI discovery enabled, the harness reads your spec and works out the calls itself — rename /accounts and this test still passes.
There's no dialect registry and nothing to configure — if your team writes RFC-style specs, checklists, or its own house style, drop those files in as-is. Anything a model can read, the harness can run.
Harness runs narrate themselves — Markdown-rendered, Gruvbox-themed console output with per-step verdicts and timings. This block is styled with the palette the harness actually ships.
SCENARIO ▶ Open an account and make a deposit ▶ RUN # Open an account and make a deposit STEP 1/3 ▶ gemma-4-31b ▶ Open the account POST /accounts … 201 Created STEP 1/3 ▶ Open the account ▶ PASS ⧖ 9.4s STEP 2/3 ▶ Make a deposit POST /accounts/1/deposits … 201 Created STEP 2/3 ▶ Make a deposit ▶ PASS ⧖ 7.1s STEP 3/3 ▶ Read the account back GET /accounts/1 … 200 OK ⚖ judge: verdict grounded in tool trace — PASS STEP 3/3 ▶ Read the account back ▶ PASS ⧖ 6.8s SCENARIO ▶ Open an account and make a deposit ▶ 3/3 PASS ⧖ 23s BUILD SUCCESSFUL in 41s
The ⚖ judge line is the optional evaluation module: a second model scoring the actor's verdict against the real tool-call trace.
A scenario is living documentation. Product owners and QA can review it in a PR, or author it themselves. No DSL to learn; Markdown is the DSL.
Intent-style scenarios don't hard-code paths or payloads. Restructure your API and the test that states the business outcome keeps passing.
Calibrated by mutation testing: the demo suite runs against deliberately buggy builds and the harness is expected to catch every injected fault — the current baseline flags 3 of 3 planted bugs.
Each step reports as its own sub-test. Your IDE, Gradle, and CI treat scenarios like any other test class — green ticks, red traces, standard reports.
Optional LLM-as-judge evaluation re-scores every verdict against the recorded
HTTP/SQL trace, and ./gradlew evaluate writes an HTML report —
so you can measure how trustworthy your oracle is.
Built and calibrated against a local dense model. No per-run API bill, no test data leaving your machine — or point it at any Spring AI-supported provider.
An LLM run can flake where a hand-written assert cannot. Treat this as your integration/acceptance layer — it complements your unit suite, it doesn't replace it.
You bring the brain: an API key (cost, latency) or a capable local model. Small and MoE models tend to hallucinate multi-step tool calls — a strong dense model is recommended.
The actor can occasionally rubber-stamp a verify step without really checking. That's precisely why the evaluation module exists — measure the false-positive rate, don't assume it's zero.
A scenario is a real conversation with a real app — think seconds per step, not milliseconds. Run scenarios where you'd run integration tests, not on every save.
APIs still move between minor versions. Pin via the BOM and read the release notes.
The actor currently speaks HTTP and SQL. Message queues and a mock server are on the roadmap, not in the box.
If your suite needs millisecond determinism, this isn't your tool — and that's fine. Inquisitor is for the layer where you'd otherwise write brittle end-to-end glue or click through the app by hand.
dependencies { testImplementation(platform("io.inquisitor:inquisitor-bom:0.6.0")) testImplementation("io.inquisitor:inquisitor-harness-junit-starter") }
# write a scenario, annotate a test class, then: you@vps:~/app$ INQUISITOR_LLM_IT=true ./gradlew test # with LLM-as-judge evaluation + HTML report: you@vps:~/app$ ./gradlew evaluate
Configure your model like any Spring AI chat client — application.yml, no harness-specific plumbing.