ADR-00023: CTN Test Strategy — layers, ownership, and CI gating

Approval Status

Status flow: Proposed → Accepted → Reviewed → Approved (or Rejected), or Not Required. A superseded ADR keeps its file; only its status changes to Superseded by NNNNN.

Governance body	Status	Last update
CTN Project Team	Proposed	2026-07-08
CTN Technical Advisory Board	Pending	—
CTN Steering Committee	Pending	—
BDI Conformance Team	Pending	—
BDI Framework Team	Pending	—

Context and Problem Statement

Several decisions already touch testing, and two of them look alike at first glance: ADR-00015 (in-repo Java browser tests) and ADR-000017 (a strategic Playwright + TypeScript automation framework). They both pick Playwright over Cypress and Selenium, which makes them read as duplicates — but they operate at different layers and answer different questions. There was, until now, no single document that says how the whole test estate fits together: what is tested at which layer, which tooling is used where, who owns and contributes each layer, and what runs on every deploy.

This ADR is that overview. It does not re-decide anything the layer-specific ADRs already settled; it places them in one picture and makes the boundaries explicit so the two Playwright ADRs no longer read as overlapping.

The guiding goal (raised by the product owner) is old-fashioned test-driven development: a growing, shared suite where the product owner, testers, and end-users can add test cases; where prototype API test cases are carried forward rather than thrown away; and where the suite runs on every deploy so regressions surface early.

Decision Outcome

CTN adopts a layered test strategy. Each layer has a defined tool, a run trigger, and an owner. The layers are complementary; nothing below replaces a lower layer.

Test layers

Layer	What it proves	Tooling	Runs	Owner
Unit	Logic of a single class/module in isolation	JUnit 5	Every build (./mvnw test), every deploy	Developers
Integration	A module against real infra (DB, Keycloak)	Quarkus @QuarkusTest *IT on Failsafe, DevServices Postgres + Keycloak, @Tag("requires-database"), Docker-gated	Every CI build where Docker is present; SKIPPED (not green) when absent	Developers
API / contract	HTTP endpoints and token flows at the wire level	RestAssured + Keycloak admin client (e.g. EHerkenningStepUpBrokerIT)	Every CI build (same gate as integration)	Developers; prototype API cases carried forward here
E2E — in-repo (tactical)	Real-browser flows of the buildless asr-demo-frontend, bound to the backend build	Playwright for Java on Maven Failsafe — see ADR-00015	Local / on-demand (@Tag("browser-e2e"), excluded from CI)	Developers
E2E — portal (strategic)	Real-browser flows of the production portal UI	Playwright + TypeScript TAaaS framework (Page Object / Component / Service / Utility layers, SOLID/DRY) — see ADR-000017	Its own CI/CD pipeline, once the portal frontend stack is chosen	TAaaS + testers + product owner / end-users
Test data	Consistent, compliant fixtures across all layers	See ADR-00020 (tiered: real data for acceptance, synthetic elsewhere)	Underpins all layers	Developers + testers

Why two end-to-end tiers (and why that is not a contradiction)

The two Playwright ADRs deliberately choose different toolchains because they test different surfaces:

• In-repo Java tier (ADR-00015) keeps a single Maven toolchain inside the backend repo, whose only frontend today is a throwaway static demo with no build step. Binding one real-browser tracer-bullet test to Failsafe — gated exactly like the database integration tests — buys real OIDC/PKCE coverage without dragging a Node build into a backend repo. ADR-00015 explicitly rejects a separate Node/JS framework for this purpose.
• Strategic TypeScript tier (ADR-000017) is the standalone UI-automation framework for the real portal, owned by the TAaaS team and contributed to by testers and the product owner. It targets the portal’s frontend stack — an open question still — and lives in its own repository and pipeline.

They are sequential and complementary, not competing: the Java tier covers what exists now (the demo and backend contract flows); the TypeScript tier is the durable home for portal UI automation once the frontend stack lands. Whether the two toolchains eventually converge onto one is a future decision, taken when the portal stack is decided; this strategy does not force convergence and does not block it.

Contribution and TDD model

• Developers keep the unit, integration, API, and in-repo e2e layers green as part of normal development; these run on every deploy through CI.
• The strategic TypeScript suite is the place where the product owner, testers, and end-users add and grow test cases — smoke tests, critical business workflows, and regression scenarios — so the suite expands with the product.
• Prototype API test cases are migrated into the API/contract layer rather than discarded, preserving accumulated coverage.

Consequences

• Positive: one shared picture of the test estate; the 00015 / 000017 overlap is resolved as a deliberate per-layer split; a clear path for non-developers to contribute tests; regressions surface on every deploy.
• Negative: two Playwright toolchains (Java and TypeScript) must be maintained until (or unless) a convergence decision is taken; ownership spans developers and a separate TAaaS/test function.
• Neutral: this ADR is an umbrella — changes to any single layer are still made in that layer’s own ADR, and this document is updated to keep the overview current.

Compliance and Validation

• Security review completed
• Performance impact assessed
• Integration impact evaluated
• Documentation updated
• Stakeholder approval obtained

Related Documents

• ADR-00015: End-to-End Browser Test Tooling — Playwright + Chromium, Failsafe-bound — the in-repo Java e2e tier
• ADR-000017: Test Automation Using Playwright with TypeScript — the strategic portal UI-automation framework
• ADR-00020: Test Data Handling Strategy — fixtures across all layers
• Coding standards §7 Testing — layer conventions and the open frontend-stack question

Status History

Date	Status	Notes
2026-07-08	Proposed	Initial umbrella strategy tying together ADR-00015 (in-repo Java e2e) and ADR-000017 (strategic TypeScript framework), plus ADR-00020 (test data)