website refactor
This commit is contained in:
185
plans/website-testing-gap-closure.md
Normal file
185
plans/website-testing-gap-closure.md
Normal file
@@ -0,0 +1,185 @@
|
||||
# Concept: Close all testing gaps for [`apps/website`](apps/website:1)
|
||||
|
||||
This plan defines a **route-driven** test strategy to prove the website is **fully working** without any **visual UI tests** (no screenshots/video/trace assertions). It leverages the existing unified Docker E2E environment described in [`README.docker.md`](README.docker.md:94).
|
||||
|
||||
## 1) Definition of done (route-driven)
|
||||
|
||||
For **every route** defined by [`routes`](apps/website/lib/routing/RouteConfig.ts:120) and enumerated by [`WebsiteRouteManager.getWebsiteRouteInventory()`](tests/shared/website/WebsiteRouteManager.ts:41), we prove:
|
||||
|
||||
1. **SSR behavior** (HTTP black-box)
|
||||
- Correct status: 200, 302, 404, 500.
|
||||
- Correct redirect targets for unauthenticated and wrong-role cases.
|
||||
- SSR HTML sanity markers and absence of Next.js error markers.
|
||||
|
||||
2. **Client-side behavior**
|
||||
- Client navigation works for representative transitions (no brittle selectors).
|
||||
- No unexpected console errors (allowlist only for known benign warnings).
|
||||
|
||||
3. **RBAC correctness**
|
||||
- Roles: unauthenticated, authenticated, admin, sponsor.
|
||||
- For every route: allowed roles can access, disallowed roles are redirected/forbidden as defined.
|
||||
|
||||
4. **Negative cases are covered**
|
||||
- Invalid route params (non-existent IDs).
|
||||
- Expired/invalid session.
|
||||
- API 5xx/timeouts and other failure injections.
|
||||
- Missing/invalid env configuration (fail fast rather than silently falling back).
|
||||
|
||||
The single source of truth is the route contract inventory, evolving from [`getWebsiteRouteContracts()`](tests/shared/website/RouteContractSpec.ts:44).
|
||||
|
||||
## 2) Architecture: one contract, three enforcement layers
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Route inventory
|
||||
RouteConfig + WebsiteRouteManager] --> B[Route contracts
|
||||
RouteContractSpec]
|
||||
B --> C[Unit tests
|
||||
invariants]
|
||||
B --> D[Integration tests
|
||||
SSR HTTP black box]
|
||||
B --> E[E2E tests
|
||||
Playwright in Docker]
|
||||
```
|
||||
|
||||
### 2.1 Single source of truth: Route contracts
|
||||
|
||||
Current implementation: [`getWebsiteRouteContracts()`](tests/shared/website/RouteContractSpec.ts:44)
|
||||
|
||||
Conceptual evolution:
|
||||
|
||||
- Extend the contract to be **scenario-based**:
|
||||
- unauth
|
||||
- auth
|
||||
- admin
|
||||
- sponsor
|
||||
- wrong-role variants
|
||||
- Make expectations explicit per scenario:
|
||||
- expectedStatus: ok, redirect, notFoundAllowed, errorRoute
|
||||
- expectedRedirectTo (pathname)
|
||||
- SSR assertions: `ssrMustContain`, `ssrMustNotContain`, `minTextLength`
|
||||
- runtime assertions: `consoleErrorPolicy` (allowlist + denylist)
|
||||
|
||||
This keeps tests from becoming a pile of one-off scripts; instead, tests are **generated** from the contract.
|
||||
|
||||
### 2.2 Unit tests: invariants that keep the system honest
|
||||
|
||||
Target:
|
||||
|
||||
- Routing classification and RBAC rules:
|
||||
- [`routeMatchers.isPublic()`](apps/website/lib/routing/RouteConfig.ts:263)
|
||||
- [`routeMatchers.requiresRole()`](apps/website/lib/routing/RouteConfig.ts:326)
|
||||
|
||||
- API coupling correctness:
|
||||
- base URL selection via [`getWebsiteApiBaseUrl()`](apps/website/lib/config/apiBaseUrl.ts:6)
|
||||
- HTTP client error mapping via [`BaseApiClient`](apps/website/lib/api/base/BaseApiClient.ts:11)
|
||||
|
||||
Desired outcome:
|
||||
|
||||
- A refactor of routing or API base URL logic breaks unit tests immediately.
|
||||
|
||||
### 2.3 Integration tests: SSR is an HTTP contract
|
||||
|
||||
We validate SSR using a black-box approach:
|
||||
|
||||
- Bring up the website server using the existing harness patterns under [`tests/integration/harness`](tests/integration/harness/index.ts:1).
|
||||
- For each route contract:
|
||||
- Issue an HTTP request to the website.
|
||||
- Assert status/redirect.
|
||||
- Assert SSR markers and absence of `__NEXT_ERROR__` as already encoded in [`DEFAULT_SSR_MUST_NOT_CONTAIN`](tests/shared/website/RouteContractSpec.ts:35).
|
||||
|
||||
Why this layer matters:
|
||||
|
||||
- It catches failures that E2E might hide (for example, client-side redirecting after a bad SSR response).
|
||||
|
||||
### 2.4 E2E tests: runtime and navigation, in unified Docker
|
||||
|
||||
We rely on the unified Docker stack described in [`README.docker.md`](README.docker.md:94) and run tests via [`npm run test:e2e:website`](package.json:120).
|
||||
|
||||
Key properties:
|
||||
|
||||
- Playwright visuals are disabled by configuration: [`playwright.website.config.ts`](playwright.website.config.ts:41).
|
||||
- E2E must validate:
|
||||
- route loads and final URL matches expectation
|
||||
- RBAC works (unauth redirects, wrong-role redirects)
|
||||
- no unexpected console errors using [`ConsoleErrorCapture`](tests/shared/website/ConsoleErrorCapture.ts:1)
|
||||
- representative navigation flows (href-based selectors) like [`navigation.e2e.test.ts`](tests/e2e/website/navigation.e2e.test.ts:7)
|
||||
|
||||
Additionally:
|
||||
|
||||
- Promote “nightly exhaustive” execution by enabling [`RUN_EXHAUSTIVE_E2E`](playwright.website.config.ts:25) to include heavier suites (like [`website-pages.e2e.test.ts`](tests/nightly/website/website-pages.e2e.test.ts:65)).
|
||||
|
||||
## 3) Close gaps by building a route coverage matrix
|
||||
|
||||
We produce a matrix derived from [`WebsiteRouteManager.getWebsiteRouteInventory()`](tests/shared/website/WebsiteRouteManager.ts:41):
|
||||
|
||||
- Rows: each route
|
||||
- Columns: scenarios
|
||||
- unauth → expected redirect or ok
|
||||
- auth → ok or role redirect
|
||||
- admin → ok on admin routes
|
||||
- sponsor → ok on sponsor routes
|
||||
- invalid param → notFoundAllowed or inline error behavior
|
||||
- API failure injection → error boundaries behave, no runtime crash
|
||||
|
||||
This matrix becomes a checklist that prevents “coverage by vibe”.
|
||||
|
||||
## 4) Failure mode strategy (no visual tests)
|
||||
|
||||
### 4.1 Invalid IDs and missing resources
|
||||
|
||||
Use [`WebsiteRouteManager.getParamEdgeCases()`](tests/shared/website/WebsiteRouteManager.ts:83) to ensure invalid IDs lead to:
|
||||
|
||||
- 404 where appropriate, or
|
||||
- 200 with an inline not-found message for CSR-heavy pages
|
||||
|
||||
### 4.2 Session drift and wrong-role
|
||||
|
||||
Drive auth contexts using [`WebsiteAuthManager.createAuthContext()`](tests/shared/website/WebsiteAuthManager.ts:1) and assert redirect behavior for:
|
||||
|
||||
- auth user hitting admin route
|
||||
- auth user hitting sponsor route
|
||||
- unauth user hitting protected routes
|
||||
|
||||
### 4.3 API failures and timeouts
|
||||
|
||||
Test website behavior when the API returns 5xx or times out:
|
||||
|
||||
- SSR path: prove the server renders an error boundary (or specific 500 route) with the expected status.
|
||||
- CSR path: prove it does not crash and produces deterministic error messaging.
|
||||
|
||||
This should be done without mocking UI visuals; we assert status, URL, and text markers.
|
||||
|
||||
## 5) Test isolation: prevent real external calls
|
||||
|
||||
We must ensure tests never hit real third-party services (analytics, payments, email):
|
||||
|
||||
- For E2E: Playwright network interception with a denylist of external hosts.
|
||||
- For Node tests: fetch interception in test setup like [`tests/setup.ts`](tests/setup.ts:1).
|
||||
|
||||
Outcome: if a new external dependency is added, tests fail fast with a clear message.
|
||||
|
||||
## 6) CI pipeline shape
|
||||
|
||||
One deterministic pipeline (PR gating):
|
||||
|
||||
- Lint: [`website:lint`](package.json:145)
|
||||
- Typecheck: [`typecheck:targets`](package.json:141)
|
||||
- Unit tests: [`test:unit`](package.json:135)
|
||||
- Integration tests: [`test:integration`](package.json:129)
|
||||
- E2E unified Docker: [`test:e2e:website`](package.json:120)
|
||||
|
||||
Nightly:
|
||||
|
||||
- Run exhaustive E2E with [`RUN_EXHAUSTIVE_E2E`](playwright.website.config.ts:25).
|
||||
|
||||
## 7) Proposed implementation sequence
|
||||
|
||||
1. Extend the route contract structure in [`RouteContractSpec`](tests/shared/website/RouteContractSpec.ts:1) to include per-role scenarios and explicit negative-case expectations.
|
||||
2. Generate tests from contracts:
|
||||
- integration SSR suite
|
||||
- e2e runtime and RBAC suite
|
||||
3. Add failure-mode suites (invalid IDs, expired session, API 5xx/timeout).
|
||||
4. Add network denylist guards.
|
||||
5. Wire CI scripts and keep nightly exhaustive separate.
|
||||
|
||||
Reference in New Issue
Block a user