mmintel/gridpilot.gg
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

wip

Browse Source
This commit is contained in:
Marc Mintel
2025-11-30 23:00:48 +01:00
parent 4b8c70978f
commit 645f537895
41 changed files with 738 additions and 1631 deletions
Download Patch File Download Diff File Expand all files Collapse all files

157
docs/TESTS.md
View File

@@ -700,15 +700,15 @@ test('should create league', async ({ page }) => {
## Real E2E Testing Strategy (No Mocks)
GridPilot requires two distinct E2E testing strategies due to the nature of its automation adapters:
GridPilot focuses its real E2E testing strategy on browser-driven automation:
1. **Strategy A (Docker)**: Test `BrowserDevToolsAdapter` with Puppeteer against a fixture server
2. **Strategy B (Native macOS)**: Test `NutJsAutomationAdapter` on real hardware with display access
1. **Strategy A (Docker)**: Test `BrowserDevToolsAdapter` with Playwright or similar browser tooling against a fixture server
2. **Strategy B (Native macOS, legacy)**: Historical native OS-level automation on real hardware (now removed)
### Constraint: iRacing Terms of Service
- **Production**: nut.js OS-level automation only (no Puppeteer/CDP for actual iRacing automation)
- **Testing**: Puppeteer CAN be used to test `BrowserDevToolsAdapter` against static HTML fixtures
- **Production**: Native OS-level automation only (no browser DevTools/CDP for actual iRacing automation)
- **Testing**: Playwright-driven automation CAN be used against static HTML fixtures
### Test Architecture Overview
@@ -720,11 +720,7 @@ graph TB
HC --> BDA[BrowserDevToolsAdapter Tests]
end
subgraph Native E2E - macOS Runner
SCR[Screen Capture] --> TM[Template Matching Tests]
WF[Window Focus Tests] --> NJA[NutJsAutomationAdapter Tests]
KB[Keyboard/Mouse Tests] --> NJA
end
%% Legacy native OS-level automation tests have been removed.
```
---
@@ -920,7 +916,7 @@ describe('E2E: BrowserDevToolsAdapter - Docker Environment', () => {
### Strategy B: Native macOS E2E Tests
#### Purpose
Test OS-level screen automation using nut.js on real hardware. These tests CANNOT run in Docker because nut.js requires actual display access.
Test OS-level screen automation on real hardware. These tests CANNOT run in Docker because native automation requires actual display access.
#### Requirements
- macOS CI runner with display access
@@ -928,128 +924,17 @@ Test OS-level screen automation using nut.js on real hardware. These tests CANNO
- Accessibility permissions enabled
- Real Chrome/browser window visible
#### BDD Scenarios for Native E2E
#### BDD Scenarios for Native E2E (Legacy)
```gherkin
Feature: NutJsAutomationAdapter OS-Level Automation
As the automation engine
I want to perform OS-level screen automation
So that I can interact with iRacing without browser DevTools
> Historical note: previous native OS-level automation scenarios have been retired.
> Real-world coverage is now provided by Playwright-based workflows and fixture-backed
> automation; native OS-level adapters are no longer part of the supported stack.
Background:
Given I am running on macOS with display access
And accessibility permissions are granted
And screen recording permissions are granted
#### Test Implementation Structure (Legacy)
Scenario: Screen capture functionality
When I capture the full screen
Then a valid image buffer should be returned
And the image dimensions should match screen resolution
Scenario: Window focus management
Given a Chrome window titled "iRacing" is open
When I focus the browser window
Then the Chrome window should become the active window
Scenario: Template matching detection
Given I have a template image for the "Create Race" button
And the iRacing hosted racing page is visible
When I search for the template on screen
Then the template should be found
And the location should have confidence > 0.8
Scenario: Mouse click at detected location
Given I have detected a button at coordinates 500,300
When I click at that location
Then the mouse should move to 500,300
And a left click should be performed
Scenario: Keyboard input simulation
Given a text field is focused
When I type "Test Session Name"
Then the text should be entered character by character
With appropriate delays between keystrokes
Scenario: Login state detection
Given the iRacing login page is displayed
When I detect the login state
Then the result should indicate logged out
And the login form indicator should be detected
Scenario: Safe automation - no checkout
Given I am on the Track Conditions step
When I execute step 18
Then no click should be performed on the checkout button
And the automation should report safety stop
```
#### Test Implementation Structure
```typescript
// tests/e2e/native/nutJsAdapter.e2e.test.ts
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { NutJsAutomationAdapter } from '@infrastructure/adapters/automation/NutJsAutomationAdapter';
describe('E2E: NutJsAutomationAdapter - Native macOS', () => {
let adapter: NutJsAutomationAdapter;
beforeAll(async () => {
// Skip if not on macOS with display
if (process.platform !== 'darwin' || !process.env.DISPLAY_AVAILABLE) {
return;
}
adapter = new NutJsAutomationAdapter({
mouseSpeed: 500,
keyboardDelay: 25,
defaultTimeout: 10000,
});
await adapter.connect();
});
afterAll(async () => {
if (adapter?.isConnected()) {
await adapter.disconnect();
}
});
describe('Screen Capture', () => {
it('should capture full screen', async () => {
const result = await adapter.captureScreen();
expect(result.success).toBe(true);
expect(result.imageData).toBeDefined();
expect(result.dimensions.width).toBeGreaterThan(0);
});
it('should capture specific region', async () => {
const region = { x: 100, y: 100, width: 200, height: 200 };
const result = await adapter.captureScreen(region);
expect(result.success).toBe(true);
});
});
describe('Window Focus', () => {
it('should focus Chrome window', async () => {
const result = await adapter.focusBrowserWindow('Chrome');
// May fail if Chrome not open, which is acceptable
expect(result).toBeDefined();
});
});
describe('Template Matching', () => {
it('should find element by template', async () => {
const template = {
id: 'test-button',
imagePath: './resources/templates/test-button.png',
confidence: 0.8,
};
const location = await adapter.findElement(template);
// Template may not be on screen - test structure only
expect(location === null || location.confidence > 0).toBe(true);
});
});
});
```
Previous native OS-level adapter tests have been removed. The current
E2E coverage relies on Playwright-driven automation and fixture-backed
flows as described in the Docker-based strategy above.
---
@@ -1218,10 +1103,8 @@ CHROME_WS_ENDPOINT=ws://localhost:9222
FIXTURE_BASE_URL=http://localhost:3456
E2E_TIMEOUT=120000
# Native E2E Configuration
# Native E2E Configuration (legacy)
DISPLAY_AVAILABLE=true
NUT_JS_MOUSE_SPEED=500
NUT_JS_KEYBOARD_DELAY=25
```
---
@@ -1265,9 +1148,9 @@ For the iRacing hosted-session automation, confidence is provided by these concr
- Example: [`steps-07-09-cars-flow.e2e.test.ts`](tests/e2e/workflows/steps-07-09-cars-flow.e2e.test.ts:1) exercises cross-step cars flow, while [`full-hosted-session.workflow.e2e.test.ts`](tests/e2e/workflows/full-hosted-session.workflow.e2e.test.ts:1) runs a full 1–18 workflow via [`MockAutomationEngineAdapter`](packages/infrastructure/adapters/automation/engine/MockAutomationEngineAdapter.ts:1) and [`StartAutomationSessionUseCase`](packages/application/use-cases/StartAutomationSessionUseCase.ts:1), asserting final `SessionState` and step position.
- Additional workflow scenarios cover mid-flow failure using [`MockBrowserAutomationAdapter`](packages/infrastructure/adapters/automation/engine/MockBrowserAutomationAdapter.ts:1), ensuring failure states and diagnostics are surfaced without emitting false confirmations.
- **Opt-in real-world automation smoke tests (true iRacing / NutJs)**
- The legacy real iRacing automation suite [`automation.e2e.test.ts`](tests/e2e/automation.e2e.test.ts:1) is treated as a smoke-only, opt-in layer.
- It is gated by `RUN_REAL_AUTOMATION_SMOKE=1` and should not run in normal CI; it exists to validate that NutJs / template-based automation still matches the real UI for a small number of manual smoke runs on a prepared macOS environment.
- **Opt-in real-world automation smoke tests (legacy)**
- The legacy real iRacing automation suite [`automation.e2e.test.ts`](tests/e2e/automation.e2e.test.ts:1) is now a skipped, documentation-only layer.
- It is gated by `RUN_REAL_AUTOMATION_SMOKE=1` but no longer performs native OS-level automation; real confidence comes from Playwright-based fixture and workflow suites.
#### Confidence expectations
@@ -1277,7 +1160,7 @@ For the iRacing hosted-session automation, confidence is provided by these concr
- All **step E2E tests** under [`tests/e2e/steps`](tests/e2e/steps:1).
- All **workflow E2E tests** under [`tests/e2e/workflows`](tests/e2e/workflows:1).
- The **real-world smoke suite** in [`tests/e2e/automation.e2e.test.ts`](tests/e2e/automation.e2e.test.ts:1) is an additional, manual confidence layer and should be run only on configured machines when validating large changes to NutJs automation, template packs, or iRacing UI assumptions.
- The **real-world smoke suite** in [`tests/e2e/automation.e2e.test.ts`](tests/e2e/automation.e2e.test.ts:1) remains as historical documentation and should not be relied upon for validating changes; instead, update and extend the Playwright-based E2E and workflow suites.
- When adding new behavior:
- Prefer **unit tests** for domain/application changes.