mmintel/gridpilot.gg
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(e2e): add Docker-based E2E test infrastructure

Browse Source
This commit is contained in:
Marc Mintel
2025-11-22 15:40:23 +01:00
parent 2b0e7b5976
commit bb8b152b8a
6 changed files with 969 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

530
docs/TESTS.md
View File

@@ -698,6 +698,534 @@ test('should create league', async ({ page }) => {
---
## Real E2E Testing Strategy (No Mocks)
GridPilot requires two distinct E2E testing strategies due to the nature of its automation adapters:
1. **Strategy A (Docker)**: Test `BrowserDevToolsAdapter` with Puppeteer against a fixture server
2. **Strategy B (Native macOS)**: Test `NutJsAutomationAdapter` on real hardware with display access
### Constraint: iRacing Terms of Service
- **Production**: nut.js OS-level automation only (no Puppeteer/CDP for actual iRacing automation)
- **Testing**: Puppeteer CAN be used to test `BrowserDevToolsAdapter` against static HTML fixtures
### Test Architecture Overview
```mermaid
graph TB
subgraph Docker E2E - CI
FX[Static HTML Fixtures] --> FS[Fixture Server Container]
FS --> HC[Headless Chrome Container]
HC --> BDA[BrowserDevToolsAdapter Tests]
end
subgraph Native E2E - macOS Runner
SCR[Screen Capture] --> TM[Template Matching Tests]
WF[Window Focus Tests] --> NJA[NutJsAutomationAdapter Tests]
KB[Keyboard/Mouse Tests] --> NJA
end
```
---
### Strategy A: Docker-Based E2E Tests
#### Purpose
Test the complete 18-step workflow using `BrowserDevToolsAdapter` against real HTML fixtures without mocks.
#### Architecture
```yaml
# docker/docker-compose.e2e.yml
services:
# Headless Chrome with remote debugging enabled
chrome:
image: browserless/chrome:latest
ports:
- "9222:3000"
environment:
- CONNECTION_TIMEOUT=600000
- MAX_CONCURRENT_SESSIONS=1
- PREBOOT_CHROME=true
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3000/json/version"]
interval: 5s
timeout: 10s
retries: 3
# Static server for iRacing HTML fixtures
fixture-server:
build:
context: ./fixture-server
dockerfile: Dockerfile
ports:
- "3456:80"
volumes:
- ../resources/iracing-hosted-sessions:/usr/share/nginx/html:ro
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:80/01-hosted-racing.html"]
interval: 5s
timeout: 10s
retries: 3
```
#### Fixture Server Configuration
```dockerfile
# docker/fixture-server/Dockerfile
FROM nginx:alpine
# Configure nginx for static HTML serving
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
```
```nginx
# docker/fixture-server/nginx.conf
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
location / {
try_files $uri $uri/ =404;
add_header Access-Control-Allow-Origin *;
}
}
```
#### BDD Scenarios for Docker E2E
```gherkin
Feature: BrowserDevToolsAdapter Workflow Automation
As the automation engine
I want to execute the 18-step hosted session workflow
So that I can verify browser automation against real HTML fixtures
Background:
Given the Docker E2E environment is running
And the fixture server is serving iRacing HTML pages
And the headless Chrome container is connected
Scenario: Complete workflow navigation through all 18 steps
Given the BrowserDevToolsAdapter is connected to Chrome
When I execute step 2 HOSTED_RACING
Then the adapter should navigate to the hosted racing page
And the page should contain the create race button
When I execute step 3 CREATE_RACE
Then the wizard modal should open
When I execute step 4 RACE_INFORMATION
And I fill the session name field with "Test Race"
Then the form field should contain "Test Race"
# ... steps 5-17 follow same pattern
When I execute step 18 TRACK_CONDITIONS
Then the automation should stop at the safety checkpoint
And the checkout button should NOT be clicked
Scenario: Modal step handling - Add Car modal
Given the automation is at step 8 SET_CARS
When I click the "Add Car" button
Then the ADD_CAR modal should open
When I search for "Dallara F3"
And I select the first result
Then the modal should close
And the car should be added to the selection
Scenario: Form field validation with real selectors
Given I am on the RACE_INFORMATION page
Then the selector "input[name='sessionName']" should exist
And the selector ".form-group:has label:has-text Session Name input" should exist
Scenario: Error handling when element not found
Given I am on a blank page
When I try to click selector "#nonexistent-element"
Then the result should indicate failure
And the error message should contain "not found"
```
#### Test Implementation Structure
```typescript
// tests/e2e/docker/browserDevToolsAdapter.e2e.test.ts
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { BrowserDevToolsAdapter } from '@infrastructure/adapters/automation/BrowserDevToolsAdapter';
import { StepId } from '@domain/value-objects/StepId';
describe('E2E: BrowserDevToolsAdapter - Docker Environment', () => {
let adapter: BrowserDevToolsAdapter;
const CHROME_WS_ENDPOINT = process.env.CHROME_WS_ENDPOINT || 'ws://localhost:9222';
const FIXTURE_BASE_URL = process.env.FIXTURE_BASE_URL || 'http://localhost:3456';
beforeAll(async () => {
adapter = new BrowserDevToolsAdapter({
browserWSEndpoint: CHROME_WS_ENDPOINT,
defaultTimeout: 30000,
});
await adapter.connect();
});
afterAll(async () => {
await adapter.disconnect();
});
describe('Step Workflow Execution', () => {
it('should navigate to hosted racing page - step 2', async () => {
const result = await adapter.navigateToPage(`${FIXTURE_BASE_URL}/01-hosted-racing.html`);
expect(result.success).toBe(true);
});
it('should fill race information form - step 4', async () => {
await adapter.navigateToPage(`${FIXTURE_BASE_URL}/03-race-information.html`);
const stepId = StepId.create(4);
const result = await adapter.executeStep(stepId, {
sessionName: 'E2E Test Session',
password: 'testpass123',
description: 'Automated E2E test session',
});
expect(result.success).toBe(true);
});
// ... additional step tests
});
describe('Modal Operations', () => {
it('should handle ADD_CAR modal - step 9', async () => {
await adapter.navigateToPage(`${FIXTURE_BASE_URL}/09-add-a-car.html`);
const stepId = StepId.create(9);
const result = await adapter.handleModal(stepId, 'open');
expect(result.success).toBe(true);
});
});
describe('Safety Checkpoint', () => {
it('should stop at step 18 without clicking checkout', async () => {
await adapter.navigateToPage(`${FIXTURE_BASE_URL}/18-track-conditions.html`);
const stepId = StepId.create(18);
const result = await adapter.executeStep(stepId, {});
expect(result.success).toBe(true);
expect(result.metadata?.safetyStop).toBe(true);
});
});
});
```
---
### Strategy B: Native macOS E2E Tests
#### Purpose
Test OS-level screen automation using nut.js on real hardware. These tests CANNOT run in Docker because nut.js requires actual display access.
#### Requirements
- macOS CI runner with display access
- Screen recording permissions granted
- Accessibility permissions enabled
- Real Chrome/browser window visible
#### BDD Scenarios for Native E2E
```gherkin
Feature: NutJsAutomationAdapter OS-Level Automation
As the automation engine
I want to perform OS-level screen automation
So that I can interact with iRacing without browser DevTools
Background:
Given I am running on macOS with display access
And accessibility permissions are granted
And screen recording permissions are granted
Scenario: Screen capture functionality
When I capture the full screen
Then a valid image buffer should be returned
And the image dimensions should match screen resolution
Scenario: Window focus management
Given a Chrome window titled "iRacing" is open
When I focus the browser window
Then the Chrome window should become the active window
Scenario: Template matching detection
Given I have a template image for the "Create Race" button
And the iRacing hosted racing page is visible
When I search for the template on screen
Then the template should be found
And the location should have confidence > 0.8
Scenario: Mouse click at detected location
Given I have detected a button at coordinates 500,300
When I click at that location
Then the mouse should move to 500,300
And a left click should be performed
Scenario: Keyboard input simulation
Given a text field is focused
When I type "Test Session Name"
Then the text should be entered character by character
With appropriate delays between keystrokes
Scenario: Login state detection
Given the iRacing login page is displayed
When I detect the login state
Then the result should indicate logged out
And the login form indicator should be detected
Scenario: Safe automation - no checkout
Given I am on the Track Conditions step
When I execute step 18
Then no click should be performed on the checkout button
And the automation should report safety stop
```
#### Test Implementation Structure
```typescript
// tests/e2e/native/nutJsAdapter.e2e.test.ts
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { NutJsAutomationAdapter } from '@infrastructure/adapters/automation/NutJsAutomationAdapter';
describe('E2E: NutJsAutomationAdapter - Native macOS', () => {
let adapter: NutJsAutomationAdapter;
beforeAll(async () => {
// Skip if not on macOS with display
if (process.platform !== 'darwin' || !process.env.DISPLAY_AVAILABLE) {
return;
}
adapter = new NutJsAutomationAdapter({
mouseSpeed: 500,
keyboardDelay: 25,
defaultTimeout: 10000,
});
await adapter.connect();
});
afterAll(async () => {
if (adapter?.isConnected()) {
await adapter.disconnect();
}
});
describe('Screen Capture', () => {
it('should capture full screen', async () => {
const result = await adapter.captureScreen();
expect(result.success).toBe(true);
expect(result.imageData).toBeDefined();
expect(result.dimensions.width).toBeGreaterThan(0);
});
it('should capture specific region', async () => {
const region = { x: 100, y: 100, width: 200, height: 200 };
const result = await adapter.captureScreen(region);
expect(result.success).toBe(true);
});
});
describe('Window Focus', () => {
it('should focus Chrome window', async () => {
const result = await adapter.focusBrowserWindow('Chrome');
// May fail if Chrome not open, which is acceptable
expect(result).toBeDefined();
});
});
describe('Template Matching', () => {
it('should find element by template', async () => {
const template = {
id: 'test-button',
imagePath: './resources/templates/test-button.png',
confidence: 0.8,
};
const location = await adapter.findElement(template);
// Template may not be on screen - test structure only
expect(location === null || location.confidence > 0).toBe(true);
});
});
});
```
---
### Test File Structure
```
tests/
├── e2e/
│ ├── docker/ # Docker-based E2E tests
│ │ ├── browserDevToolsAdapter.e2e.test.ts
│ │ ├── workflowSteps.e2e.test.ts
│ │ ├── modalHandling.e2e.test.ts
│ │ └── selectorValidation.e2e.test.ts
│ ├── native/ # Native OS automation tests
│ │ ├── nutJsAdapter.e2e.test.ts
│ │ ├── screenCapture.e2e.test.ts
│ │ ├── templateMatching.e2e.test.ts
│ │ └── windowFocus.e2e.test.ts
│ ├── automation.e2e.test.ts # Existing selector validation
│ └── features/ # Gherkin feature files
│ └── hosted-session-automation.feature
├── integration/
│ └── infrastructure/
│ └── BrowserDevToolsAdapter.test.ts
└── unit/
└── ...
docker/
├── docker-compose.e2e.yml # E2E test environment
└── fixture-server/
├── Dockerfile
└── nginx.conf
.github/
└── workflows/
├── e2e-docker.yml # Docker E2E workflow
└── e2e-macos.yml # macOS native E2E workflow
```
---
### CI/CD Integration
#### Docker E2E Workflow
```yaml
# .github/workflows/e2e-docker.yml
name: E2E Tests - Docker
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
e2e-docker:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Start Docker E2E environment
run: |
docker compose -f docker/docker-compose.e2e.yml up -d
docker compose -f docker/docker-compose.e2e.yml ps
- name: Wait for services to be healthy
run: |
timeout 60 bash -c 'until curl -s http://localhost:9222/json/version; do sleep 2; done'
timeout 60 bash -c 'until curl -s http://localhost:3456/01-hosted-racing.html; do sleep 2; done'
- name: Run Docker E2E tests
run: npm run test:e2e:docker
env:
CHROME_WS_ENDPOINT: ws://localhost:9222
FIXTURE_BASE_URL: http://localhost:3456
- name: Stop Docker environment
if: always()
run: docker compose -f docker/docker-compose.e2e.yml down -v
```
#### macOS Native E2E Workflow
```yaml
# .github/workflows/e2e-macos.yml
name: E2E Tests - macOS Native
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
e2e-macos:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Grant screen recording permissions
run: |
# Note: GitHub Actions macOS runners have limited permission support
# Some tests may be skipped if permissions cannot be granted
sudo sqlite3 /Library/Application\ Support/com.apple.TCC/TCC.db \
"INSERT OR REPLACE INTO access VALUES('kTCCServiceScreenCapture','com.apple.Terminal',0,2,0,1,NULL,NULL,0,'UNUSED',NULL,0,$(date +%s));" 2>/dev/null || true
- name: Run native E2E tests
run: npm run test:e2e:native
env:
DISPLAY_AVAILABLE: "true"
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/native/screenshots/
```
---
### NPM Scripts
```json
{
"scripts": {
"test:e2e": "vitest run --config vitest.e2e.config.ts",
"test:e2e:docker": "vitest run --config vitest.e2e.config.ts tests/e2e/docker/",
"test:e2e:native": "vitest run --config vitest.e2e.config.ts tests/e2e/native/",
"docker:e2e:up": "docker compose -f docker/docker-compose.e2e.yml up -d",
"docker:e2e:down": "docker compose -f docker/docker-compose.e2e.yml down -v",
"docker:e2e:logs": "docker compose -f docker/docker-compose.e2e.yml logs -f"
}
}
```
---
### Environment Configuration
```bash
# .env.test.example
# Docker E2E Configuration
CHROME_WS_ENDPOINT=ws://localhost:9222
FIXTURE_BASE_URL=http://localhost:3456
E2E_TIMEOUT=120000
# Native E2E Configuration
DISPLAY_AVAILABLE=true
NUT_JS_MOUSE_SPEED=500
NUT_JS_KEYBOARD_DELAY=25
```
---
## Cross-References
- **[`ARCHITECTURE.md`](./ARCHITECTURE.md)** — Layer boundaries, port definitions, and dependency rules that guide test structure
@@ -712,5 +1240,7 @@ GridPilot's testing strategy ensures:
- **Business logic is correct** (unit tests for domain/application layers)
- **Infrastructure works reliably** (integration tests for repositories/adapters)
- **User workflows function end-to-end** (E2E tests for full stack)
- **Browser automation works correctly** (Docker E2E tests with real fixtures)
- **OS-level automation works correctly** (Native macOS E2E tests with display access)
By following BDD principles and maintaining clear test organization, the team can confidently evolve GridPilot while preserving correctness and stability.