rating plan

2025-12-27 19:39:23 +01:00
parent 8d2b17d9a8
commit 5dc8c2399c
1 changed files with 540 additions and 0 deletions
--- a/plans/ratings-architecture-concept.md
+++ b/plans/ratings-architecture-concept.md
@@ -0,0 +1,540 @@
+# Ratings Architecture Concept (Multi-Rating + Transparency + Eligibility)
+
+This concept defines a **clean, extendable architecture** for ratings in GridPilot with:
+- Our own platform ratings (computed only from GridPilot league activity).
+- External per-game ratings (e.g. iRacing iRating/SR) stored separately for **display + eligibility filtering** only.
+- A **transparent rating ledger** so users can see exactly why they gained/lost rating.
+
+It is designed to fit the project’s Clean Architecture + CQRS Light rules in:
+- [`ARCHITECTURE.md`](docs/ARCHITECTURE.md:1)
+- [`Domain Objects`](docs/architecture/DOMAIN_OBJECTS.md:1)
+- [`CQRS Light`](docs/architecture/CQRS.md:1)
+- [`Use Cases`](docs/architecture/USECASES.md:1)
+- [`View Models`](docs/architecture/VIEW_MODELS.md:1)
+
+It is also aligned with the principles in:
+- [`GridPilot Rating`](docs/concept/RATING.md:1)
+- [`Stats`](docs/concept/STATS.md:1)
+
+---
+
+## 1. Requirements Summary
+
+### 1.1 Must Have (now)
+- **Platform ratings**
+  - `driving`: combines clean + fast driving (and also accounts for AFK/DNS/DNF/DSQ).
+  - `adminTrust`: administrative trust score.
+- **Per-game ratings**
+  - Stored per game (e.g. iRacing `iRating`, `safetyRating`) for display + eligibility filters.
+  - Not used to compute platform ratings.
+- **Transparency**
+  - UI must show “why did my rating change” with plus/minus, reason, and reference context.
+  - A persisted rating ledger is required.
+
+### 1.2 Future (design for, do not implement now)
+- `stewardTrust`
+- `broadcasterTrust`
+
+### 1.3 Non-Functional
+- Architecture is **easy to maintain** and **easy to access** (used across many locations).
+- Strong separation of concerns: domain is pure; commands enforce invariants; queries are pragmatic.
+- Extendability: new rating dimensions and new event types should not cause rewrites.
+
+---
+
+## 2. Key Architectural Decisions
+
+### 2.1 Platform ratings are computed only from GridPilot events
+External game ratings are:
+- Stored independently,
+- Displayed and queried,
+- Usable in eligibility filters,
+- Not inputs to platform rating computation.
+
+### 2.2 Ledger-first transparency
+Every rating adjustment is represented as an immutable **rating event** in a ledger, with:
+- Who: userId (subject)
+- What: dimension (driving/adminTrust/…)
+- Why: reason code + human-readable summary + structured metadata
+- How much: delta (+/-) and optional weight
+- Where: reference to a domain object (raceId, penaltyId, voteId, adminActionId)
+
+Snapshots are derived from ledger events, not the other way around.
+
+### 2.3 CQRS Light split
+- Commands record rating events and recompute snapshots.
+- Queries provide fast read models for UI and eligibility evaluation, without loading domain aggregates.
+
+### 2.4 Evolution path from existing code
+There is already a multi-dimensional value object [`UserRating`](core/identity/domain/value-objects/UserRating.ts:1) and a domain service [`RatingUpdateService`](core/identity/domain/services/RatingUpdateService.ts:1) triggered by [`CompleteRaceUseCaseWithRatings.execute()`](core/racing/application/use-cases/CompleteRaceUseCaseWithRatings.ts:47).
+
+This concept treats the existing [`UserRating`](core/identity/domain/value-objects/UserRating.ts:1) as an early “snapshot-like” model and proposes a controlled evolution:
+- Keep a snapshot object (can stay named `UserRating` or be renamed later).
+- Add a ledger model + repositories + calculators.
+- Gradually redirect the write flow from “direct updates” to “record events + recompute snapshot”.
+
+No “big bang rewrite”.
+
+---
+
+## 3. Domain Model (Core Concepts)
+
+### 3.1 Bounded contexts
+- **Identity context** owns user reputation/ratings (consistent with current placement of [`UserRating`](core/identity/domain/value-objects/UserRating.ts:1)).
+- **Racing context** emits race outcomes (finishes, incidents, statuses) and penalties/DSQ information; it does not own rating logic.
+- **Admin/Competition context** emits admin actions and vote outcomes; it does not own rating logic.
+
+### 3.2 Rating dimensions (extendable)
+Define a canonical dimension key set (enum-like union) for platform ratings:
+
+- `driving`
+- `adminTrust`
+- `stewardTrust` (future)
+- `broadcasterTrust` (future)
+
+Rule: adding a dimension should require:
+- A new calculator strategy, and
+- New event taxonomy entries,
+not structural redesign.
+
+### 3.3 Domain objects (suggested)
+Domain objects below follow the rules in [`Domain Objects`](docs/architecture/DOMAIN_OBJECTS.md:1).
+
+**Value Objects**
+- `RatingDimensionKey` (e.g. `driving`, `adminTrust`)
+- `RatingValue` (0..100 or 0..N; pick one standard scale; recommend 0..100 aligned with [`UserRating`](core/identity/domain/value-objects/UserRating.ts:1))
+- `RatingDelta` (signed float/decimal; stored and displayed)
+- `RatingEventId` (uuid-like string)
+- `RatingReference` (typed reference union: raceId, penaltyId, voteId, adminActionId)
+- `ExternalRating` (per-game rating data point, e.g. iRating, safety rating)
+- `GameKey` (e.g. `iracing`, future `acc`, etc.)
+
+**Entities / Aggregate Roots**
+- `RatingLedger` (aggregate root for a user’s rating events)
+  - Identity: `userId`
+  - Holds a list/stream of `RatingEvent` (not necessarily loaded fully; repository can stream)
+- `RatingEvent` (entity inside ledger or separate entity persisted in table)
+  - Identity: `ratingEventId`
+  - Immutable once persisted
+- `AdminVoteSession` (aggregate root, scoped to league + admin candidate + window)
+  - Identity: `voteSessionId`
+  - Controls who can vote, dedup, time window, and closure
+  - Emits outcome events that convert to rating ledger events
+- `ExternalGameRatingProfile` (aggregate root per user)
+  - Identity: `userId + gameKey`
+  - Stores latest known per-game ratings + provenance
+
+**Domain Services**
+- `DrivingRatingCalculator` (pure, stateless)
+- `AdminTrustRatingCalculator` (pure, stateless)
+- `RatingSnapshotCalculator` (applies ordered events to snapshot)
+- `RatingEventFactory` (turns domain facts into rating events)
+- `EligibilityEvaluator` (pure evaluation over rating snapshots and external ratings, but invoked from application layer for “decisions”)
+- Keep services similar in spirit to [`AverageStrengthOfFieldCalculator.calculate()`](core/racing/domain/services/StrengthOfFieldCalculator.ts:29) and constraints typical of value objects like [`StrengthOfField.create()`](core/racing/domain/value-objects/StrengthOfField.ts:22).
+
+### 3.4 Rating snapshot (current `UserRating`)
+A snapshot is what most screens need:
+- latest rating value per dimension,
+- confidence/sample size/trend,
+- lastUpdated.
+
+This already exists in [`UserRating`](core/identity/domain/value-objects/UserRating.ts:1). Conceptually, the snapshot is derived from events:
+- `value`: derived
+- `confidence` + `sampleSize`: derived from count/weights and recentness rules
+- `trend`: derived from recent deltas
+
+Snapshots are persisted for fast reads; events are persisted for transparency.
+
+---
+
+## 4. Rating Ledger (Transparency Backbone)
+
+### 4.1 Rating event structure (conceptual schema)
+A `RatingEvent` should contain:
+
+- `id`: `RatingEventId`
+- `userId`: subject of the rating
+- `dimension`: `RatingDimensionKey`
+- `delta`: `RatingDelta`
+- `weight`: numeric (optional; for sample size / confidence)
+- `occurredAt`: Date
+- `createdAt`: Date
+- `source`:
+  - `sourceType`: `race` | `penalty` | `vote` | `adminAction` | `manualAdjustment`
+  - `sourceId`: string
+- `reason`:
+  - `code`: stable machine code (for i18n and filtering)
+  - `summary`: human text (or key + template params)
+  - `details`: structured JSON (for UI)
+- `visibility`:
+  - `public`: boolean (default true)
+  - `redactedFields`: list (for sensitive moderation info)
+- `version`: schema version for forward compatibility
+
+### 4.2 Ledger invariants
+- Immutable events (append-only); corrections happen via compensating events.
+- Deterministic ordering rule (by `occurredAt`, then `createdAt`, then `id`).
+- The snapshot is always reproducible from events (within the same calculator version).
+
+### 4.3 Calculator versioning
+To remain maintainable over time:
+- Events reference a `calculatorVersion` used when they were generated (optional but recommended).
+- Snapshot stores the latest `calculatorVersion`.
+- When the algorithm changes, snapshots can be recomputed in background; events remain unchanged.
+
+---
+
+## 5. Platform Rating Definitions
+
+### 5.1 Driving rating (clean + fast + reliability)
+Driving rating is the platform’s main driver identity rating (as described in [`GridPilot Rating`](docs/concept/RATING.md:1)).
+
+It is derived from ledger events sourced from race facts:
+- Finishing position vs field strength (fast driving component)
+- Incidents and penalty involvement (clean driving component)
+- Attendance and reliability (DNS/DNF/DSQ/AFK)
+
+#### 5.1.1 Driver status inputs
+We must explicitly model:
+- AFK
+- DNS (did not start)
+- DNF (did not finish)
+- DSQ (disqualified)
+
+These should become explicit event types, not hidden inside one “performance score”.
+
+#### 5.1.2 Driving event taxonomy (initial)
+Examples of ledger event reason codes (illustrative; final list is a product decision):
+
+Performance:
+- `DRIVING_FINISH_STRENGTH_GAIN`
+- `DRIVING_POSITIONS_GAINED_BONUS`
+- `DRIVING_PACE_RELATIVE_GAIN` (optional)
+
+Clean driving:
+- `DRIVING_INCIDENTS_PENALTY`
+- `DRIVING_MAJOR_CONTACT_PENALTY` (if severity exists)
+- `DRIVING_PENALTY_INVOLVEMENT_PENALTY`
+
+Reliability:
+- `DRIVING_DNS_PENALTY`
+- `DRIVING_DNF_PENALTY`
+- `DRIVING_DSQ_PENALTY`
+- `DRIVING_AFK_PENALTY`
+- `DRIVING_SEASON_ATTENDANCE_BONUS` (optional later)
+
+Each event must reference source facts:
+- `raceId` always for race-derived events
+- `penaltyId` for steward/admin penalty events
+- additional metadata: start position, finish position, incident count, etc.
+
+#### 5.1.3 Field strength support
+Driving performance should consider strength of field similar to the existing value object [`StrengthOfField`](core/racing/domain/value-objects/StrengthOfField.ts:1) and its service pattern in [`StrengthOfFieldCalculator`](core/racing/domain/services/StrengthOfFieldCalculator.ts:1).
+
+Concept: the driving calculator receives:
+- driver finish data
+- field rating inputs (which can be platform driving snapshot values or external iRating for SoF only, depending on product choice)
+
+Given the earlier decision “platform rating does not use external ratings”, we can still compute SoF using:
+- platform driving snapshot values (for users with sufficient data), and/or
+- a neutral default for new users
+without using external ratings as an input to driving rating itself.
+
+(If SoF must use iRating for accuracy, it still does not violate “independent” as long as SoF is a *race context signal* and not a *direct driver rating input*. This is a design choice to confirm later.)
+
+### 5.2 Admin trust rating (hybrid system signals + votes)
+Admin trust is separate from driving.
+
+It must include:
+- System-derived actions (timeliness, reversals, consistency, completion of tasks)
+- Driver votes among participants in a league
+
+#### 5.2.1 Voting model (anti-abuse, league-scoped)
+Votes are generated within a league, but the rating is global. To avoid abuse:
+- Only eligible voters: drivers who participated in the league (membership + minimum participation threshold).
+- 1 vote per voter per admin per voting window.
+- Voting windows are timeboxed (e.g. weekly/monthly/season-end).
+- Votes have reduced weight if the voter has low trust (optional later).
+- Votes should be explainable: aggregated outcome + distribution; individual votes may be private.
+
+Votes produce ledger events:
+- `ADMIN_VOTE_OUTCOME_POSITIVE`
+- `ADMIN_VOTE_OUTCOME_NEGATIVE`
+with reference `voteSessionId` and metadata including:
+- leagueId
+- eligibleVoterCount
+- voteCount
+- percentPositive
+
+#### 5.2.2 Admin system-signal taxonomy (initial)
+Examples:
+- `ADMIN_ACTION_SLA_BONUS` (responded within SLA)
+- `ADMIN_ACTION_REVERSAL_PENALTY` (frequent reversals)
+- `ADMIN_ACTION_RULE_CLARITY_BONUS` (published rules/changes; if tracked)
+- `ADMIN_ACTION_ABUSE_REPORT_PENALTY` (validated abuse reports)
+
+All of these should be “facts” emitted by admin/competition workflows, not computed in rating domain from raw infra signals.
+
+---
+
+## 6. External Game Ratings (Per-Game Profiles)
+
+### 6.1 Purpose
+External ratings exist to:
+- Display on user profiles
+- Be used in eligibility filters
+
+They do not affect platform ratings.
+
+### 6.2 Data model (conceptual)
+`ExternalGameRatingProfile` per `userId + gameKey` stores:
+- `gameKey`: e.g. `iracing`
+- `ratings`: map of rating type -> numeric value
+  - e.g. `iracing.iRating`, `iracing.safetyRating`
+- `provenance`:
+  - `source`: `iracing-api` | `manual` | `import`
+  - `lastSyncedAt`
+  - `confidence`/`verified` flag (optional)
+
+### 6.3 Read surfaces
+Queries should provide:
+- “latest ratings by game”
+- “rating history by game” (optional future)
+- “last sync status”
+
+---
+
+## 7. Application Layer (Commands and Queries)
+
+### 7.1 Command side (write model)
+Commands are use-cases that:
+- validate permissions
+- load required domain facts (race outcomes, votes)
+- create rating events
+- append to ledger
+- recompute snapshot(s)
+- persist results
+
+Must follow [`Use Cases`](docs/architecture/USECASES.md:1): output via presenter/output port, no DTO leakage.
+
+#### 7.1.1 Command use cases (proposed)
+Driving:
+- `RecordRaceRatingEventsUseCase`
+  - Input: `raceId`
+  - Loads race results (positions, incidents, statuses)
+  - Produces ledger events for driving
+- `ApplyPenaltyRatingEventUseCase`
+  - Input: `penaltyId`
+  - Produces event(s) affecting driving and/or fairness dimension
+
+Admin trust:
+- `OpenAdminVoteSessionUseCase`
+- `CastAdminVoteUseCase`
+- `CloseAdminVoteSessionUseCase`
+  - On close: create ledger event(s) from aggregated vote outcome
+- `RecordAdminActionRatingEventUseCase`
+  - Called by admin workflows to translate system events into rating events
+
+Snapshots:
+- `RecomputeUserRatingSnapshotUseCase`
+  - Input: `userId` (or batch)
+  - Replays ledger events through calculator to update snapshot
+
+External ratings:
+- `UpsertExternalGameRatingUseCase`
+  - Input: userId, gameKey, rating values, provenance
+
+### 7.2 Query side (read model)
+Queries must be pragmatic per [`CQRS Light`](docs/architecture/CQRS.md:1), and should not use domain entities.
+
+#### 7.2.1 Query use cases (proposed)
+User-facing:
+- `GetUserRatingsSummaryQuery`
+  - returns current platform snapshot values + external game ratings + last updated timestamps
+- `GetUserRatingLedgerQuery`
+  - returns paginated ledger events, filterable by dimension, date range, reason code
+- `GetUserRatingChangeExplanationQuery`
+  - returns a “why” view for a time window (e.g. last race), pre-grouped by race/vote/penalty
+
+League-facing:
+- `GetLeagueEligibilityPreviewQuery`
+  - evaluates candidate eligibility for a league filter and returns explanation (which condition failed)
+
+Leaderboards:
+- `GetTopDrivingRatingsQuery`
+- `GetTopAdminTrustQuery`
+
+---
+
+## 8. Eligibility Filters (Leagues)
+
+### 8.1 Requirements
+Leagues can define eligibility filters against:
+- Platform `driving` rating (and future dimensions)
+- External per-game ratings (e.g. iRating threshold)
+
+Eligibility decisions should be explainable (audit trail and UI explanation).
+
+### 8.2 Filter DSL (typed, explainable)
+Define a small filter language that supports:
+
+- target:
+  - `platform.driving`
+  - `platform.adminTrust`
+  - `external.iracing.iRating`
+  - `external.iracing.safetyRating`
+
+- operators:
+  - `>=`, `>`, `<=`, `<`, `between`, `exists`
+
+- composition:
+  - `and`, `or`
+
+Each evaluation returns:
+- `eligible: boolean`
+- `reasons: []` each with:
+  - `target`
+  - `operator`
+  - `expected`
+  - `actual`
+  - `pass/fail`
+
+This makes it UI-transparent and debuggable.
+
+---
+
+## 9. Website / UI Transparency Contract
+
+Per [`View Models`](docs/architecture/VIEW_MODELS.md:1), UI should consume view models built from query DTOs.
+
+### 9.1 “Ratings” surfaces (suggested)
+- User profile:
+  - Platform driving rating + trend + confidence
+  - Admin trust rating (if relevant)
+  - External game ratings section (iRating/SR)
+- “Why did my rating change?” page:
+  - Ledger list with grouping by race/vote/penalty
+  - Each entry: delta, reason, context (race link), and explanation
+- League eligibility panel:
+  - Filter configured + explanation of pass/fail for a given user
+  - Should be able to show: “iRating 2200 is below required 2500” and/or “driving 61 is above required 55”
+
+---
+
+## 10. Event Flow Examples
+
+### 10.1 Race completion updates driving rating
+Triggered today by [`CompleteRaceUseCaseWithRatings.execute()`](core/racing/application/use-cases/CompleteRaceUseCaseWithRatings.ts:47) which calls [`RatingUpdateService.updateDriverRatingsAfterRace()`](core/identity/domain/services/RatingUpdateService.ts:21).
+
+Target flow (conceptually):
+
+```mermaid
+flowchart LR
+  RaceCompleted[Race completed]
+  Cmd[RecordRaceRatingEventsUseCase]
+  Ledger[Append rating events]
+  Calc[DrivingRatingCalculator]
+  Snap[Persist snapshot]
+  Query[GetUserRatingLedgerQuery]
+  UI[Profile and Why view]
+
+  RaceCompleted --> Cmd
+  Cmd --> Ledger
+  Cmd --> Calc
+  Calc --> Snap
+  Snap --> Query
+  Ledger --> Query
+  Query --> UI
+```
+
+### 10.2 Admin vote updates admin trust
+```mermaid
+flowchart LR
+  Open[OpenAdminVoteSessionUseCase]
+  Cast[CastAdminVoteUseCase]
+  Close[CloseAdminVoteSessionUseCase]
+  Ledger[Append vote outcome event]
+  Calc[AdminTrustRatingCalculator]
+  Snap[Persist snapshot]
+  UI[Admin trust breakdown]
+
+  Open --> Cast
+  Cast --> Close
+  Close --> Ledger
+  Close --> Calc
+  Calc --> Snap
+  Snap --> UI
+  Ledger --> UI
+```
+
+---
+
+## 11. Maintainability Notes
+
+### 11.1 Keep calculators pure
+All rating computations should be pure functions of:
+- Events
+- Inputs (like race facts)
+- Current snapshot (optional)
+No repositories, no IO.
+
+### 11.2 Stable reason codes
+Reason codes must be stable to support:
+- filtering
+- analytics
+- translations
+- consistent UI explanation
+
+### 11.3 Explicit extendability
+Adding `stewardTrust` later should follow the same template:
+- Add event taxonomy
+- Add calculator
+- Add ledger reasons
+- Add snapshot dimension
+- Add queries and UI
+
+No architecture changes.
+
+---
+
+## 12. Fit with existing `UserRating` and `RatingUpdateService`
+
+### 12.1 Current state
+- Snapshot-like model exists as [`UserRating`](core/identity/domain/value-objects/UserRating.ts:1)
+- Race completion triggers rating updates via [`RatingUpdateService`](core/identity/domain/services/RatingUpdateService.ts:1)
+
+### 12.2 Recommended evolution
+- Introduce ledger persistence and repositories first.
+- Update the write path so [`RatingUpdateService.updateDriverRatingsAfterRace()`](core/identity/domain/services/RatingUpdateService.ts:21) becomes:
+  - event generation + append + snapshot recalculation
+  - not direct “set newValue”
+
+This preserves the public API while improving transparency and extensibility.
+
+---
+
+## 13. Open Decisions (to confirm before implementation)
+
+1. Strength of Field inputs:
+   - Should SoF use platform driving snapshots only, or may it use external iRating as a contextual “field difficulty” signal while still keeping platform ratings independent?
+
+2. Scale:
+   - Keep 0..100 scale for platform ratings (consistent with [`UserRating`](core/identity/domain/value-objects/UserRating.ts:1))?
+
+3. Privacy:
+   - Which admin trust vote details are public (aggregates only) vs private (individual votes)?
+
+4. Penalty integration:
+   - Which penalties affect driving vs admin trust, and how do we ensure moderation-sensitive info can be redacted while keeping rating transparency?
+
+---
+
+## 14. Next Step: Implementation Planning Checklist
+
+Implementation should proceed in small vertical slices:
+- Ledger persistence + query read models
+- Driving rating events from race completion including DNS/DNF/DSQ/AFK
+- Admin vote sessions and rating events
+- Eligibility filter DSL + evaluation query
+
+All aligned with the project’s CQRS Light patterns in [`CQRS Light`](docs/architecture/CQRS.md:1).