Commit Graph

170 Commits

Author SHA1 Message Date
39185593dd test(e2e): Fix E2E context generator and harden StoryView plugin
- Updated e2e_workflow_ctx in conftest to structurally parse screen_type
- Fixed StoryViewPlugin to gracefully handle pre-existing story screens
- Resolved 'LIE DETECTED' failure in story test suite
- Validated structural path transitions for story ring and story exit
2026-05-05 01:50:40 +02:00
c0cfa24384 feat(core): P0 radical evolution — kill blank_start, complete ContextGate matrix, fix HD Map back transitions
P0-2: Kill blank_start: true in config — the nuclear option that wipes ALL learned
knowledge on every run. Replaced with memory_hygiene() selective amnesia that
prunes low-confidence entries while preserving high-confidence learned patterns.

P0-3: Purge all root-level garbage scripts (scratch.py, test_run.py, profile_dump.xml,
resp_dump.json, pytest_output.log, test_output.log, coverage.xml, coverage_e2e.json).
Updated .gitignore to prevent reaccumulation.

P2-2: Replace piecemeal BANNED_SCREENS/REQUIRED_MARKERS with complete Action
Compatibility Matrix (VALID_SCREENS whitelist). Every interaction intent now has
an explicit set of valid screens. Covers like, comment, follow, unfollow, save,
repost across all 14 screen types.

P2-3: Remove non-deterministic 'press back' transitions from HD Map topology for
OTHER_PROFILE, POST_DETAIL, and SEARCH_RESULTS. Add tab transitions to POST_DETAIL.

P2-4: Replace ScreenMemoryDB nuclear 200-entry wipe with LRU eviction.
store_screen() now accepts confidence parameter.

TDD: 22 new tests (all GREEN), 240 unit/tdd/integration passed, 0 regressions.
E2E: 288 passed (+2 fixed), 6 pre-existing LIE DETECTED failures.
2026-05-04 18:19:00 +02:00
3800254fe3 feat: structural hardening, grid fast-paths, and state-aware adaptive snap recovery 2026-05-04 17:43:25 +02:00
22216cbd2d feat(core): finalize P2 & P3 hardening - Topology HD Map, Empathy Filter, and Git Hygiene 2026-05-04 15:03:42 +02:00
79e5784b93 feat(core): complete P1 hardening phase - ContextGate, Localization Purge, SoM Safety Clamp 2026-05-04 14:51:54 +02:00
5c9ea0e5e2 fix(resonance): remediate 0.85 score flatlining (P1-4)
- Raised ContentMemoryDB cache threshold from 0.95 to 0.98 to prevent cross-post poisoning.
- Implemented continuous resonance scoring by storing and retrieving 'resonance_score'
  in ContentMemoryDB.
- ResonanceEngine now prioritizes high-fidelity raw scores from cache over
  discrete 'high/medium/low' mapping.

TDD: 3 tests verifying threshold and scoring logic integrity.
2026-05-04 14:42:59 +02:00
91fa426b70 chore(git): purge tracked bytecode and optimize .gitignore
- Fixed .gitignore to correctly handle __pycache__ even when inside whitelisted directories (like tests/).
- Moved pycache ignore patterns to the end of the file to ensure they take absolute precedence over any directory-level whitelists.
- Purged all previously tracked .pyc and __pycache__ files from the git index.
- Cleaned up .hypothesis and .pytest_cache artifacts.

This enforces 'Militärische Git-Disziplin' (Rule 7) and keeps the repo clean of environment-specific junk.
2026-05-04 14:42:22 +02:00
a265dee3d0 fix(memory): ScreenMemory threshold hardening + stale layout purge (P0-3)
- Raised similarity_threshold from 0.90 to 0.95 in get_screen_type() to eliminate
  POST_DETAIL false-positives caused by embedding saturation.
- Implemented purge_stale_screens() with 24h TTL to remove layout drift.
- Added collection size cap (200 points) to store_screen() with automatic wipe
  when saturated to clear overfitting bias.
- Triggered purge_stale_screens() on ScreenIdentity initialization.

TDD: 3 new tests for threshold defaults and purge method integrity.
2026-05-04 14:39:55 +02:00
d81a5d4e81 fix(perception): structural Resource-ID bypass gate + dead code purge in ActionMemory
P0-1: Toggle intents (follow/like/save) resolved via Resource-ID fast-path
now skip VLM verification entirely. Eliminates the #1 session failure:
VLM hallucinating Follow→Like, causing zero follows per session.

P0-2: Purge 52 lines of unreachable dead code after unconditional return
at line 290. Non-toggle structural delta verification now properly reachable.
Remove 4 DEBUG-prefixed logger.info pollution statements.

TDD: 7 new tests covering bypass gate, negative guard, non-toggle delta,
and debug log pollution detection.
2026-05-04 14:35:19 +02:00
f384fbb749 feat: purge ALL remaining German/localized strings from action_memory, telepathic_engine, darwin_engine, resonance_engine — enforce zero-maintenance structural determinism with TDD compliance tests 2026-05-03 23:33:28 +02:00
565bdaa568 chore: purge garbage scripts from repo and harden .gitignore 2026-05-03 23:29:14 +02:00
c641204a6b feat: enforce zero-maintenance autonomous navigation by purging hardcoded string searches and localized translations; harden VLM perception via robust JSON fallback; fix OTHER_PROFILE topological routing 2026-05-03 23:28:00 +02:00
4b645c6fb2 Hardening: Centralized zero-trust CognitiveStack validation and removed synthetic e2e test mocks 2026-05-03 20:30:30 +02:00
c7c7ce29f8 fix(qdrant): increase embedding API timeout to 30s to allow Ollama VRAM cold-starts 2026-05-03 18:38:27 +02:00
b36dde77d8 fix(navigation): harden autonomous perimeter guards against Play Store trap 2026-05-03 18:29:25 +02:00
93b2140844 docs: update ARCHITECTURE.md with SAE 3-layer modal cascade
Ensuring 100% code-documentation parity per strict architectural rules.
2026-05-03 17:15:16 +02:00
67c3d464e0 fix(navigation): harden ScreenIdentity priority cascade and GOAP structural action detection
1. ScreenIdentity: Restructured priority cascade to prevent Qdrant semantic
   cache from overriding deterministic structural heuristics. Cached types
   now resolve AFTER structural checks (Priority 3) instead of before.
   Story/Reels hallucinations from both cache and VLM are rejected if
   structural markers are absent.

2. bot_flow: Added error handling for story ring avatar tap and curiosity
   HomeFeed navigation failures — prevents silent continuation into
   undefined states.

3. GOAP: Extended is_navigation detection to include ScreenTopology
   structural actions, ensuring HD Map routes are correctly classified.

4. action_memory: Tightened _parse_yes_no to prevent JSON fall-through
   into ambiguous text matching. Structured responses with 'success'
   boolean are now handled directly.
2026-05-03 17:00:27 +02:00
d298f03891 fix(sae): structural fast-path for Instagram survey/interstitial modal detection
The SAE perceive() had no structural fast-path for Instagram-internal
modal overlays (surveys, rating prompts, interstitials). These modals
live inside com.instagram.android and were invisible to the foreign-app
detector. The LLM fallback frequently misclassified them as NORMAL,
causing the bot to get trapped in survey dialogs.

Added two O(1) structural detection layers:
1. Resource-ID markers: survey_overlay_container, interstitial_container,
   mystery_interstitial, nux_overlay, rating_prompt, feedback_dialog
2. Dismiss-button heuristic: cross-validates dismiss text with overlay
   container structure to prevent caption false positives

Also removed dead code: xml_dump.lower() no-op.

Fixes: test_perceive_instagram_survey_modal
Zero regressions: 464 passed, 6 pre-existing failures
2026-05-03 16:59:52 +02:00
604f2d7341 fix(perception): avoid skipping first highlight story by differentiating from home feed tray 2026-05-03 16:38:02 +02:00
cd8f35056c fix(perception): structural fast-path skips own profile story ring on home feed to prevent modal traps 2026-05-03 16:35:59 +02:00
800fb1da98 fix(perception): replace string matching with structural feed marker check for perfect snapping 2026-05-03 16:31:32 +02:00
6cd068f951 test(curiosity): force CHECK_CURIOSITY triggering and harden HD Map routing 2026-05-03 16:20:18 +02:00
f46b0b7bcb chore: harden verify_success fallbacks to Fail-Fast and apply aggressive GOAP penalty 2026-05-03 15:42:09 +02:00
5fbbe3d273 feat(e2e): ACHIEVED ABSOLUTE REALISM - 100% Production Parity Purge. Removed all synthetic guards and mocks from core. Enabled honest Qdrant and Disk I/O in tests. Empty KNOWN_DIVERGENCES. 2026-05-03 14:44:01 +02:00
f85d0a8a76 test(e2e): eliminate create_emulator_facade monkeypatching and replace with make_real_device_with_xml where applicable 2026-05-03 13:44:58 +02:00
f0a54d4e20 test(e2e): enforce 100% production-parity in dm_engine and follow verification tests 2026-05-03 11:24:25 +02:00
0a73c35809 purge: eliminate last MagicMock and all lambda dump_hierarchy overrides
- device_emulator.py: Replace MagicMock(watcher) with explicit WatcherStub
  that implements exact u2 watcher chain (when/click/start). Any API drift
  now causes AttributeError instead of silent acceptance.
- test_system_sae.py: Inject XML into emulator state instead of bypassing
  dump_hierarchy with lambda
- test_behavior_ad_guard.py: Pass XML to fixture, remove lambda override
- test_behavior_scrape_profile.py: Pass XML to fixture, remove lambda override

Result: ZERO unittest.mock imports, ZERO MagicMock instances,
ZERO dump_hierarchy lambdas across entire E2E suite (50 files).
2026-05-03 10:21:23 +02:00
e535c10b65 fix(perception): structural fast-path and VLM bounds extraction for Reels 2026-05-02 23:17:12 +02:00
91effbc843 test: add E2E coverage for GOAP trap recovery and restart state reset 2026-05-02 23:11:31 +02:00
3da3849ca1 fix: purge GOAP tracking state when Instagram is forced to restart 2026-05-02 22:50:57 +02:00
51ee7a6793 refactor: remove mock_semantic_engine from q_nav_graph production code 2026-05-02 22:44:53 +02:00
936da47f61 test: SAE execution loop coverage 2026-05-02 22:40:52 +02:00
d1e0995148 test: e2e regression suite for SAE structural fast-paths 2026-05-02 22:40:20 +02:00
2f8eebb7e9 fix: stochastic failure in test_autonomous_goals.py 2026-05-02 22:37:34 +02:00
9c6f80de9d test: add 63-test GOAP navigation engine suite
Production bug 2026-05-02 22:30: Bot got trapped in a restart loop:
  1. 'tap reels tab' → VLM clicked wrong element → EXPLORE_GRID
  2. After 2 failures → action masked
  3. HD Map: 'REELS_FEED unreachable due to masked edges'
  4. GOAP: 'completely trapped' → force Instagram restart
  5. Repeat forever

ALL TESTS WERE GREEN because NONE of this logic was tested.

NEW: test_system_goap_navigation.py — 63 tests across 8 contracts:

CONTRACT 1: Screen Identification from XML (16 tests)
  - Home, Explore, Reels, Profile, DM, Story, Modal, Foreign App
  - Post detail vs Home feed (action_bar discriminator)
  - Reels full-screen (no tab bar, structural markers only)

CONTRACT 2: Available Actions Extraction (4 tests)
  - All 5 tab actions present on home feed
  - Screen-specific actions (first post on Explore)

CONTRACT 3: HD Map BFS Routing (7 tests)
  - Direct routes, multi-step routes, already-there
  - Masked edges, all-edges-masked → None

CONTRACT 4: Goal → Target Screen Mapping (14 tests)
  - All navigation goals + non-navigation goals return None

CONTRACT 5: Action Failure Masking (4 tests)
  - Mask after MAX_RETRIES, no mask under threshold
  - Full production chain reproduction

CONTRACT 6: Goal Achievement Detection (9 tests)
  - Screen-based, context-based, non-nav goals

CONTRACT 7: Structural Action Protection (4 tests)
  - HD Map actions are structural, random actions are not

CONTRACT 8: Step-Aware Navigation Validation (5 tests)
  - Expected screen for action, mismatch detection
2026-05-02 22:34:45 +02:00
cff7e976e0 fix: Tab Height Guard false-positive on author intent with 'tab' substring
Production bug 2026-05-02 22:19:
Intent 'post author username text (exclude bottom tabs)' contained the
word 'tab' in its parenthetical qualifier. The old check:
    is_tab_intent = 'tab' in intent_lower
matched this intent as a TAB navigation intent, activating the Tab
Height Guard which excluded ALL nodes with center_y < 85% of screen.

This nuked the actual clips_author_username node at y=2012, leaving
only 4 irrelevant items for the VLM → bot stuck in a loop unable to
find the author username.

Fix: Replace substring match with precise regex patterns that only
match real tab navigation intents:
  - 'tap profile tab', 'tap home tab' (tap + word + tab)
  - 'profile tab', 'explore tab' (word + tab)
  - 'tab ...' at start of intent

TDD: Red test added first, reproducing exact production state.
81 perception tests + 172 total tests pass.
2026-05-02 22:28:03 +02:00
f6f15ebd9a test: expand LLM perception suite to 80 tests — 3 new contracts
CONTRACT 6: extract_json() — LLM Response Sanitizer (9 tests)
Previously COMPLETELY untested. Every single LLM response in the entire
system passes through this function. Tests:
- Clean JSON, markdown fences (json+plain), thinking block purge
- Text prefix before JSON, truncated JSON fuzzy recovery
- Garbage/empty/None resilience

CONTRACT 7: _visual_discovery Pre-Filters (8 tests)
The guards INSIDE _visual_discovery() that run BEFORE the VLM ever
sees the candidates. If they fail, the VLM gets garbage and hallucinates.
Tests:
- Area filter: tiny nodes (<200px²) and huge containers (>400000px²)
- SystemUI filter: Android system elements, notifications, battery
- Strict Button Guard: long text captions excluded for button intents
- Grid Item Guard: non-grid elements excluded, fallthrough on no match
- Author/Username Guard: nav tabs excluded for author intents

CONTRACT 8: verify_success Structural Delta (8 tests)
The post-click verification logic that decides if a click actually worked.
Tests:
- Toggle massive XML shift = navigation error (False)
- Toggle small diff = success (True)
- Toggle zero diff = inconclusive (None)
- Follow success with 'Requested' marker (private accounts)
- Follow success with German locale ('Abonniert')
- View post with clips_viewer marker (Reels)
- View post still on grid = inconclusive (None)
- Semantic gate blocks wrong toggle element (caption != like button)
2026-05-02 22:10:57 +02:00
1cc367697e test: add 55-test LLM perception pipeline integrity suite
Validates the deterministic processing logic around the VLM without
needing a real LLM. Five contracts covering the full perception stack:

1. VLM Response Parsing (20 parametrized formats):
   Tests _parse_yes_no against every response format observed in production:
   clean YES/NO, JSON variants, explanations, edge cases (now/not/nothing),
   empty strings, free-form JSON. Documents the startswith('yes') early-exit
   behavior for ambiguous responses.

2. Box Index Extraction (7 formats):
   Validates JSON parsing of VLM box selections across all observed key
   variants: 'box', 'selected_index', 'box_index', 'index', null, and
   out-of-range values.

3. Structural Guards (11 tests):
   Tests every hallucination prevention guard in IntentResolver:
   - Tab intent excludes Back button (bug 2026-04-30)
   - Back intent preserves Back button
   - Tab height guard (y > 85% screen)
   - Author intent excludes nav tabs (bug 2026-05-01)
   - Structural fast-paths (message input, send, author, comment)
   - Abstract goals return None
   - Semantic quoted-target guard

4. Semantic Match Validation (10 tests):
   Prevents memory poisoning via _intent_matches_node:
   - Correct matches (follow/like/save + German locale)
   - Poisoning attempts: Reel thumbnails, photo content, comments
   - Non-toggle intents always pass through

5. ActionMemory Lifecycle (7 tests):
   Full track → confirm/reject cycle with FakeUIMemoryDB:
   - Correct follow stores in memory
   - Poisoned follow is BLOCKED (production bug reproduction)
   - Rejected click triggers confidence decay
   - No-track confirm is noop
   - Mismatched intent is ignored
   - Follow/view-post structural verification
2026-05-02 22:07:12 +02:00
738a59ac8d test: add E2E coverage gate and full lifecycle integration test
Two critical test additions:

1. test_system_full_lifecycle.py — Full A-to-Z integration test:
   Exercises the ENTIRE bot pipeline across 4 screen types (HomeFeed →
   Explore → Post Detail → Profile) with a real InstagramEmulator state
   machine. Validates plugin execution, session state accumulation,
   cognitive stack survival, dopamine session timeout, and post-session
   serialization. This is the single most important test in the suite.

2. test_system_coverage_gate.py — Critical Path Coverage Enforcement:
   Reads coverage_e2e.json and enforces minimum coverage thresholds on
   production-critical modules. Each threshold is tied to a real production
   incident. If persistent_list.py drops below 20% coverage, the build
   fails — because that's exactly how the sessions.json corruption
   bug went undetected.

   Usage: pytest tests/e2e --cov=GramAddict.core \
     --cov-report=json:coverage_e2e.json

   Critical paths enforced:
   - session_state.py (>=50%): serialization crash prevention
   - persistent_list.py (>=20%): persist() path must be tested
   - dopamine_engine.py (>=40%): session timeout logic
   - screen_identity.py (>=60%): screen identification
   - spatial_parser.py (>=70%): UI element parsing
   - intent_resolver.py (>=50%): click decision logic
   - q_nav_graph.py (>=25%): navigation integrity
   - device_facade.py (>=40%): device interface
2026-05-02 21:56:19 +02:00
cd6cecbe27 test: add proactive system integrity contracts
Replaces reactive bug-specific tests with 4 structural CONTRACTS
that make entire categories of bugs impossible:

1. Serialization Contract (13 parametrized hostile payloads):
   Proves SessionStateEncoder NEVER crashes regardless of what
   gets injected onto args (datetime, lambda, bytes, inf, nan, etc.)

2. Persistence Contract:
   Forces the REAL persist() path by removing PYTEST_CURRENT_TEST guard.
   Validates write → read round-trip with poisoned sessions.

3. Production Parity Contract:
   Scans ALL GramAddict source for pytest/test-mode divergence guards.
   Any unaudited divergence point fails the build. New guards require
   explicit justification in KNOWN_DIVERGENCES registry.

4. Import Integrity Contract:
   Imports every single GramAddict module to catch syntax errors,
   circular imports, and missing dependencies at test time.

This addresses the root systemic failure: tests were running in a
parallel universe where serialization was silently skipped, allowing
a datetime injection to corrupt sessions.json and kill production.
2026-05-02 20:15:35 +02:00
4af4ddb060 fix: prevent SessionStateEncoder crash from non-serializable datetime on args
Root cause: bot_flow.py injected configs.args.global_start_time = datetime.now()
which polluted the args namespace. SessionStateEncoder blindly serialized
args.__dict__ via json.dump, which crashed mid-write on the datetime object,
leaving sessions.json truncated/corrupt. Every subsequent restart failed.

Fixes:
- Remove global_start_time from configs.args (only lives on engine class vars)
- Harden SessionStateEncoder with _sanitize_value() to convert any
  non-JSON-serializable type (datetime, timedelta, arbitrary objects) to strings
- Add test_system_session_persistence.py with 4 tests covering the exact
  production crash scenario (datetime injection → json.dumps → round-trip)
- Fix test_engine_timeout.py broken GoalExecutor module reference
2026-05-02 19:57:33 +02:00
f32ee46d8c test: finalize E2E timeout verification and purge global state leaks 2026-05-02 18:59:35 +02:00
d2de5f91de test(e2e): Fix DM Engine E2E test XML sequences to eliminate Alien Context drift 2026-05-02 00:26:31 +02:00
9a13216064 test: enforce real VLM execution in explore tab guard and disable fallback hallucination 2026-05-01 23:41:51 +02:00
aa5184786e test(e2e): Fix LLM mock hallucinations to use structurally merged search string for 100% truthful navigation tests 2026-05-01 22:27:38 +02:00
2e1edec56a fix(perception): 3 production bug regressions from 2026-05-01 run
Bug 1 — VLM Profile Tab Hallucination:
  - VLM confused 'Profile' nav tab with post author username
  - Added Author Tab Guard to filter_navigation_conflicts()
  - Fixed mock LLM to use structural row_feed_photo_profile matching
    instead of hardcoded username (was the test lie)

Bug 2 — Like Button 1-Byte Delta Kill:
  - Toggle actions (like/save) produce 1-byte XML deltas
  - GOAP's MIN_UI_CHANGE_BYTES=50 threshold killed them as 'no change'
  - Split interaction gate: interactions use ANY xml diff, navigations
    keep the 50-byte threshold

Bug 3 — Empty Username Silently Accepted:
  - PostDataExtraction returned 'Post by @' with no warning
  - Added 'username_missing' reliability flag to result dict
  - Downstream consumers can now detect degraded data quality

Cleanup:
  - Removed all debug print() statements from production code
  - Replaced with structured logger.debug() calls
2026-05-01 21:57:56 +02:00
df18a48a84 build(deps): bump requests version to fix urllib3 warning 2026-05-01 16:57:03 +02:00
f1a8573be8 test(e2e): eliminate lying tests via behavioral truth assertions 2026-05-01 16:55:53 +02:00
da7201117c fix(tests): purge MagicMock, fix ad fixtures, improve pre-commit E2E fallback
- test_device_connection.py: replace MagicMock with SimpleNamespace to satisfy mock ban
- test_is_ad_substring.py: add feed context markers — is_ad only checks exact labels in feed context
- pre_commit_tests.sh: smart E2E test discovery by module name words, preventing false coverage failures
- conftest.py: fix profile tab visual discovery regex (case-insensitive desc match)
- test_production_bug_regression.py: fix TelepathicEngine singleton poisoning via monkeypatch
2026-05-01 12:09:18 +02:00
fddf14fd67 test(e2e): purge FakeSAENormal and instantiate real Cognitive Stack
PRODUCTION FIX 2026-04-30 12:45:
The entire E2E suite was systematically lying by testing a shell of the bot:
1. FakeSAENormal was masking the actual SituationalAwarenessEngine, meaning structural perception failures (like the 'stuck on Other Profile' bug) were completely invisible to tests.
2. The e2e_cognitive_stack was hardcoded to return None for 80% of engines (NavGraph, ZeroEngine, Telepathic, Darwin, ActiveInference), causing plugins to silently bypass their core logic during tests.
3. get_screenshot_b64 and screenshot() returned None, preventing the VLM stack from even running structural fallback code.

Fix:
- Systematically stripped FakeSAENormal from all 29 E2E workflow tests.
- Built e2e_cognitive_stack_factory to instantiate the REAL Cognitive Stack (GrowthBrain, QNavGraph, ZeroLatencyEngine, etc.) just like in bot_flow.py.
- Patched ONLY the external LLM network requests in conftest.py via mock_llm_network_calls so that structural execution stays 100% real but avoids non-deterministic LLM timeouts.
- Injected a valid 1x1 black image for screenshots.

All 29 E2E tests now execute the true production logic path and pass in 49 seconds.
2026-04-30 12:46:01 +02:00