instagram-bot/TESTING.md

🧪 Instagram Bot Testing Standards

This project follows a strict Test-Driven Development (TDD) philosophy. We do not write features blindly; we ground our development in real-world observations and automated verification.

🔴🟢🔵 The TDD Workflow (Red-Green-Refactor)

Every new feature or bugfix should follow this cycle:

1. RED: Start by obtaining a Real XML Dump (using the Testing Toolkit) of the target UI state. Write a test that fails against this dump or a --live device.
2. GREEN: Implement the minimum amount of code (logic in TelepathicEngine, QNavGraph, etc.) to make the test pass.
3. REFACTOR: Clean up the code. Ensure it adheres to our Best Practices, is well-documented, and doesn't introduce regressions.

Tip

TDD-specific tests, regression fixes, and edge-case hardening should be placed in the tests/tdd/ directory.

---

1. Mock Testing (Offline Mode)

This is the default mode used in CI/CD pipelines and for rapid local development iterations.

• Concept: The DeviceFacade is fully mocked. UI states are loaded from static XML fixtures located in tests/fixtures/.
• Primary Benefit: Extremely fast (<1s per test) and requires no physical device or emulator.
• Command:

  pytest
  

---

2. Fixture Management (The Toolkit)

To prevent offline tests from validating against outdated Instagram layouts, XML fixtures must be periodically synchronized. Use the Testing Toolkit located in scripts/.

Interactive Guide (Full Sync)

This guide walks you through 13 critical views (Home, Explore, DMs, etc.) and automatically captures the required dumps.

• Command:

  python3 scripts/sync_fixtures.py --config test_config.yml --interactive
  

Single Fixture Update

If only a specific screen has changed:

• Command:

  python3 scripts/sync_fixtures.py --config test_config.yml --fixture explore_feed_dump.xml
  

---

3. Live Hardware Testing (Real Device Mode)

Validates that the bot correctly interacts with a real device (emulator or physical phone) by performing actual clicks and swipes.

• Concept: Disables mocks. pytest connects via ADB to the active device.
• Primary Benefit: Detects UI synchronization issues, animation delays, and ADB connection drops.
• Command:

  pytest --live
  

  (Note: Uses the device ID specified in your config or conftest.py defaults).

---

4. AI & LLM Validation

Verifies that the bot's "brain" (Telepathic Engine) still understands XML structures and correctly maps elements to actions (e.g., finding the "Like Button").

• Concept: Sends real prompts to your local LLM server (Ollama/Qwen).
• Primary Benefit: Protects against "prompt drift" or performance regressions after model updates.
• Command:

  RUN_LIVE_AI_TESTS=1 pytest tests/integration/test_live_telepathy.py
  

---

5. E2E Functional Sequences

These tests simulate full automation loops to ensure that different components (Goap, Navigation, SAE) play together correctly.

• Concept: Targeted scenario tests that verify a complete user flow from start to finish.
• Example Flows:
  • tests/e2e/test_e2e_explore_feed.py: Validates the full "Explore -> Analyze -> Interact" loop.
  • tests/e2e/test_e2e_dm_sequence.py: Validates handling of message threads.
• Command:

  pytest tests/e2e/
  

---

6. Cognitive Benchmarking

Measures the "IQ" and latency of your LLM models to ensure they are suitable for autonomous navigation.

• Concept: Runs a series of 13+ UI-parsing scenarios against your configured models and scores them on accuracy and speed.
• Benefit: Identifies models that are too slow or "hallucinate" UI coordinates before you let them loose on your real account.
• Command (Ollama):

  python3 benchmarks/run_competitive_benchmark.py --all-ollama
  

• Command (Existing Config):

  python3 benchmarks/run_competitive_benchmark.py --config test_config.yml
  

---

7. Latency and Adaptive Snap Validation

Ensuring the agent handles slow network responses or missing feed markers (e.g., getting trapped in a Story) is critical for Full Self-Driving autonomy.

• Concept: Simulates UI rendering delays to trigger the post_load_timeout and verify the Adaptive Snap recovery logic.
• Implementation: When testing bot_flow.py, mock _wait_for_post_loaded or the underlying device.dump_hierarchy() to return an incomplete or missing feed XML (like reel_viewer_root) to verify the bot presses back or wobbles successfully.
• Key Assertions:
  • Verify that nav_graph.do('align') or device.press("back") is called when _wait_for_post_loaded fails to find FEED_MARKERS.
  • Validate that the timeout gracefully escapes loop-locks rather than blindly proceeding with bad UI state.

---

🛠 Troubleshooting

• Device offline: Ensure that adb devices lists your device and it is authorized.
• LLM Timeout: Verify that Ollama is running (ollama list) and the required model (e.g., qwen3.5:latest) is loaded.
• Missing Fixture: If a test fails with MISSING REAL DUMP, use the Toolkit (Step 2) to capture the missing screen.
• Benchmark Failures: If a model fails benchmarks, it is automatically marked as is_unsuitable and should not be used for critical navigation tasks.

---

💎 Golden Rules of Implementation

To maintain 100% reliability and "Tesla-level" autonomy, every developer (and AI agent) MUST follow these rules:

1. Strict Green Light Policy: All tests (both existing and new) MUST be green before a task is considered finished. No exceptions.
2. No Fix Without a Red Test: Never implement a fix or a feature without first having a failing test that demonstrates the problem or the missing capability.
3. Explicit Test Summary: Every completion summary must explicitly list exactly which tests were added or modified to verify the change.
4. Exhaustive Edge-Case Coverage: Consider and test for failure modes: "What if the DB is down?", "What if the screen is empty?", "What if the user is in a state we've never seen?".
5. Efficient, Fail-Fast Testing:
   • Do not run the entire suite if you know where the failure is.
   • Run targeted tests immediately after a change.
   • Fail fast: fix the first failing test before moving to the next.
   • Maintain a mental (or written) list of remaining failing tests to ensure none are forgotten.

---

💎 Best Practices & No-Gos

• Use Golden Fixtures: Always use real, freshly pulled XML dumps. If the Instagram UI changes, update the fixtures immediately using the Testing Toolkit.
• Singleton Isolation: Ensure all core singletons (TelepathicEngine, GoalExecutor) are reset between tests in conftest.py.
• Hermetic Tests: Each test must be independent. Ensure on-disk caches (JSON files) are wiped before each run.
• Layered Validation: Start with fast mock tests for logic, then verify with --live hardware tests for physical interaction.
• Relative Pathing: Use os.path.join relative to __file__ for all fixture loading.

❌ No-Gos

• "Lying" Mocks: Never create hand-written or "guessed" XML structures. If you don't have a dump, pull a real one.
• Hardcoded Absolute Paths: Never use paths like /Users/name/.... These break CI and other developers' environments.
• State Leakage: Never rely on the side effects of a previous test. If a test fails, it should not cause subsequent tests to fail.
• Implicit Timing in Mocks: Do not use time.sleep() for UI waiting in offline tests. Rely on the VirtualClock or state-based assertions.
• Mocking Navigation Logic: In E2E tests, do not mock the internal decision-making of the TelepathicEngine or GrowthBrain. Force them to process real (fixture) data.