Compare commits
186 Commits
0aeed11186
...
feat/radic
| Author | SHA1 | Date | |
|---|---|---|---|
| 5389369cef | |||
| 1fbc2140c7 | |||
| cf21dd3f76 | |||
| 8a2c93fb64 | |||
| 749e82ea14 | |||
| 8d58c3cf89 | |||
| f8fc8ace8e | |||
| dd79e07fb3 | |||
| da1a308c5d | |||
| b626dc6488 | |||
| 5ca5777c4b | |||
| 8729995338 | |||
| d57857d444 | |||
| ac272eadce | |||
| 59e7967458 | |||
| 59cd5477af | |||
| c3f84088aa | |||
| 548fc374ae | |||
| 5ada31c77e | |||
| 39185593dd | |||
| c0cfa24384 | |||
| 3800254fe3 | |||
| 22216cbd2d | |||
| 79e5784b93 | |||
| 5c9ea0e5e2 | |||
| 91fa426b70 | |||
| a265dee3d0 | |||
| d81a5d4e81 | |||
| f384fbb749 | |||
| 565bdaa568 | |||
| c641204a6b | |||
| 4b645c6fb2 | |||
| c7c7ce29f8 | |||
| b36dde77d8 | |||
| 93b2140844 | |||
| 67c3d464e0 | |||
| d298f03891 | |||
| 604f2d7341 | |||
| cd8f35056c | |||
| 800fb1da98 | |||
| 6cd068f951 | |||
| f46b0b7bcb | |||
| 5fbbe3d273 | |||
| f85d0a8a76 | |||
| f0a54d4e20 | |||
| 0a73c35809 | |||
| e535c10b65 | |||
| 91effbc843 | |||
| 3da3849ca1 | |||
| 51ee7a6793 | |||
| 936da47f61 | |||
| d1e0995148 | |||
| 2f8eebb7e9 | |||
| 9c6f80de9d | |||
| cff7e976e0 | |||
| f6f15ebd9a | |||
| 1cc367697e | |||
| 738a59ac8d | |||
| cd6cecbe27 | |||
| 4af4ddb060 | |||
| f32ee46d8c | |||
| d2de5f91de | |||
| 9a13216064 | |||
| aa5184786e | |||
| 2e1edec56a | |||
| df18a48a84 | |||
| f1a8573be8 | |||
| da7201117c | |||
| fddf14fd67 | |||
| 392abff313 | |||
| 556bd181fa | |||
| 2c44331f03 | |||
| b83b55e02b | |||
| db226ed7c2 | |||
| 47f94b699c | |||
| 96fdbd7db7 | |||
| ca91ae4b33 | |||
| a560225dc9 | |||
| 849fb63426 | |||
| fc44633ebc | |||
| 0f5b71708d | |||
| 0ef2840f79 | |||
| 068a6a616a | |||
| effb1f5ae1 | |||
| 0e43996ccd | |||
| b6846ab0fe | |||
| 6db579f45b | |||
| 0ed12303ac | |||
| 6abb519e3b | |||
| 4e91db01c9 | |||
| e55abc5a8a | |||
| 03105437b8 | |||
| b9c29a5a2d | |||
| 71310b8e84 | |||
| 073a90c38c | |||
| 44fae37cc7 | |||
| 48071cc9b8 | |||
| 10a85a91f1 | |||
| 5bf0053884 | |||
| a846462d02 | |||
| 0dbafd0a82 | |||
| 83e5b94ddf | |||
| dd8285e1ce | |||
| ac5d5351a6 | |||
| ad012b4cd4 | |||
| 5fcf1f180b | |||
| 9a74d89477 | |||
| dc4b576bc1 | |||
| e94dfe8c5c | |||
| 7aa6bfccf6 | |||
| 5fef014cb4 | |||
| 0bdfd999d2 | |||
| 4ad559e107 | |||
| f220e09193 | |||
| de2a1c104f | |||
| 52c553827f | |||
| cd64794f55 | |||
| bd9148e6e9 | |||
| 1e1bba6b16 | |||
| 2b992cf2a8 | |||
| c051c3a4c3 | |||
| 3006020106 | |||
| 3b9465a3bc | |||
| 5d50228945 | |||
| 7277f27fae | |||
| ee3de811d3 | |||
| c93333928a | |||
| 12937cb2c1 | |||
| 097a5753f9 | |||
| 9ee6aab831 | |||
| da804b174a | |||
| 93175b7caf | |||
| e37d92cdfd | |||
| 1c38dabe79 | |||
| 7b8daa7670 | |||
| a7449a1db3 | |||
| 746eeb767d | |||
| 36a8683643 | |||
| 888136f733 | |||
| ae36b6e196 | |||
| e70ce0f52d | |||
| 22ca93c988 | |||
| 740f8f1f56 | |||
| f148efd2a0 | |||
| ac95dec9d8 | |||
| 0b68d4bc77 | |||
| 8c37290bc3 | |||
| b4bafb59be | |||
| 41450c4eaf | |||
| e9201e0e30 | |||
| ae046be3b1 | |||
| a2a4a75603 | |||
| 714c914432 | |||
| 294403d590 | |||
| 117e7a22e7 | |||
| 0fbd1b1678 | |||
| b5cca06ce2 | |||
| 3c4dd84a61 | |||
| 9ad49500f9 | |||
| 4de087ae45 | |||
| 42a11107fd | |||
| b916b86bc5 | |||
| 0bfda47561 | |||
| ddbe8f8e99 | |||
| 5b53a7e4c0 | |||
| 42eabb7bda | |||
| 144d6401b5 | |||
| 77e8251aa7 | |||
| ad1af4edfe | |||
| 018b615829 | |||
| d69da4c974 | |||
| 82bf931b0e | |||
| 8f8efe6f2a | |||
| 5266b8b290 | |||
| 87df8d21a9 | |||
| 6edd2a18fb | |||
| 2b0d0840a8 | |||
| f1590631a1 | |||
| 30724d3c03 | |||
| 75009d91a2 | |||
| fff9ec5b0a | |||
| ee3022e95d | |||
| 0a02e901b6 | |||
| 2c6404f387 | |||
| fc3209bdc1 | |||
| ba4d7ffda2 |
28
.agents/rules/ai_integration.md
Normal file
28
.agents/rules/ai_integration.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# AI & LLM Integration Standards
|
||||
|
||||
This document defines how LLM calls and AI features must be implemented within GramPilot.
|
||||
|
||||
## 1. Centralized Provider
|
||||
- **Never** make raw `requests.post` calls to Ollama or OpenRouter directly in business logic.
|
||||
- **Always** use the centralized `GramAddict.core.llm_provider.query_llm` or `query_telepathic_llm` wrappers. This ensures consistent timeout handling, logging, and fallback logic.
|
||||
|
||||
## 2. Determinism over Creativity
|
||||
- GramPilot uses LLMs for *structural classification*, not creative writing.
|
||||
- **Always** force structured JSON outputs (`format_json=True`).
|
||||
- **Always** set `temperature=0.0` to ensure deterministic, repeatable classifications of the UI.
|
||||
|
||||
## 3. Resource Hygiene (VRAM)
|
||||
- Local LLMs (via Ollama) consume massive VRAM.
|
||||
- Always implement the `keep_alive: 0` pattern (via `unload_ollama_models`) during bot shutdown or catastrophic crashes in the `finally` block to prevent GPU memory fragmentation.
|
||||
|
||||
## 4. The Resolution Cascade
|
||||
- The LLM is the **last resort** (Level 3).
|
||||
- Always try CPU Fast-Paths (Level 1) and Qdrant Vector Similarity (Level 2) before waking up an LLM.
|
||||
- If an LLM is used, its result must be cached in Qdrant to ensure it is never called twice for the same UI state.
|
||||
|
||||
## 5. Benchmark Guard (Safety Pre-Checks)
|
||||
- **Model Validation:** The `check_model_benchmarks` function in `benchmark_guard.py` enforces a strict quality gate before the bot even starts.
|
||||
- **Scoring System:** It checks the configured models against `benchmarks/data/llm_benchmarks.json`.
|
||||
- **< 50 Score:** Critical Failure. The agent will hallucinate and compromise account safety. The bot warns the user not to run unattended.
|
||||
- **< 80 Score:** Sub-Standard. The model might occasionally fail at precise XML structural parsing (`TelepathicScore`) or persona-matching (`ResonanceScore`).
|
||||
- **Purpose:** Because FSD (Full Self-Driving) relies heavily on structural AI fallback, running untested small parameter models (like an un-tuned 1B model) can lead to infinite loops or incorrect clicks. The Benchmark Guard ensures only capable models (like `qwen3.5:latest` or `llama3.2-vision`) are trusted for autonomous navigation.
|
||||
23
.agents/rules/android_automation.md
Normal file
23
.agents/rules/android_automation.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# Android Automation & Interaction Standards
|
||||
|
||||
This document defines how the bot interacts with the Android OS and the Instagram UI.
|
||||
|
||||
## 1. No Fixed Coordinates
|
||||
- **Never** hardcode `(x, y)` coordinates for clicks, swipes, or interactions.
|
||||
- UI layouts change across devices (dpi, aspect ratios). All coordinates must be derived dynamically from the XML bounds of the target node.
|
||||
|
||||
## 2. Stealth & Interaction Physics (Bypass Bot Detection)
|
||||
- **ABSOLUTE RULE:** You must **never** use raw, robotic input methods like `device.click()` or `device.swipe()`. Instagram will detect these mathematically perfect straight lines and constant speeds immediately and shadowban the account.
|
||||
- **Mandatory Functions:**
|
||||
- For clicking: Use `device.human_click(x, y)` which injects biological jitter and realistic touch down/up timings via sendevent.
|
||||
- For swiping: Use `device.human_swipe(start_x, start_y, end_x, end_y)` or `humanized_scroll()`. These utilize Bezier curves and variable acceleration (Dopamine Pacing Engine) to simulate human thumbs.
|
||||
- **Micro-Delays:** Always inject `random_sleep()` between interactions to simulate human perception and reaction time. Never execute zero-delay sequential clicks.
|
||||
|
||||
## 3. UIAutomator2 / DeviceFacade
|
||||
- All hardware interactions must go through the `DeviceFacade`.
|
||||
- Do not instantiate raw `uiautomator2` connections deep in the business logic.
|
||||
- If the app crashes or the connection drops, rely on the global error handling and recovery loops in `run.py` to restart the ADB server or the app.
|
||||
|
||||
## 4. Node Validation
|
||||
- **Do not trust text matching alone.** Text can be user-generated (e.g., a bio saying "Follow me").
|
||||
- Always validate the structural identity of a node using its `resource-id` or its hierarchical position within the XML tree to prevent malicious user-generated content from triggering bot actions.
|
||||
21
.agents/rules/diagnostics_and_tracing.md
Normal file
21
.agents/rules/diagnostics_and_tracing.md
Normal file
@@ -0,0 +1,21 @@
|
||||
# Diagnostics, Tracing & Logging Standards
|
||||
|
||||
This document defines how GramPilot records its autonomous sessions for debugging and replay purposes.
|
||||
|
||||
## 1. Frame-by-Frame Session Tracing (The "Black Box")
|
||||
- **Trace Directory:** During a live run, the bot continuously dumps every seen XML layout into `debug/session_traces/<timestamp>/`.
|
||||
- **Sequential Reconstruction:** Every `dump_hierarchy()` call is saved as a sequential file (e.g., `00001.xml`, `00002.xml`).
|
||||
- **Purpose:** Since the bot is 100% autonomous and makes its own decisions via LLM/Qdrant, we cannot rely on stacktraces alone if navigation fails. The session traces act as a "Black Box" flight recorder. If the bot gets stuck, developers can step through the `session_traces` XML files to see exactly what the bot "saw" and why the `SituationalAwarenessEngine` or `GoalPlanner` made a specific decision.
|
||||
|
||||
## 2. Standardized Logging
|
||||
- **Visual Log Prefixes:** Always use clear, emoji-prefixed tags in the logger to instantly identify which subsystem is acting:
|
||||
- `🧠 [SAE]`: Situational Awareness Engine (Perception & Escape Planning)
|
||||
- `🗺️ [GOAP]`: Goal-Oriented Action Planning (Intent routing)
|
||||
- `👁️ [Telepathic]`: The fast-path / structural LLM reasoning
|
||||
- `👆 [Physics]`: Swipes, clicks, and physical interactions
|
||||
- `❄️ [VRAM Cleanup]`: Resource management
|
||||
- **Log Files:** Standard execution logs are saved in the `logs/` directory for long-term auditing.
|
||||
|
||||
## 3. Fixture Harvesting
|
||||
- **Trace to Test Pipeline:** If a session trace reveals a novel UI state that the bot failed to navigate, that specific `<sequence>.xml` file from `debug/session_traces/` must be copied to `tests/fixtures/` and integrated via `scripts/sync_fixtures.py`.
|
||||
- **Never Synthesize:** You must use the raw, failed session trace to build the failing TDD test. This guarantees that the fix addresses the actual real-world DOM structure Instagram served, not a developer's assumption.
|
||||
21
.agents/rules/goap_navigation.md
Normal file
21
.agents/rules/goap_navigation.md
Normal file
@@ -0,0 +1,21 @@
|
||||
# GOAP & Dynamic Navigation Standards
|
||||
|
||||
This document defines how GramPilot handles high-level pathfinding and navigation across the Instagram app.
|
||||
|
||||
## 1. Goal-Oriented Action Planning (GOAP)
|
||||
- **No Hardcoded Paths:** The bot must never follow rigid, procedural step-by-step instructions (e.g., "click home, then click search, then type").
|
||||
- **State-Driven Execution:** Navigation is handled by the `GoalPlanner` and `GoalExecutor`. The agent evaluates its *current state* (via the `TelepathicEngine`) and defines a *target state* (the Goal).
|
||||
- **Dynamic Routing:** The `GoalPlanner` queries the `QNavGraph` (backed by Qdrant memory) to find the shortest/optimal sequence of actions to bridge the gap between the current state and the target state.
|
||||
|
||||
## 2. Intent Over Execution
|
||||
- **Navigation Intent:** When the bot wants to move, it sets an `Intent` (e.g., "NAVIGATE_TO_USER_PROFILE"). It does *not* care about how to get there. The GOAP engine calculates the intermediate hops required based on its learned memory of the UI graph.
|
||||
- **Fall-Through Healing:** If an intermediate hop fails (e.g., the bot expected to see the Explore tab but saw a Modal), the `SituationalAwarenessEngine` clears the modal, the `TelepathicEngine` re-evaluates the state, and the GOAP loop *re-plans* dynamically.
|
||||
|
||||
## 3. The QNavGraph (Qdrant Navigation Memory)
|
||||
- **Graph Nodes:** Every unique UI screen perceived is a node in the graph, hashed by its structural XML signature.
|
||||
- **Graph Edges:** Every successful interaction (`EscapeAction` or `Intent` execution) that transitions the bot from Node A to Node B is recorded as a directed edge.
|
||||
- **Continuous Discovery:** If GOAP cannot find a path to the goal in `QNavGraph`, the bot switches to "Discovery Mode", randomly exploring safe UI elements (guided by `available_actions` and LLM hints) until it maps a path to the target.
|
||||
|
||||
## 4. Infinite Recursion Guards
|
||||
- **Synthetic Intent Tracking:** The `GoalPlanner` must track failed or cycling intents to prevent infinite loops (the "feed refresh trap").
|
||||
- If the agent detects it is bouncing between the same states without making progress toward the Goal, it must escalate to a higher-level reset (e.g., `app_start`) or fail gracefully rather than doomscrolling indefinitely.
|
||||
29
.agents/rules/grampilot_core.md
Normal file
29
.agents/rules/grampilot_core.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# GramPilot Core Development Rules
|
||||
|
||||
This document codifies the core architectural and development principles for the GramPilot (Instagram Bot) project.
|
||||
|
||||
## 1. 100% AUTONOMOUS "TESLA" FSD PHILOSOPHY (ABSOLUTE DIRECTIVE)
|
||||
- **Zero Static Navigation Code:** GramPilot is a "Full Self-Driving" (FSD) agent. You are FORBIDDEN from using brittle heuristics, static UI locators (XPaths), fixed string searches, or hardcoded navigation sequences.
|
||||
- **Structural Perception Only:** The bot must rely entirely on its `SituationalAwarenessEngine` to structurally "read" the XML dump, understand context, and resolve the correct layout using Qdrant vector memory.
|
||||
- **Dynamic Fallbacks:** If the UI updates, the bot must not crash. It must fall back to LLM reasoning, dynamically find the right button, and learn the new layout asynchronously.
|
||||
- **Self-Healing Memory:** If the Qdrant DB learns a false positive (e.g., misclassifying a normal screen as an `OBSTACLE_MODAL`), the LLM must detect this, emit `false_positive`, and the engine will autonomously overwrite the corrupted vector back to `NORMAL`.
|
||||
|
||||
## 2. Strict Test-Driven Development (TDD)
|
||||
- **Red, Green, Refactor:** Never write a single line of production code without a failing test proving its necessity.
|
||||
- **Real-World Fixtures Only:** Tests must use high-fidelity, real-world XML UI dumps (`tests/fixtures/`). Mocks must never fake or obscure structural UI realities.
|
||||
- **Hermetic Test Isolation:** Tests must be deterministic. Use centralized stubs (e.g., `mock_sae_perceive` in `conftest.py`) to bypass local LLM/Qdrant latency for pure logic tests, while keeping full E2E perception tests in `test_e2e_sae.py`.
|
||||
- **Zero Flakiness:** The E2E test suite is the ultimate gatekeeper. State leakage between tests is unacceptable.
|
||||
|
||||
## 3. Qdrant as the Neural Brain
|
||||
- **Persistent Perception:** Qdrant is the persistent memory of the bot (`ScreenMemoryDB`, `NavigationMemoryDB`). It is the absolute source of truth for UI layouts.
|
||||
- **Vector-Based Sub-Second Recalls:** Instead of running an LLM on every screen, the bot hashes and compresses the XML layout, converts it to an embedding, and queries Qdrant. If the similarity threshold (>0.90) is met, the bot knows instantly what to do based on past experience.
|
||||
- **Learning Loop:** Only when Qdrant fails (a novel screen) does the LLM step in. The LLM's solution is then verified, and if successful, embedded into Qdrant. Thus, the bot gets faster and smarter with every run, transitioning from expensive AI reasoning to instant vector recalls.
|
||||
|
||||
## 4. Tooling and Infrastructure
|
||||
- **Memory Purging:** Use `blank_start: true` in `test_config.yml` to trigger a system-wide Qdrant wipe when the persistent navigation graph becomes irreparably poisoned.
|
||||
- **Fixture Synchronization:** Use `scripts/sync_fixtures.py` to capture and integrate real-world XML dumps into the test suite.
|
||||
- **LLM Fallback:** The LLM is a *fallback* for novel UI states, not the primary navigation driver. It is used to suggest escape plans (`EscapeAction`) which are then executed, verified, and learned by Qdrant.
|
||||
|
||||
## 5. Code Quality
|
||||
- **Modular Plugin Architecture:** Interaction loops (e.g., Reels, Story viewing, Profiling) must remain decoupled via the `PluginRegistry`.
|
||||
- **Max 500 LoC:** No module should exceed 500 lines of code. Divide and conquer responsibilities immediately if approaching this limit.
|
||||
18
.agents/rules/testing_standards.md
Normal file
18
.agents/rules/testing_standards.md
Normal file
@@ -0,0 +1,18 @@
|
||||
# Testing Standards & Fixtures
|
||||
|
||||
This document dictates the specific testing implementation standards for GramPilot.
|
||||
|
||||
## 1. Hermetic Testing & Mocks
|
||||
- **No Sleep:** Never use `time.sleep()` in unit tests. Time-based flakiness is unacceptable. Mock the clock or the `random_sleep` function.
|
||||
- **Centralized Stubs:** If a test does not explicitly test the `SituationalAwarenessEngine` (SAE), the SAE must be stubbed via `conftest.py` (`mock_sae_perceive`) to return `SituationType.NORMAL`. This prevents E2E logic tests (like swiping logic) from failing due to missing local Ollama/Qdrant instances.
|
||||
|
||||
## 2. UI Dumps as Truth
|
||||
- **No Synthetic XML:** You must never write synthetic or "guessed" XML strings for tests.
|
||||
- **Real Fixtures:** All tests involving perception must use real-world XML dumps extracted from physical devices. Store them in `tests/fixtures/` and sync them using `scripts/sync_fixtures.py`.
|
||||
|
||||
## 3. Coverage
|
||||
- **100% Pass Rate:** The build is considered broken if a single test fails.
|
||||
- **Test Categories:**
|
||||
- `tests/e2e/`: Full end-to-end navigational sequences.
|
||||
- `tests/anomalies/`: Edge cases (e.g., action blocks, network drops).
|
||||
- `tests/property/`: Property-based invariants (e.g., ensuring swipe physics never go out of bounds).
|
||||
43
.gitignore
vendored
43
.gitignore
vendored
@@ -10,9 +10,13 @@
|
||||
!test_config.yml
|
||||
*.json
|
||||
*.xml
|
||||
!tests/fixtures/*.xml
|
||||
!tests/fixtures/*.jpg
|
||||
!tests/fixtures/*.json
|
||||
!tests/e2e/fixtures/*.xml
|
||||
!tests/e2e/fixtures/*.jpg
|
||||
!tests/e2e/fixtures/*.json
|
||||
logs/
|
||||
*.pyc
|
||||
__pycache__/
|
||||
.DS_Store
|
||||
crashes
|
||||
accounts
|
||||
@@ -24,3 +28,38 @@ Pipfile.lock
|
||||
*.log*
|
||||
*.ini
|
||||
*.db
|
||||
|
||||
# Debug artifacts & garbage scripts (Rule 5: KRIEG DEM MÜLL)
|
||||
scratch*.py
|
||||
rewrite_*.py
|
||||
test_*.py
|
||||
!tests/**
|
||||
update_*.py
|
||||
profile_dump.*
|
||||
resp_dump.*
|
||||
test_compress.py
|
||||
test_fixtures.py
|
||||
output.txt
|
||||
e2e_*.log
|
||||
traceback.log
|
||||
|
||||
# Coverage
|
||||
htmlcov/
|
||||
.coverage
|
||||
coverage.xml
|
||||
coverage_e2e.json
|
||||
.hypothesis/
|
||||
|
||||
# Local diagnostic traces
|
||||
debug/
|
||||
|
||||
# Bytecode & Cache (Enforce at bottom to override whitelists)
|
||||
**/__pycache__/
|
||||
**/*.pyc
|
||||
**/*.pyo
|
||||
**/*.pyd
|
||||
.pytest_cache/
|
||||
.hypothesis/
|
||||
.coverage
|
||||
htmlcov/
|
||||
coverage.xml
|
||||
|
||||
24
.pre-commit-config.yaml
Normal file
24
.pre-commit-config.yaml
Normal file
@@ -0,0 +1,24 @@
|
||||
repos:
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v4.6.0
|
||||
hooks:
|
||||
- id: trailing-whitespace
|
||||
- id: end-of-file-fixer
|
||||
- id: check-yaml
|
||||
- id: check-added-large-files
|
||||
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: v0.4.1
|
||||
hooks:
|
||||
- id: ruff
|
||||
args: [ --fix ]
|
||||
- id: ruff-format
|
||||
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: run-tests-and-coverage
|
||||
name: Run fast tests & check coverage drops
|
||||
entry: ./scripts/pre_commit_tests.sh
|
||||
language: system
|
||||
types: [python]
|
||||
pass_filenames: true
|
||||
@@ -22,11 +22,31 @@ When Stage 3 successfully resolves an unknown interaction, the bot records the s
|
||||
Found in `active_inference.py`. Based on the free-energy principle, the bot calculates "Surprise" (prediction errors).
|
||||
- **Shadow Mode**: Before transitioning screens, the bot predicts the target UI. If it lands somewhere unexpected (a popup), it registers a prediction error, hits "Back", and averts a crash.
|
||||
|
||||
### 🛡️ Honeypot Radome
|
||||
### 🛡️ Honeypot Radome & Anti-Trap Sensors
|
||||
Found in `sensors/honeypot_radome.py`.
|
||||
- Instagram deploys 1x1 pixel invisible traps to detect bots. The Radome parses the raw XML and topologically removes any nodes with `bounds="[0,0][0,0]"` *before* the bot's navigation engine evaluates it.
|
||||
- **Topological Traps**: Instagram deploys 1x1 pixel or 0x0 traps to detect bots. The Radome strictly strips these nodes prior to processing.
|
||||
- **The Interceptor Sentinel**: Detects and purges full-screen invisible `clickable="true"` overlays that act as touch traps (e.g., bounds >= 90% with no content description).
|
||||
- **Ghost Engagement Guard**: Strips DOM nodes explicitly tagged with `visible-to-user="false"` to prevent triggering Accessibility Hooks.
|
||||
- **VLM Sanity Guard**: Woven into `telepathic_engine.py`, it sends semantic matches for destructive actions (Like/Follow) through a Vision Language Model step to prevent executing semantic "Bait and Switch" tricks.
|
||||
|
||||
### 🧠 Situational Awareness Engine (SAE)
|
||||
Found in `situational_awareness.py`. Handles autonomous obstacle detection, recovery, and learning without hardcoded rules.
|
||||
- **3-Layer Modal Fast-Path**: Eliminates LLM hallucination traps for Instagram-internal modals (surveys, rating prompts) via O(1) deterministic structural checks:
|
||||
1. **Resource-ID Guard**: Detects internal blocking overlays (e.g., `survey_overlay_container`, `nux_overlay`).
|
||||
2. **Dismiss-Button Heuristic**: Cross-validates typical negative actions ("Not Now", "Take Survey") with overlay structures to prevent false positives in post captions.
|
||||
3. **Zero-Deception Fallback**: If structural markers fail, falls back to `ScreenMemoryDB` and ultimately the LLM. Structured invariants always override the semantic cache.
|
||||
|
||||
### 🦾 Biometric Facade (Gaussian Clicks)
|
||||
Found in `device_facade.py`.
|
||||
- Human touches do not follow a flat mathematical uniform grid. The GramPilot simulates genuine **biometric dispersion** using `random.gauss(mu, sigma)`, strictly centering clicks inside a thumb-bias radius (bottom-left skew for right-handers). In tests, this hits a 68% standard deviation precision.
|
||||
|
||||
### 💉 Dopamine Engine & Resonance Oracle
|
||||
Instead of hardcoding limits like `max_likes = 50`, the bot stops interacting based on **simulated boredom**.
|
||||
- The `ResonanceEngine` calculates the aesthetic score of content.
|
||||
- The `DopamineEngine` uses this score to modulate pace. High resonance = engagement. Low resonance over multiple posts = early session termination (simulating human fatigue).
|
||||
|
||||
## 4. The 100% Autonomy Directive (Zero Hardcoding)
|
||||
GramPilot is designed as a true agent, not a state-machine script. It operates on **absolute zero hardcoded UI states or edge cases**.
|
||||
- **No Manual Guards**: Features like `if "row_feed_button_like" not in xml:` or `if state == "ReelsFeed":` are strictly prohibited. The bot must understand the screen via its Vision-Language-Action (VLA) pipeline.
|
||||
- **No Hand-Holding**: If the LLM makes a mistake (e.g., clicking the wrong button in a DM), the solution is to improve the VLM prompt, the system architecture, or the Visual Critic. We never insert `if is_dm_thread:` hacks.
|
||||
- **Smart like a human**: The bot navigates by visually confirming targets, detecting obstacles when the UI organically stops responding, and inferring context precisely like a real user scrolling.
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
"""Human-like Instagram bot powered by UIAutomator2"""
|
||||
|
||||
from GramAddict.core.version import __version__, __tested_ig_version__
|
||||
|
||||
from GramAddict.core.bot_flow import start_bot
|
||||
from GramAddict.core.version import __tested_ig_version__, __version__
|
||||
|
||||
|
||||
def run(**kwargs):
|
||||
start_bot(**kwargs)
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
from GramAddict.core.agentic_views import *
|
||||
import argparse
|
||||
from os import getcwd, path
|
||||
|
||||
from GramAddict import __version__
|
||||
from GramAddict.core.agentic_views import *
|
||||
from GramAddict.core.bot_flow import start_bot
|
||||
from GramAddict.core.download_from_github import download_from_github
|
||||
|
||||
@@ -13,9 +13,7 @@ def cmd_init(args):
|
||||
for username in args.account_name:
|
||||
if not path.exists("./run.py"):
|
||||
print("Creating run.py ...")
|
||||
download_from_github(
|
||||
"https://github.com/GramAddict/bot/blob/master/run.py"
|
||||
)
|
||||
download_from_github("https://github.com/GramAddict/bot/blob/master/run.py")
|
||||
if not path.exists(f"./accounts/{username}"):
|
||||
print(
|
||||
f"Creating 'accounts/{username}' folder with a config starting point inside. You have to edit these files according with https://docs.gramaddict.org/#/configuration"
|
||||
@@ -53,8 +51,10 @@ def cmd_dump(args):
|
||||
os.popen("adb shell pkill atx-agent").close()
|
||||
try:
|
||||
d = u2.connect(args.device)
|
||||
except RuntimeError as err:
|
||||
raise SystemExit(err)
|
||||
except Exception as err:
|
||||
raise SystemExit(
|
||||
f"⚠️ [ADB ConnectError] Could not connect to device: {err}\nPlease check if ADB is running and your device is authorized."
|
||||
)
|
||||
|
||||
def dump_hierarchy(device, path):
|
||||
xml_dump = device.dump_hierarchy()
|
||||
@@ -71,11 +71,7 @@ def cmd_dump(args):
|
||||
dump_hierarchy(d, "dump/cur/hierarchy.xml")
|
||||
archive_name = int(time.time())
|
||||
make_archive(archive_name)
|
||||
print(
|
||||
Fore.GREEN
|
||||
+ Style.BRIGHT
|
||||
+ "\nCurrent screen dump generated successfully! Please, send me this file:"
|
||||
)
|
||||
print(Fore.GREEN + Style.BRIGHT + "\nCurrent screen dump generated successfully! Please, send me this file:")
|
||||
print(Fore.BLUE + Style.BRIGHT + f"{os.getcwd()}\\screen_{archive_name}.zip")
|
||||
|
||||
|
||||
@@ -126,9 +122,7 @@ def main() -> None:
|
||||
prog="GramAddict",
|
||||
description="free human-like Instagram bot",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-v", "--version", action="version", version=f"{parser.prog} {__version__}"
|
||||
)
|
||||
parser.add_argument("-v", "--version", action="version", version=f"{parser.prog} {__version__}")
|
||||
subparser = parser.add_subparsers(dest="subparser")
|
||||
actions = {}
|
||||
for c in _commands:
|
||||
|
||||
132
GramAddict/core/account_switcher.py
Normal file
132
GramAddict/core/account_switcher.py
Normal file
@@ -0,0 +1,132 @@
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def verify_and_switch_account(device, nav_graph, target_username):
|
||||
logger.info(f"🛂 [Identity Guard] Verifying if active account matches target: '{target_username}'")
|
||||
|
||||
# 1. Navigate to OwnProfile to reliably check identity
|
||||
from GramAddict.core.goap import GoalExecutor
|
||||
|
||||
goap = GoalExecutor.get_instance(device, target_username)
|
||||
success = goap.achieve("open profile")
|
||||
if not success:
|
||||
logger.error("❌ [Identity Guard] Failed to reach OwnProfile to verify account.")
|
||||
return False
|
||||
|
||||
time.sleep(2.0)
|
||||
xml_dump = device.dump_hierarchy()
|
||||
|
||||
# 2. Check if already active
|
||||
# The action_bar_title on OwnProfile contains the username.
|
||||
is_active = False
|
||||
try:
|
||||
clean_xml = re.sub(r"<\?xml.*?\?>", "", xml_dump).strip()
|
||||
root = ET.fromstring(clean_xml)
|
||||
for elem in root.iter("node"):
|
||||
res_id = elem.attrib.get("resource-id", "")
|
||||
text = elem.attrib.get("text", "").lower()
|
||||
if "action_bar_title" in res_id and target_username.lower() in text:
|
||||
is_active = True
|
||||
break
|
||||
except Exception as e:
|
||||
logger.warning(f"Error parsing XML for identity check: {e}")
|
||||
|
||||
if is_active:
|
||||
logger.info(f"✅ [Identity Guard] Successfully verified active account is already '{target_username}'.")
|
||||
return True
|
||||
|
||||
logger.warning(f"🔄 [Identity Guard] Account mismatch detected! Switching to '{target_username}'...")
|
||||
|
||||
# 3. Find the Profile Tab to long press using deterministic structural markers
|
||||
profile_tab = None
|
||||
try:
|
||||
# Priority 1: Structural ID
|
||||
tab_view = device.find(resourceIdMatches=".*profile_tab.*")
|
||||
if tab_view.exists():
|
||||
bounds = tab_view.info.get("bounds")
|
||||
if bounds:
|
||||
left, top, right, bottom = bounds["left"], bounds["top"], bounds["right"], bounds["bottom"]
|
||||
profile_tab = ((left + right) // 2, (top + bottom) // 2)
|
||||
|
||||
# Priority 2: Geometric Fallback (Bottom Right)
|
||||
if not profile_tab:
|
||||
info = device.get_info()
|
||||
width = info.get("displayWidth", 1080)
|
||||
height = info.get("displayHeight", 2400)
|
||||
# Profile tab is typically in the bottom right corner (last 20% of width, bottom 10% of height)
|
||||
profile_tab = (int(width * 0.9), int(height * 0.95))
|
||||
logger.info(f"📐 [Identity Guard] Using geometric fallback for profile tab: {profile_tab}")
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Error resolving profile tab structurally: {e}")
|
||||
|
||||
if not profile_tab:
|
||||
logger.error("❌ [Identity Guard] Cannot find profile_tab to initiate account switch!")
|
||||
return False
|
||||
|
||||
# Long press to open account selector
|
||||
device.long_click(profile_tab[0], profile_tab[1], 1.5)
|
||||
time.sleep(3.0)
|
||||
|
||||
# 4. Find the target account in the selector list
|
||||
xml_dump = device.dump_hierarchy()
|
||||
account_node = None
|
||||
try:
|
||||
clean_xml = re.sub(r"<\?xml.*?\?>", "", xml_dump).strip()
|
||||
root = ET.fromstring(clean_xml)
|
||||
for elem in root.iter("node"):
|
||||
text = elem.attrib.get("text", "").lower()
|
||||
content_desc = elem.attrib.get("content-desc", "").lower()
|
||||
|
||||
# Exact match or starts with username followed by spaces/punctuation
|
||||
target_l = target_username.lower()
|
||||
is_match = False
|
||||
|
||||
if text == target_l or content_desc == target_l:
|
||||
is_match = True
|
||||
elif target_l in text.split() or target_l in content_desc.split():
|
||||
is_match = True
|
||||
elif text.startswith(target_l + "\n") or text.startswith(target_l + " "):
|
||||
is_match = True
|
||||
elif target_l in text or target_l in content_desc:
|
||||
# Fallback purely to literal inclusion (might match backups, but better than failing)
|
||||
is_match = True
|
||||
|
||||
if is_match:
|
||||
bounds_str = elem.attrib.get("bounds")
|
||||
if bounds_str:
|
||||
coords = re.findall(r"\d+", bounds_str)
|
||||
if len(coords) == 4:
|
||||
x = (int(coords[0]) + int(coords[2])) // 2
|
||||
y = (int(coords[1]) + int(coords[3])) // 2
|
||||
account_node = (x, y)
|
||||
break
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if account_node:
|
||||
logger.info(f"🖱️ [Identity Guard] Found account '{target_username}' in selector. Tapping!")
|
||||
device.click(account_node[0], account_node[1])
|
||||
time.sleep(6.0) # Wait heavily for app to reload context
|
||||
nav_graph.current_state = "UNKNOWN" # Force graph to re-evaluate after massive state shift
|
||||
return True
|
||||
else:
|
||||
logger.error(
|
||||
f"❌ [Identity Guard] Target account '{target_username}' not found in the account switcher! Is it logged in?"
|
||||
)
|
||||
try:
|
||||
from GramAddict.core.diagnostic_dump import dump_ui_state
|
||||
|
||||
dump_ui_state(
|
||||
device, "identity_guard", {"reason": "account_not_found_in_bottom_sheet", "target": target_username}
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
# Escape the bottom sheet
|
||||
device.press("back")
|
||||
return False
|
||||
@@ -1,42 +1,69 @@
|
||||
"""
|
||||
Active Inference Engine v2 — The Bot's Immune System.
|
||||
|
||||
Bayesian Active Inference: predicts future UI states before acting,
|
||||
evaluates predictions against reality, and steers behavior based on
|
||||
accumulated surprise (Free Energy).
|
||||
|
||||
v2 Enhancements:
|
||||
- Consecutive error tracking → automatic policy escalation
|
||||
- Interaction throttling → reduces follow/like probability under high surprise
|
||||
- Session abort recommendation → when environment is fundamentally unstable
|
||||
- Rich prediction context → tracks WHAT was expected vs. WHAT was found
|
||||
"""
|
||||
|
||||
import logging
|
||||
import time
|
||||
import math
|
||||
from datetime import datetime
|
||||
import time
|
||||
|
||||
from colorama import Fore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ActiveInferenceEngine:
|
||||
"""
|
||||
Bayesian Active Inference Engine.
|
||||
Calculates Free Energy (Surprise) based on prediction errors in the
|
||||
Calculates Free Energy (Surprise) based on prediction errors in the
|
||||
Instagram environment. Steers the agent's 'Thermodynamic Policy'.
|
||||
|
||||
Policies:
|
||||
- STABLE: Free energy < 0.75. Normal operation. All interactions enabled.
|
||||
- CAUTIOUS: Free energy 0.75-1.2. Reduced interaction probability. Longer waits.
|
||||
- DORMANT: Free energy > 1.2. Minimal interactions. Maximum sleep. May recommend abort.
|
||||
"""
|
||||
|
||||
def __init__(self, username):
|
||||
self.username = username
|
||||
self.free_energy = 0.0
|
||||
self.surprise_threshold = 0.75
|
||||
self.last_update = time.time()
|
||||
self.policy = "STABLE" # STABLE, CAUTIOUS, DORMANT
|
||||
self.policy = "STABLE" # STABLE, CAUTIOUS, DORMANT
|
||||
self.expectation_history = []
|
||||
|
||||
|
||||
# v2: Consecutive error tracking for escalation
|
||||
self._consecutive_prediction_errors = 0
|
||||
self._total_predictions = 0
|
||||
self._total_errors = 0
|
||||
self._session_start = time.time()
|
||||
|
||||
def calculate_surprise(self, predicted_outcome: float, observed_outcome: float):
|
||||
"""
|
||||
Bayesian surprise calculation (simplified Kullback-Leibler divergence).
|
||||
"""
|
||||
# prediction error
|
||||
error = abs(predicted_outcome - observed_outcome)
|
||||
|
||||
|
||||
# Free energy accumulation
|
||||
self.free_energy = (self.free_energy * 0.7) + (error * 0.3)
|
||||
|
||||
|
||||
# Decay free energy over time (Thermodynamic relaxation)
|
||||
now = time.time()
|
||||
hours_passed = (now - self.last_update) / 3600.0
|
||||
decay = math.exp(-0.1 * hours_passed)
|
||||
self.free_energy *= decay
|
||||
self.last_update = now
|
||||
|
||||
|
||||
# Policy steering
|
||||
if self.free_energy > 1.2:
|
||||
self.policy = "DORMANT"
|
||||
@@ -44,8 +71,11 @@ class ActiveInferenceEngine:
|
||||
self.policy = "CAUTIOUS"
|
||||
else:
|
||||
self.policy = "STABLE"
|
||||
|
||||
logger.info(f"⚖️ [Active Inference] Surprise: {self.free_energy:.4f} | Policy: {self.policy}", extra={"color": f"{Fore.BLUE}"})
|
||||
|
||||
logger.info(
|
||||
f"⚖️ [Active Inference] Surprise: {self.free_energy:.4f} | Policy: {self.policy}",
|
||||
extra={"color": f"{Fore.BLUE}"},
|
||||
)
|
||||
return self.free_energy
|
||||
|
||||
def predict_state(self, expected_signature: list):
|
||||
@@ -54,42 +84,71 @@ class ActiveInferenceEngine:
|
||||
expected_signature: list of terms expected in the resulting XML.
|
||||
"""
|
||||
self.expectation_history.append(expected_signature)
|
||||
logger.debug(f"⚖️ [Shadow Mode] Predicting future state containing: {expected_signature}", extra={"color": f"{Fore.BLUE}"})
|
||||
logger.debug(
|
||||
f"⚖️ [Shadow Mode] Predicting future state containing: {expected_signature}", extra={"color": f"{Fore.BLUE}"}
|
||||
)
|
||||
|
||||
def evaluate_prediction(self, context_xml: str) -> bool:
|
||||
"""
|
||||
Evaluates the last prediction against reality.
|
||||
Returns True if reality matches prediction, False otherwise (Prediction Error).
|
||||
|
||||
v2: Tracks consecutive errors and escalates policy automatically.
|
||||
"""
|
||||
if not self.expectation_history:
|
||||
return True
|
||||
|
||||
|
||||
expected_signature = self.expectation_history.pop()
|
||||
self._total_predictions += 1
|
||||
matched = any(sig.lower() in context_xml.lower() for sig in expected_signature)
|
||||
|
||||
|
||||
if matched:
|
||||
self._consecutive_prediction_errors = 0
|
||||
self.calculate_surprise(1.0, 1.0)
|
||||
return True
|
||||
else:
|
||||
logger.warning(f"⚖️ [Shadow Mode] Prediction Error! Did not find {expected_signature} in resulting UI.", extra={"color": f"{Fore.RED}"})
|
||||
self._consecutive_prediction_errors += 1
|
||||
self._total_errors += 1
|
||||
logger.warning(
|
||||
f"⚖️ [Shadow Mode] Prediction Error #{self._consecutive_prediction_errors}! "
|
||||
f"Did not find {expected_signature} in resulting UI.",
|
||||
extra={"color": f"{Fore.RED}"},
|
||||
)
|
||||
self.calculate_surprise(1.0, 0.0)
|
||||
|
||||
|
||||
# v2: Consecutive error escalation
|
||||
if self._consecutive_prediction_errors >= 5:
|
||||
self.policy = "DORMANT"
|
||||
logger.error(
|
||||
f"🚨 [Active Inference] {self._consecutive_prediction_errors} consecutive prediction errors! "
|
||||
f"Environment is fundamentally unstable. DORMANT mode engaged.",
|
||||
extra={"color": f"{Fore.RED}"},
|
||||
)
|
||||
elif self._consecutive_prediction_errors >= 3:
|
||||
self.policy = "CAUTIOUS"
|
||||
logger.warning(
|
||||
f"⚠️ [Active Inference] {self._consecutive_prediction_errors} consecutive errors. "
|
||||
f"Switching to CAUTIOUS policy.",
|
||||
extra={"color": f"{Fore.YELLOW}"},
|
||||
)
|
||||
|
||||
# ── Dojo Data Engine Hook ──
|
||||
# When prediction fails, explicitly submit the snapshot for shadow-compilation
|
||||
try:
|
||||
from GramAddict.core.dojo_engine import DojoEngine
|
||||
|
||||
# Note: get_instance() works without passing device as it was already initialized in bot_flow by this point.
|
||||
dojo = DojoEngine.get_instance()
|
||||
dojo.submit_snapshot(
|
||||
heuristic_name=str(expected_signature),
|
||||
context_xml=context_xml,
|
||||
intent_prompt=f"Locate the missing elements or correct the heuristic predicting state: {expected_signature}"
|
||||
intent_prompt=f"Locate the missing elements or correct the heuristic predicting state: {expected_signature}",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to offload snapshot to Dojo Engine: {e}")
|
||||
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def get_sleep_modifier(self):
|
||||
"""
|
||||
Returns a multiplier for sleep durations based on surprise.
|
||||
@@ -99,3 +158,58 @@ class ActiveInferenceEngine:
|
||||
if self.policy == "CAUTIOUS":
|
||||
return 2.0
|
||||
return 1.0
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# v2: New behavioral steering methods
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def get_interaction_probability(self) -> float:
|
||||
"""
|
||||
Returns a probability multiplier [0.0 - 1.0] for interaction decisions.
|
||||
|
||||
Under STABLE: 1.0 (full interaction rate)
|
||||
Under CAUTIOUS: 0.5 (halved interaction rate)
|
||||
Under DORMANT: 0.1 (minimal interaction — only high-confidence targets)
|
||||
|
||||
This directly modifies follow/like/comment probability in the feed loop.
|
||||
"""
|
||||
if self.policy == "DORMANT":
|
||||
return 0.1
|
||||
if self.policy == "CAUTIOUS":
|
||||
return 0.5
|
||||
return 1.0
|
||||
|
||||
def should_abort_session(self) -> bool:
|
||||
"""
|
||||
Recommends session abort when the environment is fundamentally broken.
|
||||
|
||||
Triggers:
|
||||
- 5+ consecutive prediction errors (UI is completely unexpected)
|
||||
- Free energy > 2.0 (accumulated instability beyond recovery)
|
||||
|
||||
The caller (bot_flow) can choose to honor this or override.
|
||||
"""
|
||||
if self._consecutive_prediction_errors >= 5:
|
||||
return True
|
||||
if self.free_energy > 2.0:
|
||||
return True
|
||||
return False
|
||||
|
||||
def get_error_rate(self) -> float:
|
||||
"""Returns the session-wide prediction error rate."""
|
||||
if self._total_predictions == 0:
|
||||
return 0.0
|
||||
return self._total_errors / self._total_predictions
|
||||
|
||||
def get_diagnostics(self) -> dict:
|
||||
"""Returns a diagnostic snapshot for logging/telemetry."""
|
||||
return {
|
||||
"free_energy": round(self.free_energy, 4),
|
||||
"policy": self.policy,
|
||||
"consecutive_errors": self._consecutive_prediction_errors,
|
||||
"total_predictions": self._total_predictions,
|
||||
"total_errors": self._total_errors,
|
||||
"error_rate": round(self.get_error_rate(), 4),
|
||||
"session_uptime_minutes": round((time.time() - self._session_start) / 60, 1),
|
||||
"should_abort": self.should_abort_session(),
|
||||
}
|
||||
|
||||
274
GramAddict/core/behaviors/__init__.py
Normal file
274
GramAddict/core/behaviors/__init__.py
Normal file
@@ -0,0 +1,274 @@
|
||||
"""
|
||||
Behavior Plugin Architecture — Composable, Testable Bot Actions.
|
||||
|
||||
Design goals:
|
||||
1. Each behavior is a self-contained plugin with a clear lifecycle
|
||||
2. Plugins declare prerequisites (what screen state they require)
|
||||
3. Plugins are registered in a priority-sorted registry
|
||||
4. The feed loop queries the registry: "which plugins want to act on this post?"
|
||||
5. Each plugin can be tested in complete isolation
|
||||
|
||||
This is the "neural pathway" system — instead of one monolithic brain (bot_flow.py),
|
||||
the bot has specialized pathways that fire when their conditions are met.
|
||||
|
||||
Tesla analogy: Instead of one "drive" function, there are composable behaviors
|
||||
(lane-keep, auto-park, summon) that activate when relevant.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class BehaviorContext:
|
||||
"""
|
||||
Shared context passed to every behavior plugin.
|
||||
Contains everything a behavior needs to make decisions and act.
|
||||
"""
|
||||
|
||||
device: Any # Android device facade
|
||||
configs: Any # User configuration
|
||||
session_state: Any # Current session state
|
||||
cognitive_stack: Dict[str, Any] # Cognitive engines (growth, resonance, etc.)
|
||||
shared_state: Dict[str, Any] = field(default_factory=dict) # State shared between plugins
|
||||
context_xml: str = "" # Current screen XML dump
|
||||
sleep_mod: float = 1.0 # Active Inference sleep multiplier
|
||||
post_data: Optional[Dict] = None # Extracted post content
|
||||
username: str = "" # Current target username (if applicable)
|
||||
|
||||
|
||||
@dataclass
|
||||
class BehaviorResult:
|
||||
"""
|
||||
Result returned by a behavior plugin after execution.
|
||||
Used by the orchestrator to decide what happens next.
|
||||
"""
|
||||
|
||||
executed: bool = False # Did the behavior actually do something?
|
||||
should_continue: bool = True # Should the feed loop continue to next post?
|
||||
should_skip: bool = False # Should we skip to the next post immediately?
|
||||
skip_type: str = "normal" # "normal" (humanized) or "fast" (ad evasion)
|
||||
interactions: int = 0 # Number of interactions performed
|
||||
metadata: Dict[str, Any] = field(default_factory=dict) # Plugin-specific data
|
||||
|
||||
|
||||
class BehaviorPlugin(ABC):
|
||||
"""
|
||||
Base class for all behavior plugins.
|
||||
|
||||
Lifecycle:
|
||||
1. `can_activate(ctx)` — Should this behavior fire for this context?
|
||||
2. `priority` — If multiple behaviors can activate, higher priority goes first.
|
||||
3. `execute(ctx)` — Run the behavior.
|
||||
|
||||
Rules:
|
||||
- Plugins must be stateless between posts (state lives in session_state)
|
||||
- Plugins must handle their own errors (never crash the feed loop)
|
||||
- Plugins must respect session limits via ctx.session_state
|
||||
"""
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def name(self) -> str:
|
||||
"""Unique identifier for this behavior."""
|
||||
...
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
"""
|
||||
Execution priority. Higher = runs first.
|
||||
|
||||
Guidelines:
|
||||
- 100+: Safety/guard behaviors (ad detection, block detection)
|
||||
- 50-99: Primary interactions (like, follow, comment)
|
||||
- 10-49: Secondary interactions (carousel, story view)
|
||||
- 1-9: Observational behaviors (scraping, analytics)
|
||||
"""
|
||||
return 50
|
||||
|
||||
@property
|
||||
def exclusive(self) -> bool:
|
||||
"""
|
||||
If True, no other behavior can run after this one on the same post.
|
||||
Used for guard behaviors that abort interaction (e.g., ad detection).
|
||||
"""
|
||||
return False
|
||||
|
||||
@abstractmethod
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
"""
|
||||
Returns True if this behavior should fire for the given context.
|
||||
Must be cheap to evaluate (no device interactions).
|
||||
"""
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
"""
|
||||
Execute the behavior. Must handle all errors internally.
|
||||
Returns a BehaviorResult describing what happened.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_config(self, ctx: BehaviorContext) -> dict:
|
||||
"""Helper to retrieve plugin-specific configuration."""
|
||||
return ctx.configs.get_plugin_config(self.name)
|
||||
|
||||
def __repr__(self):
|
||||
return f"<{self.__class__.__name__} name={self.name} priority={self.priority}>"
|
||||
|
||||
|
||||
class PluginRegistry:
|
||||
"""
|
||||
Central registry for behavior plugins.
|
||||
|
||||
Manages plugin registration, priority sorting, and orchestrated execution.
|
||||
Thread-safe singleton.
|
||||
"""
|
||||
|
||||
_instance = None
|
||||
|
||||
@classmethod
|
||||
def get_instance(cls) -> "PluginRegistry":
|
||||
if cls._instance is None:
|
||||
cls._instance = cls()
|
||||
return cls._instance
|
||||
|
||||
@classmethod
|
||||
def reset(cls):
|
||||
"""Wipe the registry singleton instance."""
|
||||
cls._instance = None
|
||||
|
||||
def __init__(self):
|
||||
self._plugins: List[BehaviorPlugin] = []
|
||||
self._sorted = False
|
||||
|
||||
def register(self, plugin: BehaviorPlugin):
|
||||
"""Register a behavior plugin."""
|
||||
# Prevent duplicate registration
|
||||
for existing in self._plugins:
|
||||
if existing.name == plugin.name:
|
||||
logger.debug(f"Plugin '{plugin.name}' already registered. Skipping.")
|
||||
return
|
||||
|
||||
self._plugins.append(plugin)
|
||||
self._sorted = False
|
||||
logger.debug(f"🧩 [Plugin] Registered: {plugin.name} (priority={plugin.priority})")
|
||||
|
||||
def unregister(self, name: str):
|
||||
"""Remove a plugin by name."""
|
||||
self._plugins = [p for p in self._plugins if p.name != name]
|
||||
self._sorted = False
|
||||
|
||||
def _ensure_sorted(self):
|
||||
"""Sort plugins by priority (highest first)."""
|
||||
if not self._sorted:
|
||||
self._plugins.sort(key=lambda p: p.priority, reverse=True)
|
||||
self._sorted = True
|
||||
|
||||
@property
|
||||
def plugins(self) -> List[BehaviorPlugin]:
|
||||
"""Returns all plugins, sorted by priority."""
|
||||
self._ensure_sorted()
|
||||
return list(self._plugins)
|
||||
|
||||
def get_active_plugins(self, ctx: BehaviorContext) -> List[BehaviorPlugin]:
|
||||
"""Returns plugins that can activate for the given context, sorted by priority."""
|
||||
self._ensure_sorted()
|
||||
active = []
|
||||
for plugin in self._plugins:
|
||||
try:
|
||||
if plugin.can_activate(ctx):
|
||||
active.append(plugin)
|
||||
except Exception as e:
|
||||
logger.error(f"🧩 [Plugin] Error checking {plugin.name}.can_activate: {e}")
|
||||
return active
|
||||
|
||||
def execute_all(self, ctx: BehaviorContext) -> List[BehaviorResult]:
|
||||
"""
|
||||
Execute all active plugins in priority order.
|
||||
|
||||
Stops early if an exclusive plugin fires (e.g., ad guard).
|
||||
Returns list of results from all executed plugins.
|
||||
"""
|
||||
self._ensure_sorted()
|
||||
results = []
|
||||
|
||||
for plugin in self._plugins:
|
||||
try:
|
||||
if not plugin.can_activate(ctx):
|
||||
continue
|
||||
|
||||
logger.debug(f"🧩 [PluginRegistry] TRACE: Calling execute() on {plugin.name}")
|
||||
result = plugin.execute(ctx)
|
||||
results.append(result)
|
||||
if result.executed:
|
||||
logger.debug(
|
||||
f"🧩 [PluginRegistry] Plugin {plugin.name} executed successfully. Metadata: {result.metadata}"
|
||||
)
|
||||
|
||||
if (plugin.exclusive and result.executed) or result.should_skip:
|
||||
logger.debug(
|
||||
f"🧩 [Plugin] {plugin.name} triggered chain termination (exclusive={plugin.exclusive}, should_skip={result.should_skip})."
|
||||
)
|
||||
break
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🧩 [Plugin] Error executing {plugin.name}: {e}")
|
||||
results.append(BehaviorResult(executed=False, metadata={"error": str(e)}))
|
||||
|
||||
return results
|
||||
|
||||
def __len__(self):
|
||||
return len(self._plugins)
|
||||
|
||||
def __contains__(self, name: str):
|
||||
return any(p.name == name for p in self._plugins)
|
||||
|
||||
|
||||
# Import plugins at the bottom to avoid circular imports
|
||||
from GramAddict.core.behaviors.ad_guard import AdGuardPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.anomaly_handler import AnomalyHandlerPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.close_friends_guard import CloseFriendsGuardPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.comment import CommentPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.darwin_dwell import DarwinDwellPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.like import LikePlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.obstacle_guard import ObstacleGuardPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.perfect_snapping import PerfectSnappingPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.post_data_extraction import PostDataExtractionPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.post_interaction import PostInteractionPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.profile_visit import ProfileVisitPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.rabbit_hole import RabbitHolePlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.repost import RepostPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.resonance_evaluator import ResonanceEvaluatorPlugin # noqa: E402
|
||||
from GramAddict.core.behaviors.scrape_profile import ScrapeProfilePlugin # noqa: E402
|
||||
|
||||
# Note: We do not automatically instantiate all of them globally here to avoid circular
|
||||
# dependencies during initial load. The bot_flow.py engine should explicitly register them.
|
||||
|
||||
|
||||
def load_all_plugins():
|
||||
"""
|
||||
Registers all available core behavior plugins into the global registry.
|
||||
Useful for testing or full-agent initialization.
|
||||
"""
|
||||
registry = PluginRegistry.get_instance()
|
||||
registry.register(AdGuardPlugin())
|
||||
registry.register(AnomalyHandlerPlugin())
|
||||
registry.register(CloseFriendsGuardPlugin())
|
||||
registry.register(CommentPlugin())
|
||||
registry.register(DarwinDwellPlugin())
|
||||
registry.register(LikePlugin())
|
||||
registry.register(ObstacleGuardPlugin())
|
||||
registry.register(PerfectSnappingPlugin())
|
||||
registry.register(PostDataExtractionPlugin())
|
||||
registry.register(PostInteractionPlugin())
|
||||
registry.register(ProfileVisitPlugin())
|
||||
registry.register(RabbitHolePlugin())
|
||||
registry.register(RepostPlugin())
|
||||
registry.register(ResonanceEvaluatorPlugin())
|
||||
registry.register(ScrapeProfilePlugin())
|
||||
69
GramAddict/core/behaviors/ad_guard.py
Normal file
69
GramAddict/core/behaviors/ad_guard.py
Normal file
@@ -0,0 +1,69 @@
|
||||
import logging
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
from GramAddict.core.utils import is_ad
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AdGuardPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Checks for ads in the feed and scrolls past them.
|
||||
Implements a deadlock escape after 5 consecutive ads.
|
||||
|
||||
Priority: 100 (Safety guard, runs first).
|
||||
Exclusive: True (if ad detected, stop other interactions).
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
self.consecutive_ads = 0
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "ad_guard"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 100
|
||||
|
||||
@property
|
||||
def exclusive(self) -> bool:
|
||||
return True
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
|
||||
# We check for ad presence here to decide if we activate.
|
||||
# This is a bit more expensive than a percentage check but necessary for a guard.
|
||||
# Optimization: Only check if context_xml is available or do a quick string search.
|
||||
if ctx.context_xml:
|
||||
return is_ad(ctx.context_xml, ctx.cognitive_stack)
|
||||
|
||||
return False
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
self.consecutive_ads += 1
|
||||
|
||||
if self.consecutive_ads >= 5:
|
||||
logger.warning("🛡️ [AdGuard] Deadlock detected: 5 consecutive ads. Escaping to HomeFeed.")
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
zero_engine = ctx.cognitive_stack.get("zero_latency_engine")
|
||||
if nav_graph:
|
||||
nav_graph.navigate_to("HomeFeed", zero_engine)
|
||||
self.consecutive_ads = 0
|
||||
return BehaviorResult(executed=True, should_skip=True, skip_type="fast")
|
||||
|
||||
logger.info(f"🛡️ [AdGuard] Ad detected ({self.consecutive_ads}). Delegating skip to orchestrator...")
|
||||
|
||||
# Aggressive double skip for triple ad
|
||||
if self.consecutive_ads >= 3:
|
||||
logger.info("🛡️ [AdGuard] Requesting aggressive double skip for consecutive ads.")
|
||||
return BehaviorResult(executed=True, should_skip=True, skip_type="double_fast")
|
||||
|
||||
return BehaviorResult(executed=True, should_skip=True, skip_type="fast")
|
||||
|
||||
def reset_counter(self):
|
||||
self.consecutive_ads = 0
|
||||
48
GramAddict/core/behaviors/anomaly_handler.py
Normal file
48
GramAddict/core/behaviors/anomaly_handler.py
Normal file
@@ -0,0 +1,48 @@
|
||||
import logging
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
from GramAddict.core.physics.humanized_input import humanized_scroll
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AnomalyHandlerPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Handles anomalies like zero interactive nodes on screen.
|
||||
|
||||
Priority: 98 (Runs after AdGuard, before others).
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "anomaly_handler"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 98
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
return getattr(self, "_enabled", True)
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
telepathic = ctx.cognitive_stack.get("telepathic") or TelepathicEngine.get_instance()
|
||||
xml = ctx.context_xml if ctx.context_xml else ctx.device.dump_hierarchy()
|
||||
nodes = telepathic._extract_semantic_nodes(xml)
|
||||
|
||||
ctx.shared_state["interactive_nodes"] = nodes
|
||||
|
||||
if len(nodes) == 0:
|
||||
logger.warning("🚨 [Anomaly] Zero interactive nodes found. Executing recovery...")
|
||||
ctx.device.press("back")
|
||||
sleep(1.0 * ctx.sleep_mod)
|
||||
humanized_scroll(ctx.device)
|
||||
sleep(1.0 * ctx.sleep_mod)
|
||||
return BehaviorResult(executed=True, should_skip=True)
|
||||
|
||||
return BehaviorResult(executed=False)
|
||||
102
GramAddict/core/behaviors/carousel_browsing.py
Normal file
102
GramAddict/core/behaviors/carousel_browsing.py
Normal file
@@ -0,0 +1,102 @@
|
||||
"""
|
||||
Carousel Browsing Behavior — Plugin Implementation.
|
||||
|
||||
Migrated from bot_flow.py's _interact_with_carousel function.
|
||||
Now independently testable and composable.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import random
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
from GramAddict.core.perception.feed_analysis import has_carousel_in_view
|
||||
from GramAddict.core.physics.humanized_input import humanized_horizontal_swipe
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CarouselBrowsingPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Browses carousel posts with humanized swiping and curiosity dwells.
|
||||
|
||||
Priority: 70 (Primary interaction).
|
||||
"""
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "carousel_browsing"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 70
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
|
||||
# Analysis requires XML
|
||||
xml = ctx.context_xml if ctx.context_xml else ctx.device.dump_hierarchy()
|
||||
if not has_carousel_in_view(xml):
|
||||
return False
|
||||
|
||||
if ctx.shared_state.get("carousel_browsed"):
|
||||
return False
|
||||
|
||||
config = self.get_config(ctx)
|
||||
percentage = float(config.get("percentage", getattr(ctx.configs.args, "carousel_percentage", 0)))
|
||||
return random.random() < (percentage / 100.0)
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
"""Browse carousel with humanized swiping."""
|
||||
from colorama import Fore
|
||||
|
||||
config = self.get_config(ctx)
|
||||
carousel_count_str = config.get("count", getattr(ctx.configs.args, "carousel_count", "1-2"))
|
||||
try:
|
||||
min_c, max_c = map(int, carousel_count_str.split("-"))
|
||||
count = random.randint(min_c, max_c)
|
||||
except Exception:
|
||||
count = 1
|
||||
|
||||
logger.info(
|
||||
f"📸 [Carousel] Interacting with carousel. Swiping {count} times...", extra={"color": f"{Fore.CYAN}"}
|
||||
)
|
||||
|
||||
info = ctx.device.get_info()
|
||||
w = info.get("displayWidth", 1080)
|
||||
h = info.get("displayHeight", 2400)
|
||||
|
||||
# Curiosity Peak: One slide gets extra attention
|
||||
curiosity_slide = random.randint(0, count - 1) if count > 0 else 0
|
||||
|
||||
for i in range(count):
|
||||
# Normal transition wait
|
||||
sleep(random.uniform(1.5, 3.5) * ctx.sleep_mod)
|
||||
|
||||
# ── Curiosity Dwell ──
|
||||
if i == curiosity_slide:
|
||||
dwell = random.uniform(3.0, 7.0)
|
||||
logger.debug(f"📸 [Carousel] Curiosity Peak hit on slide {i+1}. Gazing for {dwell:.1f}s...")
|
||||
sleep(dwell * ctx.sleep_mod)
|
||||
|
||||
xml_before = ctx.device.dump_hierarchy()
|
||||
|
||||
# Horizontal swipe: Right to left
|
||||
humanized_horizontal_swipe(ctx.device, start_x=w * 0.8, end_x=w * 0.2, y=h * 0.5, duration_ms=250)
|
||||
|
||||
# Brief wait for transition to complete
|
||||
sleep(random.uniform(1.5, 2.5) * ctx.sleep_mod)
|
||||
|
||||
xml_after = ctx.device.dump_hierarchy()
|
||||
xml_delta = abs(len(xml_before) - len(xml_after))
|
||||
|
||||
if xml_before == xml_after or xml_delta < 50:
|
||||
logger.info(f"📸 [Carousel] End of carousel detected on slide {i+1} (UI stable). Stopping swipe.")
|
||||
break
|
||||
|
||||
ctx.shared_state["carousel_browsed"] = True
|
||||
|
||||
return BehaviorResult(
|
||||
executed=True, interactions=count, metadata={"slides_viewed": count, "curiosity_slide": curiosity_slide}
|
||||
)
|
||||
50
GramAddict/core/behaviors/close_friends_guard.py
Normal file
50
GramAddict/core/behaviors/close_friends_guard.py
Normal file
@@ -0,0 +1,50 @@
|
||||
import logging
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
from GramAddict.core.physics.humanized_input import humanized_scroll
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CloseFriendsGuardPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Checks for close friends badge and skips.
|
||||
|
||||
Priority: 99.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "close_friends_guard"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 99
|
||||
|
||||
@property
|
||||
def exclusive(self) -> bool:
|
||||
return True
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
|
||||
xml = ctx.context_xml if ctx.context_xml else ctx.device.dump_hierarchy()
|
||||
xml_lower = xml.lower()
|
||||
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepathic = TelepathicEngine.get_instance()
|
||||
classification = telepathic.classify_screen_content(xml_lower, "close_friends_content")
|
||||
return classification == "close_friends"
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
logger.info("💚 [CloseFriendsGuard] Close friends post detected. Skipping...")
|
||||
humanized_scroll(ctx.device, is_skip=True)
|
||||
sleep(1.0 * ctx.sleep_mod)
|
||||
return BehaviorResult(executed=True, should_skip=True)
|
||||
97
GramAddict/core/behaviors/comment.py
Normal file
97
GramAddict/core/behaviors/comment.py
Normal file
@@ -0,0 +1,97 @@
|
||||
import logging
|
||||
import random
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CommentPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Handles commenting on posts.
|
||||
|
||||
Priority: 55.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "comment"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 55
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
"""Determines if we should comment on this post."""
|
||||
from GramAddict.core.session_state import SessionState
|
||||
|
||||
if ctx.session_state.check_limit(SessionState.Limit.COMMENTS):
|
||||
return False
|
||||
|
||||
# ── STRUCTURAL GUARD ──
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
if screen_type not in (ScreenType.HOME_FEED, ScreenType.POST_DETAIL, ScreenType.REELS_FEED):
|
||||
return False
|
||||
|
||||
config = self.get_config(ctx)
|
||||
comment_pct = float(config.get("percentage", getattr(ctx.configs.args, "comment_percentage", 0))) / 100.0
|
||||
|
||||
if comment_pct <= 0:
|
||||
return False
|
||||
|
||||
# Probability gate (includes resonance weighting if available in shared_state)
|
||||
res_score = ctx.shared_state.get("res_score", 1.0)
|
||||
chance = comment_pct * res_score
|
||||
|
||||
if random.random() >= chance:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
"""Comment on the current post."""
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
if not nav_graph:
|
||||
from GramAddict.core.q_nav_graph import QNavGraph
|
||||
|
||||
nav_graph = QNavGraph(ctx.device)
|
||||
|
||||
config = self.get_config(ctx)
|
||||
|
||||
# 1. Open comment section
|
||||
if nav_graph.do("open comments"):
|
||||
# 2. Generate comment text
|
||||
writer = ctx.cognitive_stack.get("writer")
|
||||
if not writer:
|
||||
logger.warning("✍️ [Comment] No 'writer' found in cognitive stack. Cannot generate comment.")
|
||||
ctx.device.press("back")
|
||||
return BehaviorResult(executed=False)
|
||||
|
||||
text = writer.generate_comment(ctx.post_data)
|
||||
logger.info(f"✍️ [Comment] Generated: '{text}'")
|
||||
|
||||
# 3. Handle Dry Run
|
||||
if config.get("dry_run", getattr(ctx.configs.args, "dry_run_comments", False)):
|
||||
logger.info("🧪 [Comment] Dry run enabled. Skipping actual post.")
|
||||
ctx.device.press("back")
|
||||
return BehaviorResult(executed=True, interactions=0, metadata={"text": text, "dry_run": True})
|
||||
|
||||
# 4. Type and post
|
||||
if nav_graph.do("type and post comment", text=text):
|
||||
logger.info(f"💬 [Comment] Posted to @{ctx.username} ✓")
|
||||
ctx.session_state.add_interaction(source=ctx.username, succeed=True, followed=False, scraped=False)
|
||||
ctx.session_state.totalComments += 1
|
||||
return BehaviorResult(executed=True, interactions=1, metadata={"text": text})
|
||||
|
||||
return BehaviorResult(executed=False)
|
||||
67
GramAddict/core/behaviors/darwin_dwell.py
Normal file
67
GramAddict/core/behaviors/darwin_dwell.py
Normal file
@@ -0,0 +1,67 @@
|
||||
import logging
|
||||
import random
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class DarwinDwellPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Simulates human dwelling using the Darwin engine.
|
||||
|
||||
Priority: 60 (Runs after evaluation, before interactions).
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "darwin_dwell"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 60
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
valid_screens = [ScreenType.HOME_FEED, ScreenType.POST_DETAIL, ScreenType.REELS_FEED, ScreenType.STORY_VIEW]
|
||||
if screen_type not in valid_screens:
|
||||
return False
|
||||
|
||||
config = self.get_config(ctx)
|
||||
percentage = float(config.get("percentage", 100))
|
||||
return random.random() < (percentage / 100.0)
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
darwin = ctx.cognitive_stack.get("darwin")
|
||||
if darwin:
|
||||
logger.info("🐢 [DarwinDwell] Executing organic dwell behaviors...")
|
||||
darwin.execute_micro_wobble(ctx.device)
|
||||
res_score = ctx.shared_state.get("res_score", 1.0)
|
||||
darwin.execute_proof_of_resonance(
|
||||
ctx.device,
|
||||
res_score,
|
||||
nav_graph=ctx.cognitive_stack.get("nav_graph"),
|
||||
configs=ctx.configs,
|
||||
resonance_oracle=ctx.cognitive_stack.get("oracle"),
|
||||
username=ctx.username,
|
||||
context_xml=ctx.context_xml or ctx.device.dump_hierarchy(),
|
||||
)
|
||||
else:
|
||||
logger.info("🐢 [DarwinDwell] Darwin engine missing. Falling back to static sleep.")
|
||||
sleep(2.5 * ctx.sleep_mod)
|
||||
|
||||
return BehaviorResult(executed=True)
|
||||
89
GramAddict/core/behaviors/follow.py
Normal file
89
GramAddict/core/behaviors/follow.py
Normal file
@@ -0,0 +1,89 @@
|
||||
import logging
|
||||
import random
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class FollowPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Follows a target user from their profile page or feed.
|
||||
|
||||
Priority: 40.
|
||||
"""
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "follow"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 40
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
"""Activates when follow is enabled and limits not reached."""
|
||||
from GramAddict.core.session_state import SessionState
|
||||
|
||||
config = self.get_config(ctx)
|
||||
follow_pct = float(config.get("percentage", getattr(ctx.configs.args, "follow_percentage", 0))) / 100.0
|
||||
|
||||
if follow_pct <= 0:
|
||||
return False
|
||||
|
||||
if ctx.session_state.check_limit(SessionState.Limit.FOLLOWS):
|
||||
return False
|
||||
|
||||
# ── STRUCTURAL GUARD ──
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
if screen_type not in (ScreenType.OTHER_PROFILE, ScreenType.FOLLOW_LIST):
|
||||
return False
|
||||
|
||||
# Probability gate
|
||||
if random.random() >= follow_pct:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
"""Follow the target user. ONLY clicks 'Follow' buttons, never 'Following'."""
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
if not nav_graph:
|
||||
from GramAddict.core.q_nav_graph import QNavGraph
|
||||
|
||||
nav_graph = QNavGraph(ctx.device)
|
||||
|
||||
# ── CRITICAL SAFETY GUARD ──
|
||||
# Pre-check: verify the button actually says "Follow" (not "Following" or "Requested").
|
||||
# Clicking "Following" opens a dangerous bottom sheet (Unfollow / Add to Favorites / Close Friends).
|
||||
xml = ctx.context_xml if ctx.context_xml else ctx.device.dump_hierarchy()
|
||||
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepathic = ctx.cognitive_stack.get("telepathic") or TelepathicEngine.get_instance()
|
||||
|
||||
classification = telepathic.classify_screen_content(xml.lower(), "profile_follow_status")
|
||||
if classification in ("following", "requested"):
|
||||
logger.info(
|
||||
f"🛡️ [Follow] Profile status is '{classification}' — user already followed. Skipping to avoid bottom sheet."
|
||||
)
|
||||
return BehaviorResult(executed=False, metadata={"reason": "already_following"})
|
||||
|
||||
if nav_graph.do("tap follow button"):
|
||||
logger.info(f"🤝 [Follow] Followed @{ctx.username} ✓")
|
||||
ctx.session_state.add_interaction(source=ctx.username, succeed=True, followed=True, scraped=False)
|
||||
|
||||
# Buffer for follow animations to close
|
||||
sleep(random.uniform(1.8, 3.2) * ctx.sleep_mod)
|
||||
|
||||
return BehaviorResult(executed=True, interactions=1, metadata={"followed": ctx.username})
|
||||
|
||||
return BehaviorResult(executed=False, metadata={"reason": "nav_failed"})
|
||||
146
GramAddict/core/behaviors/grid_like.py
Normal file
146
GramAddict/core/behaviors/grid_like.py
Normal file
@@ -0,0 +1,146 @@
|
||||
import logging
|
||||
import random
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
from GramAddict.core.physics.humanized_input import humanized_click, humanized_scroll
|
||||
from GramAddict.core.physics.timing import wait_for_post_loaded
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GridLikePlugin(BehaviorPlugin):
|
||||
"""
|
||||
Opens profile grid and likes posts with humanized behavior.
|
||||
|
||||
Priority: 30.
|
||||
"""
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "grid_like"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 30
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
"""Activates when likes are enabled, limits not reached, and probability met."""
|
||||
from GramAddict.core.session_state import SessionState
|
||||
|
||||
config = self.get_config(ctx)
|
||||
likes_pct = float(config.get("percentage", getattr(ctx.configs.args, "likes_percentage", 0))) / 100.0
|
||||
|
||||
if likes_pct <= 0:
|
||||
return False
|
||||
|
||||
if ctx.session_state.check_limit(SessionState.Limit.LIKES):
|
||||
return False
|
||||
|
||||
# ── STRUCTURAL GUARD ──
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
if screen_type not in (ScreenType.OWN_PROFILE, ScreenType.OTHER_PROFILE, ScreenType.EXPLORE_GRID):
|
||||
return False
|
||||
|
||||
# Probability gate
|
||||
if random.random() >= likes_pct:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
"""Open grid and like posts."""
|
||||
config = self.get_config(ctx)
|
||||
|
||||
# Parse like count
|
||||
likes_count_str = config.get("count", getattr(ctx.configs.args, "likes_count", "1-2"))
|
||||
try:
|
||||
if "-" in likes_count_str:
|
||||
min_l, max_l = map(int, likes_count_str.split("-"))
|
||||
count = random.randint(min_l, max_l)
|
||||
else:
|
||||
count = int(likes_count_str)
|
||||
except Exception:
|
||||
count = 1
|
||||
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
if not nav_graph:
|
||||
from GramAddict.core.q_nav_graph import QNavGraph
|
||||
|
||||
nav_graph = QNavGraph(ctx.device)
|
||||
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
nav_action = (
|
||||
"tap first image in explore grid"
|
||||
if screen_type == ScreenType.EXPLORE_GRID
|
||||
else "tap first image post in profile grid"
|
||||
)
|
||||
|
||||
if not nav_graph.do(nav_action):
|
||||
return BehaviorResult(executed=False, metadata={"reason": "grid_nav_failed"})
|
||||
|
||||
if not wait_for_post_loaded(ctx.device, timeout=5, nav_graph=nav_graph):
|
||||
logger.warning(f"❌ [GridLike] Post failed to open from profile grid of @{ctx.username}.")
|
||||
return BehaviorResult(executed=False, metadata={"reason": "post_load_failed"})
|
||||
|
||||
logger.info(f"❤️ [GridLike] Dropping {count} likes on @{ctx.username} profile grid...")
|
||||
|
||||
info = ctx.device.get_info()
|
||||
w = info.get("displayWidth", 1080)
|
||||
h = info.get("displayHeight", 2400)
|
||||
|
||||
growth = ctx.cognitive_stack.get("growth_brain")
|
||||
total_liked = 0
|
||||
|
||||
for i in range(count):
|
||||
xml_dump = ctx.device.dump_hierarchy()
|
||||
xml_dump_lower = xml_dump.lower()
|
||||
|
||||
is_reel = "reel_viewer" in xml_dump_lower or "clips_viewer" in xml_dump_lower
|
||||
|
||||
# Use growth brain for decision making (double tap vs heart button)
|
||||
use_double_tap = growth.wants_to_double_tap(is_reel=is_reel) if growth else False
|
||||
|
||||
if use_double_tap:
|
||||
offset_x = random.randint(int(w * 0.2), int(w * 0.8))
|
||||
offset_y = random.randint(int(h * 0.3), int(h * 0.7))
|
||||
humanized_click(ctx.device, offset_x, offset_y, double=True, sleep_mod=ctx.sleep_mod)
|
||||
ctx.session_state.totalLikes += 1
|
||||
total_liked += 1
|
||||
logger.debug(f"Liked grid post {i+1}/{count} via Double-Tap")
|
||||
else:
|
||||
if nav_graph.do("tap like button"):
|
||||
ctx.session_state.totalLikes += 1
|
||||
total_liked += 1
|
||||
logger.debug(f"Liked grid post {i+1}/{count} via Heart Button")
|
||||
else:
|
||||
logger.debug(f"Skipped liking grid post {i+1}/{count}")
|
||||
|
||||
sleep(random.uniform(1.0, 2.0) * ctx.sleep_mod)
|
||||
|
||||
if i < count - 1:
|
||||
if is_reel:
|
||||
humanized_scroll(ctx.device, is_skip=True)
|
||||
else:
|
||||
humanized_scroll(ctx.device, is_skip=False)
|
||||
sleep(random.uniform(1.5, 3.0) * ctx.sleep_mod)
|
||||
|
||||
ctx.device.press("back")
|
||||
sleep(random.uniform(1.0, 2.0) * ctx.sleep_mod)
|
||||
|
||||
return BehaviorResult(
|
||||
executed=True, interactions=total_liked, metadata={"posts_viewed": count, "posts_liked": total_liked}
|
||||
)
|
||||
74
GramAddict/core/behaviors/like.py
Normal file
74
GramAddict/core/behaviors/like.py
Normal file
@@ -0,0 +1,74 @@
|
||||
import logging
|
||||
import random
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class LikePlugin(BehaviorPlugin):
|
||||
"""
|
||||
Handles liking posts.
|
||||
|
||||
Priority: 50.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "likes"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 50
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
"""Determines if we should like this post."""
|
||||
from GramAddict.core.session_state import SessionState
|
||||
|
||||
if ctx.session_state.check_limit(SessionState.Limit.LIKES):
|
||||
logger.error("LikePlugin: limit check failed")
|
||||
return False
|
||||
|
||||
config = self.get_config(ctx)
|
||||
likes_pct = float(config.get("percentage", getattr(ctx.configs.args, "likes_percentage", 80))) / 100.0
|
||||
|
||||
if likes_pct <= 0:
|
||||
return False
|
||||
|
||||
# ── STRUCTURAL GUARD ──
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
if screen_type not in (ScreenType.HOME_FEED, ScreenType.POST_DETAIL, ScreenType.REELS_FEED):
|
||||
return False
|
||||
|
||||
# Probability gate
|
||||
if random.random() >= likes_pct:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
"""Like the current post."""
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
if not nav_graph:
|
||||
from GramAddict.core.q_nav_graph import QNavGraph
|
||||
|
||||
nav_graph = QNavGraph(ctx.device)
|
||||
|
||||
if nav_graph.do("tap like button"):
|
||||
logger.info(f"❤️ [Like] Liked post by @{ctx.username} ✓")
|
||||
ctx.session_state.add_interaction(source=ctx.username, succeed=True, followed=False, scraped=False)
|
||||
ctx.session_state.totalLikes += 1
|
||||
return BehaviorResult(executed=True, interactions=1)
|
||||
|
||||
return BehaviorResult(executed=False)
|
||||
101
GramAddict/core/behaviors/obstacle_guard.py
Normal file
101
GramAddict/core/behaviors/obstacle_guard.py
Normal file
@@ -0,0 +1,101 @@
|
||||
import logging
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
from GramAddict.core.diagnostic_dump import dump_ui_state
|
||||
from GramAddict.core.situational_awareness import SituationalAwarenessEngine, SituationType
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ObstacleGuardPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Guards against modals and checks marker presence to prevent infinite loops.
|
||||
|
||||
Priority: 95.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "obstacle_guard"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 95
|
||||
|
||||
@property
|
||||
def exclusive(self) -> bool:
|
||||
return True
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
return getattr(self, "_enabled", True)
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
sae = SituationalAwarenessEngine.get_instance(ctx.device)
|
||||
xml = ctx.context_xml if ctx.context_xml else ctx.device.dump_hierarchy()
|
||||
situation = sae.perceive(xml)
|
||||
|
||||
misses = ctx.shared_state.get("consecutive_marker_misses", 0)
|
||||
|
||||
# ── System Dialog / Permission Modal (e.g. "Allow Instagram to record audio?") ──
|
||||
if situation == SituationType.OBSTACLE_SYSTEM:
|
||||
logger.warning("⚠️ [ObstacleGuard] System permission dialog detected. Dismissing with BACK...")
|
||||
ctx.device.press("back")
|
||||
sleep(1.5 * ctx.sleep_mod)
|
||||
res = BehaviorResult(executed=True, should_skip=True)
|
||||
res.skip_type = "no_scroll"
|
||||
return res
|
||||
|
||||
# ── Foreign App Takeover (e.g. browser opened, wrong app in foreground) ──
|
||||
if situation == SituationType.OBSTACLE_FOREIGN_APP:
|
||||
logger.warning("⚠️ [ObstacleGuard] Foreign app detected. Pressing BACK to recover...")
|
||||
ctx.device.press("back")
|
||||
sleep(1.5 * ctx.sleep_mod)
|
||||
res = BehaviorResult(executed=True, should_skip=True)
|
||||
res.skip_type = "no_scroll"
|
||||
return res
|
||||
|
||||
# ── On-Screen Keyboard (e.g. hallucinated click on comment field) ──
|
||||
if situation == SituationType.OBSTACLE_KEYBOARD:
|
||||
logger.warning("⚠️ [ObstacleGuard] On-screen Keyboard is open. Pressing BACK to dismiss...")
|
||||
ctx.device.press("back")
|
||||
sleep(1.0 * ctx.sleep_mod)
|
||||
res = BehaviorResult(executed=True, should_skip=True)
|
||||
res.skip_type = "no_scroll"
|
||||
return res
|
||||
|
||||
# ── Instagram Modal / Overlay (survey, "Not Now" prompt, creation flow) ──
|
||||
if situation == SituationType.OBSTACLE_MODAL:
|
||||
if misses >= 2:
|
||||
logger.error("🛑 [ObstacleGuard] Failed to recover from OBSTACLE_MODAL after multiple attempts.")
|
||||
sae.unlearn_current_state(xml)
|
||||
dump_ui_state(ctx.device, f"fatal_obstacle_{ctx.session_state.job_target}")
|
||||
return BehaviorResult(executed=True, should_skip=True, metadata={"return_code": "CONTEXT_LOST"})
|
||||
|
||||
logger.warning("⚠️ [ObstacleGuard] OBSTACLE_MODAL detected. Attempting to dismiss...")
|
||||
ctx.device.press("back")
|
||||
sleep(1.5 * ctx.sleep_mod)
|
||||
|
||||
# Check recovery
|
||||
new_xml = ctx.device.dump_hierarchy()
|
||||
tele = TelepathicEngine.get_instance()
|
||||
best_node = tele.find_best_node(new_xml, intent_description="Dismiss obstacle", device=ctx.device)
|
||||
if best_node:
|
||||
ctx.device.click(best_node.get("x", 0), best_node.get("y", 0))
|
||||
|
||||
if "row_feed_button_like" in new_xml:
|
||||
logger.info("✅ [ObstacleGuard] Successfully recovered from OBSTACLE_MODAL.")
|
||||
ctx.shared_state["consecutive_marker_misses"] = 0
|
||||
else:
|
||||
ctx.shared_state["consecutive_marker_misses"] = misses + 1
|
||||
|
||||
res = BehaviorResult(executed=True, should_skip=True) # Restart loop for same post or next
|
||||
res.skip_type = "no_scroll"
|
||||
return res
|
||||
|
||||
return BehaviorResult(executed=False)
|
||||
52
GramAddict/core/behaviors/perfect_snapping.py
Normal file
52
GramAddict/core/behaviors/perfect_snapping.py
Normal file
@@ -0,0 +1,52 @@
|
||||
import logging
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
from GramAddict.core.bot_flow import _align_active_post
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PerfectSnappingPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Aligns the current post in the viewport.
|
||||
|
||||
Priority: 90 (Runs after guards, before extraction).
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "perfect_snapping"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 90
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
|
||||
# Perfect snapping is only for feed posts.
|
||||
# Do not snap if we are on a profile page, explore grid, or modal.
|
||||
from GramAddict.core.perception.feed_analysis import has_feed_markers
|
||||
|
||||
if not has_feed_markers(ctx.context_xml):
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
aligned = _align_active_post(ctx.device)
|
||||
if aligned:
|
||||
logger.info("🎯 [PerfectSnapping] Post aligned. Refreshing context XML...")
|
||||
new_xml = ctx.device.dump_hierarchy()
|
||||
radome = ctx.cognitive_stack.get("radome")
|
||||
if radome:
|
||||
new_xml = radome.sanitize_xml(new_xml)
|
||||
ctx.context_xml = new_xml
|
||||
return BehaviorResult(executed=True)
|
||||
|
||||
return BehaviorResult(executed=False)
|
||||
50
GramAddict/core/behaviors/post_data_extraction.py
Normal file
50
GramAddict/core/behaviors/post_data_extraction.py
Normal file
@@ -0,0 +1,50 @@
|
||||
import logging
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
from GramAddict.core.perception.feed_analysis import extract_post_content
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PostDataExtractionPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Extracts post data (caption, hashtags, user) for later evaluation.
|
||||
|
||||
Priority: 85 (Runs after guards, before evaluation).
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "post_data_extraction"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 85
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
from GramAddict.core.perception.feed_analysis import has_feed_markers
|
||||
|
||||
return getattr(self, "_enabled", True) and ctx.context_xml is not None and has_feed_markers(ctx.context_xml)
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
logger.debug("🧩 [PostDataExtraction] Extracting post metadata...")
|
||||
post_data = extract_post_content(ctx.context_xml, device=ctx.device)
|
||||
|
||||
if post_data:
|
||||
ctx.post_data = post_data
|
||||
ctx.username = post_data.get("username", "")
|
||||
|
||||
if post_data.get("username_missing") or not ctx.username:
|
||||
logger.error(
|
||||
"❌ [PostDataExtraction] FAILED: Post author username is empty or missing! Halting interaction."
|
||||
)
|
||||
return BehaviorResult(executed=False, metadata={"error": "Empty username extracted"})
|
||||
|
||||
logger.info(f"📝 [PostDataExtraction] Post by @{ctx.username} extracted.")
|
||||
return BehaviorResult(executed=True)
|
||||
|
||||
return BehaviorResult(executed=False)
|
||||
57
GramAddict/core/behaviors/post_interaction.py
Normal file
57
GramAddict/core/behaviors/post_interaction.py
Normal file
@@ -0,0 +1,57 @@
|
||||
import logging
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PostInteractionPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Runs after all interactions on a post are complete.
|
||||
Handles scrolling to the next post and logging outcomes.
|
||||
|
||||
Priority: 10 (lowest, runs last).
|
||||
Exclusive: True (ends the behavior chain for this post).
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "post_interaction"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 10 # Lowest priority, runs last
|
||||
|
||||
@property
|
||||
def exclusive(self) -> bool:
|
||||
return True # Ends the behavior chain for this post
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
valid_screens = [ScreenType.HOME_FEED, ScreenType.POST_DETAIL, ScreenType.REELS_FEED, ScreenType.STORY_VIEW]
|
||||
return screen_type in valid_screens
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
logger.info("🏁 [PostInteraction] Interactions complete. Moving to next post...")
|
||||
|
||||
# Log to CRM or telemetry if active
|
||||
telemetry = ctx.cognitive_stack.get("telemetry")
|
||||
if telemetry:
|
||||
telemetry.log_post_interaction(ctx.post_data, ctx.shared_state.get("session_outcomes", []))
|
||||
|
||||
return BehaviorResult(
|
||||
executed=True, should_skip=True, skip_type="normal"
|
||||
) # should_skip=True signals the feed loop to restart for the next post
|
||||
109
GramAddict/core/behaviors/profile_guard.py
Normal file
109
GramAddict/core/behaviors/profile_guard.py
Normal file
@@ -0,0 +1,109 @@
|
||||
"""
|
||||
Profile Guard Behavior — Plugin Implementation.
|
||||
|
||||
Safety guards that reject profiles before any interactions occur:
|
||||
- Private accounts
|
||||
- Empty accounts
|
||||
- Close friends (when configured)
|
||||
- Visual vibe check (AI aesthetic quality)
|
||||
|
||||
Priority 100 (highest, exclusive) — if a guard fires, no other behavior runs.
|
||||
"""
|
||||
|
||||
import logging
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ProfileGuardPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Guards against interacting with profiles that should be skipped.
|
||||
Exclusive: if this fires, no further interactions happen on this profile.
|
||||
"""
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "profile_guard"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 100 # Highest — runs before everything
|
||||
|
||||
@property
|
||||
def exclusive(self) -> bool:
|
||||
return True # Stop all other plugins if guard fires
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
"""Only activates on Profile screens to prevent false-positives in Feed/Reels."""
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
is_profile = nav_graph and nav_graph.current_state == "ProfileView"
|
||||
return bool(ctx.username) and is_profile
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
"""Check profile guards. Returns executed=True + should_skip=True if rejected."""
|
||||
from colorama import Fore
|
||||
|
||||
xml_check = ctx.context_xml
|
||||
if not xml_check:
|
||||
return BehaviorResult(executed=False)
|
||||
|
||||
xml_check_lower = xml_check.lower()
|
||||
|
||||
# Self-interaction guard
|
||||
if hasattr(ctx.session_state, "my_username") and ctx.username == ctx.session_state.my_username:
|
||||
logger.info(f"🤝 [Profile Guard] Skipping own profile @{ctx.username}.")
|
||||
return BehaviorResult(executed=True, should_skip=True, metadata={"reason": "self_profile"})
|
||||
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepathic = TelepathicEngine.get_instance()
|
||||
|
||||
# Private account guard
|
||||
if telepathic.classify_screen_content(xml_check_lower, "private_account") == "private":
|
||||
logger.info(f"🔒 [Profile Guard] @{ctx.username} is private.", extra={"color": f"{Fore.YELLOW}"})
|
||||
return BehaviorResult(executed=True, should_skip=True, metadata={"reason": "private"})
|
||||
|
||||
# Empty account guard
|
||||
if telepathic.classify_screen_content(xml_check_lower, "empty_account") == "empty":
|
||||
logger.info(f"📭 [Profile Guard] @{ctx.username} has no posts.", extra={"color": f"{Fore.YELLOW}"})
|
||||
return BehaviorResult(executed=True, should_skip=True, metadata={"reason": "empty"})
|
||||
|
||||
# Close friends guard
|
||||
if getattr(ctx.configs.args, "ignore_close_friends", False):
|
||||
if telepathic.classify_screen_content(xml_check_lower, "close_friends_content") == "close_friends":
|
||||
logger.info(
|
||||
f"💚 [Profile Guard] @{ctx.username} is a Close Friend. Ignoring.", extra={"color": "\033[32m"}
|
||||
)
|
||||
return BehaviorResult(executed=True, should_skip=True, metadata={"reason": "close_friend"})
|
||||
|
||||
# Visual Vibe Check (AI Aesthetic Quality Guard)
|
||||
import random
|
||||
|
||||
vibe_check_pct = float(getattr(ctx.configs.args, "visual_vibe_check_percentage", 0)) / 100.0
|
||||
if vibe_check_pct > 0 and random.random() < vibe_check_pct:
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepathic = ctx.cognitive_stack.get("telepathic") or TelepathicEngine.get_instance()
|
||||
persona_interests = ctx.cognitive_stack.get("persona_interests", []) if ctx.cognitive_stack else []
|
||||
vibe_result = telepathic.evaluate_profile_vibe(ctx.device, persona_interests)
|
||||
|
||||
if vibe_result:
|
||||
score = vibe_result.get("quality_score", 5)
|
||||
matches_niche = vibe_result.get("matches_niche", True)
|
||||
if score < 5 or not matches_niche:
|
||||
logger.warning(
|
||||
f"🚫 [Vibe Check] Profile @{ctx.username} rejected (Score: {score}, Niche: {matches_niche}). Reason: {vibe_result.get('reason')}"
|
||||
)
|
||||
return BehaviorResult(
|
||||
executed=True, should_skip=True, metadata={"reason": "vibe_check_failed", "score": score}
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
f"✅ [Vibe Check] Profile @{ctx.username} approved (Score: {score}). Continuing interaction.",
|
||||
extra={"color": "\033[36m"},
|
||||
)
|
||||
|
||||
# All guards passed — don't block further plugins
|
||||
return BehaviorResult(executed=False)
|
||||
114
GramAddict/core/behaviors/profile_visit.py
Normal file
114
GramAddict/core/behaviors/profile_visit.py
Normal file
@@ -0,0 +1,114 @@
|
||||
import logging
|
||||
import random
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ProfileVisitPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Handles visiting a user's profile from the feed.
|
||||
|
||||
Priority: 35.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "profile_visit"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 35
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
"""Determines if we should visit the profile."""
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
|
||||
# 1. Screen Guard: Only activate on feed screens
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
valid_screens = [ScreenType.HOME_FEED, ScreenType.EXPLORE_GRID, ScreenType.REELS_FEED]
|
||||
if screen_type not in valid_screens:
|
||||
return False
|
||||
|
||||
# 2. Guard against recursive calls or being already on profile
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
if nav_graph and nav_graph.current_state == "ProfileView":
|
||||
return False
|
||||
|
||||
# 3. Probability gate
|
||||
config = self.get_config(ctx)
|
||||
visit_pct = float(config.get("percentage", getattr(ctx.configs.args, "profile_visit_percentage", 30))) / 100.0
|
||||
|
||||
if visit_pct <= 0:
|
||||
return False
|
||||
|
||||
# 3. Probability gate (weighted by resonance)
|
||||
res_score = ctx.shared_state.get("res_score", 1.0)
|
||||
chance = visit_pct * res_score
|
||||
|
||||
if random.random() >= chance:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
"""Visit the user's profile and execute nested plugins."""
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
if not nav_graph:
|
||||
from GramAddict.core.q_nav_graph import QNavGraph
|
||||
|
||||
nav_graph = QNavGraph(ctx.device)
|
||||
|
||||
if nav_graph.do("tap post username"):
|
||||
logger.info(f"👤 [ProfileVisit] Visiting @{ctx.username}...")
|
||||
sleep(2.0 * ctx.sleep_mod)
|
||||
|
||||
# Create a new context for the profile interaction
|
||||
from GramAddict.core.behaviors import BehaviorContext, PluginRegistry
|
||||
|
||||
# Update nav state to ProfileView
|
||||
original_state = nav_graph.current_state
|
||||
nav_graph.current_state = "ProfileView"
|
||||
|
||||
profile_xml = ctx.device.dump_hierarchy()
|
||||
profile_ctx = BehaviorContext(
|
||||
device=ctx.device,
|
||||
configs=ctx.configs,
|
||||
session_state=ctx.session_state,
|
||||
cognitive_stack=ctx.cognitive_stack,
|
||||
context_xml=profile_xml,
|
||||
sleep_mod=ctx.sleep_mod,
|
||||
post_data=ctx.post_data,
|
||||
username=ctx.username,
|
||||
shared_state=ctx.shared_state,
|
||||
)
|
||||
|
||||
logger.info(f"🕵️ [ProfileVisit] Executing interactions on @{ctx.username}'s profile...")
|
||||
registry = PluginRegistry.get_instance()
|
||||
|
||||
# Execute all active plugins on the profile view (including ProfileGuard)
|
||||
registry.execute_all(profile_ctx)
|
||||
|
||||
# Restore nav state
|
||||
nav_graph.current_state = original_state
|
||||
|
||||
logger.info(f"🔙 [ProfileVisit] Returning from @{ctx.username}.")
|
||||
ctx.device.press("back")
|
||||
sleep(1.0 * ctx.sleep_mod)
|
||||
|
||||
return BehaviorResult(executed=True, interactions=1)
|
||||
|
||||
return BehaviorResult(executed=False)
|
||||
63
GramAddict/core/behaviors/rabbit_hole.py
Normal file
63
GramAddict/core/behaviors/rabbit_hole.py
Normal file
@@ -0,0 +1,63 @@
|
||||
import logging
|
||||
import random
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class RabbitHolePlugin(BehaviorPlugin):
|
||||
"""
|
||||
Randomly jumps into a user's profile if resonance is high.
|
||||
|
||||
Priority: 20 (Secondary interaction).
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "rabbit_hole"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 20
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
|
||||
res_score = ctx.shared_state.get("res_score", 0.0)
|
||||
if res_score < 0.8:
|
||||
return False
|
||||
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
valid_screens = [ScreenType.HOME_FEED, ScreenType.EXPLORE_GRID, ScreenType.REELS_FEED]
|
||||
if screen_type not in valid_screens:
|
||||
return False
|
||||
|
||||
config = self.get_config(ctx)
|
||||
percentage = float(config.get("percentage", 15))
|
||||
return random.random() < (percentage / 100.0)
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
logger.info("🕳️ [RabbitHole] Falling down the rabbit hole! Investigating user profile...")
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
if nav_graph:
|
||||
success = nav_graph.do("tap post username")
|
||||
if success:
|
||||
sleep(2.0 * ctx.sleep_mod)
|
||||
# Just a quick peek
|
||||
ctx.device.press("back")
|
||||
sleep(1.0 * ctx.sleep_mod)
|
||||
return BehaviorResult(executed=True)
|
||||
|
||||
return BehaviorResult(executed=False)
|
||||
72
GramAddict/core/behaviors/repost.py
Normal file
72
GramAddict/core/behaviors/repost.py
Normal file
@@ -0,0 +1,72 @@
|
||||
import logging
|
||||
import random
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class RepostPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Handles reposting (sharing to story) for posts.
|
||||
|
||||
Priority: 45.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "repost"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 45
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
"""Determines if we should repost this post."""
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
|
||||
# 1. Screen Guard: Only activate on post-containing screens
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
valid_screens = [ScreenType.HOME_FEED, ScreenType.POST_DETAIL, ScreenType.REELS_FEED]
|
||||
if screen_type not in valid_screens:
|
||||
return False
|
||||
|
||||
config = self.get_config(ctx)
|
||||
repost_pct = float(config.get("percentage", getattr(ctx.configs.args, "repost_percentage", 20))) / 100.0
|
||||
|
||||
if repost_pct <= 0:
|
||||
return False
|
||||
|
||||
# Probability gate
|
||||
if random.random() >= repost_pct:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
"""Repost the current post."""
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
if not nav_graph:
|
||||
from GramAddict.core.q_nav_graph import QNavGraph
|
||||
|
||||
nav_graph = QNavGraph(ctx.device)
|
||||
|
||||
# We must click the send post button first
|
||||
if nav_graph.do("tap send post button"):
|
||||
# A modal should appear, now click add to story
|
||||
if nav_graph.do("tap add to story"):
|
||||
logger.info(f"📤 [Repost] Shared post by @{ctx.username} to story ✓")
|
||||
return BehaviorResult(executed=True, interactions=1)
|
||||
|
||||
return BehaviorResult(executed=False)
|
||||
116
GramAddict/core/behaviors/resonance_evaluator.py
Normal file
116
GramAddict/core/behaviors/resonance_evaluator.py
Normal file
@@ -0,0 +1,116 @@
|
||||
import logging
|
||||
import random
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ResonanceEvaluatorPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Evaluates how much the bot likes a post based on its descriptions, vibes, etc.
|
||||
Decides whether to proceed with interactions or skip the post.
|
||||
|
||||
Priority: 80 (Runs after data extraction).
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "resonance_evaluator"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 80
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
screen_type = ctx.shared_state.get("current_screen_type")
|
||||
if not screen_type and ctx.context_xml:
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity
|
||||
|
||||
screen_type = ScreenIdentity(getattr(ctx, "username", "")).identify(ctx.context_xml).get("screen_type")
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
valid_screens = [ScreenType.HOME_FEED, ScreenType.POST_DETAIL, ScreenType.REELS_FEED, ScreenType.STORY_VIEW]
|
||||
return screen_type in valid_screens
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
resonance = ctx.cognitive_stack.get("resonance")
|
||||
if not resonance:
|
||||
logger.error("🧠 [Resonance] CRITICAL: Engine missing from cognitive stack. Defaulting to 0.5 (neutral).")
|
||||
res_score = 0.5
|
||||
else:
|
||||
post_data = ctx.post_data or {}
|
||||
res_score = resonance.calculate_resonance(post_data)
|
||||
|
||||
# Check visual vibe
|
||||
config = self.get_config(ctx)
|
||||
visual_chance = float(
|
||||
config.get("visual_vibe_check_percentage", getattr(ctx.configs.args, "visual_vibe_check_percentage", 0))
|
||||
)
|
||||
if visual_chance > 0 and random.random() < (visual_chance / 100.0):
|
||||
tele = ctx.cognitive_stack.get("telepathic")
|
||||
if tele:
|
||||
logger.info("✨ [Resonance] Performing visual vibe check...")
|
||||
|
||||
# BUG 5 Fix: Read target_audience or persona_interests
|
||||
raw_interests = getattr(ctx.configs.args, "persona_interests", "")
|
||||
if not raw_interests:
|
||||
raw_interests = getattr(ctx.configs.args, "target_audience", "")
|
||||
|
||||
if isinstance(raw_interests, list):
|
||||
persona_interests = [str(i).strip() for i in raw_interests if str(i).strip()]
|
||||
else:
|
||||
persona_interests = [i.strip() for i in str(raw_interests).split(",") if i.strip()]
|
||||
|
||||
vibe = tele.evaluate_post_vibe(ctx.device, persona_interests)
|
||||
if vibe is None:
|
||||
logger.warning(
|
||||
"✨ [Resonance] VLM vibe check returned None (truncated JSON?). Keeping neutral score."
|
||||
)
|
||||
else:
|
||||
if vibe.get("is_ad"):
|
||||
logger.info("🛡️ [Resonance Oracle] Visually identified post as an Ad! Skipping...")
|
||||
marker = vibe.get("ad_marker_text")
|
||||
if marker and marker.strip():
|
||||
from GramAddict.core.utils import learn_ad_marker
|
||||
|
||||
learn_ad_marker(marker, ctx.context_xml)
|
||||
return BehaviorResult(executed=True, should_skip=True, skip_type="fast")
|
||||
|
||||
# BUG 6 Fix: VLM returns {"should_like": true/false}, not "quality_score"
|
||||
should_like = vibe.get("should_like", False)
|
||||
vibe_score = 1.0 if should_like else 0.2
|
||||
res_score = (res_score * 0.3) + (vibe_score * 0.7)
|
||||
|
||||
ctx.shared_state["res_score"] = res_score
|
||||
logger.info(f"📊 [Resonance] Post Score: {res_score:.2f}")
|
||||
|
||||
interact_chance = float(getattr(ctx.configs.args, "interact_percentage", 100))
|
||||
|
||||
# Determine if we should skip the entire post
|
||||
# Threshold could be dynamic, but let's say 0.2 is the floor for absolute garbage
|
||||
if res_score < 0.2 or random.random() >= (interact_chance / 100.0):
|
||||
logger.info(f"⏭️ [Resonance] Skipping post (score={res_score:.2f}, chance check failed).")
|
||||
|
||||
if "session_outcomes" not in ctx.shared_state:
|
||||
ctx.shared_state["session_outcomes"] = []
|
||||
|
||||
ctx.shared_state["session_outcomes"].append(
|
||||
{"username": ctx.username, "resonance": res_score, "action": "skip"}
|
||||
)
|
||||
|
||||
# Delegate scrolling to the orchestrator
|
||||
return BehaviorResult(executed=True, should_skip=True, skip_type="fast")
|
||||
|
||||
dopamine = ctx.cognitive_stack.get("dopamine")
|
||||
if dopamine:
|
||||
quality = "high" if res_score > 0.7 else ("medium" if res_score > 0.4 else "low")
|
||||
dopamine.process_content({"score": res_score * 10, "quality": quality})
|
||||
|
||||
return BehaviorResult(executed=True, should_skip=False)
|
||||
78
GramAddict/core/behaviors/scrape_profile.py
Normal file
78
GramAddict/core/behaviors/scrape_profile.py
Normal file
@@ -0,0 +1,78 @@
|
||||
import logging
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ScrapeProfilePlugin(BehaviorPlugin):
|
||||
"""
|
||||
Extracts profile metadata (followers, following, bio) when visiting a profile.
|
||||
|
||||
Priority: 45. (Runs after ProfileGuard, before deep interactions like GridLike)
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._enabled = True
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "scrape_profile"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 45
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
if not getattr(self, "_enabled", True):
|
||||
return False
|
||||
|
||||
# Only activate if scrape_profiles is True in config
|
||||
if not getattr(ctx.configs.args, "scrape_profiles", False):
|
||||
return False
|
||||
|
||||
# Only activate when we are actively visiting a profile (via ProfileVisitPlugin)
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
if not nav_graph or nav_graph.current_state != "ProfileView":
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
from colorama import Fore
|
||||
|
||||
logger.info(f"📊 [Scraping] Extracting metadata for @{ctx.username}...", extra={"color": f"{Fore.CYAN}"})
|
||||
|
||||
telepathic = ctx.cognitive_stack.get("telepathic") or TelepathicEngine.get_instance()
|
||||
crm = ctx.cognitive_stack.get("crm")
|
||||
|
||||
xml_check = ctx.context_xml or ctx.device.dump_hierarchy()
|
||||
|
||||
f_node = telepathic.find_best_node(xml_check, "Followers count text or number", device=ctx.device)
|
||||
fg_node = telepathic.find_best_node(xml_check, "Following count text or number", device=ctx.device)
|
||||
bio_node = telepathic.find_best_node(xml_check, "User biography or description text", device=ctx.device)
|
||||
|
||||
scraped_data = {
|
||||
"username": ctx.username,
|
||||
"followers": f_node.get("text") if f_node else "unknown",
|
||||
"following": fg_node.get("text") if fg_node else "unknown",
|
||||
"bio": bio_node.get("text") if bio_node else "No bio",
|
||||
}
|
||||
|
||||
logger.info(
|
||||
f"✅ [Scraping] Data acquired: {scraped_data['followers']} followers, {scraped_data['following']} following."
|
||||
)
|
||||
|
||||
ctx.session_state.add_interaction(source=ctx.username, succeed=False, followed=False, scraped=True)
|
||||
|
||||
if crm:
|
||||
try:
|
||||
crm.enrich_lead(ctx.username, scraped_data)
|
||||
logger.info(f"💾 [CRM] Enriched lead @{ctx.username} in database.")
|
||||
except Exception as e:
|
||||
logger.error(f"❌ [CRM] Failed to enrich lead @{ctx.username}: {e}")
|
||||
|
||||
# Return executed=True, but we don't return interactions=1 since it's just data extraction
|
||||
return BehaviorResult(executed=True)
|
||||
170
GramAddict/core/behaviors/story_view.py
Normal file
170
GramAddict/core/behaviors/story_view.py
Normal file
@@ -0,0 +1,170 @@
|
||||
import logging
|
||||
import random
|
||||
import re
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.behaviors import BehaviorContext, BehaviorPlugin, BehaviorResult
|
||||
from GramAddict.core.physics.humanized_input import humanized_click
|
||||
from GramAddict.core.physics.timing import wait_for_story_loaded
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class StoryViewPlugin(BehaviorPlugin):
|
||||
"""
|
||||
Views a target user's stories from their profile.
|
||||
|
||||
Priority: 25.
|
||||
"""
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "story_view"
|
||||
|
||||
@property
|
||||
def priority(self) -> int:
|
||||
return 25
|
||||
|
||||
def can_activate(self, ctx: BehaviorContext) -> bool:
|
||||
"""Activates when story viewing is enabled and probability met."""
|
||||
config = self.get_config(ctx)
|
||||
stories_pct = float(config.get("percentage", getattr(ctx.configs.args, "stories_percentage", 0))) / 100.0
|
||||
|
||||
if stories_pct <= 0:
|
||||
return False
|
||||
|
||||
# Probability gate
|
||||
if random.random() >= stories_pct:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def execute(self, ctx: BehaviorContext) -> BehaviorResult:
|
||||
"""View stories with humanized timing."""
|
||||
config = self.get_config(ctx)
|
||||
|
||||
# Parse story count
|
||||
stories_count_str = config.get("count", getattr(ctx.configs.args, "stories_count", "1-2"))
|
||||
try:
|
||||
if "-" in stories_count_str:
|
||||
min_st, max_st = map(int, stories_count_str.split("-"))
|
||||
count = random.randint(min_st, max_st)
|
||||
else:
|
||||
count = int(stories_count_str)
|
||||
except Exception:
|
||||
count = 1
|
||||
|
||||
from GramAddict.core.goap import ScreenType
|
||||
|
||||
is_already_in_story = getattr(ctx, "screen_type", None) == ScreenType.STORY_VIEW
|
||||
|
||||
# Check for story ring
|
||||
xml = ctx.context_xml or ctx.device.dump_hierarchy()
|
||||
xml_lower = xml.lower()
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepathic = ctx.cognitive_stack.get("telepathic") or TelepathicEngine.get_instance()
|
||||
|
||||
has_story_ring = telepathic.classify_screen_content(xml_lower, "story_ring_presence") == "has_unseen_story"
|
||||
|
||||
if not has_story_ring and not is_already_in_story:
|
||||
return BehaviorResult(executed=False, metadata={"reason": "no_story"})
|
||||
|
||||
if not is_already_in_story:
|
||||
# Navigate to story
|
||||
nav_graph = ctx.cognitive_stack.get("nav_graph")
|
||||
if not nav_graph:
|
||||
from GramAddict.core.q_nav_graph import QNavGraph
|
||||
|
||||
nav_graph = QNavGraph(ctx.device)
|
||||
|
||||
if not nav_graph.do("tap story ring avatar"):
|
||||
return BehaviorResult(executed=False, metadata={"reason": "nav_failed"})
|
||||
|
||||
# Wait for story to load
|
||||
if not wait_for_story_loaded(ctx.device, timeout=5):
|
||||
logger.warning(f"❌ [StoryView] Story failed to open for @{ctx.username}.")
|
||||
return BehaviorResult(executed=False, metadata={"reason": "load_timeout"})
|
||||
|
||||
logger.info(f"📸 [StoryView] Viewing @{ctx.username}'s story ({count} segments)...")
|
||||
|
||||
info = ctx.device.get_info()
|
||||
w = info.get("displayWidth", 1080)
|
||||
h = info.get("displayHeight", 2400)
|
||||
|
||||
for i in range(count):
|
||||
sleep(random.uniform(2.0, 5.0) * ctx.sleep_mod)
|
||||
if i < count - 1:
|
||||
# Atomic state validation before click
|
||||
xml_dump = ctx.device.dump_hierarchy()
|
||||
if not xml_dump:
|
||||
continue
|
||||
|
||||
# Query VLM to find the interactive area for the next segment
|
||||
intent = "tap right side of screen to view next story segment"
|
||||
node = ctx.telepathic.find_best_node(xml_dump, intent, device=ctx.device, track=False)
|
||||
|
||||
if node:
|
||||
logger.debug(
|
||||
f"📸 [StoryView] VLM selected node '{node.resource_id or node.content_desc or 'unknown'}' for next segment."
|
||||
)
|
||||
# If VLM selects a large container (e.g. the entire screen or story viewer),
|
||||
# we must tap its right side, not its exact center, to avoid pausing the story.
|
||||
if getattr(node, "area", 0) > (w * h * 0.4):
|
||||
target_x = node.x1 + int((node.x2 - node.x1) * 0.85)
|
||||
target_y = node.y1 + int((node.y2 - node.y1) * 0.25)
|
||||
logger.debug(
|
||||
f"📸 [StoryView] Adjusting click to top-right quadrant of large container: ({target_x}, {target_y})"
|
||||
)
|
||||
else:
|
||||
target_x = node.center_x
|
||||
target_y = node.center_y
|
||||
|
||||
humanized_click(ctx.device, target_x, target_y, sleep_mod=ctx.sleep_mod)
|
||||
else:
|
||||
logger.warning(
|
||||
"📸 [StoryView] VLM could not resolve next story segment. Falling back to geometric safety quadrant."
|
||||
)
|
||||
# Click top-right to avoid 'reply' input fields and most stickers
|
||||
humanized_click(ctx.device, int(w * 0.85), int(h * 0.25), sleep_mod=ctx.sleep_mod)
|
||||
|
||||
# Verify we didn't leave Instagram
|
||||
xml_dump_after = ctx.device.dump_hierarchy()
|
||||
if not xml_dump_after:
|
||||
continue
|
||||
packages = set(re.findall(r'package="([^"]+)"', xml_dump_after))
|
||||
app_id = getattr(ctx.device, "app_id", "com.instagram.android")
|
||||
if packages and app_id not in packages:
|
||||
logger.error(
|
||||
f"🚨 [StoryView] FOREIGN APP DETECTED! Packages: {packages}. "
|
||||
f"A link likely opened an external app. Aborting loop."
|
||||
)
|
||||
ctx.device.press("back")
|
||||
sleep(1.5)
|
||||
break
|
||||
|
||||
ctx.device.press("back")
|
||||
sleep(random.uniform(1.0, 2.0) * ctx.sleep_mod)
|
||||
|
||||
# Post-interaction verification: verify we successfully exited the story overlay
|
||||
for attempt in range(3):
|
||||
xml_dump = ctx.device.dump_hierarchy()
|
||||
if not xml_dump:
|
||||
break
|
||||
xml_lower = xml_dump.lower()
|
||||
if "com.instagram.android" not in xml_dump:
|
||||
break
|
||||
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepathic = ctx.cognitive_stack.get("telepathic") or TelepathicEngine.get_instance()
|
||||
if telepathic.classify_screen_content(xml_lower, "main_feed_presence") == "main_feed":
|
||||
# Successfully back to a main view
|
||||
break
|
||||
logger.warning(
|
||||
f"⚠️ [StoryView] Still trapped in story/overlay after back press (attempt {attempt+1}). Pressing back again."
|
||||
)
|
||||
ctx.device.press("back")
|
||||
sleep(1.5)
|
||||
|
||||
return BehaviorResult(executed=True, interactions=count, metadata={"stories_viewed": count})
|
||||
@@ -1,11 +1,15 @@
|
||||
import os
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
|
||||
from colorama import Fore, Style
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
BENCHMARKS_FILE = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), "benchmarks", "data", "llm_benchmarks.json")
|
||||
BENCHMARKS_FILE = os.path.join(
|
||||
os.path.dirname(os.path.dirname(os.path.dirname(__file__))), "benchmarks", "data", "llm_benchmarks.json"
|
||||
)
|
||||
|
||||
|
||||
def check_model_benchmarks(configs):
|
||||
"""
|
||||
@@ -27,17 +31,17 @@ def check_model_benchmarks(configs):
|
||||
def _eval_model(model_name: str, context: str):
|
||||
if not model_name:
|
||||
return
|
||||
|
||||
|
||||
if model_name not in benchmarks:
|
||||
logger.warning(
|
||||
f"⚠️ [Benchmark Guard] Model '{model_name}' (for {context}) is COMPLETELY UNTESTED "
|
||||
f"for Singularity V8. Expect severe hallucinations or crashed agents.",
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.RED}"}
|
||||
f"for the Agent. Expect severe hallucinations or crashed agents.",
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.RED}"},
|
||||
)
|
||||
return
|
||||
|
||||
scores = benchmarks[model_name]
|
||||
|
||||
|
||||
# Telepathic/Vision tasks require high structural strictness
|
||||
if context == "Vision/Telepathic":
|
||||
score = scores.get("telepathic_score", 0)
|
||||
@@ -48,29 +52,29 @@ def check_model_benchmarks(configs):
|
||||
logger.error(
|
||||
f"⛔ [Benchmark Guard] Model '{model_name}' (for {context}) achieved a CRITICAL FAILURE score "
|
||||
f"of {score}/100. Autonomous safety is compromised. DO NOT RUN UNATTENDED.",
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.RED}"}
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.RED}"},
|
||||
)
|
||||
elif score < 80:
|
||||
logger.warning(
|
||||
f"⚠️ [Benchmark Guard] Model '{model_name}' (for {context}) achieved a SUB-STANDARD score "
|
||||
f"of {score}/100. It may occasionally hallucinate UI elements or misinterpret semantics.",
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.YELLOW}"}
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.YELLOW}"},
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
f"✅ [Benchmark Guard] Model '{model_name}' (for {context}) passes safety benchmarks ({score}/100).",
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.GREEN}"}
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.GREEN}"},
|
||||
)
|
||||
|
||||
# Which models did the user configure?
|
||||
telepathic_model = getattr(configs.args, "ai_telepathic_model", None)
|
||||
text_model = getattr(configs.args, "ai_model", None)
|
||||
condenser_model = getattr(configs.args, "ai_condenser_model", None)
|
||||
|
||||
|
||||
_eval_model(telepathic_model, "Vision/Telepathic")
|
||||
|
||||
|
||||
if text_model and text_model != telepathic_model:
|
||||
_eval_model(text_model, "Dopamine/Resonance")
|
||||
|
||||
|
||||
if condenser_model and condenser_model != text_model and condenser_model != telepathic_model:
|
||||
_eval_model(condenser_model, "Context Condensation")
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,16 +1,17 @@
|
||||
import logging
|
||||
import json
|
||||
from io import BytesIO
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class VLMCompilerEngine:
|
||||
"""
|
||||
Project Singularity V7: The Self-Compiling Heuristics Engine
|
||||
The Self-Compiling Heuristics Engine
|
||||
This engine leverages a massive VLM to analyze failures in the Zero-Latency Engine.
|
||||
It takes a screenshot + XML dump, finds the missing intent, and generates a new,
|
||||
blazing-fast deterministic Regex/XPath rule to be cached and executed next time.
|
||||
"""
|
||||
|
||||
def __init__(self, device):
|
||||
self.device = device
|
||||
|
||||
@@ -19,17 +20,30 @@ class VLMCompilerEngine:
|
||||
Calls the VLM to visually find the intent in the screen, then cross-reference it
|
||||
with the provided XML to generate a deterministic extraction rule.
|
||||
"""
|
||||
logger.warning(f"🧠 [Compiler Engine] Deterministic heuristic failed for: '{intent_description}'. Synthesizing new rule...", extra={"color": "\x1b[1m\x1b[35m"})
|
||||
|
||||
# Sanitize intent to avoid confusing the LLM with python list syntax
|
||||
clean_intent = intent_description
|
||||
if "['" in clean_intent:
|
||||
clean_intent = clean_intent.replace("['", "").replace("']", "").replace("', '", " AND ")
|
||||
|
||||
logger.warning(
|
||||
f"🧠 [Compiler Engine] Deterministic heuristic failed for: '{clean_intent}'. Synthesizing new rule...",
|
||||
extra={"color": "\x1b[1m\x1b[35m"},
|
||||
)
|
||||
|
||||
args = getattr(self.device, "args", None)
|
||||
model = getattr(args, "ai_telepathic_model", "llama3.2:1b") if args else "llama3.2:1b"
|
||||
url = getattr(args, "ai_telepathic_url", "http://localhost:11434/api/generate") if args else "http://localhost:11434/api/generate"
|
||||
url = (
|
||||
getattr(args, "ai_telepathic_url", "http://localhost:11434/api/generate")
|
||||
if args
|
||||
else "http://localhost:11434/api/generate"
|
||||
)
|
||||
use_local = "11434" in url or "localhost" in url
|
||||
|
||||
simplified_xml = self._simplify_xml(context_xml)
|
||||
|
||||
# --- Model Trust Logging ---
|
||||
from GramAddict.core.benchmark_guard import BENCHMARKS_FILE
|
||||
|
||||
trust_log = f"Using {model}"
|
||||
try:
|
||||
if os.path.exists(BENCHMARKS_FILE):
|
||||
@@ -39,13 +53,19 @@ class VLMCompilerEngine:
|
||||
score = bench_data.get("telepathic_score", 0)
|
||||
passed = "PASS" if bench_data.get("passed_all", False) else "FAIL"
|
||||
unsuitable = bench_data.get("is_unsuitable", False)
|
||||
trust_level = "HIGH" if score >= 80 and not unsuitable else "MEDIUM" if score >= 50 and not unsuitable else "LOW/UNSAFE"
|
||||
trust_level = (
|
||||
"HIGH"
|
||||
if score >= 80 and not unsuitable
|
||||
else "MEDIUM"
|
||||
if score >= 50 and not unsuitable
|
||||
else "LOW/UNSAFE"
|
||||
)
|
||||
trust_log += f" [Benchmark: {score}/100 | {passed} | Trust: {trust_level}]"
|
||||
if unsuitable:
|
||||
logger.error(f"⛔ [Safety Alert] {model} is marked as UNSUITABLE for this task!")
|
||||
except Exception:
|
||||
pass
|
||||
logger.info(f"🧠 [Compiler] Intent: '{intent_description}' -> {trust_log}")
|
||||
logger.info(f"🧠 [Compiler] Intent: '{clean_intent}' -> {trust_log}")
|
||||
# ---------------------------
|
||||
|
||||
system_prompt = (
|
||||
@@ -53,37 +73,38 @@ class VLMCompilerEngine:
|
||||
"Rules:\n"
|
||||
"1. Output ONLY a raw JSON object.\n"
|
||||
"2. NO markdown, NO triple backticks.\n"
|
||||
"3. Format: {\"rule_type\": \"regex\", \"target_attribute\": \"resource-id\", \"pattern\": \".*regex.*\", \"confidence\": 0.95, \"reasoning\": \"string\"}"
|
||||
'3. Format: {"rule_type": "regex", "target_attribute": "resource-id", "pattern": ".*regex.*", "confidence": 0.95, "reasoning": "string"}'
|
||||
)
|
||||
|
||||
user_prompt = f"TARGET INTENT: {intent_description}\n\nUI XML:\n{simplified_xml[:2000]}"
|
||||
user_prompt = f"TARGET INTENT: {clean_intent}\n\nUI XML:\n{simplified_xml[:2000]}"
|
||||
|
||||
try:
|
||||
from GramAddict.core.llm_provider import query_telepathic_llm
|
||||
|
||||
res_text = query_telepathic_llm(
|
||||
model=model,
|
||||
url=url,
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=user_prompt,
|
||||
temperature=0.1,
|
||||
use_local_edge=use_local
|
||||
use_local_edge=use_local,
|
||||
)
|
||||
|
||||
|
||||
if not res_text:
|
||||
logger.error("Compiler LLM returned empty response.")
|
||||
return None
|
||||
|
||||
|
||||
if "```json" in res_text:
|
||||
res_text = res_text.split("```json")[1].split("```")[0].strip()
|
||||
elif res_text.startswith("```"):
|
||||
res_text = "\n".join(res_text.strip().split("\n")[1:-1])
|
||||
|
||||
|
||||
try:
|
||||
decision = json.loads(res_text)
|
||||
except json.JSONDecodeError:
|
||||
logger.error(f"Compiler LLM returned invalid JSON: {res_text[:100]}...")
|
||||
return None
|
||||
|
||||
|
||||
# If LLM returned a list, take the first item if it's a dict
|
||||
if isinstance(decision, list):
|
||||
if len(decision) > 0 and isinstance(decision[0], dict):
|
||||
@@ -91,26 +112,31 @@ class VLMCompilerEngine:
|
||||
else:
|
||||
logger.error(f"Compiler LLM returned unexpected list format: {decision}")
|
||||
return None
|
||||
|
||||
|
||||
if not isinstance(decision, dict):
|
||||
logger.error(f"Compiler LLM returned non-object response: {type(decision)}")
|
||||
return None
|
||||
|
||||
pattern = decision.get('pattern')
|
||||
|
||||
pattern = decision.get("pattern")
|
||||
if not pattern:
|
||||
logger.error("Compiler LLM returned empty rule pattern. Aborting heuristic generation.")
|
||||
return None
|
||||
|
||||
logger.info(f"✨ [Compiler] New Heuristic Synthesized! Rule: {decision.get('rule_type')} -> {pattern}", extra={"color": "\x1b[1m\x1b[32m"})
|
||||
|
||||
|
||||
logger.info(
|
||||
f"✨ [Compiler] New Heuristic Synthesized! Rule: {decision.get('rule_type')} -> {pattern}",
|
||||
extra={"color": "\x1b[1m\x1b[32m"},
|
||||
)
|
||||
|
||||
if decision.get("rule_type") == "xpath":
|
||||
logger.error("Compiler LLM returned 'xpath'. Rejecting rule because it causes xml.etree crashes. Will fallback/retry.")
|
||||
logger.error(
|
||||
"Compiler LLM returned 'xpath'. Rejecting rule because it causes xml.etree crashes. Will fallback/retry."
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
return {
|
||||
"rule_type": "regex",
|
||||
"target_attribute": decision.get("target_attribute", "text"),
|
||||
"pattern": pattern
|
||||
"pattern": pattern,
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
@@ -119,6 +145,7 @@ class VLMCompilerEngine:
|
||||
|
||||
def _simplify_xml(self, xml_tree: str) -> str:
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
nodes = []
|
||||
try:
|
||||
root = ET.fromstring(xml_tree)
|
||||
|
||||
@@ -17,8 +17,12 @@ class Config:
|
||||
self.args = kwargs
|
||||
self.module = True
|
||||
else:
|
||||
self.args = sys.argv
|
||||
self.args = list(sys.argv)
|
||||
self.module = False
|
||||
|
||||
if not self.module and "--config" not in self.args:
|
||||
if os.path.exists("config.yml"):
|
||||
self.args.extend(["--config", "config.yml"])
|
||||
self.config = None
|
||||
self.config_list = None
|
||||
self.actions = {}
|
||||
@@ -75,6 +79,9 @@ class Config:
|
||||
self.username = self.username[0]
|
||||
self.debug = self.config.get("debug", False)
|
||||
self.app_id = self.config.get("app_id", "com.instagram.android")
|
||||
|
||||
# Autonomous goals removed — the bot now derives tasks from mission + plugins
|
||||
# via GoalDecomposer. See GramAddict/core/goal_decomposer.py.
|
||||
else:
|
||||
if "--debug" in self.args:
|
||||
self.debug = True
|
||||
@@ -93,10 +100,8 @@ class Config:
|
||||
|
||||
# Configure ArgParse
|
||||
self.parser = configargparse.ArgumentParser(
|
||||
config_file_open_func=lambda filename: open(
|
||||
filename, "r+", encoding="utf-8"
|
||||
),
|
||||
description="GramAddict Instagram Bot - Singularity V7",
|
||||
config_file_open_func=lambda filename: open(filename, "r+", encoding="utf-8"),
|
||||
description="GramAddict Instagram Bot",
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--config",
|
||||
@@ -123,7 +128,7 @@ class Config:
|
||||
action="store_true",
|
||||
help="Enable Tesla E2E Vision 'Shadow Mode' Telemetry daemon.",
|
||||
)
|
||||
|
||||
|
||||
# Core Singularity Jobs
|
||||
self.parser.add_argument("--feed", help="Amount of feed posts to interact with", default=None)
|
||||
self.parser.add_argument("--explore", help="Amount of explore posts to interact with", default=None)
|
||||
@@ -133,16 +138,30 @@ class Config:
|
||||
self.parser.add_argument("--total-sessions", help="Total amount of sessions", default="-1")
|
||||
self.parser.add_argument("--working-hours", help="Working hours", default=None)
|
||||
self.parser.add_argument("--time-delta-session", help="Time delta between sessions", default=None)
|
||||
self.parser.add_argument(
|
||||
"--max-runtime-minutes", type=int, help="Maximum runtime in minutes before bot auto-exits", default=None
|
||||
)
|
||||
self.parser.add_argument("--restart-atx-agent", action="store_true", help="Restart atx agent")
|
||||
self.parser.add_argument("--allow-untested-ig-version", action="store_true", help="Allow untested IG version")
|
||||
self.parser.add_argument("--capture-e2e-dumps", action="store_true", help="Automatically navigate through the app and capture missing XML dumps for the test suite")
|
||||
self.parser.add_argument(
|
||||
"--blank-start",
|
||||
action="store_true",
|
||||
help="Wipe all learned navigation and telepathic memories on boot to start 100%% blank.",
|
||||
)
|
||||
|
||||
self.parser.add_argument(
|
||||
"--goal",
|
||||
type=str,
|
||||
help="High-level autonomous goal for the bot (Tesla-style). Overrides config.yml goals.",
|
||||
default=None,
|
||||
)
|
||||
|
||||
# Interaction settings
|
||||
self.parser.add_argument("--likes-count", help="Likes count", default="2-3")
|
||||
self.parser.add_argument("--likes-percentage", help="Likes percentage", default="100")
|
||||
self.parser.add_argument("--stories-count", help="Stories count", default="0")
|
||||
self.parser.add_argument("--stories-percentage", help="Stories percentage", default="0")
|
||||
|
||||
|
||||
# Total Limits (Legacy names preserved for SessionState compatibility)
|
||||
self.parser.add_argument("--total-likes-limit", help="Total likes limit", default="300")
|
||||
self.parser.add_argument("--total-follows-limit", help="Total follows limit", default="50")
|
||||
@@ -150,45 +169,137 @@ class Config:
|
||||
self.parser.add_argument("--total-comments-limit", help="Total comments limit", default="10")
|
||||
self.parser.add_argument("--total-pm-limit", help="Total pm limit", default="10")
|
||||
self.parser.add_argument("--total-watches-limit", help="Total watches limit", default="50")
|
||||
self.parser.add_argument("--total-successful-interactions-limit", help="Total successful interactions limit", default="100")
|
||||
self.parser.add_argument(
|
||||
"--total-successful-interactions-limit", help="Total successful interactions limit", default="100"
|
||||
)
|
||||
self.parser.add_argument("--total-interactions-limit", help="Total interactions limit", default="1000")
|
||||
self.parser.add_argument("--total-scraped-limit", help="Total scraped limit", default="200")
|
||||
self.parser.add_argument("--total-crashes-limit", help="Total crashes limit", default="5")
|
||||
self.parser.add_argument("--speed-multiplier", help="Speed multiplier", default="1.0")
|
||||
|
||||
# AI Model Configuration (centralized — no hardcoded model names anywhere)
|
||||
self.parser.add_argument("--ai-model", "--ai-text-model", help="Primary LLM model (OpenRouter or Ollama)", default="llama3.2:1b")
|
||||
self.parser.add_argument("--ai-model-url", "--ai-text-url", help="Primary LLM endpoint URL", default="http://localhost:11434/api/generate")
|
||||
self.parser.add_argument("--ai-telepathic-model", help="Text-based model for Telepathic Engine Fallbacks", default="llama3.2:1b")
|
||||
self.parser.add_argument("--ai-telepathic-url", help="Telepathic model endpoint URL", default="http://localhost:11434/api/generate")
|
||||
self.parser.add_argument("--ai-fallback-model", "--ai-text-fallback-model", help="Fallback model when primary fails", default="llama3.2:1b")
|
||||
self.parser.add_argument("--ai-fallback-url", "--ai-text-fallback-url", help="Fallback model endpoint URL", default="http://localhost:11434/api/generate")
|
||||
self.parser.add_argument("--ai-embedding-model", help="Embedding model for vector operations", default="nomic-embed-text")
|
||||
self.parser.add_argument("--ai-embedding-url", help="Embedding endpoint URL", default="http://localhost:11434/api/embeddings")
|
||||
|
||||
self.parser.add_argument(
|
||||
"--ai-model", "--ai-text-model", help="Primary LLM model (OpenRouter or Ollama)", default="qwen3.5:latest"
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-model-url",
|
||||
"--ai-text-url",
|
||||
help="Primary LLM endpoint URL",
|
||||
default="http://localhost:11434/api/generate",
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-telepathic-model", help="Text-based model for Telepathic Engine Fallbacks", default="qwen3.5:latest"
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-telepathic-url", help="Telepathic model endpoint URL", default="http://localhost:11434/api/generate"
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-fallback-model",
|
||||
"--ai-text-fallback-model",
|
||||
help="Fallback model when primary fails",
|
||||
default="qwen3.5:latest",
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-fallback-url",
|
||||
"--ai-text-fallback-url",
|
||||
help="Fallback model endpoint URL",
|
||||
default="http://localhost:11434/api/generate",
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-embedding-model", help="Embedding model for vector operations", default="nomic-embed-text"
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-embedding-url", help="Embedding endpoint URL", default="http://localhost:11434/api/embeddings"
|
||||
)
|
||||
|
||||
# Persona & Resonance (drives ALL content evaluation and interaction decisions)
|
||||
self.parser.add_argument("--persona-interests", help="Comma-separated niche interests for content matching", default="")
|
||||
self.parser.add_argument("--ai-target-audience", help="Target audience used interchangeably with persona interests", default="")
|
||||
self.parser.add_argument("--interact-percentage", help="Overall interaction probability percentage", default="80")
|
||||
self.parser.add_argument(
|
||||
"--persona-interests", help="Comma-separated niche interests for content matching", default=""
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-target-audience", help="Target audience used interchangeably with persona interests", default=""
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--target-audience", help="Target audience used interchangeably with persona interests", default=""
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--interact-percentage", help="Overall interaction probability percentage", default="80"
|
||||
)
|
||||
self.parser.add_argument("--comment-percentage", help="Comment probability percentage", default="0")
|
||||
self.parser.add_argument("--follow-percentage", help="Follow probability percentage", default="0")
|
||||
self.parser.add_argument("--dry-run-comments", action="store_true", help="Generate AI comments but do not actually post them (debug/logging only)")
|
||||
self.parser.add_argument(
|
||||
"--dry-run-comments",
|
||||
action="store_true",
|
||||
help="Generate AI comments but do not actually post them (debug/logging only)",
|
||||
)
|
||||
self.parser.add_argument("--search", help="Comma-separated keywords to search for", default="")
|
||||
self.parser.add_argument("--scrape-profiles", action="store_true", help="Extract and store profile metadata in CRM")
|
||||
|
||||
self.parser.add_argument(
|
||||
"--scrape-profiles", action="store_true", help="Extract and store profile metadata in CRM"
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--profile-learning-percentage", help="Percentage of profiles to deeply scan before engaging", default="0"
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--visual-vibe-check-percentage",
|
||||
help="Percentage of profiles to visually evaluate via screenshot before engaging",
|
||||
default="0",
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ignore-close-friends",
|
||||
action="store_true",
|
||||
help="Completely ignore posts, stories, and profiles of Close Friends (Enge Freunde)",
|
||||
)
|
||||
|
||||
# Biomechanical Physics
|
||||
self.parser.add_argument(
|
||||
"--handedness",
|
||||
help="Dominant hand: 'right' or 'left'. Affects thumb arc direction and tap bias.",
|
||||
default="right",
|
||||
)
|
||||
|
||||
# Phase 10: RAG Comment Learning & Extractor Settings
|
||||
self.parser.add_argument("--ai-condenser-model", help="LLM used for condensing text/comments", default="llama3.2:1b")
|
||||
self.parser.add_argument("--ai-condenser-url", help="URL for the condenser model", default="http://localhost:11434/api/generate")
|
||||
self.parser.add_argument("--ai-learn-comments", action="store_true", help="Extract and learn from comment sections")
|
||||
self.parser.add_argument(
|
||||
"--ai-condenser-model", help="LLM used for condensing text/comments", default="qwen3.5:latest"
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-condenser-url", help="URL for the condenser model", default="http://localhost:11434/api/generate"
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-learn-comments", action="store_true", help="Extract and learn from comment sections"
|
||||
)
|
||||
self.parser.add_argument("--ai-learn-niche-posts", action="store_true", help="Learn from niche posts")
|
||||
self.parser.add_argument("--ai-learn-own-profile", action="store_true", help="Learn from your own profile interactions")
|
||||
self.parser.add_argument("--ai-learn-only", action="store_true", help="Run the bot in a pure read-only learning mode")
|
||||
self.parser.add_argument("--ai-vibe", help="The specific vibe to extract from comments (e.g., friendly, controversial)", default="")
|
||||
self.parser.add_argument("--ai-blacklist-topics", help="Comma-separated topics heavily penalized or skipped", default="")
|
||||
self.parser.add_argument("--ai-quality-filter", action="store_true", help="Use AI to strictly filter the quality of posts and comments")
|
||||
self.parser.add_argument("--smart-unfollow", action="store_true", help="Enable agentic decision making for clearing the following list")
|
||||
self.parser.add_argument("--ai-vision-navigation", action="store_true", help="Capture and send base64 UI screenshots to the LLM for structural element finding")
|
||||
self.parser.add_argument("--ai-vision-context", action="store_true", help="Capture and send base64 post/DM screenshots to the LLM for contextual semantic generation")
|
||||
self.parser.add_argument(
|
||||
"--ai-learn-own-profile", action="store_true", help="Learn from your own profile interactions"
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-learn-only", action="store_true", help="Run the bot in a pure read-only learning mode"
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-vibe", help="The specific vibe to extract from comments (e.g., friendly, controversial)", default=""
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-blacklist-topics", help="Comma-separated topics heavily penalized or skipped", default=""
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-quality-filter",
|
||||
action="store_true",
|
||||
help="Use AI to strictly filter the quality of posts and comments",
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--smart-unfollow",
|
||||
action="store_true",
|
||||
help="Enable agentic decision making for clearing the following list",
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-vision-navigation",
|
||||
action="store_true",
|
||||
help="Capture and send base64 UI screenshots to the LLM for structural element finding",
|
||||
)
|
||||
self.parser.add_argument(
|
||||
"--ai-vision-context",
|
||||
action="store_true",
|
||||
help="Capture and send base64 post/DM screenshots to the LLM for contextual semantic generation",
|
||||
)
|
||||
|
||||
# on first run, we must wait to proceed with loading
|
||||
if not self.first_run:
|
||||
@@ -208,18 +319,38 @@ class Config:
|
||||
logger.debug(f"Arguments used: {' '.join(sys.argv[1:])}")
|
||||
if self.config:
|
||||
logger.debug(f"Config used: {self.config}")
|
||||
if len(sys.argv) <= 1:
|
||||
if len(sys.argv) <= 1 and not self.config:
|
||||
self.parser.print_help()
|
||||
exit(0)
|
||||
if self.config:
|
||||
cleaned_config = {}
|
||||
for k, v in self.config.items():
|
||||
# Replace dictionaries with a placeholder to avoid argparse crashing
|
||||
# We'll resolve the actual values later in specialize()
|
||||
|
||||
def flatten_dict(d, parent_key="", sep="_"):
|
||||
items = []
|
||||
for k, v in d.items():
|
||||
# Special handling for 'plugins' key: we want 'like: count' to become 'like_count'
|
||||
if k == "plugins" and not parent_key:
|
||||
if isinstance(v, dict):
|
||||
for pk, pv in v.items():
|
||||
items.extend(flatten_dict(pv, pk, sep=sep).items())
|
||||
continue
|
||||
|
||||
if isinstance(v, dict) and k not in ["username", "passwords"]:
|
||||
# If we are inside a plugin, continue prefixing
|
||||
next_prefix = f"{parent_key}{sep}{k}" if parent_key else ""
|
||||
items.extend(flatten_dict(v, next_prefix, sep=sep).items())
|
||||
else:
|
||||
full_key = f"{parent_key}{sep}{k}" if parent_key else k
|
||||
items.append((full_key, v))
|
||||
return dict(items)
|
||||
|
||||
flat_config = flatten_dict(self.config)
|
||||
|
||||
for k, v in flat_config.items():
|
||||
val = v
|
||||
if isinstance(v, dict):
|
||||
val = "SPECIALIZED"
|
||||
|
||||
|
||||
cleaned_config[k.replace("-", "_")] = val
|
||||
self.parser.set_defaults(**cleaned_config)
|
||||
|
||||
@@ -232,12 +363,14 @@ class Config:
|
||||
self.args, self.unknown_args = self.parser.parse_known_args(args=arg_str)
|
||||
else:
|
||||
self.args, self.unknown_args = self.parser.parse_known_args()
|
||||
|
||||
|
||||
self.device_id = self.args.device
|
||||
|
||||
# Map actions for Singularity V7
|
||||
if getattr(self.args, "feed", None): self.enabled.append("feed")
|
||||
if getattr(self.args, "explore", None): self.enabled.append("explore")
|
||||
|
||||
# Map actions
|
||||
if getattr(self.args, "feed", None):
|
||||
self.enabled.append("feed")
|
||||
if getattr(self.args, "explore", None):
|
||||
self.enabled.append("explore")
|
||||
|
||||
def specialize(self, username):
|
||||
if self.config is None:
|
||||
@@ -258,6 +391,32 @@ class Config:
|
||||
# Handle the case where username itself is a list - we specialize it to the current target
|
||||
self.args.username = [username] if isinstance(self.args.username, list) else username
|
||||
|
||||
def get_plugin_config(self, plugin_name: str) -> dict:
|
||||
"""
|
||||
Retrieves configuration for a specific plugin.
|
||||
First checks the 'plugins' dict. If not found, falls back to flat config values
|
||||
using the plugin_name as a prefix for backward compatibility.
|
||||
"""
|
||||
if self.config and "plugins" in self.config:
|
||||
plugin_dict = self.config["plugins"].get(plugin_name, {})
|
||||
if plugin_dict:
|
||||
return plugin_dict
|
||||
|
||||
# Backward compatibility / flat config fallback
|
||||
# e.g., for "follow" plugin, check if "follow_percentage" exists
|
||||
fallback = {}
|
||||
if hasattr(self.args, f"{plugin_name}_percentage"):
|
||||
fallback["percentage"] = getattr(self.args, f"{plugin_name}_percentage")
|
||||
|
||||
# specific hardcoded fallbacks
|
||||
if plugin_name == "close_friends_guard" and hasattr(self.args, "ignore_close_friends"):
|
||||
fallback["enabled"] = getattr(self.args, "ignore_close_friends")
|
||||
|
||||
if plugin_name == "comment_interaction" and hasattr(self.args, "dry_run_comments"):
|
||||
fallback["dry_run"] = getattr(self.args, "dry_run_comments")
|
||||
|
||||
return fallback
|
||||
|
||||
|
||||
def get_time_last_save(file_path) -> str:
|
||||
try:
|
||||
|
||||
@@ -1,179 +1,238 @@
|
||||
import logging
|
||||
import random
|
||||
import os
|
||||
import math
|
||||
import uuid
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from GramAddict.core.physics.biomechanics import BezierGesture, PhysicsBody
|
||||
from GramAddict.core.physics.sendevent_injector import SendEventInjector
|
||||
from GramAddict.core.qdrant_memory import QdrantBase
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class DarwinEngine(QdrantBase):
|
||||
"""
|
||||
Project Singularity: Continuous Bayesian Evolutionary Engine V3 (Proof of Resonance).
|
||||
Determines mathematically how to act on a per-post basis, generating custom
|
||||
Dwell Times and nonlinear scroll sequences to maximize the RL Reward Matrix.
|
||||
"""
|
||||
|
||||
def __init__(self, username: str, config_path: str = "config.yml"):
|
||||
self.username = username
|
||||
self.config_path = config_path
|
||||
super().__init__(collection_name="bot_darwin_mdp_resonance", vector_size=5) # 5 corresponds to behavior_bounds length
|
||||
|
||||
super().__init__(
|
||||
collection_name="bot_darwin_mdp_resonance", vector_size=5
|
||||
) # 5 corresponds to behavior_bounds length
|
||||
|
||||
# We replace naive percentages with Markovian Dwell Behaviors
|
||||
self.behavior_bounds = {
|
||||
"initial_dwell_sec": (1.0, 15.0, 2.0),
|
||||
"scroll_velocity": (0.1, 2.0, 0.3), # 1.0 is normal
|
||||
"scroll_velocity": (0.1, 2.0, 0.3), # 1.0 is normal
|
||||
"back_swipe_prob": (0.0, 0.4, 0.1),
|
||||
"profile_visit_prob": (0.0, 0.8, 0.2),
|
||||
"comment_read_dwell": (0.0, 20.0, 4.0)
|
||||
"comment_read_dwell": (0.0, 20.0, 4.0),
|
||||
}
|
||||
self.current_behavior = {}
|
||||
|
||||
def synthesize_interaction_profile(self, target_resonance: float, text_length: int = 0) -> dict:
|
||||
"""
|
||||
Given an AI aesthetic resonance score (0.0 to 1.0) and caption length,
|
||||
Given an AI aesthetic resonance score (0.0 to 1.0) and caption length,
|
||||
this generates a deterministic topological interaction behavior.
|
||||
"""
|
||||
history = self._get_historical_landscape()
|
||||
epsilon = 0.15 # 15% pure exploration
|
||||
|
||||
epsilon = 0.15 # 15% pure exploration
|
||||
|
||||
if not history or random.random() < epsilon:
|
||||
logger.info("🧬 [Darwin Engine] EXPLORE: Generating chaotic non-linear behavioral vector.")
|
||||
center = {k: (v[0]+v[1])/2 for k, v in self.behavior_bounds.items()}
|
||||
center = {k: (v[0] + v[1]) / 2 for k, v in self.behavior_bounds.items()}
|
||||
self.current_behavior = self._mutate(center)
|
||||
else:
|
||||
# Exploitation: Nearest neighbor matching the resonance profile closely
|
||||
best_node = max(history, key=lambda x: x[1]) # x[1] is the Reward
|
||||
best_node = max(history, key=lambda x: x[1]) # x[1] is the Reward
|
||||
best_params = best_node[0]
|
||||
logger.info(f"🧬 [Darwin Engine] EXPLOIT: Adapting proven behavioral vector from highest Peak Reward ({best_node[1]:.2f}).")
|
||||
logger.info(
|
||||
f"🧬 [Darwin Engine] EXPLOIT: Adapting proven behavioral vector from highest Peak Reward ({best_node[1]:.2f})."
|
||||
)
|
||||
self.current_behavior = self._mutate(best_params)
|
||||
|
||||
|
||||
# Modulate behavior directly by resonance
|
||||
# E.g., if resonance is 0.9 (amazing post), read comments longer!
|
||||
self.current_behavior["initial_dwell_sec"] *= max(0.5, target_resonance * 1.5)
|
||||
self.current_behavior["profile_visit_prob"] *= max(0.2, target_resonance * 2.0)
|
||||
|
||||
|
||||
# ── Generative Dwell-Time ──
|
||||
# Humans take longer to finish "reading" long captions.
|
||||
# Average reading speed is ~15-20 chars per second.
|
||||
if text_length > 20:
|
||||
reading_latency = min(15.0, text_length / 25.0) # Cap extra reading time at 15s
|
||||
logger.debug(f"🧬 [Darwin Engine] Generative Dwell spike: +{reading_latency:.1f}s (Caption: {text_length} chars)")
|
||||
reading_latency = min(15.0, text_length / 25.0) # Cap extra reading time at 15s
|
||||
logger.debug(
|
||||
f"🧬 [Darwin Engine] Generative Dwell spike: +{reading_latency:.1f}s (Caption: {text_length} chars)"
|
||||
)
|
||||
self.current_behavior["initial_dwell_sec"] += reading_latency
|
||||
|
||||
# Clip bounds
|
||||
for k, (b_min, b_max, _) in self.behavior_bounds.items():
|
||||
self.current_behavior[k] = max(b_min, min(b_max, self.current_behavior[k]))
|
||||
|
||||
|
||||
return self.current_behavior
|
||||
|
||||
def execute_proof_of_resonance(self, device, resonance: float, text_length: int = 0, nav_graph=None, zero_engine=None, configs=None, resonance_oracle=None, username=None):
|
||||
def execute_proof_of_resonance(
|
||||
self,
|
||||
device,
|
||||
resonance: float,
|
||||
text_length: int = 0,
|
||||
nav_graph=None,
|
||||
configs=None,
|
||||
resonance_oracle=None,
|
||||
username=None,
|
||||
context_xml: str = "",
|
||||
):
|
||||
"""
|
||||
Translates the mathematical interaction profile directly into device actions
|
||||
Translates the mathematical interaction profile directly into device actions
|
||||
to prove engagement to the platform's anti-bot heuristic algorithm.
|
||||
"""
|
||||
profile = self.synthesize_interaction_profile(resonance, text_length=text_length)
|
||||
|
||||
|
||||
logger.info("🧬 [Darwin MDP] Executing Proof of Resonance Sequence...")
|
||||
|
||||
|
||||
# Pre-compute screen dimensions for all sub-phases
|
||||
info = device.get_info()
|
||||
h = info.get("displayHeight", 2400)
|
||||
w = info.get("displayWidth", 1080)
|
||||
|
||||
# 1. Initial Dwell
|
||||
dwell = profile["initial_dwell_sec"]
|
||||
logger.debug(f" -> Dwelling for {dwell:.1f}s")
|
||||
time.sleep(dwell)
|
||||
|
||||
|
||||
# 2. Non-linear cognitive latency (Micro-Jitters)
|
||||
if profile["scroll_velocity"] != 1.0:
|
||||
logger.debug(f" -> Simulating cognitive read latency (Micro-Jitters, Velocity: {profile['scroll_velocity']:.2f})")
|
||||
info = device.get_info()
|
||||
h = info.get("displayHeight", 2400)
|
||||
w = info.get("displayWidth", 1080)
|
||||
logger.debug(
|
||||
f" -> Simulating cognitive read latency (Micro-Jitters, Velocity: {profile['scroll_velocity']:.2f})"
|
||||
)
|
||||
body = PhysicsBody.get_session_instance(device)
|
||||
injector = SendEventInjector.get_instance(device)
|
||||
|
||||
# Thumb starts on the right side of the screen to avoid clicking polls/tags in the center
|
||||
cx = int(w * 0.8) + device.cm_to_pixels(random.uniform(-0.3, 0.3))
|
||||
cy = h // 2
|
||||
|
||||
|
||||
# Keep distance microscopic (0.1 to 0.3 cm) so we DO NOT lose visual alignment
|
||||
distance = device.cm_to_pixels(random.uniform(0.1, 0.3))
|
||||
duration = max(0.5, 1.0 / max(0.1, profile["scroll_velocity"]))
|
||||
start_y = int(cy + distance / 2)
|
||||
end_y = int(cy - distance / 2)
|
||||
|
||||
# Add some x-axis noise for nonlinear human realism (~0.1 cm)
|
||||
noise_x = device.cm_to_pixels(random.uniform(-0.1, 0.1))
|
||||
|
||||
device.deviceV2.swipe(cx, start_y, cx + noise_x, end_y, duration=duration)
|
||||
|
||||
|
||||
# Use Bézier curve for the jitter
|
||||
points = BezierGesture.scroll_curve((cx, start_y), (cx, end_y), body, n_points=6)
|
||||
timing = BezierGesture.compute_sigmoid_timing(len(points), duration * 1000)
|
||||
|
||||
injector.inject_gesture(points, timing, touch_major=body.get_touch_major())
|
||||
|
||||
# 3. Micro Back-swipe (The Human Wobble)
|
||||
if random.random() < profile["back_swipe_prob"]:
|
||||
logger.debug(" -> Executing cognitive wobble (Trace swipe)")
|
||||
# small rapid corrective swipe (approx 0.1-0.2 cm downward slip)
|
||||
slip_distance = device.cm_to_pixels(random.uniform(0.1, 0.2))
|
||||
noise_x = device.cm_to_pixels(random.uniform(-0.1, 0.1))
|
||||
# small rapid corrective swipe (approx 0.4-0.8 cm downward slip to exceed Touch Slop)
|
||||
slip_distance = device.cm_to_pixels(random.uniform(0.4, 0.8))
|
||||
noise_x = device.cm_to_pixels(random.uniform(-0.2, 0.2))
|
||||
cx = w // 2 + device.cm_to_pixels(random.uniform(-0.5, 0.5))
|
||||
cy = h // 2
|
||||
|
||||
device.deviceV2.swipe(cx, cy, cx + noise_x, cy + slip_distance, duration=random.uniform(0.2, 0.5))
|
||||
|
||||
dur_ms = int(random.uniform(200, 500))
|
||||
|
||||
# Use physics-based injector instead of algorithmic 'input swipe'
|
||||
body = PhysicsBody.get_session_instance(device)
|
||||
injector = SendEventInjector.get_instance(device)
|
||||
start_pt = (int(cx), int(cy))
|
||||
end_pt = (int(cx + noise_x), int(cy + slip_distance))
|
||||
|
||||
points = BezierGesture.scroll_curve(start_pt, end_pt, body, n_points=5)
|
||||
timing = BezierGesture.compute_sigmoid_timing(len(points), dur_ms)
|
||||
injector.inject_gesture(points, timing, touch_major=body.get_touch_major())
|
||||
|
||||
time.sleep(random.uniform(0.5, 1.2))
|
||||
|
||||
|
||||
# 4. Comment depth simulation (probabilistic & resonance-correlated)
|
||||
if profile["comment_read_dwell"] > 1.0 and resonance > 0.4 and random.random() < 0.3:
|
||||
if nav_graph and zero_engine:
|
||||
logger.debug(f" -> Opening comments section for {profile['comment_read_dwell']:.1f}s depth simulation")
|
||||
|
||||
# Capture image context of post BEFORE opening comment sheet
|
||||
b64_img_payload = None
|
||||
if configs and getattr(configs.args, "ai_vision_context", False):
|
||||
try:
|
||||
import base64
|
||||
raw = device.screenshot()
|
||||
if raw:
|
||||
b64_img_payload = [base64.b64encode(raw).decode('utf-8')]
|
||||
logger.debug("👁️ [Vision Context] Captured post screenshot for True Vision semantic analysis.")
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ [Vision Context] Failed to capture screenshot: {e}")
|
||||
|
||||
success = nav_graph._execute_transition("tap_comment_button")
|
||||
if success:
|
||||
# ---- Phase 10: RAG Comment Extraction ----
|
||||
if configs and resonance_oracle and getattr(configs.args, "ai_learn_comments", False):
|
||||
# Limit scraping to 15% to avoid mechanical persistence
|
||||
if random.random() < 0.05:
|
||||
logger.debug(" -> Dumping UI hierarchy for Comment Extraction...")
|
||||
try:
|
||||
xml_data = device.dump_hierarchy()
|
||||
t0 = time.time()
|
||||
resonance_oracle.extract_and_learn_comments(xml_data, configs, author=username or "unknown", images_b64=b64_img_payload)
|
||||
t1 = time.time()
|
||||
remaining_sleep = profile["comment_read_dwell"] - (t1 - t0)
|
||||
if remaining_sleep > 0:
|
||||
time.sleep(remaining_sleep)
|
||||
except Exception as e:
|
||||
logger.error(f" -> Comment extraction failed: {e}")
|
||||
time.sleep(profile["comment_read_dwell"])
|
||||
else:
|
||||
logger.debug(" -> Skipping RAG Extraction (Probabilistic Evasion)")
|
||||
time.sleep(profile["comment_read_dwell"])
|
||||
else:
|
||||
time.sleep(profile["comment_read_dwell"])
|
||||
# ------------------------------------------
|
||||
|
||||
logger.debug(" -> Closing comments section")
|
||||
device.deviceV2.press("back")
|
||||
time.sleep(1.0)
|
||||
# Instead of relying on a fragile bottom_sheet_container ID,
|
||||
# we verify if the feed is visible. If not, the comment sheet is still open (or keyboard).
|
||||
ui_dump = device.dump_hierarchy()
|
||||
if 'resource-id="com.instagram.android:id/row_feed"' not in ui_dump and 'resource-id="com.instagram.android:id/button_like"' not in ui_dump:
|
||||
logger.debug(" -> Not back on Home feed, pressing back again to close comment sheet/keyboard")
|
||||
device.deviceV2.press("back")
|
||||
time.sleep(1.0)
|
||||
else:
|
||||
logger.debug(f" -> Could not find comment button, falling back to dwell simulation for {profile['comment_read_dwell']:.1f}s")
|
||||
time.sleep(profile["comment_read_dwell"])
|
||||
else:
|
||||
logger.debug(f" -> Simulating comment section processing for {profile['comment_read_dwell']:.1f}s")
|
||||
time.sleep(profile["comment_read_dwell"])
|
||||
|
||||
if nav_graph:
|
||||
if not self._has_comments(context_xml):
|
||||
logger.debug(" -> 🚫 [Darwin Engine] Skipping comment depth simulation (Post has 0 comments).")
|
||||
else:
|
||||
logger.debug(
|
||||
f" -> Opening comments section for {profile['comment_read_dwell']:.1f}s depth simulation"
|
||||
)
|
||||
|
||||
# Capture image context of post BEFORE opening comment sheet
|
||||
b64_img_payload = None
|
||||
if configs and getattr(configs.args, "ai_vision_context", False):
|
||||
try:
|
||||
import base64
|
||||
|
||||
raw = device.screenshot()
|
||||
if raw:
|
||||
import io
|
||||
|
||||
buf = io.BytesIO()
|
||||
raw.save(buf, format="JPEG")
|
||||
b64_img_payload = [base64.b64encode(buf.getvalue()).decode("utf-8")]
|
||||
logger.debug(
|
||||
"👁️ [Vision Context] Captured post screenshot for True Vision semantic analysis."
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ [Vision Context] Failed to capture screenshot: {e}")
|
||||
|
||||
success = nav_graph.do("tap comment button")
|
||||
if success:
|
||||
# ---- Phase 10: RAG Comment Extraction ----
|
||||
if configs and resonance_oracle and getattr(configs.args, "ai_learn_comments", False):
|
||||
# Limit scraping to 15% to avoid mechanical persistence
|
||||
if random.random() < 0.05:
|
||||
logger.debug(" -> Dumping UI hierarchy for Comment Extraction...")
|
||||
try:
|
||||
xml_data = device.dump_hierarchy()
|
||||
t0 = time.time()
|
||||
resonance_oracle.extract_and_learn_comments(
|
||||
xml_data, configs, author=username or "unknown", images_b64=b64_img_payload
|
||||
)
|
||||
t1 = time.time()
|
||||
remaining_sleep = profile["comment_read_dwell"] - (t1 - t0)
|
||||
if remaining_sleep > 0:
|
||||
time.sleep(remaining_sleep)
|
||||
except Exception as e:
|
||||
logger.error(f" -> Comment extraction failed: {e}")
|
||||
time.sleep(profile["comment_read_dwell"])
|
||||
else:
|
||||
logger.debug(" -> Skipping RAG Extraction (Probabilistic Evasion)")
|
||||
time.sleep(profile["comment_read_dwell"])
|
||||
else:
|
||||
time.sleep(profile["comment_read_dwell"])
|
||||
# ------------------------------------------
|
||||
|
||||
logger.debug(" -> Closing comments section")
|
||||
device.press("back")
|
||||
time.sleep(1.0)
|
||||
# Instead of relying on a fragile bottom_sheet_container ID,
|
||||
# we verify if the feed is visible. If not, the comment sheet is still open (or keyboard).
|
||||
ui_dump = device.dump_hierarchy()
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepath = TelepathicEngine.get_instance()
|
||||
if not telepath.find_best_node(
|
||||
ui_dump, "post like button heart", min_confidence=0.4, device=device
|
||||
):
|
||||
logger.debug(" -> Not back on Home feed, pressing back again to close comment sheet/keyboard")
|
||||
device.press("back")
|
||||
time.sleep(1.0)
|
||||
else:
|
||||
logger.debug(
|
||||
f" -> Could not find comment button, falling back to dwell simulation for {profile['comment_read_dwell']:.1f}s"
|
||||
)
|
||||
time.sleep(profile["comment_read_dwell"])
|
||||
else:
|
||||
logger.debug(f" -> Simulating comment section processing for {profile['comment_read_dwell']:.1f}s")
|
||||
time.sleep(profile["comment_read_dwell"])
|
||||
|
||||
logger.info("🧬 [Darwin MDP] Interaction sequence completed safely.")
|
||||
return profile
|
||||
|
||||
@@ -181,32 +240,47 @@ class DarwinEngine(QdrantBase):
|
||||
"""
|
||||
Simulates a thumb resting or slightly shifting on the glass.
|
||||
Essential for breaking the 'robotically still' dwell periods.
|
||||
Uses PhysicsBody for handedness-aware direction and fatigue-scaled amplitude.
|
||||
"""
|
||||
if random.random() < 0.2: # 20% chance for a wobble during dwell
|
||||
if random.random() < 0.2: # 20% chance for a wobble during dwell
|
||||
logger.debug("🧬 [Ghost Protocol] Micro-Wobble triggered.")
|
||||
body = PhysicsBody.get_session_instance(device)
|
||||
injector = SendEventInjector.get_instance(device)
|
||||
|
||||
info = device.get_info()
|
||||
w = info.get("displayWidth", 1080)
|
||||
info.get("displayWidth", 1080)
|
||||
h = info.get("displayHeight", 2400)
|
||||
cx = int(w * 0.8) + device.cm_to_pixels(random.uniform(-0.3, 0.3))
|
||||
|
||||
# Start position from body (session-aware)
|
||||
cx, cy = body.get_scroll_start()
|
||||
# Override Y to center for wobble
|
||||
cy = h // 2
|
||||
|
||||
# Keep the shift very small (~0.05 to 0.15 cm) so it doesn't actually scroll the feed up/down noticeably
|
||||
y_shift = device.cm_to_pixels(random.uniform(0.05, 0.15)) * random.choice([1, -1])
|
||||
x_shift = device.cm_to_pixels(random.uniform(-0.05, 0.05))
|
||||
|
||||
# Single slow slip
|
||||
if hasattr(device, "human_swipe"):
|
||||
device.human_swipe(cx, cy, cx + x_shift, cy + y_shift, duration=random.uniform(0.1, 0.2))
|
||||
|
||||
# Fatigue scales wobble amplitude (tired = more sloppy)
|
||||
amplitude = 1.0 + body.fatigue * 0.5
|
||||
|
||||
# Keep the shift small but above Android's touch slop threshold (~8dp)
|
||||
y_shift = device.cm_to_pixels(random.uniform(0.3, 0.6) * amplitude) * random.choice([1, -1])
|
||||
x_shift = device.cm_to_pixels(random.uniform(-0.2, 0.2) * amplitude)
|
||||
|
||||
# Handedness bias: right-handers wobble right-down, left-handers left-down
|
||||
if body.handedness == "right":
|
||||
x_shift += device.cm_to_pixels(random.uniform(0, 0.1))
|
||||
else:
|
||||
device.deviceV2.swipe(cx, cy, cx + x_shift, cy + y_shift, duration=random.uniform(0.1, 0.2))
|
||||
x_shift -= device.cm_to_pixels(random.uniform(0, 0.1))
|
||||
|
||||
end_x = int(cx + x_shift)
|
||||
end_y = int(cy + y_shift)
|
||||
|
||||
points = BezierGesture.scroll_curve((cx, cy), (end_x, end_y), body, n_points=5)
|
||||
duration_ms = random.uniform(150, 300)
|
||||
timing = BezierGesture.compute_sigmoid_timing(len(points), duration_ms)
|
||||
|
||||
injector.inject_gesture(points, timing, touch_major=body.get_touch_major())
|
||||
|
||||
def _get_historical_landscape(self):
|
||||
try:
|
||||
records = self.client.scroll(
|
||||
collection_name=self.collection_name,
|
||||
limit=1000,
|
||||
with_payload=True
|
||||
)[0]
|
||||
records = self.client.scroll(collection_name=self.collection_name, limit=1000, with_payload=True)[0]
|
||||
return [(r.payload.get("params", {}), r.payload.get("reward", 0.0)) for r in records]
|
||||
except Exception:
|
||||
return []
|
||||
@@ -221,25 +295,28 @@ class DarwinEngine(QdrantBase):
|
||||
|
||||
def select_arm_and_apply(self, args):
|
||||
"""
|
||||
Multi-Armed Bandit (MAB) logic to select the most promising behavioral
|
||||
Multi-Armed Bandit (MAB) logic to select the most promising behavioral
|
||||
mutation strategy for the current account phase.
|
||||
"""
|
||||
logger.info(f"🧬 [Darwin Engine] Applying MDP State channel for @{self.username}...")
|
||||
self.synthesize_interaction_profile(target_resonance=0.5) # Initial neutral bias
|
||||
self.synthesize_interaction_profile(target_resonance=0.5) # Initial neutral bias
|
||||
|
||||
def evaluate_session_end(self, duration_minutes: float, followers_gained: int):
|
||||
if duration_minutes <= 0: duration_minutes = 1.0
|
||||
if duration_minutes <= 0:
|
||||
duration_minutes = 1.0
|
||||
reward = (followers_gained / duration_minutes) * 10.0
|
||||
logger.info(f"🧬 [Darwin Engine] Session Evaluation: {followers_gained} followers gained in {duration_minutes:.1f}m. Reward: {reward:.2f}")
|
||||
logger.info(
|
||||
f"🧬 [Darwin Engine] Session Evaluation: {followers_gained} followers gained in {duration_minutes:.1f}m. Reward: {reward:.2f}"
|
||||
)
|
||||
self.emit_reward_signal(followers_gained=followers_gained, block_warnings_seen=0)
|
||||
|
||||
def emit_reward_signal(self, followers_gained: int, block_warnings_seen: int):
|
||||
if not self.current_behavior:
|
||||
return
|
||||
|
||||
|
||||
try:
|
||||
reward = followers_gained - (block_warnings_seen * 50)
|
||||
|
||||
|
||||
vector = []
|
||||
for k, (p_min, p_max, _) in self.behavior_bounds.items():
|
||||
val = self.current_behavior.get(k, p_min)
|
||||
@@ -254,10 +331,19 @@ class DarwinEngine(QdrantBase):
|
||||
"username": self.username,
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
"params": self.current_behavior,
|
||||
"reward": reward
|
||||
"reward": reward,
|
||||
},
|
||||
log_success=f"🧬 [Darwin Engine V3] MDP Reward Matrix stored. Reward Value: {reward:.2f}"
|
||||
log_success=f"🧬 [Darwin Engine V3] MDP Reward Matrix stored. Reward Value: {reward:.2f}",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"🧬 [Darwin Engine] Failed to record reward: {e}")
|
||||
|
||||
def _has_comments(self, xml_string: str) -> bool:
|
||||
"""
|
||||
Delegates detection of comments to the Telepathic Engine's VLM to ensure
|
||||
zero maintenance and no hardcoded locale strings or resource-ids.
|
||||
"""
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepathic = TelepathicEngine.get_instance()
|
||||
return telepathic.classify_screen_content(xml_string.lower(), "post_has_comments") == "has_comments"
|
||||
|
||||
@@ -1,13 +1,17 @@
|
||||
import logging
|
||||
import json
|
||||
import uiautomator2 as u2
|
||||
from time import sleep, time
|
||||
from random import uniform
|
||||
from GramAddict.core.utils import random_sleep
|
||||
import os
|
||||
from functools import wraps
|
||||
from random import uniform
|
||||
from time import sleep
|
||||
|
||||
import uiautomator2 as u2
|
||||
|
||||
from GramAddict.core.physics.biomechanics import BezierGesture, PhysicsBody
|
||||
from GramAddict.core.physics.sendevent_injector import SendEventInjector
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def adb_retry(retries=3, delay=2.0):
|
||||
def decorator(func):
|
||||
@wraps(func)
|
||||
@@ -22,47 +26,91 @@ def adb_retry(retries=3, delay=2.0):
|
||||
sleep(delay * (attempt + 1)) # Exponential backoff
|
||||
logger.error(f"❌ ADB action {func.__name__} failed after {retries} retries. Crashing gracefully.")
|
||||
raise last_err
|
||||
|
||||
return wrapper
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def create_device(device_id, app_id, args=None):
|
||||
try:
|
||||
return DeviceFacade(device_id, app_id, args)
|
||||
except Exception as e:
|
||||
err_msg = str(e)
|
||||
err_type = str(type(e))
|
||||
if any(
|
||||
keyword in err_type or keyword in err_msg
|
||||
for keyword in ["ConnectError", "ConnectionRefused", "ConnectionError", "Timeout"]
|
||||
):
|
||||
logger.error(f"⚠️ [ADB ConnectError] Could not connect to device '{device_id}'.")
|
||||
|
||||
# Proactive Discovery
|
||||
try:
|
||||
import subprocess
|
||||
|
||||
result = subprocess.run(["adb", "devices"], capture_output=True, text=True, timeout=2)
|
||||
lines = [
|
||||
line.strip()
|
||||
for line in result.stdout.split("\n")
|
||||
if line.strip() and not line.startswith("List of devices")
|
||||
]
|
||||
devices = [line.split("\t")[0] for line in lines if "device" in line]
|
||||
|
||||
if devices:
|
||||
logger.info("🔍 Proactive Discovery: I found the following devices connected:")
|
||||
for d in devices:
|
||||
if d.split(":")[0] == device_id.split(":")[0]:
|
||||
logger.info(f" 👉 {d} (MATCHING IP - Is this the same device with a different port?)")
|
||||
else:
|
||||
logger.info(f" - {d}")
|
||||
else:
|
||||
logger.warning("🔍 Proactive Discovery: No ADB devices found. Is your phone authorized?")
|
||||
except Exception as discovery_err:
|
||||
logger.debug(f"Proactive discovery failed: {discovery_err}")
|
||||
|
||||
logger.error("👉 Please verify:")
|
||||
logger.error(" 1. Your phone is connected via USB or Wi-Fi.")
|
||||
logger.error(" 2. 'USB Debugging' is enabled in Developer Options.")
|
||||
logger.error(" 3. You have authorized this computer on your phone's screen.")
|
||||
logger.error(" 4. The adb server is running ('adb devices').")
|
||||
raise SystemExit(1)
|
||||
|
||||
logger.error(f"Failed to create device: {e}")
|
||||
# In V7, we don't want to just return None and crash later.
|
||||
# We don't want to just return None and crash later.
|
||||
# We should raise so the orchestrator knows it's a fatal boot error.
|
||||
raise e
|
||||
|
||||
|
||||
def get_device_info(device):
|
||||
if not device or not device.deviceV2:
|
||||
logger.error("Cannot get device info: Device not initialized.")
|
||||
return
|
||||
info = device.deviceV2.info
|
||||
info = device.info
|
||||
logger.debug(f"Device Info: {info.get('productName')} | SDK: {info.get('sdkInt')}")
|
||||
|
||||
|
||||
class DeviceFacade:
|
||||
deviceV2 = None
|
||||
app_id = None
|
||||
device_id = None
|
||||
|
||||
def __init__(self, device_id, app_id, args):
|
||||
self.device_id = device_id
|
||||
self.app_id = app_id
|
||||
self.args = args
|
||||
self.deviceV2 = u2.connect(device_id)
|
||||
|
||||
|
||||
# Configure uiautomator2
|
||||
self.deviceV2.settings["wait_timeout"] = 3.0
|
||||
self.deviceV2.settings["post_delay"] = 0.5
|
||||
|
||||
|
||||
# System dialog handler (language-agnostic via resource-id, not text)
|
||||
try:
|
||||
# u2 v3.x: named watchers with xpath selectors
|
||||
# android:id/aerr_close = App crash "Close" button (all languages)
|
||||
self.deviceV2.watcher("crash_dialog").when(
|
||||
xpath='//*[@resource-id="android:id/aerr_close"]'
|
||||
).click()
|
||||
self.deviceV2.watcher("crash_dialog").when(xpath='//*[@resource-id="android:id/aerr_close"]').click()
|
||||
# android:id/button1 = positive system dialog button (all languages)
|
||||
self.deviceV2.watcher("system_dialog").when(
|
||||
xpath='//*[@resource-id="android:id/button1"]'
|
||||
).click()
|
||||
self.deviceV2.watcher("system_dialog").when(xpath='//*[@resource-id="android:id/button1"]').click()
|
||||
self.deviceV2.watcher.start()
|
||||
except Exception as e:
|
||||
logger.debug(f"Could not start system watcher: {e}")
|
||||
@@ -78,7 +126,7 @@ class DeviceFacade:
|
||||
@adb_retry()
|
||||
def cm_to_pixels(self, cm: float) -> int:
|
||||
info = self.deviceV2.info
|
||||
dpx = info.get("displaySizeDpX", 400)
|
||||
dpx = info.get("displaySizeDpX", 400)
|
||||
width = info.get("displayWidth", 1080)
|
||||
# Android baseline: 1 dp = 1/160 inch. 1 inch = 2.54 cm
|
||||
# PPCM (Pixels Per CM) = (width / dpx) * (160 / 2.54)
|
||||
@@ -92,26 +140,86 @@ class DeviceFacade:
|
||||
self.deviceV2.press("home")
|
||||
sleep(1)
|
||||
|
||||
@adb_retry()
|
||||
def unlock(self):
|
||||
self.deviceV2.unlock()
|
||||
|
||||
@property
|
||||
def info(self):
|
||||
return self.deviceV2.info
|
||||
|
||||
@adb_retry()
|
||||
def app_start(self, app_id=None, use_monkey=False):
|
||||
target_app = app_id or self.app_id
|
||||
if use_monkey:
|
||||
self.deviceV2.app_start(target_app, use_monkey=True)
|
||||
else:
|
||||
self.deviceV2.app_start(target_app)
|
||||
|
||||
@adb_retry()
|
||||
def app_stop(self, app_id=None):
|
||||
target_app = app_id or self.app_id
|
||||
self.deviceV2.app_stop(target_app)
|
||||
|
||||
@adb_retry()
|
||||
def shell(self, cmd):
|
||||
return self.deviceV2.shell(cmd)
|
||||
|
||||
@adb_retry()
|
||||
def swipe(self, sx, sy, ex, ey, duration=None):
|
||||
"""Pass-through strictly for non-biological bezier swiping (e.g., darwin_engine noise correction)"""
|
||||
kwargs = {}
|
||||
if duration is not None:
|
||||
kwargs["duration"] = duration
|
||||
self.deviceV2.swipe(sx, sy, ex, ey, **kwargs)
|
||||
|
||||
@adb_retry()
|
||||
def long_click(self, x, y, duration=1.5):
|
||||
self.deviceV2.long_click(x, y, duration)
|
||||
|
||||
@adb_retry()
|
||||
def press(self, key):
|
||||
self.deviceV2.press(key)
|
||||
|
||||
@adb_retry()
|
||||
def back(self):
|
||||
self.deviceV2.press("back")
|
||||
|
||||
@adb_retry()
|
||||
def click(self, x=None, y=None, obj=None):
|
||||
if obj:
|
||||
if isinstance(obj, dict) and 'x' in obj and 'y' in obj:
|
||||
self.human_click(obj['x'], obj['y'])
|
||||
if isinstance(obj, dict) and "x" in obj and "y" in obj:
|
||||
self.human_click(obj["x"], obj["y"])
|
||||
return
|
||||
try:
|
||||
left, top, right, bottom = obj.bounds()
|
||||
cx = (left + right) // 2
|
||||
cy = (top + bottom) // 2
|
||||
from random import uniform
|
||||
# Randomize hit location within inner 50% of the UI element
|
||||
w = right - left
|
||||
h = bottom - top
|
||||
cx += int(uniform(-w * 0.25, w * 0.25))
|
||||
cy += int(uniform(-h * 0.25, h * 0.25))
|
||||
|
||||
# Biological fingerprint via PhysicsBody
|
||||
body = PhysicsBody.get_session_instance(self)
|
||||
# Thumb bias: right-handers land slightly left-below center
|
||||
if body.handedness == "right":
|
||||
cx_base = left + (w * 0.45)
|
||||
cy_base = top + (h * 0.55)
|
||||
else:
|
||||
cx_base = left + (w * 0.55)
|
||||
cy_base = top + (h * 0.55)
|
||||
|
||||
from random import gauss
|
||||
|
||||
# Fatigue increases spread
|
||||
fatigue_mult = 1.0 + body.fatigue * 0.3
|
||||
sigma_x = max(1, w * 0.15 * fatigue_mult)
|
||||
sigma_y = max(1, h * 0.15 * fatigue_mult)
|
||||
|
||||
cx = int(gauss(cx_base, sigma_x))
|
||||
cy = int(gauss(cy_base, sigma_y))
|
||||
|
||||
# Math constraint to ensure it physically lands on the button
|
||||
cx = max(left + 1, min(cx, right - 1))
|
||||
cy = max(top + 1, min(cy, bottom - 1))
|
||||
|
||||
self.human_click(cx, cy)
|
||||
except Exception as e:
|
||||
logger.debug(f"Bounds extraction failed, fallback to native click: {e}")
|
||||
@@ -121,67 +229,89 @@ class DeviceFacade:
|
||||
|
||||
@adb_retry()
|
||||
def human_click(self, x, y):
|
||||
from random import uniform
|
||||
# 🛡️ [Gesture Guard] If clicking near the edges, use native click to prevent
|
||||
# triggering System Gestures (e.g., Google Assistant diagonal swipe, App Switcher)
|
||||
# and prevent network latency turning edge taps into long-presses (Circle to Search).
|
||||
if y > 2100 or y < 200 or x < 50 or x > 1030:
|
||||
self.deviceV2.shell(f"input tap {int(x)} {int(y)}")
|
||||
return
|
||||
|
||||
try:
|
||||
self.deviceV2.touch.down(x, y)
|
||||
# Human finger rest time (squish)
|
||||
sleep(uniform(0.05, 0.15))
|
||||
|
||||
# Sloppy slip (Containment: Don't slip horizontally at the bottom edge, prevents Android App-Switch gestures)
|
||||
slip_x = x + int(uniform(-4, 4)) if y < 2100 else x
|
||||
slip_y = y + int(uniform(-4, 4))
|
||||
|
||||
self.deviceV2.touch.move(slip_x, slip_y)
|
||||
sleep(uniform(0.01, 0.05))
|
||||
self.deviceV2.touch.up(slip_x, slip_y)
|
||||
body = PhysicsBody.get_session_instance(self)
|
||||
injector = SendEventInjector.get_instance(self)
|
||||
points = BezierGesture.tap_curve(x, y, body)
|
||||
tap_duration = uniform(40, 90)
|
||||
timing = BezierGesture.compute_sigmoid_timing(len(points), tap_duration)
|
||||
|
||||
injector.inject_gesture(points, timing, touch_major=body.get_touch_major())
|
||||
except Exception as e:
|
||||
logger.debug(f"human_click failed, fallback: {e}")
|
||||
self.deviceV2.click(x, y)
|
||||
logger.debug(f"human_click biomechanics failed, fallback: {e}")
|
||||
try:
|
||||
self.deviceV2.touch.down(x, y)
|
||||
sleep(uniform(0.05, 0.15))
|
||||
slip_x = x + int(uniform(-4, 4)) if y < 2100 else x
|
||||
slip_y = y + int(uniform(-4, 4))
|
||||
self.deviceV2.touch.move(slip_x, slip_y)
|
||||
sleep(uniform(0.01, 0.05))
|
||||
self.deviceV2.touch.up(slip_x, slip_y)
|
||||
except Exception as e2:
|
||||
logger.debug(f"human_click u2 failed, final fallback: {e2}")
|
||||
self.deviceV2.shell(f"input tap {int(x)} {int(y)}")
|
||||
|
||||
@adb_retry()
|
||||
def swipe_points(self, x1, y1, x2, y2, duration=0.1):
|
||||
self.deviceV2.swipe(x1, y1, x2, y2, duration)
|
||||
dur_ms = int(duration * 1000)
|
||||
self.deviceV2.shell(f"input swipe {int(x1)} {int(y1)} {int(x2)} {int(y2)} {dur_ms}")
|
||||
|
||||
@adb_retry()
|
||||
def human_swipe(self, start_x, start_y, end_x, end_y, duration=0.3):
|
||||
# Simulate a realistic human swipe by keeping it simple.
|
||||
# Android's ScrollView calculates fling velocity based on the final few points.
|
||||
# If we use swipe_points with non-linear distances, it breaks the fling physics and produces stuttering or backwards scrolls.
|
||||
# We just use native swipe with randomized small x-variance.
|
||||
self.deviceV2.swipe(start_x, start_y, end_x, end_y, duration)
|
||||
# 🛡️ [Gesture Guard] If swiping near the very edges, use native swipe to prevent
|
||||
# system gesture clashes, unless it's a feed scroll (which is usually safe).
|
||||
dur_ms = int(duration * 1000)
|
||||
if start_x < 50 or start_x > 1030 or start_y < 200 or start_y > 2100:
|
||||
self.deviceV2.shell(f"input swipe {int(start_x)} {int(start_y)} {int(end_x)} {int(end_y)} {dur_ms}")
|
||||
return
|
||||
|
||||
try:
|
||||
body = PhysicsBody.get_session_instance(self)
|
||||
injector = SendEventInjector.get_instance(self)
|
||||
|
||||
# Use scroll_curve for vertical swipes, horizontal_swipe_curve for horizontal
|
||||
is_horizontal = abs(end_x - start_x) > abs(end_y - start_y)
|
||||
if is_horizontal:
|
||||
points = BezierGesture.horizontal_swipe_curve((start_x, start_y), (end_x, end_y), body)
|
||||
else:
|
||||
points = BezierGesture.scroll_curve((start_x, start_y), (end_x, end_y), body)
|
||||
|
||||
# Use fling timing (J-curve) to ensure high terminal velocity so Android scroll physics works natively
|
||||
timing = BezierGesture.compute_fling_timing(len(points), dur_ms)
|
||||
|
||||
injector.inject_gesture(points, timing, touch_major=body.get_touch_major())
|
||||
except Exception as e:
|
||||
logger.debug(f"human_swipe biomechanics failed, fallback to native swipe: {e}")
|
||||
self.deviceV2.shell(f"input swipe {int(start_x)} {int(start_y)} {int(end_x)} {int(end_y)} {dur_ms}")
|
||||
|
||||
@adb_retry()
|
||||
def _get_current_app(self):
|
||||
"""
|
||||
Hardened app package detection.
|
||||
Transient notifications (e.g. Amazon, WhatsApp, SystemUI) can spoof uiautomator2's app_current() report.
|
||||
We verify the package with multiple retries and a grace period if it doesn't match our expected app_id.
|
||||
SAE-aware app detection.
|
||||
Instead of maintaining a hardcoded list of 'transient' packages,
|
||||
we check the actual package and let the SAE handle recovery if needed.
|
||||
Transient notifications (status bar, brief banners) are handled by
|
||||
a single brief retry — no hardcoded app list needed.
|
||||
"""
|
||||
pkg = self.deviceV2.app_current().get("package")
|
||||
if pkg == self.app_id:
|
||||
return pkg
|
||||
|
||||
# If it doesn't match, it might be a notification banner.
|
||||
# Known transient spoofers: WhatsApp, SystemUI (status bar), Android System
|
||||
transient_packages = ["com.whatsapp", "com.android.systemui", "android"]
|
||||
|
||||
if pkg in transient_packages:
|
||||
# Check cooldown: if we just handled this package < 10s ago, don't sleep again
|
||||
now = time()
|
||||
if pkg == self.last_transient_pkg and (now - self.last_transient_time) < 10.0:
|
||||
logger.debug(f"Perimeter: Consecutive hit for transient package '{pkg}'. Skipping cooldown wait.")
|
||||
return self.app_id
|
||||
|
||||
logger.debug(f"⚠️ [Perimeter] Detected transient package '{pkg}'. Waiting for banner to clear...")
|
||||
self.last_transient_pkg = pkg
|
||||
self.last_transient_time = now
|
||||
sleep(1.5) # Give the notification/animation time to fade
|
||||
|
||||
pkg = self.deviceV2.app_current().get("package")
|
||||
if pkg in transient_packages:
|
||||
# If it persists, we trust the drift logic to handle it if it blocks the UI,
|
||||
# but for focus detection, we return the target app to avoid infinite wait loops.
|
||||
return self.app_id
|
||||
|
||||
# Brief retry: many false positives come from <500ms notification banners
|
||||
# A single short wait handles ALL transient overlays regardless of source app
|
||||
sleep(0.5)
|
||||
pkg = self.deviceV2.app_current().get("package")
|
||||
|
||||
# If still not our app, check if it's just SystemUI (always present, never a real takeover)
|
||||
if pkg in ("com.android.systemui", "android"):
|
||||
return self.app_id
|
||||
|
||||
return pkg
|
||||
|
||||
@@ -192,35 +322,77 @@ class DeviceFacade:
|
||||
|
||||
@adb_retry()
|
||||
def dump_hierarchy(self):
|
||||
xml = self.deviceV2.dump_hierarchy()
|
||||
|
||||
# Compressed=True dramatically speeds up UIAutomator2 dumps by skipping invisible elements!
|
||||
xml = self.deviceV2.dump_hierarchy(compressed=True)
|
||||
|
||||
# Continuous Session Tracing
|
||||
import os
|
||||
import shutil
|
||||
from datetime import datetime
|
||||
|
||||
try:
|
||||
traces_root = os.path.join("debug", "session_traces")
|
||||
if not hasattr(self, "_trace_counter"):
|
||||
self._trace_counter = 0
|
||||
ts = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
|
||||
self._trace_dir = os.path.join("debug", "session_traces", ts)
|
||||
self._trace_dir = os.path.join(traces_root, ts)
|
||||
os.makedirs(self._trace_dir, exist_ok=True)
|
||||
|
||||
|
||||
# Cleanup: keep only last 5 session folders
|
||||
try:
|
||||
if os.path.exists(traces_root):
|
||||
folders = [
|
||||
os.path.join(traces_root, d)
|
||||
for d in os.listdir(traces_root)
|
||||
if os.path.isdir(os.path.join(traces_root, d))
|
||||
]
|
||||
folders.sort(key=os.path.getmtime)
|
||||
while len(folders) > 5:
|
||||
oldest = folders.pop(0)
|
||||
shutil.rmtree(oldest, ignore_errors=True)
|
||||
logger.info(f"🧹 [Cleanup] Removed old session trace: {oldest}")
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to cleanup old traces: {e}")
|
||||
|
||||
self._trace_counter += 1
|
||||
trace_path = os.path.join(self._trace_dir, f"{self._trace_counter:05d}.xml")
|
||||
with open(trace_path, "w", encoding="utf-8") as f:
|
||||
f.write(xml)
|
||||
|
||||
# Dump screenshot as well
|
||||
try:
|
||||
import base64
|
||||
|
||||
screenshot_b64 = self.get_screenshot_b64()
|
||||
if screenshot_b64:
|
||||
screenshot_data = base64.b64decode(screenshot_b64)
|
||||
screenshot_path = trace_path.replace(".xml", ".jpg")
|
||||
with open(screenshot_path, "wb") as f:
|
||||
f.write(screenshot_data)
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to capture screenshot for session trace: {e}")
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to write session trace: {e}")
|
||||
|
||||
|
||||
return xml
|
||||
|
||||
@adb_retry()
|
||||
def screenshot(self):
|
||||
return self.deviceV2.screenshot()
|
||||
def get_screenshot_b64(self):
|
||||
import base64
|
||||
from io import BytesIO
|
||||
|
||||
img = self.deviceV2.screenshot()
|
||||
if img is None:
|
||||
return None
|
||||
buffered = BytesIO()
|
||||
img.save(buffered, format="JPEG", quality=70) # Compressed for target latency
|
||||
return base64.b64encode(buffered.getvalue()).decode("utf-8")
|
||||
|
||||
# Telepathic Semantic UI Integration
|
||||
@adb_retry()
|
||||
def find_semantic(self, intent_description: str):
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
engine = TelepathicEngine.get_instance()
|
||||
xml = self.dump_hierarchy()
|
||||
# Passing self (DeviceFacade) enables the Vision Cortex VLM fallback
|
||||
|
||||
@@ -9,54 +9,64 @@ and a structured reason tag for easy triage.
|
||||
|
||||
Retention: Keeps the last 50 dumps per reason category to avoid disk bloat.
|
||||
"""
|
||||
import os
|
||||
import logging
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DUMP_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), "debug", "xml_dumps")
|
||||
MAX_DUMPS_PER_CATEGORY = 50
|
||||
MAX_DUMPS_PER_CATEGORY = 5
|
||||
|
||||
|
||||
def dump_ui_state(device, reason: str, extra_context: dict = None):
|
||||
"""
|
||||
Capture and save the current UI hierarchy to disk for debugging.
|
||||
|
||||
Args:
|
||||
device: The uiautomator2 device facade.
|
||||
reason: Short tag for the failure type. Used for filename grouping.
|
||||
Examples: 'context_lost', 'vlm_hallucination', 'nav_failure',
|
||||
'stuck_on_post', 'unexpected_screen'
|
||||
extra_context: Optional dict with additional metadata (intent, expected state, etc.)
|
||||
Capture and save the current UI hierarchy and screenshot to disk for debugging.
|
||||
"""
|
||||
try:
|
||||
os.makedirs(DUMP_DIR, exist_ok=True)
|
||||
|
||||
|
||||
# Capture hierarchy
|
||||
xml = device.dump_hierarchy()
|
||||
|
||||
|
||||
# Generate filename: reason__2026-04-13_17-41-39.xml
|
||||
ts = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
|
||||
safe_reason = reason.replace(" ", "_").replace("/", "_")[:40]
|
||||
filename = f"{safe_reason}__{ts}.xml"
|
||||
filepath = os.path.join(DUMP_DIR, filename)
|
||||
|
||||
|
||||
# Write XML
|
||||
with open(filepath, "w", encoding="utf-8") as f:
|
||||
f.write(xml)
|
||||
|
||||
|
||||
# Capture and write screenshot
|
||||
try:
|
||||
import base64
|
||||
|
||||
screenshot_b64 = device.get_screenshot_b64()
|
||||
if screenshot_b64:
|
||||
screenshot_data = base64.b64decode(screenshot_b64)
|
||||
screenshot_path = filepath.replace(".xml", ".jpg")
|
||||
with open(screenshot_path, "wb") as f:
|
||||
f.write(screenshot_data)
|
||||
except Exception as e:
|
||||
logger.debug(f"[Diagnostic] Could not capture screenshot: {e}")
|
||||
|
||||
# Write companion metadata JSON
|
||||
meta = {
|
||||
"reason": reason,
|
||||
"timestamp": ts,
|
||||
"xml_file": filename,
|
||||
"screenshot_file": filename.replace(".xml", ".jpg"),
|
||||
}
|
||||
# Capture the session log if available
|
||||
try:
|
||||
import shutil
|
||||
|
||||
from GramAddict.core.log import get_log_file_config
|
||||
|
||||
log_name, log_dir, _, _ = get_log_file_config()
|
||||
if log_name and log_dir:
|
||||
active_log = os.path.join(log_dir, log_name)
|
||||
@@ -69,39 +79,68 @@ def dump_ui_state(device, reason: str, extra_context: dict = None):
|
||||
|
||||
if extra_context:
|
||||
meta["context"] = extra_context
|
||||
|
||||
|
||||
meta_path = filepath.replace(".xml", ".meta.json")
|
||||
with open(meta_path, "w", encoding="utf-8") as f:
|
||||
json.dump(meta, f, indent=2, ensure_ascii=False)
|
||||
|
||||
logger.info(f"📸 [Diagnostic] UI state and session log dumped for '{reason}': {filepath}")
|
||||
|
||||
|
||||
logger.info(f"📸 [Diagnostic] UI state, screenshot, and session log dumped for '{reason}': {filepath}")
|
||||
|
||||
# Rotate old dumps for this category
|
||||
_rotate_dumps(safe_reason)
|
||||
|
||||
|
||||
return filepath
|
||||
|
||||
|
||||
except Exception as e:
|
||||
# Dumping must NEVER crash the bot
|
||||
logger.debug(f"[Diagnostic] Could not dump UI state: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _rotate_dumps(category_prefix: str):
|
||||
"""Keep only the last MAX_DUMPS_PER_CATEGORY dumps per category."""
|
||||
def _rotate_dumps(category_prefix: str = None):
|
||||
"""Keep only the last MAX_DUMPS_PER_CATEGORY dumps per category. If no category, cleans all."""
|
||||
try:
|
||||
all_files = sorted([
|
||||
f for f in os.listdir(DUMP_DIR)
|
||||
if f.startswith(category_prefix) and f.endswith(".xml")
|
||||
])
|
||||
|
||||
if len(all_files) > MAX_DUMPS_PER_CATEGORY:
|
||||
files_to_remove = all_files[:len(all_files) - MAX_DUMPS_PER_CATEGORY]
|
||||
for f in files_to_remove:
|
||||
xml_path = os.path.join(DUMP_DIR, f)
|
||||
meta_path = xml_path.replace(".xml", ".meta.json")
|
||||
os.remove(xml_path)
|
||||
if os.path.exists(meta_path):
|
||||
os.remove(meta_path)
|
||||
except Exception:
|
||||
pass
|
||||
if not os.path.exists(DUMP_DIR):
|
||||
return
|
||||
|
||||
# Get all unique timestamps/prefixes
|
||||
all_files = os.listdir(DUMP_DIR)
|
||||
prefixes = set()
|
||||
for f in all_files:
|
||||
# Format is usually reason__timestamp.ext
|
||||
if "__" in f:
|
||||
prefix = f.split(".")[0]
|
||||
prefixes.add(prefix)
|
||||
|
||||
# Group prefixes by category
|
||||
categories = {}
|
||||
for p in prefixes:
|
||||
parts = p.split("__")
|
||||
if len(parts) >= 2:
|
||||
cat = parts[0]
|
||||
if cat not in categories:
|
||||
categories[cat] = []
|
||||
categories[cat].append(p)
|
||||
|
||||
for cat, prefs in categories.items():
|
||||
if category_prefix and cat != category_prefix:
|
||||
continue
|
||||
|
||||
prefs.sort() # chronological
|
||||
if len(prefs) > MAX_DUMPS_PER_CATEGORY:
|
||||
prefs_to_remove = prefs[: len(prefs) - MAX_DUMPS_PER_CATEGORY]
|
||||
for p_rm in prefs_to_remove:
|
||||
for ext in [".xml", ".jpg", ".log", ".meta.json"]:
|
||||
fp = os.path.join(DUMP_DIR, p_rm + ext)
|
||||
if os.path.exists(fp):
|
||||
os.remove(fp)
|
||||
|
||||
# Also clean orphaned files that don't match any known prefix pattern
|
||||
for f in all_files:
|
||||
if "__" not in f:
|
||||
fp = os.path.join(DUMP_DIR, f)
|
||||
if os.path.isfile(fp):
|
||||
os.remove(fp)
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"[Diagnostic] Error during dump rotation: {e}")
|
||||
|
||||
@@ -1,31 +1,75 @@
|
||||
import logging
|
||||
import random
|
||||
|
||||
from colorama import Fore, Style
|
||||
|
||||
from GramAddict.core.session_state import SessionState
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Hard cap: maximum DM replies per inbox visit to prevent spam.
|
||||
MAX_REPLIES_PER_INBOX_VISIT = 3
|
||||
|
||||
# Sentinel values that indicate missing message context.
|
||||
_EMPTY_CONTEXT_SENTINELS = frozenset({"no previous context", "", "none", "n/a"})
|
||||
|
||||
|
||||
# Structural resource-IDs that indicate a real "Send" button.
|
||||
def _is_send_button(node: dict) -> bool:
|
||||
"""Semantic verification: returns True if the node is identified as a Send button."""
|
||||
desc = (node.get("description") or node.get("desc", "")).lower()
|
||||
text = (node.get("text") or "").lower()
|
||||
rid = (node.get("id") or node.get("resource_id", "")).lower()
|
||||
|
||||
# Accept if semantic markers indicate sending
|
||||
if any(m in rid for m in ["send", "composer_button"]):
|
||||
return True
|
||||
if any(m in desc for m in ["send", "absenden"]):
|
||||
return True
|
||||
if text == "send" or text == "absenden":
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _run_zero_latency_dm_loop(device, zero_engine, nav_graph, configs, session_state, current_target, cognitive_stack):
|
||||
"""
|
||||
Executes the autonomous Direct Messaging logic in the Zero-Latency architecture.
|
||||
Assumes the bot is already at the "MessageInbox" UI state.
|
||||
|
||||
Safety guarantees:
|
||||
- Refuses to execute if dm_reply plugin is disabled in config.
|
||||
- Skips threads with no extractable text context.
|
||||
- Structurally verifies the Send button before logging success.
|
||||
- Hard-caps replies per inbox visit to MAX_REPLIES_PER_INBOX_VISIT.
|
||||
"""
|
||||
logger.info(f"🧠 [DM Engine] Initiating inbox processing in {current_target}...", extra={"color": f"{Style.BRIGHT}{Fore.CYAN}"})
|
||||
|
||||
# ── Kill-Switch: Respect dm_reply.enabled config ──
|
||||
dm_plugin_config = configs.get_plugin_config("dm_reply")
|
||||
if not dm_plugin_config.get("enabled", False):
|
||||
logger.warning(
|
||||
"🛑 [DM Engine] dm_reply plugin is DISABLED in config. Refusing to process inbox.",
|
||||
extra={"color": f"{Fore.RED}"},
|
||||
)
|
||||
return "BOREDOM_CHANGE_FEED"
|
||||
|
||||
logger.info(
|
||||
f"🧠 [DM Engine] Initiating inbox processing in {current_target}...",
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.CYAN}"},
|
||||
)
|
||||
|
||||
telepathic = cognitive_stack.get("telepathic")
|
||||
dopamine = cognitive_stack.get("dopamine")
|
||||
crm = cognitive_stack.get("crm")
|
||||
|
||||
from GramAddict.core.bot_flow import sleep, dump_ui_state, _humanized_click
|
||||
|
||||
from GramAddict.core.bot_flow import _humanized_click, sleep
|
||||
from GramAddict.core.llm_provider import query_llm
|
||||
from GramAddict.core.stealth_typing import ghost_type
|
||||
|
||||
|
||||
# Initialize session limits if missing
|
||||
if not hasattr(session_state, 'totalMessages'):
|
||||
if not hasattr(session_state, "totalMessages"):
|
||||
session_state.totalMessages = 0
|
||||
|
||||
|
||||
failed_attempts = 0
|
||||
|
||||
replies_this_visit = 0
|
||||
|
||||
while not dopamine.is_app_session_over():
|
||||
# Limits check
|
||||
limit_val = session_state.check_limit(SessionState.Limit.PM)
|
||||
@@ -34,80 +78,188 @@ def _run_zero_latency_dm_loop(device, zero_engine, nav_graph, configs, session_s
|
||||
return "BOREDOM_CHANGE_FEED"
|
||||
elif limit_val is True:
|
||||
return "BOREDOM_CHANGE_FEED"
|
||||
|
||||
|
||||
try:
|
||||
xml_dump = device.dump_hierarchy()
|
||||
|
||||
|
||||
# --- Zero Trust Structural Guard ---
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity, ScreenType
|
||||
|
||||
identity_engine = ScreenIdentity(getattr(configs.args, "username", ""))
|
||||
identity_engine.device = device
|
||||
screen_info = identity_engine.identify(xml_dump)
|
||||
|
||||
screen_type = screen_info["screen_type"]
|
||||
is_inbox = screen_type == ScreenType.DM_INBOX
|
||||
is_thread = screen_type == ScreenType.DM_THREAD
|
||||
|
||||
if is_thread:
|
||||
logger.warning("⚠️ [Structural Guard] DM Engine trapped in an open thread. Escaping...")
|
||||
device.press("back")
|
||||
from GramAddict.core.bot_flow import sleep
|
||||
|
||||
sleep(1.5)
|
||||
continue
|
||||
|
||||
if not is_inbox:
|
||||
# We have drifted somewhere entirely alien (like Privacy Settings)
|
||||
logger.error(
|
||||
f"🛑 [Structural Guard] Alien context detected ({screen_type}). Not in Inbox. Triggering CONTEXT_LOST."
|
||||
)
|
||||
return "CONTEXT_LOST"
|
||||
# -----------------------------------
|
||||
|
||||
# Step 1: Find unread conversation threads
|
||||
unread_threads = telepathic._extract_semantic_nodes(xml_dump, "find unread message threads or unread badges", threshold=0.7)
|
||||
|
||||
unread_threads = telepathic._extract_semantic_nodes(
|
||||
xml_dump, "find unread message threads or unread badges", threshold=0.7, device=device
|
||||
)
|
||||
|
||||
if unread_threads and not unread_threads[0].get("skip"):
|
||||
target_node = unread_threads[0]
|
||||
logger.info(f"📨 Found unread message thread. Opening.")
|
||||
logger.info("📨 Found unread message thread. Opening.")
|
||||
_humanized_click(device, target_node["x"], target_node["y"])
|
||||
sleep(2.0)
|
||||
|
||||
|
||||
# Step 2: Read the conversation context
|
||||
thread_xml = device.dump_hierarchy()
|
||||
msg_nodes = telepathic._extract_semantic_nodes(thread_xml, "find the last received message text", threshold=0.6)
|
||||
|
||||
msg_nodes = telepathic._extract_semantic_nodes(
|
||||
thread_xml, "find the last received message text", threshold=0.6, device=device
|
||||
)
|
||||
|
||||
context_text = "No previous context"
|
||||
if msg_nodes and not msg_nodes[0].get("skip") and msg_nodes[0].get("text"):
|
||||
context_text = msg_nodes[0].get("text")
|
||||
|
||||
|
||||
logger.debug(f"Last received message context: {context_text}")
|
||||
|
||||
|
||||
# ── Context Guard: Skip threads with no extractable message ──
|
||||
if context_text.strip().lower() in _EMPTY_CONTEXT_SENTINELS:
|
||||
logger.warning(
|
||||
"⏭️ [DM Engine] Thread has no extractable message context (story reply / media-only). Skipping."
|
||||
)
|
||||
device.press("back")
|
||||
sleep(1.5)
|
||||
continue
|
||||
|
||||
# Verify we aren't at limits before sending
|
||||
if not getattr(configs.args, "disable_ai_messaging", False):
|
||||
# Generate response
|
||||
prompt = f"You are replying to a direct message on Instagram. The last message you received was: '{context_text}'. Keep it short, casual, and friendly. Do not use hashtags."
|
||||
response_text = query_llm(prompt)
|
||||
|
||||
if response_text:
|
||||
# Find the input field
|
||||
input_nodes = telepathic._extract_semantic_nodes(thread_xml, "find the message input text field", threshold=0.7)
|
||||
if input_nodes and not input_nodes[0].get("skip"):
|
||||
in_node = input_nodes[0]
|
||||
_humanized_click(device, in_node["x"], in_node["y"])
|
||||
sleep(1.0)
|
||||
|
||||
# Type the message
|
||||
ghost_type(device, response_text, speed="fast")
|
||||
sleep(1.0)
|
||||
|
||||
# Find Send button
|
||||
send_xml = device.dump_hierarchy()
|
||||
send_nodes = telepathic._extract_semantic_nodes(send_xml, "find the send message button", threshold=0.8)
|
||||
|
||||
if send_nodes and not send_nodes[0].get("skip"):
|
||||
s_node = send_nodes[0]
|
||||
# ── Iteration Cap: Prevent DM spam ──
|
||||
if replies_this_visit >= MAX_REPLIES_PER_INBOX_VISIT:
|
||||
logger.info(
|
||||
f"🛑 [DM Engine] Reached max replies per inbox visit ({MAX_REPLIES_PER_INBOX_VISIT}). Exiting."
|
||||
)
|
||||
device.press("back")
|
||||
sleep(1.0)
|
||||
return "BOREDOM_CHANGE_FEED"
|
||||
|
||||
# Configure models
|
||||
model = getattr(configs.args, "ai_condenser_model", "llama3.2:1b")
|
||||
url = getattr(configs.args, "ai_condenser_url", "http://localhost:11434/api/generate")
|
||||
|
||||
# Generate response
|
||||
prompt = f"You are replying to a direct message on Instagram. The last message you received was: '{context_text}'. Keep it short, casual, and friendly. Do not use hashtags."
|
||||
|
||||
logger.info(">>> [DM Engine] ABOUT TO CALL LLM")
|
||||
response_dict = query_llm(
|
||||
url=url,
|
||||
model=model,
|
||||
prompt=prompt,
|
||||
format_json=False,
|
||||
timeout=120,
|
||||
max_tokens=100,
|
||||
temperature=0.7,
|
||||
)
|
||||
logger.info(f">>> [DM Engine] LLM RETURNED: {response_dict}")
|
||||
|
||||
if response_dict and "response" in response_dict:
|
||||
response_text = response_dict["response"].strip()
|
||||
# Find the input field
|
||||
input_nodes = telepathic._extract_semantic_nodes(
|
||||
thread_xml, "find the message input text field", threshold=0.7, device=device
|
||||
)
|
||||
if input_nodes and not input_nodes[0].get("skip"):
|
||||
in_node = input_nodes[0]
|
||||
_humanized_click(device, in_node["x"], in_node["y"])
|
||||
sleep(1.0)
|
||||
|
||||
# Type the message
|
||||
ghost_type(device, response_text, speed="fast")
|
||||
sleep(1.0)
|
||||
|
||||
# Find Send button
|
||||
send_xml = device.dump_hierarchy()
|
||||
send_nodes = telepathic._extract_semantic_nodes(
|
||||
send_xml, "find the send message button", threshold=0.8, device=device
|
||||
)
|
||||
|
||||
if send_nodes and not send_nodes[0].get("skip"):
|
||||
s_node = send_nodes[0]
|
||||
|
||||
# ── Send Button Structural Verification ──
|
||||
if not _is_send_button(s_node):
|
||||
s_rid = s_node.get("original_attribs", {}).get("resource-id", "unknown")
|
||||
logger.warning(
|
||||
f"⚠️ [DM Engine] Refused to click non-Send element: {s_rid}. Aborting reply."
|
||||
)
|
||||
else:
|
||||
_humanized_click(device, s_node["x"], s_node["y"])
|
||||
logger.info("✅ [DM Engine] Successfully sent a generated reply.", extra={"color": Fore.GREEN})
|
||||
|
||||
logger.info(
|
||||
"✅ [DM Engine] Successfully sent a generated reply.",
|
||||
extra={"color": Fore.GREEN},
|
||||
)
|
||||
session_state.totalMessages += 1
|
||||
if crm:
|
||||
crm.log_sent_dm("unknown_target", response_text, "", [])
|
||||
|
||||
replies_this_visit += 1
|
||||
dm_memory = cognitive_stack.get("dm_memory")
|
||||
if dm_memory:
|
||||
dm_memory.log_sent_dm("unknown_target", response_text, "", [])
|
||||
|
||||
# Return back to inbox
|
||||
device.deviceV2.press("back")
|
||||
sleep(1.0)
|
||||
|
||||
device.press("back")
|
||||
sleep(1.5)
|
||||
|
||||
# If keyboard was open, the first back only closed it. Check if still in thread.
|
||||
check_xml = device.dump_hierarchy()
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity, ScreenType
|
||||
|
||||
check_identity = ScreenIdentity(getattr(configs.args, "username", ""))
|
||||
check_identity.device = device
|
||||
check_screen = check_identity.identify(check_xml)
|
||||
|
||||
if check_screen["screen_type"] == ScreenType.DM_THREAD:
|
||||
device.press("back")
|
||||
sleep(1.0)
|
||||
|
||||
dopamine.boredom += random.uniform(5.0, 15.0)
|
||||
failed_attempts = 0
|
||||
else:
|
||||
logger.info("📭 No unread threads found. Inbox clear.")
|
||||
dopamine.boredom += 50.0 # Inbox clear = massive boredom = change feed
|
||||
|
||||
|
||||
if dopamine.wants_to_change_feed() or dopamine.boredom >= 100:
|
||||
logger.info("🧠 [DM Engine] Interaction complete. Transitioning back from inbox.")
|
||||
device.deviceV2.press("back") # Go back from inbox
|
||||
device.press("back") # Go back from inbox
|
||||
return "BOREDOM_CHANGE_FEED"
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"⚠️ [FSD Anomaly Handler] Exception in DM Loop: {e}")
|
||||
device.deviceV2.press("back")
|
||||
logger.error(f"⚠️ [Anomaly Handler] Exception in DM Loop: {e}")
|
||||
device.press("back")
|
||||
sleep(1.0)
|
||||
|
||||
check_xml = device.dump_hierarchy()
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity, ScreenType
|
||||
|
||||
check_identity = ScreenIdentity(getattr(configs.args, "username", ""))
|
||||
check_identity.device = device
|
||||
check_screen = check_identity.identify(check_xml)
|
||||
|
||||
if check_screen["screen_type"] == ScreenType.DM_THREAD:
|
||||
device.press("back")
|
||||
sleep(1.0)
|
||||
|
||||
failed_attempts += 1
|
||||
if failed_attempts > 2:
|
||||
return "CONTEXT_LOST"
|
||||
|
||||
return "CONTEXT_LOST"
|
||||
|
||||
if dopamine.is_app_session_over():
|
||||
return "SESSION_OVER"
|
||||
|
||||
return "FEED_EXHAUSTED"
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
import os
|
||||
import queue
|
||||
import threading
|
||||
from datetime import datetime
|
||||
|
||||
from colorama import Fore
|
||||
|
||||
# Import existing VLM engine and Qdrant DB for operations
|
||||
@@ -12,17 +11,19 @@ from GramAddict.core.qdrant_memory import HeuristicMemoryDB
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class DojoEngine:
|
||||
"""
|
||||
Project Dojo: The Tesla FSD Data Engine.
|
||||
Project Dojo: The Data Engine.
|
||||
Handles asynchronous learning from failures (Prediction Errors).
|
||||
Instead of blocking the bot when an element is not found, the bot
|
||||
offloads the snapshot to this queue. The DojoEngine recompiles the
|
||||
offloads the snapshot to this queue. The DojoEngine recompiles the
|
||||
heuristic using a heavy VLM model in the background and updates the DB.
|
||||
"Never make a mistake twice."
|
||||
"""
|
||||
|
||||
_instance = None
|
||||
|
||||
|
||||
@classmethod
|
||||
def get_instance(cls, device=None):
|
||||
if cls._instance is None:
|
||||
@@ -43,7 +44,9 @@ class DojoEngine:
|
||||
self.is_running = True
|
||||
self.worker_thread = threading.Thread(target=self._process_queue, daemon=True)
|
||||
self.worker_thread.start()
|
||||
logger.info("⛩️ [Dojo Data Engine] Background learning pipeline initialized.", extra={"color": f"{Fore.CYAN}"})
|
||||
logger.info(
|
||||
"⛩️ [Dojo Data Engine] Background learning pipeline initialized.", extra={"color": f"{Fore.CYAN}"}
|
||||
)
|
||||
|
||||
def stop(self):
|
||||
self.is_running = False
|
||||
@@ -58,10 +61,13 @@ class DojoEngine:
|
||||
"name": heuristic_name,
|
||||
"xml": context_xml,
|
||||
"intent": intent_prompt,
|
||||
"timestamp": datetime.now().isoformat()
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
}
|
||||
self.learning_queue.put(snapshot)
|
||||
logger.info(f"⛩️ [Dojo] Snapshot for '{heuristic_name}' enqueued for shadow-compilation.", extra={"color": f"{Fore.CYAN}"})
|
||||
logger.info(
|
||||
f"⛩️ [Dojo] Snapshot for '{heuristic_name}' enqueued for shadow-compilation.",
|
||||
extra={"color": f"{Fore.CYAN}"},
|
||||
)
|
||||
|
||||
def _process_queue(self):
|
||||
"""
|
||||
@@ -71,24 +77,29 @@ class DojoEngine:
|
||||
try:
|
||||
# Wait for a job
|
||||
snapshot = self.learning_queue.get(timeout=5.0)
|
||||
h_name = snapshot['name']
|
||||
xml = snapshot['xml']
|
||||
intent = snapshot['intent']
|
||||
|
||||
h_name = snapshot["name"]
|
||||
xml = snapshot["xml"]
|
||||
intent = snapshot["intent"]
|
||||
|
||||
logger.info(f"⛩️ [Dojo] Processing auto-labeling job: {h_name}...", extra={"color": f"{Fore.CYAN}"})
|
||||
|
||||
|
||||
# Heavy compilation
|
||||
new_rule = self.compiler.generate_heuristic(intent, xml)
|
||||
|
||||
|
||||
if new_rule:
|
||||
# Overwrite legacy rule in Database (Fleet update)
|
||||
self.db.cache_heuristic(h_name, new_rule)
|
||||
logger.info(f"⛩️ [Dojo] SUCCESS! Fleet Memory updated with robust heuristic for '{h_name}'.", extra={"color": f"{Fore.GREEN}"})
|
||||
logger.info(
|
||||
f"⛩️ [Dojo] SUCCESS! Fleet Memory updated with robust heuristic for '{h_name}'.",
|
||||
extra={"color": f"{Fore.GREEN}"},
|
||||
)
|
||||
else:
|
||||
logger.warning(f"⛩️ [Dojo] FAILED to compile robust heuristic for '{h_name}'.", extra={"color": f"{Fore.RED}"})
|
||||
|
||||
logger.warning(
|
||||
f"⛩️ [Dojo] FAILED to compile robust heuristic for '{h_name}'.", extra={"color": f"{Fore.RED}"}
|
||||
)
|
||||
|
||||
self.learning_queue.task_done()
|
||||
|
||||
|
||||
except queue.Empty:
|
||||
continue
|
||||
except Exception as e:
|
||||
|
||||
@@ -1,41 +1,50 @@
|
||||
import logging
|
||||
import random
|
||||
import time
|
||||
|
||||
from colorama import Fore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class DopamineEngine:
|
||||
"""
|
||||
Simulation of human neurochemistry.
|
||||
Manages boredom levels and interest-based interaction pacing.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.boredom = 0.0 # 0.0 to 100.0
|
||||
self.boredom = 0.0 # 0.0 to 100.0
|
||||
self.spike_threshold = 7.0
|
||||
self.homeostasis_rate = 0.05 # decay per minute
|
||||
self.homeostasis_rate = 0.05 # decay per minute
|
||||
self.last_spike = time.time()
|
||||
self.session_start = time.time()
|
||||
self.session_limit_seconds = random.uniform(10 * 60, 35 * 60) # 10-35 mins session
|
||||
|
||||
self.session_limit_seconds = random.uniform(10 * 60, 35 * 60) # 10-35 mins session
|
||||
|
||||
def process_content(self, classification: dict):
|
||||
"""
|
||||
classification: {'quality': 'high'|'low', 'type': 'meme'|'aesthetic'|'ad', 'score': 0-10}
|
||||
"""
|
||||
score = classification.get("score", 5.0)
|
||||
quality = classification.get("quality", "medium")
|
||||
|
||||
|
||||
# Calculate spike
|
||||
spike = score * 1.5 if quality == "high" else score * 0.5
|
||||
|
||||
|
||||
# Update boredom: negative correlation with high quality content
|
||||
if spike > self.spike_threshold:
|
||||
self.boredom = max(0.0, self.boredom - (spike * 0.2))
|
||||
logger.info(f"💉 [Dopamine] Spike detected! Interest high. Boredom decreased to {self.boredom:.1f}%", extra={"color": f"{Fore.YELLOW}"})
|
||||
logger.info(
|
||||
f"💉 [Dopamine] Spike detected! Interest high. Boredom decreased to {self.boredom:.1f}%",
|
||||
extra={"color": f"{Fore.YELLOW}"},
|
||||
)
|
||||
else:
|
||||
self.boredom = min(100.0, self.boredom + 5.0)
|
||||
logger.info(f"💉 [Dopamine] Low interest content. Boredom increased to {self.boredom:.1f}%", extra={"color": f"{Fore.YELLOW}"})
|
||||
|
||||
logger.info(
|
||||
f"💉 [Dopamine] Low interest content. Boredom increased to {self.boredom:.1f}%",
|
||||
extra={"color": f"{Fore.YELLOW}"},
|
||||
)
|
||||
|
||||
self.last_spike = time.time()
|
||||
return self.is_bored()
|
||||
|
||||
@@ -44,15 +53,60 @@ class DopamineEngine:
|
||||
|
||||
def wants_to_doomscroll(self):
|
||||
# Engage fast swiping if highly bored but not fully exhausted
|
||||
return 75.0 < self.boredom < 100.0
|
||||
|
||||
# Make the behavior probabilistic so we don't get stuck in an infinite loop
|
||||
if 75.0 < self.boredom < 100.0:
|
||||
chance = (self.boredom - 70.0) / 30.0 # Scales from ~16% to 100% chance
|
||||
if random.random() < chance:
|
||||
# Decrease boredom slightly so the agent slowly snaps out of it
|
||||
self.boredom = max(70.0, self.boredom - 1.5)
|
||||
return True
|
||||
return False
|
||||
|
||||
def wants_to_change_feed(self):
|
||||
# Spontaneous urge to change context due to extreme boredom spikes
|
||||
return self.boredom > 85.0 and random.random() < 0.2
|
||||
|
||||
# Engage context shift if highly bored
|
||||
if 80.0 < self.boredom < 100.0:
|
||||
return random.random() < 0.4
|
||||
return False
|
||||
|
||||
def reset_boredom(self, decay=0.2):
|
||||
"""
|
||||
Resets boredom after a successful context shift.
|
||||
We don't reset to 0.0 to prevent infinite looping in the same feeds.
|
||||
"""
|
||||
old = self.boredom
|
||||
self.boredom = max(0.0, self.boredom * decay)
|
||||
logger.info(
|
||||
f"💉 [Dopamine] Context shifted. Boredom cooled: {old:.1f}% -> {self.boredom:.1f}%",
|
||||
extra={"color": f"{Fore.YELLOW}"},
|
||||
)
|
||||
|
||||
def reset_session(self):
|
||||
"""
|
||||
Resets all variables for a completely new app session.
|
||||
"""
|
||||
self.boredom = 0.0
|
||||
self.session_start = time.time()
|
||||
self.last_spike = time.time()
|
||||
self.session_limit_seconds = random.uniform(10 * 60, 35 * 60)
|
||||
logger.info(
|
||||
"💉 [Dopamine] Session limits and neurochemistry reset to baseline.", extra={"color": f"{Fore.YELLOW}"}
|
||||
)
|
||||
|
||||
def is_app_session_over(self):
|
||||
# Global Hard Kill check
|
||||
if getattr(self, "global_max_runtime_minutes", None):
|
||||
if hasattr(self, "global_start_time"):
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
if datetime.now() - self.global_start_time > timedelta(minutes=self.global_max_runtime_minutes):
|
||||
logger.info(
|
||||
f"🛑 [Timeout] Maximum runtime of {self.global_max_runtime_minutes} minutes reached (checked by DopamineEngine). Force-stopping session.",
|
||||
extra={"color": f"{Fore.RED}"},
|
||||
)
|
||||
return True
|
||||
|
||||
# True if we have scrolled too long or hit absolute burnout
|
||||
return (time.time() - self.session_start) > self.session_limit_seconds or self.boredom >= 100.0
|
||||
return (time.time() - self.session_start) >= self.session_limit_seconds or self.boredom >= 100.0
|
||||
|
||||
def get_pacing_modifier(self, base_score: float):
|
||||
"""
|
||||
@@ -60,9 +114,9 @@ class DopamineEngine:
|
||||
High dopamine (high interest) = longer viewing time.
|
||||
"""
|
||||
if base_score > 8:
|
||||
return random.uniform(2.0, 4.0) # Entranced
|
||||
return random.uniform(2.0, 4.0) # Entranced
|
||||
if base_score < 3:
|
||||
return random.uniform(0.1, 0.4) # Fast-swipe
|
||||
return random.uniform(0.1, 0.4) # Fast-swipe
|
||||
return 1.0
|
||||
|
||||
def decay(self):
|
||||
|
||||
@@ -1,105 +0,0 @@
|
||||
import os
|
||||
import time
|
||||
import logging
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
def capture_all(device):
|
||||
"""
|
||||
Automated E2E Dump Capturer Sequence.
|
||||
Navigates through the Instagram UI and securely saves exact XML representations
|
||||
to satisfy the `e2e_device_dump_injector` test requirements.
|
||||
|
||||
Warning: Requires a logged-in session and active device connection.
|
||||
"""
|
||||
logger.info("📸 Initiating E2E Dump Capture Sequence!")
|
||||
|
||||
FIX_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), "tests", "fixtures")
|
||||
os.makedirs(FIX_DIR, exist_ok=True)
|
||||
|
||||
def _save_dump(filename, description):
|
||||
logger.info(f"⏳ Waiting for UI to settle for [{description}]...")
|
||||
time.sleep(3.5) # ensure animations finish
|
||||
xml_data = device.dump_hierarchy()
|
||||
path = os.path.join(FIX_DIR, filename)
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
f.write(xml_data)
|
||||
logger.info(f"✅ Saved ECHTEN DUMP to {filename}")
|
||||
|
||||
print("\n" + "="*50)
|
||||
print("🤖 MANUAL E2E DUMP CAPTURE SEQUENCE")
|
||||
print("="*50)
|
||||
print("Please follow the instructions below to capture the required fixtures.")
|
||||
print("If an IG update changed the layout, you can navigate there naturally.")
|
||||
print("="*50 + "\n")
|
||||
|
||||
try:
|
||||
# Pre-condition: Device connected
|
||||
logger.info("Verifying device connection...")
|
||||
device.deviceV2.info
|
||||
|
||||
# 1. Comment Sheet
|
||||
input("\n👉 1. COMMENT SHEET:\nOpen Instagram, scroll to any post on the HomeFeed, and open the comment section.\nWhen the comment sheet is fully visible, press ENTER to capture...")
|
||||
_save_dump("comment_sheet.xml", "Post Comment Sheet")
|
||||
|
||||
# 2. Stories Feed
|
||||
input("\n👉 2. STORIES FEED:\nGo to the HomeFeed and tap any user's story right at the top.\nWhile the story is playing (video/photo is visible), press ENTER to capture...")
|
||||
_save_dump("stories_feed_dump.xml", "Active Story Playback")
|
||||
|
||||
# 3. DM Inbox
|
||||
input("\n👉 3. DM INBOX:\nGo back to the HomeFeed and tap the message icon in the top right to open your inbox.\nWhen your list of chats is visible, press ENTER to capture...")
|
||||
_save_dump("dm_inbox_dump.xml", "DM Inbox / Threads List")
|
||||
|
||||
# 4. Profile Scraping & Unfollow List
|
||||
input("\n👉 4. OWN PROFILE:\nGo to your OWN profile by tapping your avatar in the bottom right corner.\nWhen your bio and grid are fully visible, press ENTER to capture...")
|
||||
_save_dump("scraping_profile_dump.xml", "Own Profile Root (User Info)")
|
||||
|
||||
input("\n👉 4.b FOLLOWING LIST:\nFrom your profile, tap your 'Following' (Abonniert) count to open the list of people you follow.\nWhen the list is fully loaded, press ENTER to capture...")
|
||||
_save_dump("unfollow_list_dump.xml", "Following List Iteration View")
|
||||
|
||||
# 5. Search Feed
|
||||
input("\n👉 5. EXPLORE SEARCH:\nTap the magnifying glass (Explore) tab at the bottom. Then, tap into the top 'Search' bar so your keyboard opens.\nWhen you are in the search state, press ENTER to capture...")
|
||||
_save_dump("search_feed_dump.xml", "Explore Search Input Focus")
|
||||
|
||||
# 6. Reels Feed
|
||||
input("\n👉 6. REELS FEED:\nTap the Reels (Video) tab at the bottom center. Let a video start playing.\nPress ENTER to capture...")
|
||||
_save_dump("reels_feed_dump.xml", "Reels Video Feed")
|
||||
|
||||
# 7. Notifications
|
||||
input("\n👉 7. NOTIFICATIONS (ACTIVITY):\nGo to the HomeFeed and tap the Heart icon in the top right to open notifications.\nPress ENTER to capture...")
|
||||
_save_dump("notifications_dump.xml", "Activity / Notifications tab")
|
||||
|
||||
# 8. Explore Grid
|
||||
input("\n👉 8. EXPLORE GRID:\nTap the magnifying glass (Explore) tab, but do NOT tap the search bar.\nWhen the grid of images/videos is visible, press ENTER to capture...")
|
||||
_save_dump("explore_feed_dump.xml", "Explore Discovery Grid")
|
||||
|
||||
# 9. Other User's Profile
|
||||
input("\n👉 9. ALIEN PROFILE:\nNavigate to ANY OTHER user's profile (e.g. from your Feed or Search).\nWhen their bio and grid are visible, press ENTER to capture...")
|
||||
_save_dump("user_profile_dump.xml", "Alien Profile Root")
|
||||
|
||||
# 10. Followers List
|
||||
input("\n👉 10. FOLLOWERS LIST:\nFrom that profile (or your own), tap the 'Followers' (Abonnenten) count.\nWhen the list of followers is visible, press ENTER to capture...")
|
||||
_save_dump("followers_list_dump.xml", "Followers List Iteration View")
|
||||
|
||||
# 11. Carousel Post
|
||||
input("\n👉 11. CAROUSEL POST:\nScroll your Feed until you see a Carousel (a post with multiple swipable images/videos).\nWhen it is visible, press ENTER to capture...")
|
||||
_save_dump("carousel_post_dump.xml", "Carousel Post Wrapper")
|
||||
|
||||
# 12. Sponsored Post / Ad
|
||||
input("\n👉 12. SPONSORED AD:\nScroll your Feed or Stories until you see a Sponsored / Gesponsert Post with an action button.\nWhen the Ad is visible, press ENTER to capture...")
|
||||
_save_dump("home_feed_with_ad.xml", "Sponsored Ad Post")
|
||||
|
||||
# 13. Inside DM Chat
|
||||
input("\n👉 13. DM CHAT THREAD:\nOpen any message thread in your DM inbox.\nWhen the chat messages and text input field are visible, press ENTER to capture...")
|
||||
_save_dump("dm_thread_dump.xml", "Direct Message Chat Thread")
|
||||
|
||||
print("\n" + "="*50)
|
||||
logger.info("🎉 Capture Sequence Complete! All 13 E2E dumps have been placed into tests/fixtures/")
|
||||
print("="*50 + "\n")
|
||||
|
||||
except KeyboardInterrupt:
|
||||
print("\n")
|
||||
logger.info("🛑 Capture Sequence Interrupted by User.")
|
||||
except Exception as e:
|
||||
logger.error(f"💥 Capture Sequence crashed: {e}", exc_info=True)
|
||||
287
GramAddict/core/evolution_engine.py
Normal file
287
GramAddict/core/evolution_engine.py
Normal file
@@ -0,0 +1,287 @@
|
||||
"""
|
||||
Evolution Engine — Autonomous Parameter Tuning via Genetic Algorithm.
|
||||
|
||||
Instead of hardcoded behavioral parameters (scroll probability, boredom decay,
|
||||
resonance thresholds), this engine EVOLVES them based on real session outcomes.
|
||||
|
||||
Inspired by Tesla's real-time neural network weight updates from fleet data:
|
||||
- Each session is a "generation"
|
||||
- Session outcomes (follows gained, blocks, duration) determine "fitness"
|
||||
- Winning parameters are preserved; losing parameters are mutated
|
||||
- Hard safety bounds prevent the bot from evolving into dangerous territory
|
||||
|
||||
All parameters persist in Qdrant, surviving restarts.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import random
|
||||
import time
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from typing import Any
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ── Hard Safety Bounds ──
|
||||
# These are absolute limits that CANNOT be exceeded through evolution.
|
||||
# Think of them as the "physics" constraints of the system.
|
||||
SAFETY_BOUNDS = {
|
||||
"scroll_correction_probability": (0.05, 0.35), # Never below 5%, never above 35%
|
||||
"boredom_decay_rate": (0.05, 0.5), # How fast boredom accumulates
|
||||
"resonance_threshold": (0.3, 0.9), # Content quality filter
|
||||
"interaction_cooldown_seconds": (1.0, 10.0), # Min pause between interactions
|
||||
"max_follows_per_session": (5, 40), # Absolute follow cap
|
||||
"max_likes_per_session": (10, 80), # Absolute like cap
|
||||
"session_duration_target_minutes": (15, 120), # Session length target
|
||||
"story_view_probability": (0.1, 0.8), # How often to view stories
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Genome:
|
||||
"""
|
||||
The bot's behavioral DNA — a set of evolvable parameters.
|
||||
Each parameter has a current value and respects hard safety bounds.
|
||||
"""
|
||||
|
||||
scroll_correction_probability: float = 0.15
|
||||
boredom_decay_rate: float = 0.2
|
||||
resonance_threshold: float = 0.7
|
||||
interaction_cooldown_seconds: float = 2.5
|
||||
max_follows_per_session: int = 15
|
||||
max_likes_per_session: int = 30
|
||||
session_duration_target_minutes: float = 45.0
|
||||
story_view_probability: float = 0.4
|
||||
|
||||
# Metadata
|
||||
generation: int = 0
|
||||
best_fitness: float = 0.0
|
||||
last_updated: float = field(default_factory=time.time)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, d: dict) -> "Genome":
|
||||
# Filter out unknown keys for forward-compatibility
|
||||
known = {f.name for f in cls.__dataclass_fields__.values()}
|
||||
filtered = {k: v for k, v in d.items() if k in known}
|
||||
return cls(**filtered)
|
||||
|
||||
|
||||
@dataclass
|
||||
class SessionResult:
|
||||
"""
|
||||
Outcome metrics from a completed session.
|
||||
Used to calculate fitness for the current genome.
|
||||
"""
|
||||
|
||||
follows_gained: int = 0
|
||||
likes_given: int = 0
|
||||
stories_viewed: int = 0
|
||||
blocks_received: int = 0
|
||||
duration_minutes: float = 0.0
|
||||
prediction_error_rate: float = 0.0 # From Active Inference
|
||||
profiles_scraped: int = 0
|
||||
|
||||
|
||||
class EvolutionEngine:
|
||||
"""
|
||||
Genetic algorithm for behavioral parameter optimization.
|
||||
|
||||
Lifecycle:
|
||||
1. Load genome from Qdrant (or use defaults)
|
||||
2. Bot uses genome parameters during session
|
||||
3. After session, evaluate fitness
|
||||
4. If fitness improved → lock genome (preserve winning params)
|
||||
5. If fitness decreased → mutate genome (try new params)
|
||||
6. Persist genome to Qdrant
|
||||
"""
|
||||
|
||||
_instance = None
|
||||
|
||||
@classmethod
|
||||
def get_instance(cls, username: str = None) -> "EvolutionEngine":
|
||||
if cls._instance is None:
|
||||
cls._instance = cls(username or "default")
|
||||
return cls._instance
|
||||
|
||||
@classmethod
|
||||
def reset(cls):
|
||||
cls._instance = None
|
||||
|
||||
def __init__(self, username: str):
|
||||
self.username = username
|
||||
self.genome = Genome()
|
||||
self._qdrant_connected = False
|
||||
self._load_genome()
|
||||
|
||||
def _load_genome(self):
|
||||
"""Load persisted genome from Qdrant, or use defaults."""
|
||||
try:
|
||||
from GramAddict.core.qdrant_memory import QdrantBase
|
||||
|
||||
self._db = QdrantBase("evolution_genomes_v1", vector_size=128)
|
||||
|
||||
if not self._db.is_connected:
|
||||
logger.debug("[Evolution] Qdrant not available. Using default genome.")
|
||||
return
|
||||
|
||||
self._qdrant_connected = True
|
||||
|
||||
# Try to recall existing genome
|
||||
vec = self._db._get_embedding(f"genome_{self.username}")
|
||||
if not vec:
|
||||
return
|
||||
|
||||
results = self._db.client.query_points(
|
||||
collection_name=self._db.collection_name,
|
||||
query=vec,
|
||||
limit=1,
|
||||
score_threshold=0.95,
|
||||
).points
|
||||
|
||||
if results:
|
||||
payload = results[0].payload
|
||||
genome_data = payload.get("genome", {})
|
||||
if genome_data:
|
||||
self.genome = Genome.from_dict(genome_data)
|
||||
logger.info(
|
||||
f"🧬 [Evolution] Loaded genome generation {self.genome.generation} "
|
||||
f"(fitness: {self.genome.best_fitness:.3f})"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"[Evolution] Failed to load genome: {e}")
|
||||
|
||||
def _save_genome(self):
|
||||
"""Persist genome to Qdrant."""
|
||||
if not self._qdrant_connected:
|
||||
return
|
||||
|
||||
try:
|
||||
vec = self._db._get_embedding(f"genome_{self.username}")
|
||||
if not vec:
|
||||
return
|
||||
|
||||
self.genome.last_updated = time.time()
|
||||
payload = {
|
||||
"username": self.username,
|
||||
"genome": self.genome.to_dict(),
|
||||
}
|
||||
|
||||
self._db.upsert_point(
|
||||
f"genome_{self.username}",
|
||||
payload,
|
||||
vector=vec,
|
||||
log_success=f"🧬 [Evolution] Saved genome generation {self.genome.generation}",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"[Evolution] Failed to save genome: {e}")
|
||||
|
||||
def compute_fitness(self, result: SessionResult) -> float:
|
||||
"""
|
||||
Computes a fitness score [0.0 - 1.0] from session outcomes.
|
||||
|
||||
Reward:
|
||||
- Follows gained (high value)
|
||||
- Likes given (medium value)
|
||||
- Stories viewed (low value)
|
||||
- Longer sessions (moderate value)
|
||||
|
||||
Penalty:
|
||||
- Blocks received (SEVERE penalty — 50% fitness reduction per block)
|
||||
- High prediction error rate (moderate penalty)
|
||||
"""
|
||||
if result.blocks_received > 0:
|
||||
# Blocks are catastrophic — any genome that triggers a block is unfit
|
||||
block_penalty = 0.5**result.blocks_received
|
||||
logger.warning(
|
||||
f"🧬 [Evolution] BLOCK PENALTY: {result.blocks_received} blocks → "
|
||||
f"fitness multiplier {block_penalty:.3f}"
|
||||
)
|
||||
else:
|
||||
block_penalty = 1.0
|
||||
|
||||
# Normalize outcomes to [0, 1] range
|
||||
follow_score = min(result.follows_gained / 20.0, 1.0) # Cap at 20
|
||||
like_score = min(result.likes_given / 50.0, 1.0) # Cap at 50
|
||||
story_score = min(result.stories_viewed / 20.0, 1.0) # Cap at 20
|
||||
duration_score = min(result.duration_minutes / 60.0, 1.0) # Cap at 60 min
|
||||
|
||||
# Prediction accuracy bonus
|
||||
accuracy_bonus = 1.0 - result.prediction_error_rate
|
||||
|
||||
# Weighted fitness
|
||||
raw_fitness = (
|
||||
follow_score * 0.35 # Follows are most valuable
|
||||
+ like_score * 0.20 # Likes are secondary
|
||||
+ story_score * 0.05 # Stories are minor
|
||||
+ duration_score * 0.15 # Session stability matters
|
||||
+ accuracy_bonus * 0.25 # Prediction accuracy = environmental mastery
|
||||
)
|
||||
|
||||
fitness = raw_fitness * block_penalty
|
||||
fitness = max(0.0, min(1.0, fitness)) # Clamp to [0, 1]
|
||||
|
||||
return round(fitness, 4)
|
||||
|
||||
def evolve(self, result: SessionResult):
|
||||
"""
|
||||
Evaluate session and evolve the genome.
|
||||
|
||||
If fitness improved → lock parameters (exploitation)
|
||||
If fitness decreased → mutate parameters (exploration)
|
||||
"""
|
||||
fitness = self.compute_fitness(result)
|
||||
|
||||
logger.info(
|
||||
f"🧬 [Evolution] Generation {self.genome.generation} fitness: {fitness:.4f} "
|
||||
f"(best: {self.genome.best_fitness:.4f})"
|
||||
)
|
||||
|
||||
if fitness >= self.genome.best_fitness:
|
||||
# ── Exploitation: Lock winning parameters ──
|
||||
logger.info(f"🧬 [Evolution] ✅ Fitness improved! Locking generation {self.genome.generation}.")
|
||||
self.genome.best_fitness = fitness
|
||||
else:
|
||||
# ── Exploration: Mutate parameters ──
|
||||
logger.info(f"🧬 [Evolution] 🔀 Fitness regressed. Mutating for generation {self.genome.generation + 1}.")
|
||||
self._mutate()
|
||||
|
||||
self.genome.generation += 1
|
||||
self._save_genome()
|
||||
|
||||
def _mutate(self, mutation_rate: float = 0.15):
|
||||
"""
|
||||
Mutate genome parameters within safety bounds.
|
||||
|
||||
Each parameter has a `mutation_rate` chance of being modified.
|
||||
Mutations are small (±10-20% of current value) to ensure gradual evolution.
|
||||
"""
|
||||
for param_name, (low, high) in SAFETY_BOUNDS.items():
|
||||
if random.random() > mutation_rate:
|
||||
continue
|
||||
|
||||
current = getattr(self.genome, param_name, None)
|
||||
if current is None:
|
||||
continue
|
||||
|
||||
# Mutation: ±10-20% of range
|
||||
param_range = high - low
|
||||
delta = random.uniform(-0.2, 0.2) * param_range
|
||||
|
||||
new_value = current + delta
|
||||
|
||||
# Clamp to safety bounds
|
||||
if isinstance(current, int):
|
||||
new_value = int(max(low, min(high, round(new_value))))
|
||||
else:
|
||||
new_value = max(low, min(high, new_value))
|
||||
|
||||
old_value = current
|
||||
setattr(self.genome, param_name, new_value)
|
||||
logger.debug(f"🧬 [Mutation] {param_name}: {old_value} → {new_value}")
|
||||
|
||||
def get_param(self, name: str, default: Any = None) -> Any:
|
||||
"""Get a parameter value from the current genome."""
|
||||
return getattr(self.genome, name, default)
|
||||
276
GramAddict/core/goal_decomposer.py
Normal file
276
GramAddict/core/goal_decomposer.py
Normal file
@@ -0,0 +1,276 @@
|
||||
"""
|
||||
GoalDecomposer — Mission-Driven Task Planning
|
||||
|
||||
Translates the bot's `mission` config + `plugins` capabilities into
|
||||
concrete, weighted Task objects. Pure logic — no LLM, no device,
|
||||
no network, no side effects.
|
||||
|
||||
This is the bridge between:
|
||||
- "What does the user WANT?" (mission.strategy)
|
||||
- "What CAN the bot DO?" (enabled plugins + actions)
|
||||
- "What SHOULD it do NOW?" (weighted Task selection)
|
||||
|
||||
Tesla analogy: FSD doesn't have a "goal: drive safely" config.
|
||||
It derives behavior from destination + road rules + sensor capabilities.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import random
|
||||
from dataclasses import dataclass
|
||||
from typing import Dict, List
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ── Strategy Weight Tables ──
|
||||
# Each strategy defines relative weights for screen targets.
|
||||
# Higher weight = more likely to be selected by GrowthBrain.
|
||||
STRATEGY_WEIGHTS: Dict[str, Dict[str, float]] = {
|
||||
"aggressive_growth": {
|
||||
"HomeFeed": 0.15,
|
||||
"ExploreFeed": 0.45,
|
||||
"ReelsFeed": 0.15,
|
||||
"StoriesFeed": 0.10,
|
||||
"MessageInbox": 0.10,
|
||||
"FollowingList": 0.05,
|
||||
},
|
||||
"community_builder": {
|
||||
"HomeFeed": 0.40,
|
||||
"ExploreFeed": 0.10,
|
||||
"ReelsFeed": 0.05,
|
||||
"StoriesFeed": 0.25,
|
||||
"MessageInbox": 0.15,
|
||||
"FollowingList": 0.05,
|
||||
},
|
||||
"passive_learning": {
|
||||
"HomeFeed": 0.20,
|
||||
"ExploreFeed": 0.50,
|
||||
"ReelsFeed": 0.20,
|
||||
"StoriesFeed": 0.05,
|
||||
"MessageInbox": 0.00,
|
||||
"FollowingList": 0.05,
|
||||
},
|
||||
"stealth_lurker": {
|
||||
"HomeFeed": 0.35,
|
||||
"ExploreFeed": 0.25,
|
||||
"ReelsFeed": 0.15,
|
||||
"StoriesFeed": 0.15,
|
||||
"MessageInbox": 0.05,
|
||||
"FollowingList": 0.05,
|
||||
},
|
||||
}
|
||||
|
||||
# ── Plugin → Screen Mapping ──
|
||||
# Which plugins enable which screen targets.
|
||||
# A screen is only viable if at least one enabling plugin is active.
|
||||
# Some plugins work on MULTIPLE screens (likes work on home, explore, reels).
|
||||
PLUGIN_SCREENS_MAP: Dict[str, set] = {
|
||||
"likes": {"HomeFeed", "ExploreFeed", "ReelsFeed"},
|
||||
"comment": {"HomeFeed", "ExploreFeed"},
|
||||
"follow": {"HomeFeed", "ExploreFeed"},
|
||||
"repost": {"HomeFeed", "ExploreFeed"},
|
||||
"profile_visit": {"HomeFeed", "ExploreFeed"},
|
||||
"grid_like": {"HomeFeed"},
|
||||
"carousel_browsing": {"HomeFeed"},
|
||||
"rabbit_hole": {"HomeFeed", "ExploreFeed"},
|
||||
"story_view": {"StoriesFeed"},
|
||||
"dm_reply": {"MessageInbox"},
|
||||
}
|
||||
|
||||
# ── Action → Screen Mapping ──
|
||||
# The `actions:` config section maps directly to screens.
|
||||
ACTION_SCREEN_MAP: Dict[str, str] = {
|
||||
"feed": "HomeFeed",
|
||||
"explore": "ExploreFeed",
|
||||
"reels": "ReelsFeed",
|
||||
}
|
||||
|
||||
# ── Screen → Verb Mapping ──
|
||||
SCREEN_VERB_MAP: Dict[str, str] = {
|
||||
"HomeFeed": "browse_feed",
|
||||
"ExploreFeed": "browse_explore",
|
||||
"ReelsFeed": "browse_reels",
|
||||
"StoriesFeed": "view_stories",
|
||||
"MessageInbox": "check_messages",
|
||||
"FollowingList": "manage_following",
|
||||
}
|
||||
|
||||
# ── Screen → Human Intent ──
|
||||
SCREEN_INTENT_MAP: Dict[str, str] = {
|
||||
"HomeFeed": "Interact with posts in the home feed",
|
||||
"ExploreFeed": "Discover and engage with new content",
|
||||
"ReelsFeed": "Browse and interact with reels",
|
||||
"StoriesFeed": "View and react to stories",
|
||||
"MessageInbox": "Reply to unread direct messages",
|
||||
"FollowingList": "Review and manage following list",
|
||||
}
|
||||
|
||||
DEFAULT_BUDGET = 5
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Task:
|
||||
"""A concrete, executable unit of work for the bot.
|
||||
|
||||
Unlike abstract goals ("nurture community"), a Task has:
|
||||
- A specific screen to navigate to
|
||||
- A measurable budget (how many posts/items to process)
|
||||
- A weight for probabilistic selection
|
||||
- A human-readable intent for logging
|
||||
"""
|
||||
|
||||
verb: str
|
||||
target_screen: str
|
||||
intent: str
|
||||
budget_posts: int
|
||||
weight: float
|
||||
|
||||
|
||||
class GoalDecomposer:
|
||||
"""Translates mission + plugins → weighted Task list.
|
||||
|
||||
Pure logic, zero side effects. Call generate_tasks() to get
|
||||
the bot's action menu for the current session.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
plugins: Dict[str, dict],
|
||||
actions: Dict[str, str],
|
||||
mission: Dict[str, str],
|
||||
):
|
||||
self._plugins = plugins
|
||||
self._actions = actions
|
||||
self._strategy = mission.get("strategy", "aggressive_growth")
|
||||
|
||||
def generate_tasks(self) -> List[Task]:
|
||||
"""Generate weighted tasks from config.
|
||||
|
||||
Returns an empty list if no plugins are enabled —
|
||||
the bot literally has nothing to do.
|
||||
"""
|
||||
viable_screens = self._discover_viable_screens()
|
||||
if not viable_screens:
|
||||
return []
|
||||
|
||||
strategy_weights = STRATEGY_WEIGHTS.get(self._strategy, STRATEGY_WEIGHTS["aggressive_growth"])
|
||||
|
||||
tasks = []
|
||||
for screen in viable_screens:
|
||||
weight = strategy_weights.get(screen, 0.1)
|
||||
if weight <= 0:
|
||||
continue
|
||||
|
||||
budget = self._budget_for_screen(screen)
|
||||
verb = SCREEN_VERB_MAP.get(screen, "browse")
|
||||
intent = SCREEN_INTENT_MAP.get(screen, f"Interact on {screen}")
|
||||
|
||||
tasks.append(
|
||||
Task(
|
||||
verb=verb,
|
||||
target_screen=screen,
|
||||
intent=intent,
|
||||
budget_posts=budget,
|
||||
weight=weight,
|
||||
)
|
||||
)
|
||||
|
||||
return tasks
|
||||
|
||||
def _discover_viable_screens(self) -> set:
|
||||
"""Determine which screens the bot can meaningfully interact on.
|
||||
|
||||
A screen is viable if it has BOTH:
|
||||
1. A route (action config or plugin-implied), AND
|
||||
2. At least one active plugin that can DO something there.
|
||||
|
||||
Without an active plugin, navigating to a screen is pointless —
|
||||
the bot would just scroll with nothing to interact on.
|
||||
"""
|
||||
# 1. Collect screens with active plugins
|
||||
plugin_screens: set = set()
|
||||
for plugin_name, screens in PLUGIN_SCREENS_MAP.items():
|
||||
plugin_cfg = self._plugins.get(plugin_name, {})
|
||||
if not plugin_cfg:
|
||||
continue
|
||||
if not self._is_plugin_active(plugin_cfg):
|
||||
continue
|
||||
plugin_screens.update(screens)
|
||||
|
||||
# 2. Screens from actions are only viable if plugins exist for them
|
||||
action_screens: set = set()
|
||||
for action_key, screen in ACTION_SCREEN_MAP.items():
|
||||
if action_key in self._actions and self._actions[action_key]:
|
||||
action_screens.add(screen)
|
||||
|
||||
# 3. A screen must have plugin coverage to be viable
|
||||
# Action-enabled screens need at least one active plugin
|
||||
viable = action_screens & plugin_screens
|
||||
|
||||
# 4. Plugin-only screens (story_view, dm_reply) are viable
|
||||
# even without an explicit action config
|
||||
viable |= plugin_screens
|
||||
|
||||
return viable
|
||||
|
||||
def _is_plugin_active(self, plugin_cfg: dict) -> bool:
|
||||
"""Check if a plugin config represents an active plugin.
|
||||
|
||||
A plugin is active if:
|
||||
- It has `enabled: true` (explicit), OR
|
||||
- It has `percentage` > 0 (implicit enable), OR
|
||||
- It has any config keys and `enabled` is not explicitly False
|
||||
"""
|
||||
# Explicit disable
|
||||
if plugin_cfg.get("enabled") is False:
|
||||
return False
|
||||
|
||||
# Explicit enable
|
||||
if plugin_cfg.get("enabled") is True:
|
||||
return True
|
||||
|
||||
# Percentage-based: 0% means disabled
|
||||
pct = plugin_cfg.get("percentage")
|
||||
if pct is not None:
|
||||
try:
|
||||
return float(pct) > 0
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
|
||||
# Has config keys but no explicit enabled/percentage = active
|
||||
return bool(plugin_cfg)
|
||||
|
||||
def _budget_for_screen(self, screen: str) -> int:
|
||||
"""Determine the post budget for a screen.
|
||||
|
||||
Reads from actions config (e.g. feed: "5-10") and parses
|
||||
the range string into a random integer within bounds.
|
||||
"""
|
||||
# Map screen back to action key
|
||||
reverse_map = {v: k for k, v in ACTION_SCREEN_MAP.items()}
|
||||
action_key = reverse_map.get(screen)
|
||||
|
||||
if action_key and action_key in self._actions:
|
||||
return _parse_range(self._actions[action_key])
|
||||
|
||||
# Special screens get fixed budgets from plugin config
|
||||
if screen == "StoriesFeed":
|
||||
story_cfg = self._plugins.get("story_view", {})
|
||||
count_str = story_cfg.get("count", "1-3")
|
||||
return _parse_range(str(count_str))
|
||||
|
||||
if screen == "MessageInbox":
|
||||
return DEFAULT_BUDGET
|
||||
|
||||
return DEFAULT_BUDGET
|
||||
|
||||
|
||||
def _parse_range(range_str: str) -> int:
|
||||
"""Parse a range string like '5-10' into a random int within bounds."""
|
||||
try:
|
||||
if "-" in str(range_str):
|
||||
parts = str(range_str).split("-")
|
||||
low, high = int(parts[0]), int(parts[1])
|
||||
return random.randint(low, high)
|
||||
return int(range_str)
|
||||
except (ValueError, IndexError):
|
||||
return DEFAULT_BUDGET
|
||||
564
GramAddict/core/goap.py
Normal file
564
GramAddict/core/goap.py
Normal file
@@ -0,0 +1,564 @@
|
||||
"""
|
||||
Goal-Oriented Action Planner (GOAP)
|
||||
|
||||
The bot's autonomous brain. Replaces ALL hardcoded navigation with
|
||||
goal-driven behavior. The bot perceives the screen, understands where
|
||||
it is, plans what to do next, executes, verifies, and learns.
|
||||
|
||||
Like a GPS navigation system:
|
||||
- You tell it WHERE you want to go (goal)
|
||||
- It figures out the route (plan)
|
||||
- It guides you step by step (execute)
|
||||
- It reroutes if you take a wrong turn (recover)
|
||||
- It remembers shortcuts (learn)
|
||||
"""
|
||||
|
||||
import logging
|
||||
import time
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from GramAddict.core.navigation.knowledge import NavigationKnowledge
|
||||
from GramAddict.core.navigation.path_memory import PathMemory
|
||||
from GramAddict.core.navigation.planner import GoalPlanner
|
||||
from GramAddict.core.perception.context_gate import ContextGate
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity, ScreenType
|
||||
from GramAddict.core.utils import random_sleep
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Re-export for backward compatibility (optional but helps minimize import breakage)
|
||||
__all__ = ["GoalExecutor", "ScreenIdentity", "ScreenType", "PathMemory", "NavigationKnowledge", "GoalPlanner"]
|
||||
|
||||
# ══════════════════════════════════════════════════════
|
||||
# GOAL EXECUTOR — The Main Brain Loop
|
||||
# ══════════════════════════════════════════════════════
|
||||
|
||||
|
||||
class GoalExecutor:
|
||||
"""
|
||||
The autonomous brain. Achieves goals through perceive→plan→execute→verify→learn.
|
||||
|
||||
Usage:
|
||||
goap = GoalExecutor(device, bot_username="marisaundmarc")
|
||||
goap.achieve("like a post from explore")
|
||||
"""
|
||||
|
||||
_instance = None
|
||||
global_start_time = None
|
||||
global_max_runtime_minutes = None
|
||||
|
||||
@classmethod
|
||||
def get_instance(cls, device=None, bot_username=""):
|
||||
if cls._instance is None:
|
||||
cls._instance = cls(device, bot_username)
|
||||
elif device is not None:
|
||||
cls._instance.device = device
|
||||
return cls._instance
|
||||
|
||||
@classmethod
|
||||
def reset(cls):
|
||||
"""Reset the singleton instance."""
|
||||
cls._instance = None
|
||||
|
||||
def __init__(self, device, bot_username: str = ""):
|
||||
self.device = device
|
||||
self.username = bot_username
|
||||
self.screen_id = ScreenIdentity(bot_username)
|
||||
self.screen_id.device = device
|
||||
self.planner = GoalPlanner(bot_username)
|
||||
self.path_memory = PathMemory(bot_username)
|
||||
self.context_gate = ContextGate()
|
||||
self.max_steps = 15 # Safety: never execute more than 15 steps
|
||||
self._sae = None # Lazy-loaded, injectable for tests
|
||||
self.action_failures = {} # Tracking for failed actions in current goal session
|
||||
|
||||
def _get_sae(self):
|
||||
"""Get or create the SAE instance. Injectable for tests."""
|
||||
if self._sae is None:
|
||||
from GramAddict.core.situational_awareness import SituationalAwarenessEngine
|
||||
|
||||
self._sae = SituationalAwarenessEngine.get_instance(self.device)
|
||||
return self._sae
|
||||
|
||||
def perceive(self, xml_dump: str = None) -> Dict[str, Any]:
|
||||
"""Perceive the current screen state."""
|
||||
if xml_dump is None:
|
||||
xml_dump = self.device.dump_hierarchy()
|
||||
return self.screen_id.identify(xml_dump)
|
||||
|
||||
def achieve(self, goal: str, max_steps: int = None) -> bool:
|
||||
"""
|
||||
Main entry point. Achieves a goal autonomously.
|
||||
|
||||
Args:
|
||||
goal: Natural language goal like "like a post from explore"
|
||||
max_steps: Maximum steps before giving up
|
||||
|
||||
Returns:
|
||||
True if goal achieved, False if failed
|
||||
"""
|
||||
if max_steps is None:
|
||||
max_steps = self.max_steps
|
||||
|
||||
logger.info(f"🎯 [GOAP] Pursuing goal: '{goal}'")
|
||||
self.action_failures.clear()
|
||||
|
||||
# ── Try recalled path first ──
|
||||
screen = self.perceive()
|
||||
start_screen = screen["screen_type"].value
|
||||
recalled = self.path_memory.recall_path(goal, start_screen)
|
||||
|
||||
if recalled:
|
||||
logger.info(f"🧠 [GOAP] Using memorized path ({len(recalled)} steps)")
|
||||
success = self._execute_recalled_path(recalled, goal)
|
||||
if success:
|
||||
return True
|
||||
logger.warning("🧠 [GOAP] Memorized path failed. Falling back to live planning...")
|
||||
|
||||
# ── Live planning ──
|
||||
steps_taken = []
|
||||
last_action = None
|
||||
last_screen_type = None
|
||||
consecutive_back_presses = 0
|
||||
MAX_CONSECUTIVE_BACK = 3
|
||||
explored_nav_actions = set()
|
||||
visited_screens = set()
|
||||
for step_num in range(max_steps):
|
||||
# ── Global Hard Kill Check ──
|
||||
max_rt = GoalExecutor.global_max_runtime_minutes
|
||||
start_time = GoalExecutor.global_start_time
|
||||
if max_rt and start_time:
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
if datetime.now() - start_time > timedelta(minutes=max_rt):
|
||||
logger.error(
|
||||
f"🛑 [Timeout] Maximum runtime of {max_rt} minutes reached during GOAP execution. Hard stopping planner.",
|
||||
extra={"color": "\\033[31m"},
|
||||
)
|
||||
return False
|
||||
|
||||
# PERCEIVE
|
||||
screen = self.perceive()
|
||||
screen_type = screen["screen_type"]
|
||||
visited_screens.add(screen_type)
|
||||
|
||||
if last_screen_type and screen_type != last_screen_type:
|
||||
logger.debug(
|
||||
f"📍 [GOAP State] Screen transitioned from {last_screen_type.name} to {screen_type.name}. Clearing explored actions."
|
||||
)
|
||||
explored_nav_actions.clear()
|
||||
consecutive_back_presses = 0 # Progress was made
|
||||
|
||||
# ── Loop Prevention: Mask Failed Actions ──
|
||||
MAX_RETRIES = 2
|
||||
original_available = screen.get("available_actions", []).copy()
|
||||
masked_available = []
|
||||
for act in original_available:
|
||||
fail_count = self.action_failures.get((screen_type, act), 0)
|
||||
if fail_count >= MAX_RETRIES:
|
||||
logger.warning(
|
||||
f"🚫 [GOAP] Masking action '{act}' due to {fail_count} consecutive failures to prevent loops."
|
||||
)
|
||||
else:
|
||||
masked_available.append(act)
|
||||
screen["available_actions"] = masked_available
|
||||
|
||||
logger.debug(
|
||||
f"📍 [GOAP Step {step_num + 1}] Goal: '{goal}' | On: {screen_type.value} | "
|
||||
f"Available: {screen.get('available_actions', [])[:5]}"
|
||||
)
|
||||
|
||||
# Handle obstacles
|
||||
if screen_type == ScreenType.FOREIGN_APP or screen_type == ScreenType.MODAL:
|
||||
obstacle_name = "Foreign app" if screen_type == ScreenType.FOREIGN_APP else "Modal"
|
||||
logger.warning(f"🚨 [GOAP] {obstacle_name} detected. Using SAE to clear...")
|
||||
|
||||
# SAE Feedback Loop!
|
||||
# If we hit this, the LAST action caused an obstacle! Mask it!
|
||||
if last_action and last_screen_type:
|
||||
self.action_failures[(last_screen_type, last_action)] = (
|
||||
self.action_failures.get((last_screen_type, last_action), 0) + MAX_RETRIES
|
||||
) # Instantly mask it for this session
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
|
||||
if ScreenTopology.is_structural_action(last_screen_type, last_action):
|
||||
logger.warning(
|
||||
f"🛡️ [SAE Feedback] Structural action '{last_action}' caused an obstacle. "
|
||||
f"Masking for this session. (Never burned permanently)"
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
f"🛡️ [SAE Feedback] Content action '{last_action}' caused an obstacle. "
|
||||
f"Masking for this session to break loop, but preventing permanent Qdrant poisoning."
|
||||
)
|
||||
# We specifically DO NOT call self.planner.knowledge.learn_trap here anymore!
|
||||
# Burning dynamic actions like "tap follow button" permanently destroys the bot's capabilities across sessions.
|
||||
|
||||
if not self._get_sae().ensure_clear_screen():
|
||||
if screen_type == ScreenType.FOREIGN_APP:
|
||||
self.path_memory.learn_path(goal, start_screen, steps_taken, False)
|
||||
return False
|
||||
continue
|
||||
|
||||
# PLAN
|
||||
action = self.planner.plan_next_step(
|
||||
goal,
|
||||
screen,
|
||||
explored_nav_actions=explored_nav_actions,
|
||||
action_failures=self.action_failures,
|
||||
visited_screens=visited_screens,
|
||||
)
|
||||
|
||||
if action is None:
|
||||
# Goal achieved!
|
||||
logger.info(f"✅ [GOAP] Goal '{goal}' achieved in {step_num} steps!")
|
||||
self.path_memory.learn_path(goal, start_screen, steps_taken, True)
|
||||
|
||||
# Record dynamic knowledge: This goal lands us on THIS screen
|
||||
self.planner.knowledge.learn_goal_requirement(goal, screen_type)
|
||||
return True
|
||||
|
||||
logger.info(f"🧭 [GOAP Step {step_num + 1}] Action: '{action}'")
|
||||
last_action = action
|
||||
last_screen_type = screen_type
|
||||
|
||||
# EXECUTE
|
||||
success = self._execute_action(action, goal=goal, screen_state=screen)
|
||||
|
||||
if success:
|
||||
steps_taken.append({"action": action})
|
||||
|
||||
if action == "force start instagram":
|
||||
logger.info("🔄 [GOAP State] App restarted. Purging memory/traps to attempt fresh routing.")
|
||||
self.action_failures.clear()
|
||||
explored_nav_actions.clear()
|
||||
visited_screens.clear()
|
||||
consecutive_back_presses = 0
|
||||
# CRITICAL: Also clear the planner's learned traps.
|
||||
# Without this, traps learned before restart persist and
|
||||
# immediately re-trap the bot on the same (or similar) screen.
|
||||
if hasattr(self, "planner") and hasattr(self.planner, "knowledge"):
|
||||
self.planner.knowledge.clear_traps()
|
||||
continue
|
||||
|
||||
# Check if it was a navigation action (vs a goal action). If we are not on the required screen,
|
||||
# any action taken is essentially a navigation attempt.
|
||||
explored_nav_actions.add(action)
|
||||
# Reset failures for this action since it eventually succeeded
|
||||
self.action_failures[(screen_type, action)] = 0
|
||||
|
||||
if "scroll" in action.lower():
|
||||
logger.debug(
|
||||
"📍 [GOAP State] Scrolled successfully. Clearing explored actions to allow retrying off-screen elements."
|
||||
)
|
||||
explored_nav_actions.clear()
|
||||
# Keep action_failures for synthetic intents, but clear them for structural actions
|
||||
# so that the HD Map can retry route actions that might now be visible!
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
|
||||
keys_to_clear = [
|
||||
k
|
||||
for k in self.action_failures.keys()
|
||||
if k[0] == screen_type and ScreenTopology.is_structural_action(screen_type, k[1])
|
||||
]
|
||||
for k in keys_to_clear:
|
||||
del self.action_failures[k]
|
||||
|
||||
# ── Back-Press Circuit Breaker → Escalation ──
|
||||
if action == "press back":
|
||||
consecutive_back_presses += 1
|
||||
if consecutive_back_presses >= MAX_CONSECUTIVE_BACK:
|
||||
logger.warning(
|
||||
f"🛑 [GOAP] Back-pressed {MAX_CONSECUTIVE_BACK} times with no screen transition. "
|
||||
f"Escalating to force restart."
|
||||
)
|
||||
|
||||
# Unlearn the trap path
|
||||
from GramAddict.core.qdrant_memory import NavigationMemoryDB
|
||||
|
||||
if len(steps_taken) > consecutive_back_presses:
|
||||
last_real_action = steps_taken[-consecutive_back_presses - 1]["action"]
|
||||
logger.debug(
|
||||
f"[GOAP Unlearn] last_real_action={last_real_action}, " f"start_screen={start_screen}"
|
||||
)
|
||||
NavigationMemoryDB().unlearn_transition(start_screen, last_real_action)
|
||||
|
||||
# ── ESCALATION: Force restart instead of aborting ──
|
||||
app_id = getattr(self.device, "app_id", "com.instagram.android")
|
||||
self.device.app_start(app_id, use_monkey=True)
|
||||
random_sleep(2.0, 3.5)
|
||||
steps_taken.append({"action": "force start instagram"})
|
||||
|
||||
logger.info("🔄 [GOAP Escalation] App restarted. Purging all failure state for fresh attempt.")
|
||||
self.action_failures.clear()
|
||||
explored_nav_actions.clear()
|
||||
visited_screens.clear()
|
||||
consecutive_back_presses = 0
|
||||
continue
|
||||
else:
|
||||
consecutive_back_presses = 0
|
||||
else:
|
||||
self.action_failures[(screen_type, action)] = self.action_failures.get((screen_type, action), 0) + 1
|
||||
# Track failed actions in explored_nav_actions so the planner
|
||||
# knows NOT to return the same synthetic intent again.
|
||||
# Without this, synthetic intents (not in available_actions)
|
||||
# bypass the masking logic and loop forever.
|
||||
explored_nav_actions.add(action)
|
||||
|
||||
if self.action_failures[(screen_type, action)] >= MAX_RETRIES:
|
||||
# ── Topology Guard: Never poison structural HD Map actions ──
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
|
||||
if ScreenTopology.is_structural_action(screen_type, action):
|
||||
logger.warning(
|
||||
f"🛡️ [Topology Guard] NOT burning structural action '{action}' — "
|
||||
f"it's in the HD Map. VLM may have failed, but the route is valid."
|
||||
)
|
||||
else:
|
||||
self.planner.knowledge.learn_trap(screen_type, action, "repeated_failure_or_null_action")
|
||||
logger.error(
|
||||
f"💀 [GOAP Execute] Action '{action}' failed {MAX_RETRIES} times. Marked as permanent trap."
|
||||
)
|
||||
else:
|
||||
logger.warning(f"⚠️ [GOAP Execute] Action '{action}' failed. Continuing with replanning...")
|
||||
|
||||
random_sleep(0.5, 1.5)
|
||||
|
||||
logger.warning(f"⚠️ [GOAP] Goal '{goal}' failed after {max_steps} steps.")
|
||||
self.path_memory.learn_path(goal, start_screen, steps_taken, False)
|
||||
|
||||
# Memory Purge Logic: Wipe the path memory for this start_screen/goal combo
|
||||
# so it doesn't get stuck in a broken loop in future sessions!
|
||||
self.path_memory.forget_path(goal, start_screen)
|
||||
logger.warning(
|
||||
f"🧹 [Memory Purge] Wiped PathMemory cache for '{goal}' starting at '{start_screen}' to force re-discovery."
|
||||
)
|
||||
|
||||
return False
|
||||
|
||||
def _execute_action(self, action: str, goal: str = None, screen_state: dict = None) -> bool:
|
||||
"""Execute a single natural-language action using the TelepathicEngine."""
|
||||
|
||||
if action == "press back":
|
||||
self.device.press("back")
|
||||
random_sleep(0.8, 1.5)
|
||||
return True
|
||||
|
||||
if action == "scroll down":
|
||||
# Swipe up to scroll down
|
||||
self.device.swipe(540, 1600, 540, 800, duration=0.3)
|
||||
random_sleep(1.0, 2.0)
|
||||
return True
|
||||
|
||||
if action == "force start instagram":
|
||||
app_id = getattr(self.device, "app_id", "com.instagram.android")
|
||||
self.device.app_start(app_id, use_monkey=True)
|
||||
random_sleep(2.0, 3.5)
|
||||
return True
|
||||
|
||||
# ── P1-5: Context Gate ──
|
||||
# Check if the intent is structurally plausible on THIS screen before calling VLM
|
||||
if screen_state and not self.context_gate.is_allowed(action, screen_state):
|
||||
logger.warning(f"🛡️ [GOAP Execute] Action '{action}' blocked by ContextGate for this screen.")
|
||||
return False
|
||||
|
||||
# Use TelepathicEngine for any semantic click
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
engine = TelepathicEngine.get_instance()
|
||||
|
||||
xml_dump = self.device.dump_hierarchy()
|
||||
best_node = engine.find_best_node(xml_dump, action, min_confidence=0.75, device=self.device, goal=goal)
|
||||
|
||||
if not best_node or best_node.get("skip"):
|
||||
logger.warning(f"⚠️ [GOAP Execute] TelepathicEngine found nothing for '{action}'")
|
||||
return best_node.get("skip", False) if best_node else False
|
||||
|
||||
if best_node.get("blocked_by_modal"):
|
||||
logger.warning(f"🛡️ [GOAP Execute] Action '{action}' is blocked by an active modal. Aborting click.")
|
||||
# Let SAE clear the screen anomaly autonomously
|
||||
self._get_sae().ensure_clear_screen(max_attempts=3)
|
||||
return False
|
||||
|
||||
# Execute click
|
||||
self.device.click(obj=best_node)
|
||||
import random
|
||||
|
||||
time.sleep(random.uniform(1.6, 2.8))
|
||||
|
||||
# Verify success via Goal Context + Screen Feedback
|
||||
post_xml = self.device.dump_hierarchy()
|
||||
pre_action_screen = self.perceive(xml_dump) # Screen state BEFORE the click
|
||||
post_screen = self.perceive(post_xml)
|
||||
post_screen_type = post_screen["screen_type"]
|
||||
pre_action_screen_type = pre_action_screen["screen_type"]
|
||||
|
||||
# Determine if this was a navigation or an interaction
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
|
||||
is_navigation = any(k in action.lower() for k in ["tab", "open", "go to", "navigate", "following list"])
|
||||
if not is_navigation:
|
||||
is_navigation = ScreenTopology.is_structural_action(pre_action_screen_type, action)
|
||||
action_success = False
|
||||
|
||||
# ── UI Change Detection with Noise Threshold ──
|
||||
# Raw string diffs of < 50 bytes are noise (timestamps, whitespace, counters).
|
||||
# A real navigation changes the XML by hundreds/thousands of bytes.
|
||||
MIN_UI_CHANGE_BYTES = 50
|
||||
xml_delta = abs(len(post_xml) - len(xml_dump))
|
||||
ui_changed = post_xml != xml_dump and xml_delta >= MIN_UI_CHANGE_BYTES
|
||||
logger.debug(
|
||||
f"[GOAP Verify] ui_changed={ui_changed}, "
|
||||
f"xml_len_pre={len(xml_dump)}, xml_len_post={len(post_xml)}, delta={xml_delta}b"
|
||||
)
|
||||
|
||||
if is_navigation:
|
||||
if ui_changed:
|
||||
# ── Step-Aware Navigation Validation (SSOT) ──
|
||||
# Validates against the EXPECTED screen for THIS ACTION, not the final goal.
|
||||
# This enables multi-step routes: "tap profile tab" should land on
|
||||
# OWN_PROFILE (intermediate), even if the goal is "open following list".
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
|
||||
expected_step_screen = ScreenTopology.expected_screen_for_action(action, pre_action_screen_type)
|
||||
|
||||
if expected_step_screen:
|
||||
if post_screen_type == expected_step_screen:
|
||||
action_success = True
|
||||
logger.info(f"✅ [GOAP Step] '{action}' → {post_screen_type.name} (matches HD Map)")
|
||||
self.planner.knowledge.learn_screen_mapping(action, post_screen_type)
|
||||
else:
|
||||
logger.warning(
|
||||
f"❌ [GOAP Step] '{action}' expected {expected_step_screen.name}, "
|
||||
f"got {post_screen_type.name}. Rejecting."
|
||||
)
|
||||
action_success = False
|
||||
# Unlearn: purge poisoned Qdrant vectors for this transition
|
||||
# so the memory doesn't reinforce a broken path in future sessions
|
||||
try:
|
||||
from GramAddict.core.qdrant_memory import NavigationMemoryDB
|
||||
|
||||
NavigationMemoryDB().unlearn_transition(pre_action_screen_type.value, action)
|
||||
logger.info(
|
||||
f"🧹 [Unlearn] Purged Qdrant vector: " f"{pre_action_screen_type.value} → '{action}'"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"Unlearn failed (non-critical): {e}")
|
||||
else:
|
||||
# Unknown action (not in HD Map) — accept any screen change as progress
|
||||
action_success = True
|
||||
logger.info(f"✅ [GOAP Step] Navigation '{action}' → {post_screen_type.name} (discovery).")
|
||||
self.planner.knowledge.learn_screen_mapping(action, post_screen_type)
|
||||
else:
|
||||
# Check if we're already on the target screen (no-op is OK)
|
||||
goal_lower = (goal or "").lower()
|
||||
already_there = self.planner._is_goal_achieved(
|
||||
goal_lower, post_screen_type, post_screen.get("context", {})
|
||||
)
|
||||
if already_there:
|
||||
logger.info(
|
||||
f"✅ [GOAP Step] No UI change — but goal '{goal}' is already achieved on {post_screen_type.name}. Not punishing."
|
||||
)
|
||||
action_success = True
|
||||
else:
|
||||
logger.warning(f"❌ [GOAP Step] No UI change detected after '{action}'.")
|
||||
action_success = False
|
||||
else:
|
||||
# For interactions (like, follow) or unknown goals, use XML delta + semantic verify
|
||||
# REGRESSION FIX 2026-05-01: Toggle actions (like/save) produce tiny XML deltas
|
||||
# (e.g. checked="false" → "true" = 1 byte). We must NOT gate interactions on
|
||||
# MIN_UI_CHANGE_BYTES. ANY change at all warrants semantic verification.
|
||||
interaction_xml_changed = post_xml != xml_dump
|
||||
if post_screen_type == ScreenType.FOREIGN_APP:
|
||||
logger.error(
|
||||
f"❌ [GOAP Verify] Interaction '{action}' caused navigation to FOREIGN_APP (e.g. Play Store). Rejecting as catastrophic failure."
|
||||
)
|
||||
action_success = False
|
||||
elif interaction_xml_changed:
|
||||
score = best_node.get("score", 0.0) if best_node else 0.0
|
||||
verification = engine.verify_success(action, post_xml, device=self.device, confidence=score)
|
||||
if verification is True:
|
||||
action_success = True
|
||||
logger.info(f"✅ [GOAP Step] Interaction '{action}' successful.")
|
||||
elif verification is False:
|
||||
logger.warning(f"❌ [GOAP Verify] Semantic verification failed for '{action}'.")
|
||||
action_success = False
|
||||
elif verification is None:
|
||||
logger.warning(
|
||||
f"⚠️ [GOAP Verify] Semantic verification INCONCLUSIVE for '{action}'. Will not blacklist."
|
||||
)
|
||||
action_success = None
|
||||
else:
|
||||
logger.warning(f"❌ [GOAP Verify] No UI change detected after interaction '{action}'.")
|
||||
action_success = None # Inconclusive if UI didn't change at all
|
||||
|
||||
# Optional: Log if the overarching goal was met during this step
|
||||
if goal and action_success:
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
|
||||
goal_target = ScreenTopology.goal_to_target_screen(goal.lower() if goal else "")
|
||||
if goal_target and post_screen_type == goal_target:
|
||||
logger.info(f"🎉 [GOAP Verify] OVERARCHING Goal '{goal}' achieved during step '{action}'.")
|
||||
if action_success is True:
|
||||
engine.confirm_click(action)
|
||||
return True
|
||||
elif action_success is False:
|
||||
engine.reject_click(action)
|
||||
return False
|
||||
else:
|
||||
# action_success is None (INCONCLUSIVE)
|
||||
logger.warning(f"⚠️ [GOAP Execute] Applying AGGRESSIVE PENALTY for inconclusive action '{action}'.")
|
||||
engine.decay_click(action)
|
||||
# Double penalty to burn ambiguous paths faster (outer loop adds +1, so total +2 = instantly hits MAX_RETRIES)
|
||||
self.action_failures[(pre_action_screen_type, action)] = (
|
||||
self.action_failures.get((pre_action_screen_type, action), 0) + 1
|
||||
)
|
||||
return False
|
||||
|
||||
def _execute_recalled_path(self, steps: List[Dict], goal: str) -> bool:
|
||||
"""Execute a memorized path."""
|
||||
# Pre-check: Is the goal already met? Don't execute stale paths.
|
||||
screen = self.perceive()
|
||||
if self.planner.plan_next_step(goal, screen) is None:
|
||||
logger.info(f"🎯 [GOAP Recall] Goal '{goal}' already achieved. Skipping recalled path.")
|
||||
return True
|
||||
|
||||
for i, step in enumerate(steps):
|
||||
action = step.get("action", "")
|
||||
logger.info(f"🧠 [GOAP Recall Step {i + 1}/{len(steps)}] '{action}'")
|
||||
|
||||
# Re-perceive to ensure ContextGate has fresh data
|
||||
current_screen = self.perceive()
|
||||
|
||||
# Guard: Verify the action is physically available!
|
||||
# If not, the memorized path is stale/invalid for the current physical UI state.
|
||||
available = current_screen.get("available_actions", [])
|
||||
if action not in available and action != "force start instagram" and "scroll" not in action:
|
||||
logger.warning(f"⚠️ [GOAP Recall] Recalled action '{action}' is NOT available on screen! Path is stale.")
|
||||
return False
|
||||
|
||||
success = self._execute_action(action, goal=goal, screen_state=current_screen)
|
||||
if not success:
|
||||
logger.warning(f"⚠️ [GOAP Recall] Step '{action}' failed. Path may be stale.")
|
||||
return False
|
||||
|
||||
random_sleep(0.5, 1.0)
|
||||
|
||||
# Verify goal achieved
|
||||
screen = self.perceive()
|
||||
achieved = self.planner.plan_next_step(goal, screen) is None
|
||||
return achieved
|
||||
|
||||
# ── Convenience methods (backward compatibility with navigate_to) ──
|
||||
|
||||
def navigate_to_screen(self, target: str) -> bool:
|
||||
"""Navigate to a screen by name. Wrapper for achieve(). Delegates to ScreenTopology SSOT."""
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
|
||||
goal = ScreenTopology.screen_name_to_goal(target)
|
||||
return self.achieve(goal)
|
||||
|
||||
def get_current_screen_type(self) -> ScreenType:
|
||||
"""Quick screen check."""
|
||||
screen = self.perceive()
|
||||
return screen["screen_type"]
|
||||
@@ -1,36 +1,166 @@
|
||||
import logging
|
||||
import random
|
||||
from datetime import datetime
|
||||
|
||||
from colorama import Fore
|
||||
|
||||
from GramAddict.core.qdrant_memory import PersonaMemoryDB
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GrowthBrain:
|
||||
"""
|
||||
Biological Feedback and Persona Management.
|
||||
|
||||
|
||||
Two critical functions:
|
||||
1. Circadian Rhythm — modulates ALL sleep/dwell times based on time of day
|
||||
2. Persona Refinement — learns from interaction outcomes and stores insights
|
||||
"""
|
||||
|
||||
def __init__(self, username: str, persona_interests: list[str] = None):
|
||||
self.username = username
|
||||
self.persona_memory = PersonaMemoryDB()
|
||||
self.persona_interests = persona_interests or []
|
||||
self.strategy = "aggressive_growth" # Will be updated by orchestrator
|
||||
self.last_learning_at = datetime.now()
|
||||
|
||||
|
||||
def evaluate_governance(self, dopamine_engine, job_target: str, is_reels: bool = False) -> str:
|
||||
"""
|
||||
Global Strategy Oracle.
|
||||
Decides if the bot should stay in the current feed, check curiosity targets,
|
||||
or escape due to boredom.
|
||||
|
||||
Returns: "STAY", "SHIFT_CONTEXT", "CHECK_CURIOSITY"
|
||||
"""
|
||||
# 1. Boredom Check (Priority 1)
|
||||
if dopamine_engine.boredom > 85.0 and random.random() < 0.2:
|
||||
logger.info(
|
||||
"🧠 [GrowthBrain] Supreme boredom reached or periodic shift triggered. Decision: SHIFT_CONTEXT."
|
||||
)
|
||||
return "SHIFT_CONTEXT"
|
||||
|
||||
# 2. Curiosity Check (Priority 2)
|
||||
# Only in main feeds, not during deep reels sessions
|
||||
if job_target.lower() in ("homefeed", "feed", "home") and not is_reels:
|
||||
if random.random() < 0.06:
|
||||
logger.info("🧠 [GrowthBrain] Spontaneous curiosity spike. Decision: CHECK_CURIOSITY.")
|
||||
return "CHECK_CURIOSITY"
|
||||
|
||||
return "STAY"
|
||||
|
||||
def get_current_desire(self, dopamine_engine, available_targets=None) -> str:
|
||||
"""
|
||||
Agent Core: Determines what the bot actually WANTS to do right now,
|
||||
based on strategy, circadian rhythm, and dopamine/boredom levels.
|
||||
|
||||
Returns a high-level semantic Desire string.
|
||||
"""
|
||||
if dopamine_engine.boredom > 80.0:
|
||||
logger.info("🧠 [GrowthBrain] Internal drive: Context shift required.")
|
||||
return "ShiftContext"
|
||||
|
||||
weights = {}
|
||||
if self.strategy == "aggressive_growth":
|
||||
weights = {
|
||||
"DiscoverNewContent": 60, # Explore, Reels
|
||||
"NurtureCommunity": 15, # HomeFeed
|
||||
"SocialReciprocity": 25, # Follow list, DMs
|
||||
}
|
||||
elif self.strategy == "community_builder":
|
||||
weights = {
|
||||
"DiscoverNewContent": 20,
|
||||
"NurtureCommunity": 50,
|
||||
"SocialReciprocity": 30,
|
||||
}
|
||||
elif self.strategy == "passive_learning":
|
||||
weights = {
|
||||
"DiscoverNewContent": 80, # Maximize exploration to build Vector DB
|
||||
"NurtureCommunity": 20,
|
||||
"SocialReciprocity": 0,
|
||||
}
|
||||
else: # stealth_lurker
|
||||
weights = {
|
||||
"DiscoverNewContent": 40,
|
||||
"NurtureCommunity": 50,
|
||||
"SocialReciprocity": 10,
|
||||
}
|
||||
|
||||
choices = []
|
||||
for desire, weight in weights.items():
|
||||
choices.extend([desire] * weight)
|
||||
|
||||
selected_desire = random.choice(choices)
|
||||
logger.info(f"🧠 [GrowthBrain] Strategy '{self.strategy}' dictated Desire: {selected_desire}")
|
||||
return selected_desire
|
||||
|
||||
def get_current_goal(self, dopamine_engine, available_goals: list[str], success_rates: dict = None) -> str:
|
||||
"""
|
||||
Autonomously selects the next strategic goal.
|
||||
If no goals are configured, falls back to legacy desires.
|
||||
Weights goals based on session success rates if provided.
|
||||
|
||||
.. deprecated::
|
||||
Use select_task() instead for concrete, plugin-linked task selection.
|
||||
"""
|
||||
import random
|
||||
|
||||
if not available_goals:
|
||||
# Legacy Desire Mapping (Fallback)
|
||||
return self.get_current_desire(dopamine_engine)
|
||||
|
||||
if dopamine_engine.boredom > 80:
|
||||
return "ShiftContext" # High boredom triggers a context shift
|
||||
|
||||
if not success_rates:
|
||||
return random.choice(available_goals)
|
||||
|
||||
weights = []
|
||||
for goal in available_goals:
|
||||
base_weight = 1.0
|
||||
success_count = success_rates.get(goal, 0)
|
||||
weight = base_weight + float(success_count)
|
||||
weights.append(weight)
|
||||
|
||||
return random.choices(available_goals, weights=weights, k=1)[0]
|
||||
|
||||
def select_task(self, dopamine_engine, available_tasks: list) -> "Optional[Task]":
|
||||
"""Select the next concrete Task using weighted random selection.
|
||||
|
||||
This is the primary interface for the orchestrator. Unlike get_current_goal()
|
||||
which returns abstract strings, this returns a Task object with a specific
|
||||
target_screen, budget, and success metric.
|
||||
|
||||
Returns:
|
||||
Task: The selected task to execute.
|
||||
None: If no tasks available or boredom is too high (ShiftContext signal).
|
||||
"""
|
||||
if not available_tasks:
|
||||
return None
|
||||
|
||||
# High boredom = ShiftContext (take a break, switch feed)
|
||||
if dopamine_engine.boredom > 85.0:
|
||||
logger.info("🧠 [GrowthBrain] Boredom too high for task selection. ShiftContext.")
|
||||
return None
|
||||
|
||||
weights = [task.weight for task in available_tasks]
|
||||
selected = random.choices(available_tasks, weights=weights, k=1)[0]
|
||||
logger.info(
|
||||
f"🧠 [GrowthBrain] Selected task: {selected.verb} → {selected.target_screen} "
|
||||
f"(weight={selected.weight:.2f}, budget={selected.budget_posts})"
|
||||
)
|
||||
return selected
|
||||
|
||||
def get_circadian_pacing(self) -> float:
|
||||
"""
|
||||
Adjusts activity levels based on the current local time
|
||||
Adjusts activity levels based on the current local time
|
||||
to simulate human sleep/wake cycles.
|
||||
|
||||
|
||||
Returns a multiplier (0.1 to 1.0) that should be applied to ALL sleep durations.
|
||||
Lower = slower (more human-like during off-hours).
|
||||
"""
|
||||
hour = datetime.now().hour
|
||||
|
||||
|
||||
# Determine current pacing state
|
||||
if 2 <= hour <= 5:
|
||||
pacing = 0.1
|
||||
@@ -52,39 +182,39 @@ class GrowthBrain:
|
||||
pacing = 1.0
|
||||
state_id = "peak_hours"
|
||||
msg = "🧠 [GrowthBrain] Peak metabolic rate. Performance 100%."
|
||||
|
||||
|
||||
# Log intelligently (only info log on state change)
|
||||
if not hasattr(self, '_last_pacing_state') or getattr(self, '_last_pacing_state') != state_id:
|
||||
if not hasattr(self, "_last_pacing_state") or getattr(self, "_last_pacing_state") != state_id:
|
||||
logger.info(msg, extra={"color": f"{Fore.GREEN}"})
|
||||
self._last_pacing_state = state_id
|
||||
else:
|
||||
logger.debug(msg)
|
||||
|
||||
|
||||
return pacing
|
||||
|
||||
def refine_persona(self, interaction_outcomes: list[dict]):
|
||||
"""
|
||||
Learns from interaction outcomes to refine persona understanding.
|
||||
|
||||
|
||||
interaction_outcomes: [{'username': str, 'action': 'like'|'comment'|'skip', 'resonance': float}]
|
||||
|
||||
|
||||
Stores high-performing interaction patterns in PersonaMemoryDB.
|
||||
"""
|
||||
if not interaction_outcomes:
|
||||
return
|
||||
|
||||
|
||||
# Find interactions that had high resonance (those are our niche)
|
||||
high_res = [o for o in interaction_outcomes if o.get("resonance", 0) > 0.7]
|
||||
low_res = [o for o in interaction_outcomes if o.get("resonance", 0) < 0.3]
|
||||
|
||||
|
||||
if high_res:
|
||||
insight = f"High-resonance interactions in this session: {len(high_res)} posts matched niche."
|
||||
self.persona_memory.store_persona_insight("session_learning", insight)
|
||||
logger.info(
|
||||
f"🧠 [GrowthBrain] Session learning: {len(high_res)} high-resonance, {len(low_res)} low-resonance posts.",
|
||||
extra={"color": f"{Fore.GREEN}"}
|
||||
extra={"color": f"{Fore.GREEN}"},
|
||||
)
|
||||
|
||||
|
||||
self.last_learning_at = datetime.now()
|
||||
|
||||
def get_persona_context(self) -> str:
|
||||
@@ -92,8 +222,30 @@ class GrowthBrain:
|
||||
base = ""
|
||||
if self.persona_interests:
|
||||
base = f"Core interests: {', '.join(self.persona_interests)}"
|
||||
|
||||
|
||||
learned = self.persona_memory.get_persona_context()
|
||||
if learned:
|
||||
return f"{base}\n{learned}" if base else learned
|
||||
return base
|
||||
|
||||
# ── [Phase 3] Humanized Decision Logic ──
|
||||
|
||||
def wants_to_double_tap(self, is_reel: bool = False) -> bool:
|
||||
"""Determines if the bot should use double-tap for likes."""
|
||||
prob = 0.45 if self.strategy == "aggressive_growth" else 0.25
|
||||
if is_reel:
|
||||
prob += 0.20 # People double-tap reels more often
|
||||
return random.random() < prob
|
||||
|
||||
def evaluate_hesitation(self) -> bool:
|
||||
"""Simulates human 'change of mind' or hesitation before a major action."""
|
||||
# Stealthy or passive bots hesitate more
|
||||
prob = 0.15 if self.strategy in ("stealth_lurker", "passive_learning") else 0.05
|
||||
return random.random() < prob
|
||||
|
||||
def wants_to_repost(self, resonance_score: float) -> bool:
|
||||
"""Decides if content is worthy of a repost."""
|
||||
if resonance_score < 0.85:
|
||||
return False
|
||||
prob = 0.3 if self.strategy == "aggressive_growth" else 0.1
|
||||
return random.random() < prob
|
||||
|
||||
86
GramAddict/core/interaction.py
Normal file
86
GramAddict/core/interaction.py
Normal file
@@ -0,0 +1,86 @@
|
||||
import logging
|
||||
from typing import Dict
|
||||
|
||||
from GramAddict.core.llm_provider import query_llm
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class LLMWriter:
|
||||
"""
|
||||
The Creative Engine — Content Generation for Interactions.
|
||||
|
||||
Generates high-fidelity, persona-aligned comments and messages.
|
||||
Replaces legacy static 'comment_list' with dynamic, contextual resonance.
|
||||
"""
|
||||
|
||||
def __init__(self, username: str, persona_interests: list[str], configs):
|
||||
self.username = username
|
||||
self.persona_interests = persona_interests
|
||||
self.configs = configs
|
||||
self.args = getattr(configs, "args", None)
|
||||
|
||||
def generate_comment(self, post_data: Dict) -> str:
|
||||
"""
|
||||
Generates a human-like comment based on post data and persona interests.
|
||||
"""
|
||||
if not post_data:
|
||||
logger.warning("✍️ [Writer] No post data provided. Using generic fallback.")
|
||||
return "Cool!"
|
||||
|
||||
caption = post_data.get("caption", "")
|
||||
description = post_data.get("description", "")
|
||||
target_username = post_data.get("username", "the user")
|
||||
|
||||
# Build context for the LLM
|
||||
context = f"Post by @{target_username}\n"
|
||||
if caption:
|
||||
context += f"Caption: {caption}\n"
|
||||
if description:
|
||||
context += f"Visual Description: {description}\n"
|
||||
|
||||
interests_str = ", ".join(self.persona_interests) if self.persona_interests else "general interesting things"
|
||||
|
||||
prompt = (
|
||||
f"You are an Instagram user interested in: {interests_str}.\n"
|
||||
f"You want to leave a brief, friendly, and authentic comment on the following post:\n\n"
|
||||
f"{context}\n"
|
||||
f"INSTRUCTIONS:\n"
|
||||
f"1. Keep it under 10 words.\n"
|
||||
f"2. Be casual and human. Avoid overly formal language or sounding like a bot.\n"
|
||||
f"3. Do NOT use more than one emoji.\n"
|
||||
f"4. Do NOT use hashtags.\n"
|
||||
f"5. Focus on something specific in the post if possible.\n"
|
||||
f"6. Reply with ONLY the comment text."
|
||||
)
|
||||
|
||||
model = getattr(self.args, "ai_writer_model", getattr(self.args, "ai_model", "llama3.2:1b"))
|
||||
url = getattr(
|
||||
self.args, "ai_writer_url", getattr(self.args, "ai_model_url", "http://localhost:11434/api/generate")
|
||||
)
|
||||
|
||||
logger.info(f"✍️ [Writer] Generating comment for @{target_username} using {model}...")
|
||||
|
||||
try:
|
||||
response_dict = query_llm(
|
||||
url=url,
|
||||
model=model,
|
||||
prompt=prompt,
|
||||
system="You are a friendly Instagram user. You write short, authentic comments.",
|
||||
format_json=False,
|
||||
timeout=60,
|
||||
temperature=0.7, # Add some variety to avoid 'the to the' loops
|
||||
)
|
||||
|
||||
if response_dict and "response" in response_dict:
|
||||
comment = response_dict["response"].strip().strip('"')
|
||||
# Basic cleaning to remove LLM artifacts
|
||||
comment = comment.split("\n")[0] # Take only first line
|
||||
if not comment:
|
||||
return "Nice!"
|
||||
return comment
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"✍️ [Writer] Failed to generate comment: {e}")
|
||||
|
||||
return "Great post! 🔥"
|
||||
@@ -1,43 +1,77 @@
|
||||
import re
|
||||
import os
|
||||
import json
|
||||
import requests
|
||||
import logging
|
||||
from typing import Optional, List, Dict
|
||||
import os
|
||||
import re
|
||||
from typing import List, Optional
|
||||
|
||||
import requests
|
||||
|
||||
try:
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv()
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def extract_json(text: str) -> Optional[str]:
|
||||
"""
|
||||
Robustly extracts the first JSON object or array from a string that may contain
|
||||
Robustly extracts the first JSON object or array from a string that may contain
|
||||
natural language prefix/suffix. Also purges <think> blocks and markdown ticks.
|
||||
"""
|
||||
if not text:
|
||||
return None
|
||||
|
||||
|
||||
# 100% Autonomous: Scrub model's internal thinking process
|
||||
if "<think>" in text:
|
||||
text = re.sub(r'<think>.*?</think>', '', text, flags=re.DOTALL).strip()
|
||||
text = re.sub(r"<think>.*?</think>", "", text, flags=re.DOTALL).strip()
|
||||
logger.debug("🧠 [LLM] Scoped thinking block detected and purged.")
|
||||
|
||||
# Remove markdown code block formats
|
||||
text = re.sub(r'^```json\s*', '', text, flags=re.MULTILINE)
|
||||
text = re.sub(r'^```\s*', '', text, flags=re.MULTILINE)
|
||||
text = re.sub(r"^```json\s*", "", text, flags=re.MULTILINE)
|
||||
text = re.sub(r"^```\s*", "", text, flags=re.MULTILINE)
|
||||
|
||||
# Look for { ... } or [ ... ]
|
||||
match = re.search(r'(\{.*\}|\[.*\])', text, re.DOTALL)
|
||||
# Try perfect json block extraction first
|
||||
match = re.search(r"(\{.*\}|\[.*\])", text, re.DOTALL)
|
||||
if match:
|
||||
return match.group(0)
|
||||
candidate = match.group(0)
|
||||
try:
|
||||
import json
|
||||
|
||||
json.loads(candidate)
|
||||
return candidate
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Smart Fallback: Truncated JSON Healing
|
||||
# If standard validation fails (e.g., due to EOF truncation by local models),
|
||||
# run a regex extraction pass over the raw generated text to safely salvage
|
||||
# all key-value pairs that *were* successfully completed before the truncation.
|
||||
import json
|
||||
|
||||
matches = re.findall(r'"([a-zA-Z0-9_]+)"\s*:\s*(?:([0-9.-]+)|"([^"\\]*(?:\\.[^"\\]*)*)")', text)
|
||||
if matches:
|
||||
res = {}
|
||||
for k, num, obj in matches:
|
||||
if num:
|
||||
try:
|
||||
res[k] = float(num) if "." in num else int(num)
|
||||
except ValueError:
|
||||
res[k] = num
|
||||
else:
|
||||
res[k] = obj.replace('\\"', '"')
|
||||
|
||||
recovered_json = json.dumps(res)
|
||||
logger.warning(f"🔧 [Fuzzy Parse] Successfully salvaged {len(res)} keys from heavily truncated LLM output.")
|
||||
return recovered_json
|
||||
|
||||
return None
|
||||
|
||||
|
||||
_MODEL_PRICING_CACHE = None
|
||||
|
||||
|
||||
def get_model_pricing(model_id: str) -> dict:
|
||||
global _MODEL_PRICING_CACHE
|
||||
if _MODEL_PRICING_CACHE is None:
|
||||
@@ -50,79 +84,128 @@ def get_model_pricing(model_id: str) -> dict:
|
||||
_MODEL_PRICING_CACHE = {}
|
||||
except Exception:
|
||||
_MODEL_PRICING_CACHE = {}
|
||||
|
||||
|
||||
# Check if exact match exists, if not, try partial matches (e.g., if version suffixes differ)
|
||||
if _MODEL_PRICING_CACHE and model_id not in _MODEL_PRICING_CACHE:
|
||||
for k, v in _MODEL_PRICING_CACHE.items():
|
||||
if model_id in k or k in model_id:
|
||||
return v
|
||||
|
||||
|
||||
return _MODEL_PRICING_CACHE.get(model_id, {})
|
||||
|
||||
|
||||
def prewarm_ollama_models(configs):
|
||||
"""
|
||||
Sends a dummy request to the configured local Ollama API endpoints via a background thread
|
||||
Sends a dummy request to the configured local Ollama API endpoints via a background thread
|
||||
to force the models to load into VRAM during bot startup, minimizing initial connection latency
|
||||
and avoiding timeouts downstream.
|
||||
"""
|
||||
args = configs.args
|
||||
|
||||
|
||||
def _warmup():
|
||||
import threading
|
||||
models_to_warm = set()
|
||||
|
||||
|
||||
# Collect unique local models
|
||||
for attr, url_attr in [
|
||||
("ai_telepathic_model", "ai_telepathic_url"),
|
||||
("ai_fallback_model", "ai_fallback_url"),
|
||||
("ai_condenser_model", "ai_condenser_url"),
|
||||
("ai_model", "ai_model_url")
|
||||
("ai_model", "ai_model_url"),
|
||||
]:
|
||||
url = getattr(args, url_attr, "")
|
||||
model = getattr(args, attr, "")
|
||||
if model and url and ("localhost" in url or "127.0.0.1" in url):
|
||||
models_to_warm.add((url, model))
|
||||
|
||||
|
||||
for url, model in models_to_warm:
|
||||
logger.info(f"🔥 [VRAM Pre-Warm] Instructing local Ollama engine to load {model} into memory in the background...")
|
||||
logger.info(
|
||||
f"🔥 [VRAM Pre-Warm] Instructing local Ollama engine to load {model} into memory in the background..."
|
||||
)
|
||||
try:
|
||||
# Fire an ultra-short generation to force it into VRAM
|
||||
requests.post(
|
||||
url,
|
||||
json={"model": model, "prompt": "Hi", "stream": False, "options": {"num_predict": 1}},
|
||||
timeout=120
|
||||
url,
|
||||
json={"model": model, "prompt": "Hi", "stream": False, "options": {"num_predict": 1}},
|
||||
timeout=120,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
if hasattr(args, "ai_telepathic_model"):
|
||||
import threading
|
||||
|
||||
threading.Thread(target=_warmup, daemon=True).start()
|
||||
|
||||
|
||||
def unload_ollama_models(configs):
|
||||
"""
|
||||
Sends keep_alive: 0 to all configured local Ollama API endpoints via a background thread
|
||||
to force the models to unload from VRAM during bot shutdown.
|
||||
"""
|
||||
args = configs.args
|
||||
|
||||
def _unload():
|
||||
models_to_unload = set()
|
||||
|
||||
# Collect unique local models
|
||||
for attr, url_attr in [
|
||||
("ai_telepathic_model", "ai_telepathic_url"),
|
||||
("ai_fallback_model", "ai_fallback_url"),
|
||||
("ai_condenser_model", "ai_condenser_url"),
|
||||
("ai_model", "ai_model_url"),
|
||||
]:
|
||||
url = getattr(args, url_attr, "")
|
||||
model = getattr(args, attr, "")
|
||||
if model and url and ("localhost" in url or "127.0.0.1" in url):
|
||||
models_to_unload.add((url, model))
|
||||
|
||||
for url, model in models_to_unload:
|
||||
logger.info(f"❄️ [VRAM Cleanup] Instructing local Ollama engine to unload {model} from memory...")
|
||||
try:
|
||||
# Fire keep_alive: 0 to unload it from VRAM
|
||||
requests.post(url, json={"model": model, "keep_alive": 0}, timeout=5)
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to unload {model}: {e}")
|
||||
|
||||
if hasattr(args, "ai_telepathic_model"):
|
||||
import threading
|
||||
|
||||
threading.Thread(target=_unload, daemon=True).start()
|
||||
|
||||
|
||||
def log_openrouter_burn():
|
||||
"""Fetches and logs the current OpenRouter API key usage (money burned) ONLY if OpenRouter is actively used."""
|
||||
key = os.environ.get("OPENROUTER_API_KEY")
|
||||
if not key:
|
||||
return
|
||||
|
||||
|
||||
try:
|
||||
from GramAddict.core.config import Config
|
||||
|
||||
args = Config().args
|
||||
uses_openrouter = False
|
||||
|
||||
|
||||
# Check all possible model/url endpoints for 'openrouter'
|
||||
for attr in ["ai_model", "ai_model_url", "ai_telepathic_model", "ai_telepathic_url",
|
||||
"ai_fallback_model", "ai_fallback_url", "ai_condenser_model", "ai_condenser_url"]:
|
||||
for attr in [
|
||||
"ai_model",
|
||||
"ai_model_url",
|
||||
"ai_telepathic_model",
|
||||
"ai_telepathic_url",
|
||||
"ai_fallback_model",
|
||||
"ai_fallback_url",
|
||||
"ai_condenser_model",
|
||||
"ai_condenser_url",
|
||||
]:
|
||||
val = getattr(args, attr, "")
|
||||
if val and "openrouter" in str(val).lower():
|
||||
uses_openrouter = True
|
||||
break
|
||||
|
||||
|
||||
if not uses_openrouter:
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
try:
|
||||
r = requests.get("https://openrouter.ai/api/v1/auth/key", headers={"Authorization": f"Bearer {key}"}, timeout=5)
|
||||
if r.status_code == 200:
|
||||
@@ -130,11 +213,16 @@ def log_openrouter_burn():
|
||||
total_spent = data.get("usage", 0.0)
|
||||
daily_spent = data.get("usage_daily", 0.0)
|
||||
limit = data.get("limit")
|
||||
|
||||
logger.info(f"🔥 [OpenRouter Burn Rate] Daily: ${daily_spent:.4f} | Total: ${total_spent:.4f}" + (f" | Limit: ${limit}" if limit else ""), extra={"color": "\x1b[38;5;208m\x1b[1m"})
|
||||
|
||||
logger.info(
|
||||
f"🔥 [OpenRouter Burn Rate] Daily: ${daily_spent:.4f} | Total: ${total_spent:.4f}"
|
||||
+ (f" | Limit: ${limit}" if limit else ""),
|
||||
extra={"color": "\x1b[38;5;208m\x1b[1m"},
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"Could not fetch OpenRouter burn rate: {e}")
|
||||
|
||||
|
||||
def query_llm(
|
||||
url: str,
|
||||
model: str,
|
||||
@@ -144,16 +232,18 @@ def query_llm(
|
||||
format_json: bool = False,
|
||||
timeout: int = 180,
|
||||
fallback_model: Optional[str] = None,
|
||||
fallback_url: Optional[str] = None
|
||||
fallback_url: Optional[str] = None,
|
||||
temperature: Optional[float] = None,
|
||||
max_tokens: Optional[int] = None,
|
||||
) -> Optional[dict]:
|
||||
"""
|
||||
Unified LLM API Caller with configurable fallback.
|
||||
"""
|
||||
openrouter_key = os.environ.get("OPENROUTER_API_KEY")
|
||||
|
||||
|
||||
# URL-based provider detection (not model-name based — works for any model)
|
||||
is_openai_compat = "/v1/chat/completions" in url or "openrouter.ai" in url.lower() or "openai.com" in url.lower()
|
||||
|
||||
|
||||
# If using a cloud model but a local URL was passed, fix it
|
||||
if not is_openai_compat and ("openrouter" in model.lower() or "/" in model):
|
||||
# Model looks like "org/model-name" which is OpenRouter format
|
||||
@@ -161,60 +251,67 @@ def query_llm(
|
||||
url = "https://openrouter.ai/api/v1/chat/completions"
|
||||
|
||||
headers = {"Content-Type": "application/json"}
|
||||
|
||||
|
||||
if is_openai_compat:
|
||||
if openrouter_key:
|
||||
headers["Authorization"] = f"Bearer {openrouter_key}"
|
||||
|
||||
|
||||
messages = []
|
||||
if system:
|
||||
messages.append({"role": "system", "content": system})
|
||||
|
||||
|
||||
user_content = []
|
||||
if prompt:
|
||||
user_content.append({"type": "text", "text": prompt})
|
||||
|
||||
|
||||
if images_b64:
|
||||
for img in images_b64:
|
||||
user_content.append({
|
||||
"type": "image_url",
|
||||
"image_url": {"url": f"data:image/jpeg;base64,{img}"}
|
||||
})
|
||||
|
||||
user_content.append({"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{img}"}})
|
||||
|
||||
messages.append({"role": "user", "content": user_content if len(user_content) > 1 else prompt})
|
||||
|
||||
req_data = {
|
||||
"model": model,
|
||||
"messages": messages,
|
||||
"stream": False
|
||||
}
|
||||
|
||||
req_data = {"model": model, "messages": messages, "stream": False}
|
||||
if format_json:
|
||||
req_data["response_format"] = {"type": "json_object"}
|
||||
|
||||
if temperature is not None:
|
||||
req_data["temperature"] = temperature
|
||||
if max_tokens is not None:
|
||||
req_data["max_tokens"] = max_tokens
|
||||
|
||||
else:
|
||||
# Ollama /generate API
|
||||
req_data = {
|
||||
"model": model,
|
||||
"prompt": prompt,
|
||||
"stream": False
|
||||
}
|
||||
req_data = {"model": model, "prompt": prompt, "stream": False}
|
||||
if system:
|
||||
req_data["system"] = system
|
||||
if images_b64:
|
||||
req_data["images"] = images_b64
|
||||
if format_json:
|
||||
req_data["format"] = "json"
|
||||
else:
|
||||
# For free-text calls (Brain action extraction), explicitly disable
|
||||
# thinking mode. Reasoning models like qwen3.5 put EVERYTHING in
|
||||
# the thinking block and return response='', which is useless for
|
||||
# action extraction. think=false forces a direct response.
|
||||
req_data["think"] = False
|
||||
|
||||
# Ollama passes configs inside 'options'
|
||||
if temperature is not None or max_tokens is not None:
|
||||
req_data["options"] = {}
|
||||
if temperature is not None:
|
||||
req_data["options"]["temperature"] = temperature
|
||||
if max_tokens is not None:
|
||||
req_data["options"]["num_predict"] = max_tokens
|
||||
|
||||
try:
|
||||
response = requests.post(url, json=req_data, headers=headers, timeout=timeout)
|
||||
response.raise_for_status()
|
||||
resp_json = response.json()
|
||||
|
||||
|
||||
# Normalize response payload so callers don't have to distinguish
|
||||
if is_openai_compat:
|
||||
# OpenRouter returns choices[0].message.content
|
||||
content = resp_json.get("choices", [{}])[0].get("message", {}).get("content", "")
|
||||
|
||||
|
||||
usage = resp_json.get("usage", {})
|
||||
if usage:
|
||||
cost_str = ""
|
||||
@@ -225,62 +322,79 @@ def query_llm(
|
||||
pricing = get_model_pricing(model)
|
||||
if pricing:
|
||||
try:
|
||||
p_cost = float(pricing.get("prompt", 0)) * usage.get('prompt_tokens', 0)
|
||||
c_cost = float(pricing.get("completion", 0)) * usage.get('completion_tokens', 0)
|
||||
p_cost = float(pricing.get("prompt", 0)) * usage.get("prompt_tokens", 0)
|
||||
c_cost = float(pricing.get("completion", 0)) * usage.get("completion_tokens", 0)
|
||||
calc_cost = p_cost + c_cost
|
||||
if calc_cost > 0:
|
||||
cost_str = f" | 💸 Cost: ${calc_cost:.6f}"
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
p_tokens = usage.get('prompt_tokens', 0)
|
||||
c_tokens = usage.get('completion_tokens', 0)
|
||||
t_tokens = usage.get('total_tokens', 0)
|
||||
|
||||
|
||||
p_tokens = usage.get("prompt_tokens", 0)
|
||||
c_tokens = usage.get("completion_tokens", 0)
|
||||
t_tokens = usage.get("total_tokens", 0)
|
||||
|
||||
# Make it stand out!
|
||||
logger.info(f"🪙 [LLM Burn] {model} -> In: {p_tokens} | Out: {c_tokens} | Total: {t_tokens}{cost_str}", extra={"color": "\x1b[38;5;208m\x1b[1m"})
|
||||
|
||||
logger.info(
|
||||
f"🪙 [LLM Burn] {model} -> In: {p_tokens} | Out: {c_tokens} | Total: {t_tokens}{cost_str}",
|
||||
extra={"color": "\x1b[38;5;208m\x1b[1m"},
|
||||
)
|
||||
|
||||
# Validation: if JSON was expected, try to extract it
|
||||
if format_json:
|
||||
extracted = extract_json(content)
|
||||
if not extracted:
|
||||
raise ValueError(f"OpenRouter returned non-JSON content when JSON was expected: {content[:100]}...")
|
||||
raise ValueError(f"OpenRouter returned non-JSON content when JSON was expected: {content[:100]}...")
|
||||
content = extracted
|
||||
|
||||
return {"response": content}
|
||||
else:
|
||||
# Ollama returns response OR thinking (for reasoning models)
|
||||
content = resp_json.get("response") or resp_json.get("thinking") or ""
|
||||
raw_response = resp_json.get("response", "")
|
||||
raw_thinking = resp_json.get("thinking", "")
|
||||
|
||||
logger.debug(f"DEBUG LLM PAYLOAD: response='{raw_response}', thinking='{raw_thinking}'")
|
||||
|
||||
# CRITICAL: For free-text mode (format_json=False), do NOT substitute
|
||||
# thinking for empty response. The thinking block is REASONING, not
|
||||
# a decision. The Brain parser would extract random actions from it.
|
||||
# For JSON mode (format_json=True), falling back to thinking IS correct
|
||||
# because reasoning models may place structured output in the thinking block.
|
||||
if format_json:
|
||||
content = raw_response or raw_thinking or ""
|
||||
extracted = extract_json(content)
|
||||
if not extracted:
|
||||
# Log more context if JSON extraction fails
|
||||
logger.debug(f"Ollama raw content (for JSON extraction): {content[:200]}...")
|
||||
raise ValueError(f"Ollama returned non-JSON content when JSON was expected.")
|
||||
resp_json["response"] = extracted
|
||||
logger.warning(f"Failed to extract JSON from content: {content[:100]}")
|
||||
else:
|
||||
content = extracted
|
||||
else:
|
||||
content = raw_response
|
||||
|
||||
return resp_json
|
||||
return {"response": content}
|
||||
except requests.exceptions.ConnectionError:
|
||||
logger.error(f"⚠️ [LLM Provider] Connection refused for {model} at {url}. Is the service running?")
|
||||
except Exception as e:
|
||||
logger.error(f"LLM Provider Error with {model}: {e}")
|
||||
|
||||
|
||||
# Prevent infinite fallback loops
|
||||
if getattr(query_llm, "_is_fallback", False):
|
||||
return None
|
||||
|
||||
|
||||
# Decide on fallback model/url
|
||||
f_model = fallback_model
|
||||
f_url = fallback_url
|
||||
|
||||
|
||||
# Read fallback config from args if available
|
||||
if not f_model or not f_url:
|
||||
from GramAddict.core.config import Config
|
||||
|
||||
try:
|
||||
args = Config().args
|
||||
f_model = f_model or getattr(args, "ai_fallback_model", None)
|
||||
f_url = f_url or getattr(args, "ai_fallback_url", None)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# Last resort defaults
|
||||
if not f_model or not f_url:
|
||||
if is_openai_compat:
|
||||
@@ -305,12 +419,15 @@ def query_llm(
|
||||
images_b64=images_b64,
|
||||
system=system,
|
||||
format_json=format_json,
|
||||
timeout=timeout
|
||||
timeout=timeout,
|
||||
temperature=temperature,
|
||||
max_tokens=max_tokens,
|
||||
)
|
||||
finally:
|
||||
query_llm._is_fallback = False
|
||||
return None
|
||||
|
||||
|
||||
def query_telepathic_llm(
|
||||
model: str,
|
||||
url: str,
|
||||
@@ -318,7 +435,7 @@ def query_telepathic_llm(
|
||||
user_prompt: str,
|
||||
temperature: float = 0.0,
|
||||
use_local_edge: bool = False,
|
||||
images_b64: Optional[List[str]] = None
|
||||
images_b64: Optional[List[str]] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Routes UI Telepathic requests purely based on textual interpretation of the screen's XML nodes.
|
||||
@@ -330,19 +447,26 @@ def query_telepathic_llm(
|
||||
target_model = model
|
||||
|
||||
if use_local_edge:
|
||||
logger.info("⚡ [Edge Inference] Routing telepathic request to local Ollama host (0ms latency target).")
|
||||
from GramAddict.core.config import Config
|
||||
try:
|
||||
args = Config().args
|
||||
target_url = getattr(args, "ai_fallback_url", "http://localhost:11434/api/generate")
|
||||
target_model = getattr(args, "ai_fallback_model", "llama3.2:1b")
|
||||
except Exception:
|
||||
target_url = "http://localhost:11434/api/generate"
|
||||
target_model = "llama3.2:1b"
|
||||
|
||||
is_already_local = "localhost" in url or "127.0.0.1" in url
|
||||
if is_already_local:
|
||||
logger.debug(
|
||||
f"⚡ [Edge Inference] Primary model {model} is already local. Using it directly to prevent VRAM thrashing."
|
||||
)
|
||||
else:
|
||||
logger.info("⚡ [Edge Inference] Routing telepathic request to local Ollama host (0ms latency target).")
|
||||
from GramAddict.core.config import Config
|
||||
|
||||
try:
|
||||
args = Config().args
|
||||
target_url = getattr(args, "ai_fallback_url", "http://localhost:11434/api/generate")
|
||||
target_model = getattr(args, "ai_fallback_model", "llama3.2:1b")
|
||||
except Exception:
|
||||
target_url = "http://localhost:11434/api/generate"
|
||||
target_model = "llama3.2:1b"
|
||||
|
||||
is_local = "localhost" in target_url or "127.0.0.1" in target_url
|
||||
calc_timeout = 180 if is_local else 45
|
||||
|
||||
|
||||
ans = query_llm(
|
||||
url=target_url,
|
||||
model=target_model,
|
||||
@@ -350,7 +474,9 @@ def query_telepathic_llm(
|
||||
images_b64=images_b64,
|
||||
system=system_prompt,
|
||||
format_json=True,
|
||||
timeout=calc_timeout # Navigation VLM must fail fast for Cloud, but wait for Local VRAM loads
|
||||
timeout=calc_timeout, # Navigation VLM must fail fast for Cloud, but wait for Local VRAM loads
|
||||
temperature=temperature,
|
||||
max_tokens=150, # Hard stop to prevent VLM from endlessly hallucinating UI elements
|
||||
)
|
||||
if ans and "response" in ans:
|
||||
return ans["response"]
|
||||
|
||||
1
GramAddict/core/navigation/__init__.py
Normal file
1
GramAddict/core/navigation/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
# Navigation domain package
|
||||
89
GramAddict/core/navigation/brain.py
Normal file
89
GramAddict/core/navigation/brain.py
Normal file
@@ -0,0 +1,89 @@
|
||||
import logging
|
||||
from typing import List, Optional
|
||||
|
||||
from GramAddict.core.config import Config
|
||||
from GramAddict.core.llm_provider import query_llm
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def ask_brain_for_action(
|
||||
goal: str, screen_type: str, available_actions: List[str], explored_actions: set, context: dict = None
|
||||
) -> Optional[str]:
|
||||
"""Asks the VLM to decide the best available action to reach the goal, considering failures."""
|
||||
if not available_actions:
|
||||
return None
|
||||
|
||||
cfg = Config()
|
||||
url = (
|
||||
getattr(cfg.args, "ai_model_url", "http://localhost:11434/api/generate")
|
||||
if hasattr(cfg, "args")
|
||||
else "http://localhost:11434/api/generate"
|
||||
)
|
||||
model = getattr(cfg.args, "ai_model", "qwen3.5:latest") if hasattr(cfg, "args") else "qwen3.5:latest"
|
||||
|
||||
prompt = (
|
||||
f"You are an autonomous Instagram agent. Your ultimate goal is: '{goal}'.\n"
|
||||
f"You are currently on the screen: {screen_type}.\n"
|
||||
f"These actions are available to you right now: {available_actions}\n"
|
||||
)
|
||||
if explored_actions:
|
||||
prompt += f"You recently tried these actions but they failed or didn't help: {list(explored_actions)}\n"
|
||||
if context:
|
||||
prompt += f"Context: {context}\n"
|
||||
|
||||
prompt += (
|
||||
"INSTRUCTIONS:\n"
|
||||
"1. Reason about where you are. Consider the screen type and what actions make sense on that screen.\n"
|
||||
"2. If the goal requires navigating away from the current screen, choose the action that moves you closest to the goal.\n"
|
||||
"3. 'scroll down' reveals more UI elements on scrollable screens (feeds, profiles, lists). If your target is likely on this screen but not currently visible, you MUST choose 'scroll down'.\n"
|
||||
"4. 'press back' exits the current screen and returns to the previous one. Use it when you are on a screen that doesn't lead to your goal.\n"
|
||||
"5. DO NOT hallucinate actions. Reply ONLY with the exact string from the available actions list.\n"
|
||||
"6. Reply with ONLY the action string, nothing else."
|
||||
)
|
||||
|
||||
try:
|
||||
response = query_llm(
|
||||
url=url,
|
||||
model=model,
|
||||
prompt="Choose the next best action.",
|
||||
system=prompt,
|
||||
format_json=False,
|
||||
max_tokens=250,
|
||||
)
|
||||
if response:
|
||||
result = response if isinstance(response, str) else response.get("response", "")
|
||||
result = result.strip().strip("'\"").rstrip(".")
|
||||
|
||||
# 1. Exact match check (ideal case)
|
||||
for act in available_actions:
|
||||
if act.lower() == result.lower():
|
||||
return act
|
||||
|
||||
# 2. Strict line-by-line check (often the model outputs the action on the last line)
|
||||
for line in reversed(result.splitlines()):
|
||||
line = line.strip().strip("'\"").rstrip(".")
|
||||
for act in available_actions:
|
||||
if act.lower() == line.lower():
|
||||
return act
|
||||
|
||||
# 3. Fuzzy match (find the LAST mentioned action in the text, assuming it's the conclusion)
|
||||
best_act = None
|
||||
best_idx = -1
|
||||
for act in available_actions:
|
||||
idx = result.lower().rfind(act.lower())
|
||||
if idx > best_idx:
|
||||
best_idx = idx
|
||||
best_act = act
|
||||
|
||||
if best_act:
|
||||
logger.warning(f"🧠 [Brain] Extracted action '{best_act}' from verbose LLM output.")
|
||||
return best_act
|
||||
|
||||
logger.warning(
|
||||
f"🧠 [Brain] LLM returned an invalid action or no action found: '{result[:100]}...'. Falling back."
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"🧠 [Brain] Error querying LLM: {e}")
|
||||
|
||||
return None
|
||||
256
GramAddict/core/navigation/knowledge.py
Normal file
256
GramAddict/core/navigation/knowledge.py
Normal file
@@ -0,0 +1,256 @@
|
||||
import logging
|
||||
import time
|
||||
from typing import List, Optional
|
||||
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
from GramAddict.core.qdrant_memory import QdrantBase
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class NavigationKnowledge:
|
||||
"""
|
||||
Manages the bot's learned understanding of the Instagram UI.
|
||||
Discovered dynamically through exploration and success.
|
||||
"""
|
||||
|
||||
def __init__(self, username: str):
|
||||
self.username = username
|
||||
try:
|
||||
self._db = QdrantBase("navigation_knowledge", vector_size=768)
|
||||
except Exception:
|
||||
self._db = None
|
||||
|
||||
# In-memory cache for rapidly avoiding traps during exploration
|
||||
# In-memory cache for rapidly avoiding traps during exploration
|
||||
self._learned_screen_mappings = {}
|
||||
self._learned_traps = set()
|
||||
|
||||
def wipe(self):
|
||||
"""Wipe all learned knowledge from Qdrant."""
|
||||
if self._db and self._db.is_connected:
|
||||
try:
|
||||
self._db.wipe_collection()
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ [NavigationKnowledge] Could not wipe knowledge: {e}")
|
||||
|
||||
def update_username(self, username: str):
|
||||
"""Update username and reconnect DB if needed."""
|
||||
if self.username != username:
|
||||
self.username = username
|
||||
try:
|
||||
self._db = QdrantBase("navigation_knowledge", vector_size=768)
|
||||
except Exception:
|
||||
self._db = None
|
||||
|
||||
def get_requirements(self, goal: str) -> List[ScreenType]:
|
||||
"""Get required screens for a goal. Returns known requirements or empty list."""
|
||||
if not self._db or not self._db.is_connected:
|
||||
return []
|
||||
|
||||
try:
|
||||
from qdrant_client.models import FieldCondition, Filter, MatchValue
|
||||
|
||||
results = self._db.client.scroll(
|
||||
collection_name=self._db.collection_name,
|
||||
scroll_filter=Filter(must=[FieldCondition(key="goal", match=MatchValue(value=goal))]),
|
||||
limit=1,
|
||||
)[0]
|
||||
if results:
|
||||
screen_name = results[0].payload.get("required_screen")
|
||||
logger.debug(f"🧠 [Nav Knowledge] Found requirement for '{goal}': {screen_name}")
|
||||
if screen_name:
|
||||
return [ScreenType[screen_name]]
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ [Nav Knowledge] Search error: {e}")
|
||||
return []
|
||||
|
||||
def learn_goal_requirement(self, goal: str, screen_type: ScreenType):
|
||||
"""Learn that achieving 'goal' lands us on 'screen_type'."""
|
||||
if not self._db or not self._db.is_connected:
|
||||
logger.warning("⚠️ [Nav Knowledge] Cannot learn: DB not connected")
|
||||
return
|
||||
|
||||
seed = f"req_{goal}"
|
||||
vec = self._db._get_embedding(f"goal_requirement: {goal}")
|
||||
payload = {"goal": goal, "required_screen": screen_type.name, "timestamp": time.time()}
|
||||
self._db.upsert_point(seed, payload, vector=vec)
|
||||
logger.info(f"🧠 [Nav Knowledge] Learned: '{goal}' → {screen_type.name}")
|
||||
|
||||
def get_action_for_screen(self, target_screen: ScreenType) -> Optional[str]:
|
||||
"""Find which action leads to this screen."""
|
||||
for action, screen in self._learned_screen_mappings.items():
|
||||
if screen == target_screen:
|
||||
return action
|
||||
|
||||
if not self._db or not self._db.is_connected:
|
||||
return None
|
||||
|
||||
try:
|
||||
from qdrant_client.models import FieldCondition, Filter, MatchValue
|
||||
|
||||
results = self._db.client.scroll(
|
||||
collection_name=self._db.collection_name,
|
||||
scroll_filter=Filter(
|
||||
must=[FieldCondition(key="result_screen", match=MatchValue(value=target_screen.name))]
|
||||
),
|
||||
limit=1,
|
||||
)[0]
|
||||
if results:
|
||||
return results[0].payload.get("action")
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
def get_screen_for_action(self, action: str) -> Optional[ScreenType]:
|
||||
"""Find where this action leads to to avoid looping traps."""
|
||||
if action in self._learned_screen_mappings:
|
||||
return self._learned_screen_mappings[action]
|
||||
|
||||
if not self._db or not self._db.is_connected:
|
||||
return None
|
||||
|
||||
try:
|
||||
from qdrant_client.models import FieldCondition, Filter, MatchValue
|
||||
|
||||
results = self._db.client.scroll(
|
||||
collection_name=self._db.collection_name,
|
||||
scroll_filter=Filter(must=[FieldCondition(key="action", match=MatchValue(value=action))]),
|
||||
limit=1,
|
||||
)[0]
|
||||
if results:
|
||||
screen_name = results[0].payload.get("result_screen")
|
||||
if screen_name:
|
||||
return ScreenType[screen_name]
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
def learn_screen_mapping(self, action: str, result_screen: ScreenType):
|
||||
"""Learn that taking 'action' leads to 'result_screen'."""
|
||||
if not self._db or not self._db.is_connected:
|
||||
return
|
||||
|
||||
seed = f"map_{action}"
|
||||
vec = self._db._get_embedding(f"screen_mapping: {result_screen.name}")
|
||||
payload = {"action": action, "result_screen": result_screen.name, "timestamp": time.time()}
|
||||
|
||||
self._learned_screen_mappings[action] = result_screen
|
||||
|
||||
self._db.upsert_point(seed, payload, vector=vec)
|
||||
logger.info(f"🧠 [Nav Knowledge] Learned Mapping: '{action}' → {result_screen.name}")
|
||||
|
||||
def get_screen_for_tab(self, tab_id: str) -> Optional[ScreenType]:
|
||||
"""Find where this tab leads to to avoid looping traps."""
|
||||
if tab_id in self._learned_screen_mappings:
|
||||
return self._learned_screen_mappings[tab_id]
|
||||
|
||||
if not self._db or not self._db.is_connected:
|
||||
return None
|
||||
|
||||
try:
|
||||
from qdrant_client.models import FieldCondition, Filter, MatchValue
|
||||
|
||||
results = self._db.client.scroll(
|
||||
collection_name=self._db.collection_name,
|
||||
scroll_filter=Filter(must=[FieldCondition(key="tab_id", match=MatchValue(value=tab_id))]),
|
||||
limit=1,
|
||||
)[0]
|
||||
if results:
|
||||
s_name = results[0].payload.get("result_screen")
|
||||
if s_name:
|
||||
return ScreenType[s_name]
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
def clear_traps(self):
|
||||
"""Clear all in-memory traps. Called after force-restart to allow fresh routing."""
|
||||
count = len(self._learned_traps)
|
||||
self._learned_traps.clear()
|
||||
if count > 0:
|
||||
logger.info(f"🧹 [NavigationKnowledge] Cleared {count} in-memory traps for fresh routing.")
|
||||
|
||||
def learn_trap(self, screen_type: ScreenType, action: str, trap_reason: str = "softlock"):
|
||||
"""Aversively learn that an action on a screen is dangerous/useless."""
|
||||
trap_key = f"{screen_type.name}_{action}"
|
||||
self._learned_traps.add(trap_key)
|
||||
|
||||
# GUARD: Never persist traps for UNKNOWN screens to Qdrant.
|
||||
# UNKNOWN is a catch-all — persisting traps here permanently blocks
|
||||
# ALL unidentified screens, creating inescapable dead-ends.
|
||||
if screen_type == ScreenType.UNKNOWN:
|
||||
logger.warning(
|
||||
f"🛡️ [Aversive Learning] Trap '{action}' on UNKNOWN kept in-memory only (not persisted). "
|
||||
"UNKNOWN is a catch-all — permanent traps here block all unidentified screens."
|
||||
)
|
||||
return
|
||||
|
||||
if not self._db or not self._db.is_connected:
|
||||
return
|
||||
|
||||
seed = f"trap_{trap_key}"
|
||||
# Aversive vector is completely orthogonal to normal goals to prevent retrieval overlap
|
||||
vec = self._db._get_embedding(f"trap_avoidance: {trap_key} {trap_reason}")
|
||||
payload = {
|
||||
"trap_screen": screen_type.name,
|
||||
"trap_action": action,
|
||||
"trap_reason": trap_reason,
|
||||
"timestamp": time.time(),
|
||||
}
|
||||
self._db.upsert_point(seed, payload, vector=vec)
|
||||
logger.error(f"💀 [Aversive Learning] BURNED action '{action}' on {screen_type.name} due to: {trap_reason}")
|
||||
|
||||
def is_trap(self, screen_type: ScreenType, action: str) -> bool:
|
||||
"""Check if an action on this screen is a known trap.
|
||||
|
||||
Traps have time-based expiry: entries older than 30 minutes are
|
||||
auto-forgiven and deleted from Qdrant to prevent permanent dead-ends.
|
||||
"""
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
|
||||
if ScreenTopology.is_structural_action(screen_type, action):
|
||||
return False # Structural actions can NEVER be traps
|
||||
|
||||
trap_key = f"{screen_type.name}_{action}"
|
||||
if trap_key in self._learned_traps:
|
||||
return True
|
||||
|
||||
if not self._db or not self._db.is_connected:
|
||||
return False
|
||||
|
||||
TRAP_EXPIRY_SECONDS = 1800 # 30 minutes: old traps expire
|
||||
|
||||
try:
|
||||
from qdrant_client.models import FieldCondition, Filter, MatchValue
|
||||
|
||||
results = self._db.client.scroll(
|
||||
collection_name=self._db.collection_name,
|
||||
scroll_filter=Filter(
|
||||
must=[
|
||||
FieldCondition(key="trap_screen", match=MatchValue(value=screen_type.name)),
|
||||
FieldCondition(key="trap_action", match=MatchValue(value=action)),
|
||||
]
|
||||
),
|
||||
limit=1,
|
||||
)[0]
|
||||
if results:
|
||||
timestamp = results[0].payload.get("timestamp", 0)
|
||||
age_seconds = time.time() - timestamp
|
||||
|
||||
# Time-based expiry: old traps are forgiven
|
||||
if age_seconds > TRAP_EXPIRY_SECONDS:
|
||||
logger.info(
|
||||
f"🔄 [Aversive Decay] Forgave expired trap '{action}' on {screen_type.name} "
|
||||
f"(age: {age_seconds/60:.0f}min). Allowing re-exploration."
|
||||
)
|
||||
# Delete the stale trap from Qdrant
|
||||
seed = f"trap_{trap_key}"
|
||||
self._db.delete_point(seed)
|
||||
return False
|
||||
|
||||
self._learned_traps.add(trap_key)
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
return False
|
||||
117
GramAddict/core/navigation/path_memory.py
Normal file
117
GramAddict/core/navigation/path_memory.py
Normal file
@@ -0,0 +1,117 @@
|
||||
import logging
|
||||
import time
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
from GramAddict.core.qdrant_memory import QdrantBase
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PathMemory:
|
||||
"""
|
||||
Qdrant-backed memory for successful navigation paths.
|
||||
|
||||
Stores: goal → [step1, step2, ...] → success
|
||||
Enables instant recall for known goals.
|
||||
"""
|
||||
|
||||
def __init__(self, username: str = ""):
|
||||
self.username = username
|
||||
try:
|
||||
suffix = f"_{username}" if username else ""
|
||||
self._db = QdrantBase(f"goap_paths_v1{suffix}", vector_size=768)
|
||||
except Exception:
|
||||
self._db = None
|
||||
|
||||
def wipe(self):
|
||||
"""Wipe all learned navigation paths from Qdrant."""
|
||||
if self._db and self._db.is_connected:
|
||||
try:
|
||||
self._db.wipe_collection()
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ [PathMemory] Could not wipe collection: {e}")
|
||||
|
||||
def recall_path(self, goal: str, current_screen_type: str) -> Optional[List[Dict]]:
|
||||
"""
|
||||
Recall a previously successful path for this goal from this screen type.
|
||||
Returns list of steps or None.
|
||||
"""
|
||||
if not self._db or not self._db.is_connected:
|
||||
return None
|
||||
|
||||
query = f"goal: {goal} | from: {current_screen_type}"
|
||||
vec = self._db._get_embedding(query)
|
||||
if not vec:
|
||||
return None
|
||||
|
||||
try:
|
||||
from qdrant_client.models import FieldCondition, Filter, MatchValue
|
||||
|
||||
results = self._db.client.query_points(
|
||||
collection_name=self._db.collection_name,
|
||||
query=vec,
|
||||
query_filter=Filter(
|
||||
must=[FieldCondition(key="start_screen", match=MatchValue(value=current_screen_type))]
|
||||
),
|
||||
limit=3,
|
||||
score_threshold=0.85,
|
||||
).points
|
||||
|
||||
for r in results:
|
||||
p = r.payload
|
||||
if p.get("success") and p.get("steps"):
|
||||
logger.info(
|
||||
f"🧠 [GOAP Recall] Found path for '{goal}': "
|
||||
f"{len(p['steps'])} steps (confidence: {p.get('confidence', 0):.2f})"
|
||||
)
|
||||
return p["steps"]
|
||||
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.debug(f"GOAP recall error: {e}")
|
||||
return None
|
||||
|
||||
def learn_path(self, goal: str, start_screen: str, steps: List[Dict], success: bool):
|
||||
"""Store a navigation path in Qdrant."""
|
||||
if not self._db or not self._db.is_connected:
|
||||
return
|
||||
|
||||
query = f"goal: {goal} | from: {start_screen}"
|
||||
vec = self._db._get_embedding(query)
|
||||
if not vec:
|
||||
return
|
||||
|
||||
seed = f"{goal}|{start_screen}"
|
||||
payload = {
|
||||
"goal": goal,
|
||||
"start_screen": start_screen,
|
||||
"steps": steps,
|
||||
"step_count": len(steps),
|
||||
"success": success,
|
||||
"confidence": 0.85 if success else 0.0,
|
||||
"timestamp": time.time(),
|
||||
}
|
||||
|
||||
outcome = "✅" if success else "❌"
|
||||
self._db.upsert_point(
|
||||
seed,
|
||||
payload,
|
||||
vector=vec,
|
||||
log_success=f"🧠 [GOAP Learn] {outcome} Path for '{goal}': {len(steps)} steps from {start_screen}",
|
||||
)
|
||||
|
||||
def forget_path(self, goal: str, start_screen: str):
|
||||
"""Remove a cached path to force re-discovery."""
|
||||
if not self._db or not self._db.is_connected:
|
||||
return
|
||||
|
||||
seed = f"{goal}|{start_screen}"
|
||||
try:
|
||||
from qdrant_client import models
|
||||
|
||||
point_id = self._db.generate_uuid(seed)
|
||||
self._db.client.delete(
|
||||
collection_name=self._db.collection_name, points_selector=models.PointIdsList(points=[point_id])
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to forget path: {e}")
|
||||
296
GramAddict/core/navigation/planner.py
Normal file
296
GramAddict/core/navigation/planner.py
Normal file
@@ -0,0 +1,296 @@
|
||||
import logging
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from GramAddict.core.navigation.knowledge import NavigationKnowledge
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GoalPlanner:
|
||||
"""
|
||||
Given a goal and current screen state, plans the next action.
|
||||
|
||||
Uses Dynamic Discovery to navigate without hardcoded maps.
|
||||
"""
|
||||
|
||||
def __init__(self, username: str):
|
||||
self.knowledge = NavigationKnowledge(username)
|
||||
|
||||
def plan_next_step(
|
||||
self,
|
||||
goal: str,
|
||||
screen: Dict[str, Any],
|
||||
explored_nav_actions: set = None,
|
||||
action_failures: dict = None,
|
||||
visited_screens: set = None,
|
||||
) -> Optional[str]:
|
||||
"""Plans the NEXT single action to take toward the goal."""
|
||||
screen_type = screen["screen_type"]
|
||||
available = screen.get("available_actions", [])
|
||||
context = screen.get("context", {})
|
||||
goal_lower = goal.lower()
|
||||
|
||||
# ── 1. Check if goal is ALREADY achieved ──
|
||||
if self._is_goal_achieved(goal_lower, screen_type, context):
|
||||
logger.info(f"🎯 [GOAP] Goal '{goal}' already achieved on {screen_type.value}!")
|
||||
return None
|
||||
|
||||
# (Phase 5: legacy _plan_goal_action static heuristics purged,
|
||||
# all intents fall through to VLM-driven Discovery in _plan_navigation)
|
||||
|
||||
# ── 3. Am I on the right screen? If not, navigate there ──
|
||||
selected_tab = screen.get("selected_tab")
|
||||
nav_action = self._plan_navigation(
|
||||
goal_lower, screen_type, available, selected_tab, explored_nav_actions, action_failures, visited_screens
|
||||
)
|
||||
if nav_action:
|
||||
return nav_action
|
||||
|
||||
# Final fallback: back-track, UNLESS back-tracking is a known trap on this screen!
|
||||
if not self.knowledge.is_trap(screen_type, "press back"):
|
||||
return "press back"
|
||||
|
||||
# We are trapped! Can't go forward, can't go back!
|
||||
logger.error(f"💀 [GOAP] Completely trapped on {screen_type.name}. Forcing Instagram restart.")
|
||||
return "force start instagram"
|
||||
|
||||
def _is_goal_achieved(self, goal: str, screen_type: ScreenType, context: dict) -> bool:
|
||||
"""Check if the goal is already satisfied. Delegates to ScreenTopology SSOT."""
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
|
||||
# Interaction goals (context-specific, not navigation)
|
||||
if "view profile" in goal and screen_type in (ScreenType.OWN_PROFILE, ScreenType.OTHER_PROFILE):
|
||||
return True
|
||||
|
||||
if "like" in goal and "post" in goal:
|
||||
return context.get("is_liked", False) is True
|
||||
|
||||
if "follow" in goal and "user" in goal:
|
||||
return context.get("is_followed", False) is True
|
||||
|
||||
# Navigation goals — delegate to SSOT
|
||||
target = ScreenTopology.goal_to_target_screen(goal)
|
||||
if target and screen_type == target:
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def _plan_navigation(
|
||||
self,
|
||||
goal: str,
|
||||
screen_type: ScreenType,
|
||||
available: List[str],
|
||||
selected_tab: Optional[str] = None,
|
||||
explored_nav_actions: set = None,
|
||||
action_failures: dict = None,
|
||||
visited_screens: set = None,
|
||||
) -> Optional[str]:
|
||||
"""If we're on the wrong screen, figure out how to navigate.
|
||||
|
||||
Strategy (priority order):
|
||||
1. HD Map (ScreenTopology BFS) — deterministic, pre-computed routes
|
||||
2. Learned Knowledge (Qdrant) — dynamic discovery from past sessions
|
||||
3. Autonomous Discovery — linguistic matching + VLM intent
|
||||
"""
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
|
||||
# 0. Aversive Filter: Remove known traps from available actions
|
||||
safe_available = []
|
||||
for action in available:
|
||||
if not self.knowledge.is_trap(screen_type, action):
|
||||
safe_available.append(action)
|
||||
else:
|
||||
logger.debug(f"🛡️ [Aversive Filter] Masking trapped action: '{action}'")
|
||||
available = safe_available
|
||||
|
||||
visited_screens = visited_screens or set()
|
||||
|
||||
# 0b. No-Op Guard & Anti-Loop Guard:
|
||||
# - Strip tab actions that navigate to the CURRENT screen.
|
||||
# - Strip actions that navigate to PREVIOUSLY VISITED screens (except back-tracking).
|
||||
noop_actions = set()
|
||||
for action in available:
|
||||
expected = ScreenTopology.expected_screen_for_action(action, screen_type)
|
||||
if expected == screen_type:
|
||||
noop_actions.add(action)
|
||||
logger.debug(f"🛡️ [No-Op Guard] Stripping '{action}' — leads back to {screen_type.name}")
|
||||
elif expected in visited_screens and action != "press back":
|
||||
noop_actions.add(action)
|
||||
logger.debug(f"🛡️ [Anti-Loop Guard] Stripping '{action}' — leads to visited {expected.name}")
|
||||
|
||||
available = [a for a in available if a not in noop_actions]
|
||||
|
||||
# Build avoid_actions for HD Map route planning
|
||||
avoid_actions = (explored_nav_actions or set()).copy()
|
||||
if action_failures:
|
||||
for key, count in action_failures.items():
|
||||
if isinstance(key, tuple) and len(key) == 2:
|
||||
scr, act = key
|
||||
if scr == screen_type and count >= 2: # MAX_RETRIES is 2 in goap
|
||||
avoid_actions.add(act)
|
||||
else:
|
||||
if count >= 2:
|
||||
avoid_actions.add(key)
|
||||
|
||||
available = [a for a in available if a not in avoid_actions]
|
||||
|
||||
target_screen = ScreenTopology.goal_to_target_screen(goal)
|
||||
|
||||
# ── 1. HD Map Pre-Check for Dead Ends ──
|
||||
# If the topological map KNOWS the target is unreachable due to action_failures,
|
||||
# we must preempt the Brain from blindly routing into a dead end.
|
||||
if target_screen and target_screen != screen_type:
|
||||
route = ScreenTopology.find_route(screen_type, target_screen, avoid_actions=avoid_actions)
|
||||
if route is None and ScreenTopology.find_route(screen_type, target_screen):
|
||||
logger.warning(
|
||||
f"🛡️ [HD Map] Target {target_screen.name} is unreachable due to masked edges! Preventing Brain from blind routing."
|
||||
)
|
||||
return None
|
||||
|
||||
# ── 2. HD Map Routing (Primary Strategy for Navigation) ──
|
||||
# Ground UI transitions in structural invariants. If the topological map knows the route, use it.
|
||||
target_screen = ScreenTopology.goal_to_target_screen(goal)
|
||||
if target_screen and target_screen != screen_type:
|
||||
# We use a while loop to dynamically recalculate routes if proposed actions are missing from the UI
|
||||
current_avoid = avoid_actions.copy()
|
||||
while True:
|
||||
route = ScreenTopology.find_route(screen_type, target_screen, avoid_actions=current_avoid)
|
||||
if not route:
|
||||
# If we exhausted all topological routes because UI elements are missing
|
||||
# (e.g. tabs are hidden in a nested profile/post view), the deterministic
|
||||
# escape hatch is to press back to pop the navigation stack.
|
||||
if "press back" in available and "press back" not in current_avoid:
|
||||
logger.warning(
|
||||
"🛡️ [HD Map Guard] All routes blocked/missing. Falling back to 'press back' to escape nested view."
|
||||
)
|
||||
return "press back"
|
||||
break
|
||||
|
||||
next_action, next_screen = route[0]
|
||||
|
||||
# Check if the action is physically available on the screen
|
||||
if next_action not in available and next_action != "force start instagram":
|
||||
logger.warning(
|
||||
f"🛡️ [HD Map Guard] Action '{next_action}' is NOT available on screen. Recalculating route."
|
||||
)
|
||||
current_avoid.add(next_action)
|
||||
continue
|
||||
|
||||
# Verify action isn't explored/trapped
|
||||
if next_action not in (explored_nav_actions or set()):
|
||||
if not self.knowledge.is_trap(screen_type, next_action):
|
||||
route_desc = " → ".join(s.name for _, s in route)
|
||||
logger.info(
|
||||
f"🗺️ [HD Map] Route: {screen_type.name} → {route_desc}. " f"Next action: '{next_action}'"
|
||||
)
|
||||
return next_action
|
||||
else:
|
||||
logger.warning(f"🛡️ [HD Map] Route action '{next_action}' is trapped. Recalculating route.")
|
||||
current_avoid.add(next_action)
|
||||
else:
|
||||
logger.debug(
|
||||
f"🛡️ [HD Map] Route action '{next_action}' already explored and failed. Recalculating route."
|
||||
)
|
||||
current_avoid.add(next_action)
|
||||
|
||||
# ── 2.5. ContextGate Feedback Loop ──
|
||||
# Preempt the brain from hallucinating banned interaction intents.
|
||||
from GramAddict.core.perception.context_gate import ContextGate
|
||||
|
||||
cg = ContextGate()
|
||||
valid_screens = cg.get_valid_screens(goal)
|
||||
if valid_screens is not None and screen_type not in valid_screens:
|
||||
logger.warning(
|
||||
f"🛡️ [Planner Feedback] Goal '{goal}' is structurally banned on {screen_type.name} by ContextGate."
|
||||
)
|
||||
# We are trapped from doing the goal here. Must navigate to one of the valid screens.
|
||||
best_route = None
|
||||
for vs in valid_screens:
|
||||
r = ScreenTopology.find_route(screen_type, vs, avoid_actions=avoid_actions)
|
||||
if r and (not best_route or len(r) < len(best_route)):
|
||||
best_route = r
|
||||
if best_route:
|
||||
next_action, next_screen = best_route[0]
|
||||
if next_action not in (explored_nav_actions or set()):
|
||||
if not self.knowledge.is_trap(screen_type, next_action):
|
||||
logger.info(
|
||||
f"🗺️ [Planner Feedback] Auto-routing to {best_route[-1][1].name} via '{next_action}'"
|
||||
)
|
||||
return next_action
|
||||
|
||||
# If no route found, force back-tracking or skip brain to avoid hallucination.
|
||||
if "press back" in available:
|
||||
return "press back"
|
||||
return None
|
||||
|
||||
# ── 3. Brain-Driven Decision Making (Fallback / Discovery) ──
|
||||
# For non-navigation goals or when the HD Map is incomplete.
|
||||
from GramAddict.core.navigation.brain import ask_brain_for_action
|
||||
|
||||
brain_action = ask_brain_for_action(goal, screen_type.name, available, avoid_actions)
|
||||
if brain_action:
|
||||
logger.info(f"🧠 [Brain] Decided to execute: '{brain_action}' (to achieve: '{goal}')")
|
||||
return brain_action
|
||||
|
||||
# ── 2. Learned Knowledge (Qdrant) ──
|
||||
required_screens = self.knowledge.get_requirements(goal)
|
||||
|
||||
# ── 3. Autonomous Discovery (Blank Start fallback) ──
|
||||
if not required_screens:
|
||||
logger.info(f"🧠 [Nav Discovery] No known requirements for '{goal}'. Will attempt autonomous discovery.")
|
||||
|
||||
# Return raw intent for TelepathicEngine discovery (VLM)
|
||||
if explored_nav_actions and goal in explored_nav_actions:
|
||||
logger.info(
|
||||
f"🛑 [Nav Discovery] Autonomous intent '{goal}' already tried and failed/trapped. Yielding to back-tracking."
|
||||
)
|
||||
return None # Don't return goal again — force fallback to press back
|
||||
else:
|
||||
return goal
|
||||
|
||||
# 4. If we're already on an acceptable screen, no navigation needed
|
||||
if screen_type in required_screens:
|
||||
return None
|
||||
|
||||
# 5. Find the action we need to take (from learned knowledge or HD map)
|
||||
for target_screen in required_screens:
|
||||
# Try HD Map first!
|
||||
route = ScreenTopology.find_route(screen_type, target_screen, avoid_actions=avoid_actions)
|
||||
if route:
|
||||
next_action, next_screen = route[0]
|
||||
if next_action not in (explored_nav_actions or set()):
|
||||
if not self.knowledge.is_trap(screen_type, next_action):
|
||||
logger.info(f"🧭 [Nav HD Map] Routing to required {target_screen.name} via '{next_action}'")
|
||||
return next_action
|
||||
|
||||
known_action = self.knowledge.get_action_for_screen(target_screen)
|
||||
|
||||
if not known_action:
|
||||
logger.info(f"🧭 [Nav Discovery] Don't know action to reach {target_screen.name}. Asking VLM...")
|
||||
|
||||
screen_friendly_name = target_screen.name.replace("_", " ").lower()
|
||||
goal_words = [w.rstrip("s") for w in screen_friendly_name.split() if len(w) > 3]
|
||||
|
||||
for action in available:
|
||||
if any(w in action.lower() for w in goal_words):
|
||||
known_target = self.knowledge.get_screen_for_action(action)
|
||||
if known_target and known_target != target_screen:
|
||||
continue
|
||||
|
||||
logger.info(
|
||||
f"🎯 [Nav Discovery] Linguistic match on available action! '{action}' aligns with '{screen_friendly_name}'"
|
||||
)
|
||||
return action
|
||||
|
||||
return f"navigate to {screen_friendly_name}"
|
||||
else:
|
||||
if known_action in available:
|
||||
logger.info(f"🧭 [Nav Knowledge] Navigating to {target_screen.name} via '{known_action}'")
|
||||
return known_action
|
||||
|
||||
# If no targeted navigation works, try going back first
|
||||
if "press back" in available:
|
||||
return "press back"
|
||||
|
||||
return None
|
||||
13
GramAddict/core/perception/__init__.py
Normal file
13
GramAddict/core/perception/__init__.py
Normal file
@@ -0,0 +1,13 @@
|
||||
"""Perception — Feed and Content Analysis."""
|
||||
|
||||
from GramAddict.core.perception.feed_analysis import (
|
||||
extract_post_content,
|
||||
has_carousel_in_view,
|
||||
has_feed_markers,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"has_carousel_in_view",
|
||||
"extract_post_content",
|
||||
"has_feed_markers",
|
||||
]
|
||||
282
GramAddict/core/perception/action_memory.py
Normal file
282
GramAddict/core/perception/action_memory.py
Normal file
@@ -0,0 +1,282 @@
|
||||
import json
|
||||
import logging
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from GramAddict.core.perception.spatial_parser import SpatialNode
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _parse_yes_no(response: str) -> Optional[bool]:
|
||||
"""Parses a VLM response to find a definitive YES or NO without substring-matching 'not' or 'now'."""
|
||||
text = response.strip()
|
||||
|
||||
# Try parsing as JSON first
|
||||
if text.startswith("{"):
|
||||
try:
|
||||
data = json.loads(text)
|
||||
for k, v in data.items():
|
||||
if str(k).strip().upper() == "YES" or str(v).strip().upper() == "YES":
|
||||
return True
|
||||
if str(k).strip().upper() == "NO" or str(v).strip().upper() == "NO":
|
||||
return False
|
||||
if str(k).strip().lower() == "success" and isinstance(v, bool):
|
||||
return v
|
||||
|
||||
# If it is valid JSON but we couldn't definitively find YES/NO,
|
||||
# do NOT fall through to text matching
|
||||
return None
|
||||
except Exception:
|
||||
# Prevent JSON parsing fall-throughs
|
||||
return None
|
||||
|
||||
text_lower = text.lower()
|
||||
if text_lower.startswith("yes"):
|
||||
return True
|
||||
if text_lower.startswith("no") and not text_lower.startswith("now") and not text_lower.startswith("not"):
|
||||
return False
|
||||
|
||||
return None
|
||||
|
||||
|
||||
# FSD Architecture: No static string dictionaries.
|
||||
# The bot relies 100% on learned confidence and VLM/Delta verification.
|
||||
|
||||
|
||||
class ActionMemory:
|
||||
"""
|
||||
Handles the caching, tracking, and negative reinforcement (unlearning) of UI interactions.
|
||||
Decouples the memory layer from the core parsing engine.
|
||||
"""
|
||||
|
||||
def __init__(self, ui_memory=None, context_memory=None):
|
||||
# We optionally inject UIMemoryDB and ContextMemoryDB to decouple tests
|
||||
if ui_memory is None:
|
||||
from GramAddict.core.qdrant_memory import UIMemoryDB
|
||||
|
||||
self.ui_memory = UIMemoryDB()
|
||||
else:
|
||||
self.ui_memory = ui_memory
|
||||
|
||||
if context_memory is None:
|
||||
from GramAddict.core.qdrant_memory import ContextMemoryDB
|
||||
|
||||
self.context_memory = ContextMemoryDB()
|
||||
else:
|
||||
self.context_memory = context_memory
|
||||
|
||||
self._last_click_context: Optional[Dict[str, Any]] = None
|
||||
|
||||
def track_click(self, intent: str, node: SpatialNode, xml_context: str = "", screen_type: str = "UNKNOWN"):
|
||||
"""Stores the context of a click before it's actually performed."""
|
||||
semantic_string = f"text: '{node.text}', desc: '{node.content_desc}', id: '{node.resource_id}'"
|
||||
|
||||
self._last_click_context = {
|
||||
"intent": intent,
|
||||
"node_dict": node.to_dict(),
|
||||
"semantic_string": semantic_string,
|
||||
"xml_context": xml_context,
|
||||
"screen_type": screen_type,
|
||||
}
|
||||
logger.debug(f"🧠 [ActionMemory] Tracking tentative click for intent: '{intent}' -> {semantic_string}")
|
||||
|
||||
def confirm_click(self, intent: str = None):
|
||||
"""Positive Reinforcement: Confirms the last click was successful.
|
||||
|
||||
Guard: Refuses to store in Qdrant if the clicked element does not
|
||||
semantically match the intent. Prevents memory poisoning.
|
||||
"""
|
||||
ctx = self._last_click_context
|
||||
if not ctx:
|
||||
return
|
||||
|
||||
if intent and ctx["intent"] != intent:
|
||||
return
|
||||
|
||||
# Zero-Trust FSD: No semantic string mismatch guards here.
|
||||
# If the VLM/Delta verification passed, we trust it and learn.
|
||||
|
||||
logger.info(
|
||||
f"✅ [ActionMemory] Confirming success for '{ctx['intent']}'. Boosting confidence.",
|
||||
extra={"color": "\x1b[32m"},
|
||||
)
|
||||
|
||||
# Store or boost in Qdrant
|
||||
try:
|
||||
# Check if it exists first
|
||||
existing = self.ui_memory.retrieve_memory(ctx["intent"], ctx["xml_context"])
|
||||
if existing:
|
||||
self.ui_memory.boost_confidence(ctx["intent"], ctx["xml_context"])
|
||||
else:
|
||||
self.ui_memory.store_memory(ctx["intent"], ctx["xml_context"], ctx["node_dict"])
|
||||
# Boost context confidence
|
||||
screen_type = ctx.get("screen_type", "UNKNOWN")
|
||||
self.context_memory.update_confidence(ctx["intent"], screen_type, delta=0.2)
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to confirm click in Qdrant: {e}")
|
||||
|
||||
self._last_click_context = None
|
||||
|
||||
def reject_click(self, intent: str = None):
|
||||
"""Negative Reinforcement: Penalizes a failed click (Unlearning)."""
|
||||
ctx = self._last_click_context
|
||||
if not ctx:
|
||||
return
|
||||
|
||||
if intent and ctx["intent"] != intent:
|
||||
return
|
||||
|
||||
logger.warning(
|
||||
f"❌ [ActionMemory] Click failed for '{ctx['intent']}'. Applying penalty.", extra={"color": "\x1b[31m"}
|
||||
)
|
||||
|
||||
try:
|
||||
self.ui_memory.decay_confidence(ctx["intent"], ctx["xml_context"])
|
||||
|
||||
# Decay context confidence
|
||||
screen_type = ctx.get("screen_type", "UNKNOWN")
|
||||
self.context_memory.update_confidence(ctx["intent"], screen_type, delta=-0.2)
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to decay confidence in Qdrant: {e}")
|
||||
|
||||
self._last_click_context = None
|
||||
|
||||
def _compute_structural_delta(self, pre_xml: str, post_xml: str) -> dict:
|
||||
"""Computes a semantic diff between two XML states."""
|
||||
import re
|
||||
|
||||
pre_ids = set(re.findall(r'resource-id="([^"]+)"', pre_xml))
|
||||
post_ids = set(re.findall(r'resource-id="([^"]+)"', post_xml))
|
||||
|
||||
pre_selected = set(re.findall(r'selected="true"[^>]*resource-id="([^"]+)"', pre_xml))
|
||||
post_selected = set(re.findall(r'selected="true"[^>]*resource-id="([^"]+)"', post_xml))
|
||||
|
||||
return {
|
||||
"new_ids": post_ids - pre_ids,
|
||||
"removed_ids": pre_ids - post_ids,
|
||||
"selection_changed": pre_selected != post_selected,
|
||||
"id_delta_count": len(post_ids.symmetric_difference(pre_ids)),
|
||||
}
|
||||
|
||||
def verify_success(
|
||||
self, intent: str, pre_click_xml: str, post_click_xml: str, device=None, confidence: float = 0.0
|
||||
) -> Optional[bool]:
|
||||
"""
|
||||
Structural and Visual verification: Did the UI actually change after the click?
|
||||
"""
|
||||
intent_lower = intent.lower()
|
||||
|
||||
# ALL HARDCODED UI STRUCTURAL VERIFICATION GUARDS HAVE BEEN PURGED!
|
||||
# Rule: ZERO MAINTENANCE. We do not hardcode Resource IDs to verify if a navigation
|
||||
# was successful (e.g., checking for 'profile_header_container' or 'main_feed_action_bar').
|
||||
# Success verification MUST rely entirely on the VLM visual feedback and the Structural Delta diff.
|
||||
|
||||
state_toggles = ["like", "save", "follow", "heart"]
|
||||
is_toggle = any(t in intent_lower for t in state_toggles)
|
||||
|
||||
# P0-1 Bypass Gate removed in FSD architecture.
|
||||
# We NO LONGER bypass VLM verification via string matching.
|
||||
# If confidence is < 0.95, we always do VLM or Delta verification.
|
||||
|
||||
# ── VLM Verification (soft signal, NOT sole authority) ──
|
||||
|
||||
# If we are highly confident (e.g. pulled from Qdrant memory), bypass heavy VLM
|
||||
vlm_verdict = None
|
||||
if device and confidence < 0.95:
|
||||
logger.info(
|
||||
f"👁️ [ActionMemory] Confidence ({confidence:.2f}) < 0.95. Handing over verification for '{intent}' to VLM visual analysis..."
|
||||
)
|
||||
from GramAddict.core.perception.semantic_evaluator import SemanticEvaluator
|
||||
|
||||
evaluator = SemanticEvaluator()
|
||||
|
||||
# Build context of what was actually clicked
|
||||
clicked_context = ""
|
||||
if self._last_click_context:
|
||||
clicked_context = f"The element that was tapped: {self._last_click_context['semantic_string']}. "
|
||||
|
||||
prompt = (
|
||||
f"The user just attempted to perform the action: '{intent}'. "
|
||||
f"{clicked_context}"
|
||||
f"Look at the current screen carefully. Was the action successful? "
|
||||
)
|
||||
if is_toggle:
|
||||
prompt += (
|
||||
"If the intent was 'follow', does the button now indicate 'Following' or 'Requested'? "
|
||||
"If it was 'like', is the heart icon clearly active/red? "
|
||||
"If the screen shifted completely to a profile when you just wanted to like/follow from a feed, it FAILED. "
|
||||
"If the tapped element does NOT sound like a like/follow button (e.g. it's a caption, comment field, or post content), it FAILED. "
|
||||
)
|
||||
else:
|
||||
prompt += (
|
||||
f"Does the current screen match the expected outcome of '{intent}'? "
|
||||
f"For example, if the intent was to open a post/photo, are you looking at a post view (not a user profile or story)? "
|
||||
f"If the intent was to open a profile, are you on a profile page? "
|
||||
f"If the intent was to go back, are you on the previous screen? "
|
||||
)
|
||||
prompt += 'Answer ONLY with a valid JSON object exactly matching this schema: {"success": true} or {"success": false}. DO NOT add any other keys.'
|
||||
|
||||
try:
|
||||
screenshot = device.get_screenshot_b64()
|
||||
if not screenshot:
|
||||
raise ValueError("No screenshot available from device")
|
||||
response = evaluator._query_vlm(prompt, screenshot)
|
||||
|
||||
vlm_verdict = _parse_yes_no(response) if response else None
|
||||
|
||||
if vlm_verdict is True:
|
||||
logger.debug(f"🧠 [ActionMemory] VLM visually confirmed success for '{intent}'.")
|
||||
return True
|
||||
elif vlm_verdict is False:
|
||||
# VLM says false — but small local VLMs (7B) are unreliable.
|
||||
# Do NOT trust this blindly. Fallthrough to structural delta verification
|
||||
# which is the ground-truth tiebreaker.
|
||||
logger.info(
|
||||
f"🧠 [ActionMemory] VLM says '{intent}' failed — but VLM is unreliable. "
|
||||
"Falling through to structural delta for ground-truth verification."
|
||||
)
|
||||
# DO NOT return False here — let structural delta decide
|
||||
else:
|
||||
logger.debug(
|
||||
f"🧠 [ActionMemory] VLM response for '{intent}' was not YES/NO "
|
||||
f"(got: '{response[:80]}...'). Falling through to structural verification."
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to query VLM for visual verification: {e}")
|
||||
# Fallthrough to structural delta if VLM crashes
|
||||
|
||||
# Pre-Structural Semantic Gate removed in FSD architecture.
|
||||
# If the delta matches, we trust it. No more static string restrictions.
|
||||
|
||||
# ── Structural Delta Verification ──
|
||||
diff = self._compute_structural_delta(pre_click_xml, post_click_xml)
|
||||
|
||||
if is_toggle:
|
||||
if diff["id_delta_count"] > 10:
|
||||
logger.warning(
|
||||
f"⚠️ [ActionMemory] Massive structural shift ({diff['id_delta_count']} nodes) for state-toggle '{intent}'. Navigated away by mistake? Verification FAIL."
|
||||
)
|
||||
return False
|
||||
if diff["id_delta_count"] > 0 or diff["selection_changed"]:
|
||||
logger.debug(f"🧠 [ActionMemory] Structural delta detected for toggle '{intent}'. Verification PASS.")
|
||||
return True
|
||||
logger.warning(f"⚠️ [ActionMemory] Zero structural shift for state-toggle '{intent}'. Verification FAIL.")
|
||||
return False
|
||||
|
||||
# ── Non-Toggle Structural Delta ──
|
||||
# A click (non-toggle) should change something on screen (e.g., popup, screen transition)
|
||||
# Even scrolling will load new items (so new IDs will be present).
|
||||
# We look for at least a few ID changes. Let's say >= 2 to be safe against random background updates,
|
||||
# or if the selection changed.
|
||||
if diff["id_delta_count"] >= 1 or diff["selection_changed"]:
|
||||
logger.debug(
|
||||
f"🧠 [ActionMemory] Structural change detected ({diff['id_delta_count']} nodes) for '{intent}'. Verification PASS."
|
||||
)
|
||||
return True
|
||||
|
||||
logger.warning(
|
||||
f"⚠️ [ActionMemory] Insufficient structural change (delta=0) for non-toggle '{intent}'. Verification FAIL."
|
||||
)
|
||||
return False
|
||||
182
GramAddict/core/perception/context_gate.py
Normal file
182
GramAddict/core/perception/context_gate.py
Normal file
@@ -0,0 +1,182 @@
|
||||
import logging
|
||||
from typing import Any, Dict, FrozenSet, Optional
|
||||
|
||||
from GramAddict.core.perception.screen_identity import ScreenType
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════
|
||||
# Categorical Ban Matrix — Structural Impossibility
|
||||
# ══════════════════════════════════════════════════════
|
||||
# These define WHERE each interaction intent is structurally possible.
|
||||
# If a screen is NOT listed for an intent, the action is categorically banned.
|
||||
# This is a WHITELIST: unlisted = impossible. No VLM, no learning, no Qdrant.
|
||||
# This matrix is the Single Source of Truth for structural action plausibility.
|
||||
ALLOWED_SCREENS: Dict[str, FrozenSet[ScreenType]] = {
|
||||
"like": frozenset(
|
||||
{
|
||||
ScreenType.HOME_FEED,
|
||||
ScreenType.POST_DETAIL,
|
||||
ScreenType.REELS_FEED,
|
||||
ScreenType.EXPLORE_GRID, # After opening a post
|
||||
ScreenType.STORY_VIEW, # toolbar_like_button exists on stories
|
||||
}
|
||||
),
|
||||
"comment": frozenset(
|
||||
{
|
||||
ScreenType.HOME_FEED,
|
||||
ScreenType.POST_DETAIL,
|
||||
ScreenType.REELS_FEED,
|
||||
ScreenType.COMMENTS,
|
||||
ScreenType.STORY_VIEW, # reel_viewer_comments_button + message_composer
|
||||
}
|
||||
),
|
||||
"follow": frozenset(
|
||||
{
|
||||
ScreenType.OTHER_PROFILE,
|
||||
ScreenType.FOLLOW_LIST,
|
||||
ScreenType.STORY_VIEW, # reel_header_unconnected_follow_button_stub
|
||||
}
|
||||
),
|
||||
"unfollow": frozenset(
|
||||
{
|
||||
ScreenType.OTHER_PROFILE,
|
||||
ScreenType.FOLLOW_LIST,
|
||||
}
|
||||
),
|
||||
"save": frozenset(
|
||||
{
|
||||
ScreenType.HOME_FEED,
|
||||
ScreenType.POST_DETAIL,
|
||||
ScreenType.REELS_FEED,
|
||||
}
|
||||
),
|
||||
"repost": frozenset(
|
||||
{
|
||||
ScreenType.HOME_FEED,
|
||||
ScreenType.POST_DETAIL,
|
||||
ScreenType.REELS_FEED,
|
||||
}
|
||||
),
|
||||
"share": frozenset(
|
||||
{
|
||||
ScreenType.HOME_FEED,
|
||||
ScreenType.POST_DETAIL,
|
||||
ScreenType.REELS_FEED,
|
||||
ScreenType.STORY_VIEW, # toolbar_reshare_button exists on stories
|
||||
}
|
||||
),
|
||||
"tap post username": frozenset(
|
||||
{
|
||||
ScreenType.HOME_FEED,
|
||||
ScreenType.POST_DETAIL,
|
||||
ScreenType.REELS_FEED,
|
||||
ScreenType.EXPLORE_GRID,
|
||||
}
|
||||
),
|
||||
}
|
||||
|
||||
# Intent keywords that trigger the categorical ban check
|
||||
INTERACTION_KEYWORDS = frozenset(ALLOWED_SCREENS.keys())
|
||||
|
||||
|
||||
class ContextGate:
|
||||
"""
|
||||
Validates if an action (intent) is structurally possible on the current screen.
|
||||
This acts as a high-speed circuit breaker before invoking expensive VLM logic.
|
||||
|
||||
Architecture: 2-Layer Cascade
|
||||
─────────────────────────────
|
||||
Layer 0: Categorical Ban Matrix (O(1) dict lookup, zero dependencies)
|
||||
Blocks structurally impossible actions BEFORE any network call.
|
||||
e.g., "like" is impossible on STORY_VIEW — no like button exists.
|
||||
|
||||
Layer 1: Qdrant Learned Failures (optional, requires running Qdrant)
|
||||
Blocks actions that have been learned to fail consistently.
|
||||
e.g., "tap follow" on OTHER_PROFILE if that profile's follow button
|
||||
is hidden behind a "Requested" state.
|
||||
"""
|
||||
|
||||
def __init__(self, context_memory=None):
|
||||
if context_memory is None:
|
||||
try:
|
||||
from GramAddict.core.qdrant_memory import ContextMemoryDB
|
||||
|
||||
self.context_memory = ContextMemoryDB()
|
||||
except Exception:
|
||||
self.context_memory = None
|
||||
else:
|
||||
self.context_memory = context_memory
|
||||
|
||||
def is_allowed(self, intent: str, screen_state: Dict[str, Any]) -> bool:
|
||||
"""
|
||||
Evaluates the context gate.
|
||||
|
||||
Args:
|
||||
intent: The action name (e.g. 'follow', 'comment', 'tap like button')
|
||||
screen_state: The result of ScreenIdentity.identify()
|
||||
|
||||
Returns:
|
||||
bool: True if the action is plausible (or unknown), False if banned.
|
||||
"""
|
||||
intent_lower = intent.lower()
|
||||
screen_type = screen_state.get("screen_type", ScreenType.UNKNOWN)
|
||||
|
||||
# ── Layer 0: Categorical Ban Matrix (instant, no dependencies) ──
|
||||
matched_keyword = self._extract_interaction_keyword(intent_lower)
|
||||
if matched_keyword is not None and screen_type != ScreenType.UNKNOWN:
|
||||
allowed_screens = ALLOWED_SCREENS[matched_keyword]
|
||||
if screen_type not in allowed_screens:
|
||||
logger.debug(
|
||||
f"🛡️ [ContextGate] BLOCKED '{intent}' on {screen_type.name} — "
|
||||
f"structurally impossible (allowed: {[s.name for s in allowed_screens]})"
|
||||
)
|
||||
return False
|
||||
|
||||
# ── Layer 1: Qdrant Learned Failures ──
|
||||
if (
|
||||
matched_keyword is not None
|
||||
and screen_type != ScreenType.UNKNOWN
|
||||
and self.context_memory is not None
|
||||
and getattr(self.context_memory, "is_connected", False)
|
||||
):
|
||||
if not self.context_memory.is_allowed(intent_lower, screen_type.name):
|
||||
logger.debug(
|
||||
f"🛡️ [ContextGate] BLOCKED '{intent}' on {screen_type.name} — " f"learned failure from Qdrant"
|
||||
)
|
||||
return False
|
||||
|
||||
# ── Default: Allow (Exploration) ──
|
||||
return True
|
||||
|
||||
def get_valid_screens(self, intent: str) -> Optional[FrozenSet[ScreenType]]:
|
||||
"""
|
||||
Returns the set of screens where an interaction intent is structurally valid.
|
||||
Used by the Planner for auto-routing when the goal can't be achieved on
|
||||
the current screen.
|
||||
|
||||
Returns:
|
||||
FrozenSet[ScreenType] if the intent maps to a known interaction, else None.
|
||||
"""
|
||||
keyword = self._extract_interaction_keyword(intent.lower())
|
||||
if keyword is not None:
|
||||
return ALLOWED_SCREENS[keyword]
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _extract_interaction_keyword(intent_lower: str) -> Optional[str]:
|
||||
"""
|
||||
Extracts the primary interaction keyword from an intent string.
|
||||
Returns None if no interaction keyword is found (i.e., this is a navigation intent).
|
||||
|
||||
Uses word-boundary matching to prevent false positives:
|
||||
- "follow" matches "follow user" but NOT "followers" or "following list"
|
||||
- "like" matches "like post" but NOT "likelihood"
|
||||
"""
|
||||
import re
|
||||
|
||||
for kw in INTERACTION_KEYWORDS:
|
||||
if re.search(rf"\b{kw}\b", intent_lower):
|
||||
return kw
|
||||
return None
|
||||
185
GramAddict/core/perception/feed_analysis.py
Normal file
185
GramAddict/core/perception/feed_analysis.py
Normal file
@@ -0,0 +1,185 @@
|
||||
"""
|
||||
Perception — Feed Content Analysis.
|
||||
|
||||
Structural analysis of the feed: detecting markers, carousels,
|
||||
extracting post content. Zero-AI, pure structural parsing.
|
||||
|
||||
Extracted from bot_flow.py to enable isolated testing.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import re
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def has_carousel_in_view(xml_dump: str) -> bool:
|
||||
"""
|
||||
Checks if a carousel is present on screen via autonomous VLM classification.
|
||||
Zero-Maintenance Rule: No hardcoded Resource IDs.
|
||||
"""
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepathic = TelepathicEngine.get_instance()
|
||||
|
||||
# We ask the semantic engine to detect if a carousel is present
|
||||
classification = telepathic.classify_screen_content(xml_dump, "carousel_or_single_post")
|
||||
if classification == "carousel":
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def extract_post_content(context_xml: str, device=None) -> dict:
|
||||
"""
|
||||
Extracts meaningful content data from the current feed post's XML.
|
||||
This is the BOT'S EYES — what it actually "sees" about each post.
|
||||
|
||||
Returns:
|
||||
{'username': str, 'description': str, 'caption': str, 'username_missing': bool}
|
||||
"""
|
||||
result = {"username": "", "description": "", "caption": "", "username_missing": False}
|
||||
|
||||
try:
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepath = TelepathicEngine.get_instance()
|
||||
|
||||
# 1. Learn/extract post author dynamically
|
||||
# 🛡️ ZERO MAINTENANCE RULE: No hardcoded Resource IDs allowed.
|
||||
# We rely 100% on the Telepathic Engine to understand the UI layout autonomously.
|
||||
author_node = telepath.find_best_node(
|
||||
context_xml, "post author username text (exclude bottom tabs)", min_confidence=0.75, device=device
|
||||
)
|
||||
logger.debug(f"Telepathic resolution for author_node: {author_node}")
|
||||
|
||||
# 🛡️ Anti-Hallucination Guard: Ensure we actually found text.
|
||||
if author_node:
|
||||
attribs = author_node.get("original_attribs", {})
|
||||
text = attribs.get("text", "").strip()
|
||||
desc = attribs.get("content_desc", "").strip()
|
||||
|
||||
if text:
|
||||
result["username"] = text
|
||||
elif desc:
|
||||
result["username"] = desc
|
||||
else:
|
||||
# If the VLM selected a container (like clips_author_info_component),
|
||||
# extract text from its children.
|
||||
logger.debug("Author node lacks text/desc. Searching children for username...")
|
||||
bounds = attribs.get("bounds")
|
||||
if bounds:
|
||||
try:
|
||||
# Re-parse to find children within bounds
|
||||
import re
|
||||
|
||||
match = re.match(r"\[(\d+),(\d+)\]\[(\d+),(\d+)\]", bounds)
|
||||
if match:
|
||||
left, top, right, bottom = map(int, match.groups())
|
||||
|
||||
# Fallback: scan all nodes in XML and see if they are inside these bounds
|
||||
possible_texts = []
|
||||
possible_descs = []
|
||||
for n in ET.fromstring(context_xml).iter("node"):
|
||||
child_bounds = n.attrib.get("bounds")
|
||||
child_text = n.attrib.get("text", "").strip()
|
||||
child_desc = n.attrib.get("content-desc", "").strip()
|
||||
|
||||
if child_bounds and (child_text or child_desc):
|
||||
cm = re.match(r"\[(\d+),(\d+)\]\[(\d+),(\d+)\]", child_bounds)
|
||||
if cm:
|
||||
c_left, c_top, c_right, c_bottom = map(int, cm.groups())
|
||||
# Check if child is strictly inside the container
|
||||
if c_left >= left and c_top >= top and c_right <= right and c_bottom <= bottom:
|
||||
if child_text:
|
||||
possible_texts.append(child_text)
|
||||
if child_desc and "Profile picture" not in child_desc:
|
||||
possible_descs.append(child_desc)
|
||||
|
||||
if possible_texts:
|
||||
result["username"] = possible_texts[0]
|
||||
logger.debug(f"Extracted username '{result['username']}' from child node text.")
|
||||
elif possible_descs:
|
||||
result["username"] = possible_descs[0]
|
||||
logger.debug(f"Extracted username '{result['username']}' from child node desc.")
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to extract username from children: {e}")
|
||||
|
||||
# 2. Learn/extract post media description dynamically
|
||||
media_node = telepath.find_best_node(
|
||||
context_xml,
|
||||
"post media content (the actual image or video, exclude bottom tabs)",
|
||||
min_confidence=0.35,
|
||||
device=device,
|
||||
)
|
||||
if media_node and media_node.get("original_attribs", {}).get("content_desc"):
|
||||
result["description"] = media_node["original_attribs"]["content_desc"].strip()
|
||||
|
||||
# 3. Visible caption text (heuristic fallback if node isn't explicitly found)
|
||||
# Search all nodes for text that contains the username to find the caption body
|
||||
root = ET.fromstring(context_xml)
|
||||
for node in root.iter("node"):
|
||||
text = node.attrib.get("text", "").strip()
|
||||
if result["username"] and len(text) > 20 and result["username"] in text:
|
||||
result["caption"] = text
|
||||
break
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Error extracting post content autonomously: {e}")
|
||||
|
||||
# REGRESSION FIX 2026-05-01: Flag unreliable data when username is empty
|
||||
if not result["username"]:
|
||||
result["username_missing"] = True
|
||||
logger.warning("⚠️ [PostDataExtraction] Username is empty — data may be unreliable.")
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _parse_number_from_text(text: str) -> int:
|
||||
"""Extracts numeric value from strings like '1,234 likes', '1.5M views', 'Gefällt 12.345 Mal'."""
|
||||
text = text.lower()
|
||||
|
||||
# Clean up purely thousands separators but keep decimals
|
||||
# If there is a 'm' or 'k', a period is usually a decimal (e.g. 1.5m).
|
||||
# If no 'm' or 'k', a period might be a German thousands separator (12.345).
|
||||
# We will let the regex handle decimals.
|
||||
|
||||
# Remove commas (usually thousands separator in English)
|
||||
text = text.replace(",", "")
|
||||
|
||||
# Find all numbers, potentially with k or m
|
||||
matches = re.findall(r"(\d+(?:\.\d+)?)\s*([km])?", text)
|
||||
if not matches:
|
||||
return 0
|
||||
|
||||
best_val = 0
|
||||
for num_str, multiplier in matches:
|
||||
val = float(num_str)
|
||||
if multiplier == "k":
|
||||
val *= 1000
|
||||
elif multiplier == "m":
|
||||
val *= 1000000
|
||||
else:
|
||||
# If no multiplier, a period in num_str might be a German thousands separator
|
||||
if "." in num_str and val < 1000:
|
||||
# E.g. '12.345' became 12.345. Since no multiplier, it's actually 12345.
|
||||
# Heuristic: If it has 3 decimal places, it's a thousands separator.
|
||||
parts = num_str.split(".")
|
||||
if len(parts[1]) == 3:
|
||||
val = float(num_str.replace(".", ""))
|
||||
|
||||
best_val = max(best_val, int(val))
|
||||
|
||||
return best_val
|
||||
|
||||
|
||||
def has_feed_markers(xml_dump: str) -> bool:
|
||||
"""
|
||||
Checks if a post is visible via autonomous ScreenIdentity classification.
|
||||
Zero-Maintenance Rule: No hardcoded Resource IDs.
|
||||
"""
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity, ScreenType
|
||||
|
||||
identity = ScreenIdentity("")
|
||||
state = identity.identify(xml_dump)
|
||||
return state["screen_type"] in (ScreenType.POST_DETAIL, ScreenType.HOME_FEED, ScreenType.REELS_FEED)
|
||||
760
GramAddict/core/perception/intent_resolver.py
Normal file
760
GramAddict/core/perception/intent_resolver.py
Normal file
@@ -0,0 +1,760 @@
|
||||
import base64
|
||||
import json
|
||||
import logging
|
||||
from io import BytesIO
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
from GramAddict.core.perception.spatial_parser import SpatialNode
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _humanize_desc(desc: str) -> str:
|
||||
"""
|
||||
Inserts a space between numbers and letters to fix Instagram's concatenated content-desc.
|
||||
Example: "991following" -> "991 following", "140Kfollowers" -> "140K followers"
|
||||
"""
|
||||
if not desc:
|
||||
return ""
|
||||
import re
|
||||
|
||||
return re.sub(r"(\d[KMBkmb]?)([a-z])", r"\1 \2", desc)
|
||||
|
||||
|
||||
class IntentResolver:
|
||||
"""
|
||||
Vision-First Intent Resolver.
|
||||
|
||||
Resolves UI intents by SEEING the screen, not by parsing text descriptions.
|
||||
Uses Set-of-Mark (SoM) visual prompting: annotates a screenshot with numbered
|
||||
bounding boxes around clickable candidates, sends the annotated image to the VLM,
|
||||
and lets the VLM visually decide which box to tap.
|
||||
|
||||
Architecture:
|
||||
1. Navigation tabs → structural zone guard (bottom 15%, resource-id)
|
||||
2. Everything else → Visual Discovery (screenshot + numbered boxes + VLM)
|
||||
3. Fallback → text-based VLM (when no device/screenshot available)
|
||||
"""
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# Structural Guards
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def filter_navigation_conflicts(
|
||||
self, candidates: List[SpatialNode], intent_description: str, screen_height: int = 2400
|
||||
) -> List[SpatialNode]:
|
||||
"""
|
||||
Prevents VLM from confusing navigation-bar buttons (Back, Close)
|
||||
with bottom tab-bar buttons (Home, Profile, Search).
|
||||
|
||||
Production bug 2026-04-30: VLM picked action_bar_button_back
|
||||
for "tap profile tab" → account switch failed.
|
||||
|
||||
Production bug 2026-05-01: VLM picked profile_tab (desc='Profile')
|
||||
for "post author username text" → navigated to own profile instead.
|
||||
|
||||
Rules:
|
||||
- For tab intents: exclude nodes with "back" in resource_id or
|
||||
content_desc == "Back"
|
||||
- For back/close intents: no filtering (Back is the correct target)
|
||||
- For author/username intents: exclude bottom navigation tabs
|
||||
"""
|
||||
intent_lower = intent_description.lower()
|
||||
|
||||
# Only apply for REAL tab navigation intents.
|
||||
# REGRESSION FIX 2026-05-02: "tab" as a substring was too broad.
|
||||
# Intent "post author username text (exclude bottom tabs)" matched
|
||||
# because it contained "tab" → Tab Height Guard nuked the author node.
|
||||
# Now we require specific tab navigation patterns:
|
||||
# - "tap profile tab", "tap home tab", "explore tab"
|
||||
# - NOT "exclude bottom tabs", "tabbar", random mentions
|
||||
import re
|
||||
|
||||
_TAB_PATTERN = re.compile(
|
||||
r"\btap\s+\w+\s+tab\b" # "tap profile tab", "tap home tab"
|
||||
r"|\b\w+\s+tab\b" # "profile tab", "explore tab"
|
||||
r"|^tab\b", # "tab" at start of intent
|
||||
re.IGNORECASE,
|
||||
)
|
||||
filtered = []
|
||||
is_tab_intent = bool(_TAB_PATTERN.search(intent_lower)) and "back" not in intent_lower
|
||||
# REGRESSION FIX 2026-05-01: Author/username intents must never pick nav tabs
|
||||
is_author_intent = any(kw in intent_lower for kw in ["author", "username", "post media"])
|
||||
|
||||
for node in candidates:
|
||||
cls_name = (node.class_name or "").lower()
|
||||
|
||||
# Geometric and Class-based heuristics only. No language strings or Resource IDs!
|
||||
is_bottom_nav_area = node.center_y > (screen_height * 0.85)
|
||||
is_top_header_area = node.center_y < (screen_height * 0.15)
|
||||
is_input_field = "edittext" in cls_name
|
||||
is_image_or_video = "imageview" in cls_name or "textureview" in cls_name
|
||||
is_large_container = node.area > (screen_height * 0.3 * screen_height * 0.3)
|
||||
|
||||
# Tab intents should NEVER be at the top of the screen
|
||||
if is_tab_intent and is_top_header_area:
|
||||
logger.debug("🛡️ [Tab Height Guard] Excluded top element for tab intent.")
|
||||
continue
|
||||
|
||||
# Author/Username intents should not pick bottom navigation tabs
|
||||
if is_author_intent and is_bottom_nav_area:
|
||||
logger.debug("🛡️ [Author Tab Guard] Excluded bottom nav area for author intent.")
|
||||
continue
|
||||
|
||||
# Input fields should never be picked unless explicitly asked for
|
||||
is_reply_intent = "reply" in intent_lower or "message" in intent_lower or "type" in intent_lower
|
||||
if is_input_field and not is_reply_intent:
|
||||
logger.debug("🛡️ [Reply Guard] Excluded input field for non-reply intent.")
|
||||
continue
|
||||
|
||||
# Posts/Authors shouldn't be massive full-screen containers unless it's a specific media intent
|
||||
is_post_author_intent = is_author_intent and "post" in intent_lower
|
||||
if is_post_author_intent and is_large_container and not is_image_or_video:
|
||||
logger.debug("🛡️ [Author Container Guard] Excluded massive container for author intent.")
|
||||
continue
|
||||
|
||||
filtered.append(node)
|
||||
|
||||
return filtered
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# Public API
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def resolve(
|
||||
self, intent_description: str, candidates: List[SpatialNode], device=None, screen_height: int = 2400
|
||||
) -> Optional[SpatialNode]:
|
||||
if not candidates:
|
||||
return None
|
||||
|
||||
intent_lower = intent_description.lower()
|
||||
|
||||
# Block abstract goals from leaking into node clicks
|
||||
abstract_goals = ["open profile", "open explore", "open following", "learn own profile"]
|
||||
if intent_lower in abstract_goals:
|
||||
return None
|
||||
|
||||
# --- Strict Structural Fast-Paths ---
|
||||
# ALL HARDCODED UI STRUCTURAL GUARDS HAVE BEEN PURGED!
|
||||
# Rule: ZERO MAINTENANCE. We do not hardcode Resource IDs, content_desc, or text matching
|
||||
# for any UI elements (Likes, Follows, Messages, Tabs, Posts, Story Rings, etc.).
|
||||
# The bot MUST autonomously find elements via the Telepathic VLM Engine and store
|
||||
# them in Qdrant memory after successful structural delta verification.
|
||||
|
||||
# --- Fast-Path: Send Message Button ---
|
||||
if "send message button" in intent_lower:
|
||||
for node in candidates:
|
||||
rid = (node.resource_id or "").lower()
|
||||
desc = (node.content_desc or "").lower()
|
||||
if "send_button" in rid or "absenden" in desc or "send" in desc or "send" in (node.text or "").lower():
|
||||
logger.info("📬 [Fast-Path] Matched 'Send Message Button' structurally. Bypassing VLM.")
|
||||
return node
|
||||
|
||||
# --- Fast-Path: Structural Tab Resolver ---
|
||||
# PRIMARY: Match by resource-id (architectural constants of the Instagram APK).
|
||||
# FALLBACK: Pure geometry if resource-IDs are missing (e.g. custom ROMs).
|
||||
# Resource IDs like feed_tab, profile_tab, clips_tab are NOT localized strings —
|
||||
# they are compile-time Android identifiers that NEVER change across locales.
|
||||
_TAB_MAP = {
|
||||
"home": {"id": "feed_tab", "desc": "home"},
|
||||
"profile": {"id": "profile_tab", "desc": "profile"},
|
||||
"explore": {"id": "search_tab", "desc": "search and explore"},
|
||||
"search": {"id": "search_tab", "desc": "search and explore"},
|
||||
"reels": {"id": "clips_tab", "desc": "reels"},
|
||||
"messages": {"id": "direct_tab", "desc": "message"},
|
||||
}
|
||||
if "tab" in intent_lower and "tap" in intent_lower and "back" not in intent_lower:
|
||||
# Strategy 1: Structural resource-id and content-desc match (O(1), zero ambiguity)
|
||||
for keyword, targets in _TAB_MAP.items():
|
||||
if keyword in intent_lower:
|
||||
for node in candidates:
|
||||
# Bottom 20% guard to ensure it's actually a tab and not someone's name
|
||||
if node.center_y < screen_height * 0.8:
|
||||
continue
|
||||
|
||||
rid = (node.resource_id or "").lower()
|
||||
desc = (node.content_desc or "").lower()
|
||||
text = (node.text or "").lower()
|
||||
|
||||
# Use strictly equal for desc/text to avoid matching "profile picture"
|
||||
if targets["id"] in rid or targets["desc"] == desc or targets["desc"] == text:
|
||||
logger.info(
|
||||
f"📐 [Structural Tab Resolver] Matched '{keyword}' tab structurally. Bypassing VLM."
|
||||
)
|
||||
return node
|
||||
break # keyword matched but no node found — fall through to geometry
|
||||
|
||||
# Strategy 2: Geometric fallback (for edge cases where resource-IDs are stripped)
|
||||
# Requirements: bottom 5% of screen, FrameLayout, long-clickable (tabs are always long-clickable)
|
||||
tab_candidates = []
|
||||
for node in candidates:
|
||||
if node.center_y > screen_height * 0.93:
|
||||
cls_name = (node.class_name or "").lower()
|
||||
is_long_clickable = getattr(node, "long_clickable", False)
|
||||
# Tabs are FrameLayouts that are long-clickable — this excludes:
|
||||
# - Post grid thumbnails (ImageView, not long-clickable)
|
||||
# - ViewGroup containers (wrong class)
|
||||
if "framelayout" in cls_name and is_long_clickable:
|
||||
tab_candidates.append(node)
|
||||
|
||||
if tab_candidates:
|
||||
tab_candidates.sort(key=lambda n: n.center_x)
|
||||
|
||||
# Deduplicate by X cluster (icon inside container shares same X)
|
||||
unique_tabs = []
|
||||
last_x = -1000
|
||||
for t in tab_candidates:
|
||||
if t.center_x - last_x > 100: # Tabs are ~216px apart, use 100px threshold
|
||||
unique_tabs.append(t)
|
||||
last_x = t.center_x
|
||||
|
||||
if len(unique_tabs) >= 4:
|
||||
logger.info(f"📐 [Geometric Tab Fallback] Found {len(unique_tabs)} bottom tabs. Bypassing VLM.")
|
||||
if "home" in intent_lower:
|
||||
return unique_tabs[0]
|
||||
elif "profile" in intent_lower:
|
||||
return unique_tabs[-1]
|
||||
elif "explore" in intent_lower or "search" in intent_lower:
|
||||
# search_tab is 4th from left in 5-tab layout
|
||||
return unique_tabs[3] if len(unique_tabs) >= 5 else unique_tabs[1]
|
||||
elif "reels" in intent_lower:
|
||||
return unique_tabs[1] if len(unique_tabs) >= 5 else unique_tabs[-2]
|
||||
|
||||
# --- Semantic Match Guard ---
|
||||
# If the intent explicitly quotes a target (e.g., "tap 'New Message'"),
|
||||
# we strictly filter candidates to those whose text or content_desc contains the quote.
|
||||
import re
|
||||
|
||||
quotes = re.findall(r"['\"](.*?)['\"]", intent_description)
|
||||
if quotes:
|
||||
target_text = quotes[0].lower()
|
||||
|
||||
# Only use the exact target string (no manual localized translation dictionaries!)
|
||||
localized_targets = [target_text]
|
||||
|
||||
semantic_candidates = []
|
||||
for node in candidates:
|
||||
n_text = _humanize_desc((node.text or "").lower())
|
||||
n_desc = _humanize_desc((node.content_desc or "").lower())
|
||||
|
||||
# Check if any of the localized targets match
|
||||
for loc_target in localized_targets:
|
||||
pattern = r"\b" + re.escape(loc_target) + r"\b"
|
||||
|
||||
# Map interaction text to structural ID patterns for multilingual support
|
||||
res_target = loc_target
|
||||
if loc_target == "following" or loc_target == "follow":
|
||||
res_target = "follow"
|
||||
elif loc_target == "message":
|
||||
res_target = "message"
|
||||
elif loc_target == "like":
|
||||
res_target = "like"
|
||||
elif loc_target == "comment":
|
||||
res_target = "comment"
|
||||
|
||||
if (
|
||||
re.search(pattern, n_text)
|
||||
or re.search(pattern, n_desc)
|
||||
or res_target in (node.resource_id or "").lower()
|
||||
):
|
||||
semantic_candidates.append(node)
|
||||
break # Found a match, no need to check other localized targets
|
||||
|
||||
if semantic_candidates:
|
||||
if len(semantic_candidates) == 1:
|
||||
logger.debug(f"🎯 [Semantic Guard] Exact match found for '{target_text}', skipping VLM.")
|
||||
return semantic_candidates[0]
|
||||
else:
|
||||
logger.info(
|
||||
f"🎯 [Semantic Guard] {len(semantic_candidates)} matches found for '{target_text}'. Reducing candidates for VLM."
|
||||
)
|
||||
candidates = semantic_candidates
|
||||
else:
|
||||
logger.warning(
|
||||
f"⚠️ [Semantic Guard] No candidates found containing '{target_text}'. Returning None to prevent hallucination."
|
||||
)
|
||||
return None
|
||||
|
||||
# ── PRIMARY PATH: Visual Discovery ──
|
||||
# If we have a device, the VLM SEES the screen and decides.
|
||||
if device is not None and (
|
||||
hasattr(device, "screenshot") or hasattr(getattr(device, "deviceV2", None), "screenshot")
|
||||
):
|
||||
logger.info("📸 Device screenshot capability detected. Enforcing visual discovery.")
|
||||
vlm_node = self._visual_discovery(intent_description, candidates, device, screen_height=screen_height)
|
||||
if vlm_node is not None:
|
||||
return vlm_node
|
||||
logger.warning("⚠️ Visual discovery returned None. Falling through to text-based fallback.")
|
||||
|
||||
# --- Strict VLM Hallucination Guard (Text-only Fallback) ---
|
||||
# For known structural targets that the text-based VLM frequently hallucinates when they are missing,
|
||||
# we enforce a strict failure.
|
||||
structural_intents = [
|
||||
"following list",
|
||||
"followers list",
|
||||
"tap message button",
|
||||
"tab",
|
||||
"scroll",
|
||||
"back",
|
||||
"home",
|
||||
"profile",
|
||||
"reels",
|
||||
"search",
|
||||
"explore",
|
||||
"send message button",
|
||||
]
|
||||
|
||||
if any(si in intent_lower for si in structural_intents):
|
||||
logger.warning(
|
||||
f"🛡️ [Hallucination Guard] Intent '{intent_description}' is a strict structural target. "
|
||||
"Since it wasn't resolved by fast-paths, it is either missing or blocked. Rejecting VLM fallback."
|
||||
)
|
||||
return None
|
||||
|
||||
# ── FALLBACK: Text-based VLM resolution ──
|
||||
# Only used when device is unavailable (e.g., unit tests without screenshots).
|
||||
return self._text_based_resolve(intent_description, candidates, device, screen_height=screen_height)
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# Visual Discovery (Set-of-Mark Prompting)
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def _annotate_screenshot_with_candidates(
|
||||
self, device, candidates: List[SpatialNode]
|
||||
) -> Tuple[str, Dict[int, SpatialNode]]:
|
||||
"""
|
||||
Takes a screenshot and draws numbered bounding boxes around clickable candidates.
|
||||
|
||||
Returns:
|
||||
annotated_b64: Base64-encoded JPEG of the annotated screenshot.
|
||||
box_map: Dict mapping box number → SpatialNode for coordinate lookup.
|
||||
"""
|
||||
from PIL import ImageDraw
|
||||
|
||||
img = device.deviceV2.screenshot()
|
||||
|
||||
# Stage 1: Basic area filter + exclude system UI and notifications (ALREADY HANDLED in _visual_discovery)
|
||||
pre_filtered = candidates
|
||||
|
||||
# Stage 2: Spatial deduplication
|
||||
# A node could completely contain another.
|
||||
# If parent is clickable and child is not: suppress child (e.g. text inside button)
|
||||
# If parent is not clickable and child is: suppress parent (e.g. layout container around button)
|
||||
# If both are not clickable: suppress parent (keep the smaller, more specific text)
|
||||
# If both are clickable: keep both! (e.g. nested buttons like row and camera icon)
|
||||
def _contains(parent: SpatialNode, child: SpatialNode) -> bool:
|
||||
return (
|
||||
parent.x1 <= child.x1
|
||||
and parent.y1 <= child.y1
|
||||
and parent.x2 >= child.x2
|
||||
and parent.y2 >= child.y2
|
||||
and parent.node_id != child.node_id
|
||||
)
|
||||
|
||||
to_suppress = set()
|
||||
# Sort by area DESCENDING so we process largest (parents) first
|
||||
pre_filtered.sort(key=lambda n: n.area, reverse=True)
|
||||
|
||||
for i, parent in enumerate(pre_filtered):
|
||||
for j in range(i + 1, len(pre_filtered)):
|
||||
child = pre_filtered[j]
|
||||
if _contains(parent, child):
|
||||
if parent.clickable and not child.clickable:
|
||||
to_suppress.add(child.node_id)
|
||||
# Merge semantic info from child to parent if missing
|
||||
if (
|
||||
child.text
|
||||
and child.text not in (parent.text or "")
|
||||
and child.text not in (parent.content_desc or "")
|
||||
):
|
||||
parent.content_desc = f"{(parent.content_desc or '')} {child.text}".strip()
|
||||
if (
|
||||
child.content_desc
|
||||
and child.content_desc not in (parent.text or "")
|
||||
and child.content_desc not in (parent.content_desc or "")
|
||||
):
|
||||
parent.content_desc = f"{(parent.content_desc or '')} {child.content_desc}".strip()
|
||||
elif not parent.clickable and child.clickable:
|
||||
to_suppress.add(parent.node_id)
|
||||
# Pass any semantic info down just in case
|
||||
if parent.content_desc and not child.content_desc:
|
||||
child.content_desc = parent.content_desc
|
||||
if parent.text and not child.text:
|
||||
child.text = parent.text
|
||||
elif not parent.clickable and not child.clickable:
|
||||
to_suppress.add(parent.node_id)
|
||||
if parent.content_desc and not child.content_desc:
|
||||
child.content_desc = parent.content_desc
|
||||
elif parent.clickable and child.clickable:
|
||||
# Keep both, distinct nested interactables
|
||||
pass
|
||||
|
||||
visible_candidates = [n for n in pre_filtered if n.node_id not in to_suppress]
|
||||
|
||||
draw = ImageDraw.Draw(img)
|
||||
box_map: Dict[int, SpatialNode] = {}
|
||||
|
||||
# Color palette for distinct boxes
|
||||
colors = [
|
||||
(255, 0, 0),
|
||||
(0, 200, 0),
|
||||
(0, 0, 255),
|
||||
(255, 165, 0),
|
||||
(128, 0, 128),
|
||||
(0, 200, 200),
|
||||
(255, 20, 147),
|
||||
(0, 128, 0),
|
||||
(255, 215, 0),
|
||||
(70, 130, 180),
|
||||
]
|
||||
|
||||
for i, node in enumerate(visible_candidates):
|
||||
color = colors[i % len(colors)]
|
||||
|
||||
# Draw bounding box
|
||||
draw.rectangle(
|
||||
[node.x1, node.y1, node.x2, node.y2],
|
||||
outline=color,
|
||||
width=3,
|
||||
)
|
||||
|
||||
# Draw number label with background for readability
|
||||
label = str(i)
|
||||
label_x = node.x1 + 2
|
||||
label_y = max(node.y1 - 18, 0)
|
||||
|
||||
# Draw label background
|
||||
bbox = draw.textbbox((label_x, label_y), label)
|
||||
draw.rectangle(
|
||||
[bbox[0] - 2, bbox[1] - 2, bbox[2] + 2, bbox[3] + 2],
|
||||
fill=color,
|
||||
)
|
||||
draw.text((label_x, label_y), label, fill=(255, 255, 255))
|
||||
|
||||
box_map[i] = node
|
||||
|
||||
# Encode to base64 JPEG
|
||||
buffered = BytesIO()
|
||||
img.save(buffered, format="JPEG", quality=85)
|
||||
annotated_b64 = base64.b64encode(buffered.getvalue()).decode("utf-8")
|
||||
|
||||
return annotated_b64, box_map
|
||||
|
||||
def _visual_discovery(
|
||||
self, intent_description: str, candidates: List[SpatialNode], device, screen_height: int = 2400
|
||||
) -> Optional[SpatialNode]:
|
||||
"""
|
||||
Vision-first intent resolution via Set-of-Mark (SoM) prompting.
|
||||
|
||||
1. Takes a screenshot
|
||||
2. Draws numbered bounding boxes on clickable candidates
|
||||
3. Sends the annotated screenshot to the VLM
|
||||
4. VLM SEES the UI and picks which numbered box matches the intent
|
||||
5. Maps box number back to SpatialNode for precise coordinates
|
||||
"""
|
||||
from GramAddict.core.config import Config
|
||||
from GramAddict.core.llm_provider import query_telepathic_llm
|
||||
|
||||
# Pre-filter candidates by area and system UI before any semantic matching
|
||||
candidates = [
|
||||
n
|
||||
for n in candidates
|
||||
if 200 < n.area < 400000
|
||||
and "com.android.systemui" not in (n.resource_id or "")
|
||||
and "notification:" not in (n.content_desc or "").lower()
|
||||
and "per cent" not in (n.content_desc or "").lower()
|
||||
and "keyboard" not in (n.resource_id or "").lower()
|
||||
and "input_method" not in (n.resource_id or "").lower()
|
||||
and "tastatur" not in (n.content_desc or "").lower()
|
||||
and "eingabetaste" not in (n.content_desc or "").lower()
|
||||
]
|
||||
|
||||
# --- Navigation Conflict Guard ---
|
||||
# Prevents VLM from confusing Back buttons with tab buttons
|
||||
# Production bug 2026-04-30: VLM picked Back for "tap profile tab"
|
||||
candidates = self.filter_navigation_conflicts(candidates, intent_description, screen_height=screen_height)
|
||||
|
||||
# --- Strict Button Guard ---
|
||||
# If the intent specifically asks for a "button", "icon", or "tab",
|
||||
# filter out candidates that contain long text (e.g. captions, comments)
|
||||
# to prevent the VLM from hallucinating text nodes as interactive buttons.
|
||||
intent_lower = intent_description.lower()
|
||||
if "button" in intent_lower or "icon" in intent_lower or "tab" in intent_lower:
|
||||
filtered_candidates = []
|
||||
for node in candidates:
|
||||
text_len = len(node.text or "")
|
||||
if text_len < 40:
|
||||
filtered_candidates.append(node)
|
||||
else:
|
||||
logger.debug(f"🛡️ [Strict Button Guard] Filtered out node with long text: '{node.text[:20]}...'")
|
||||
candidates = filtered_candidates
|
||||
|
||||
# --- Post/Grid Item Guard ---
|
||||
# Removed hardcoded English string matching for 'row 1', 'photos by'. We trust the VLM.
|
||||
pass
|
||||
|
||||
# --- Author/Username Guard ---
|
||||
# Geometric constraint: Author and username on profiles/posts are usually in the top half.
|
||||
if "author" in intent_lower or "username" in intent_lower or "profile name" in intent_lower:
|
||||
filtered_candidates = []
|
||||
for node in candidates:
|
||||
if node.center_y > (screen_height * 0.85):
|
||||
logger.debug("🛡️ [Author Guard] Filtered out bottom area element.")
|
||||
else:
|
||||
filtered_candidates.append(node)
|
||||
candidates = filtered_candidates
|
||||
|
||||
# --- Reply Guard ---
|
||||
# Prevents VLM from clicking input fields for non-reply intents
|
||||
if (
|
||||
"reply" not in intent_lower
|
||||
and "message" not in intent_lower
|
||||
and "comment" not in intent_lower
|
||||
and "type" not in intent_lower
|
||||
and "write" not in intent_lower
|
||||
):
|
||||
filtered_candidates = []
|
||||
for node in candidates:
|
||||
cls_name = (node.class_name or "").lower()
|
||||
if "edittext" in cls_name:
|
||||
logger.debug("🛡️ [Reply Guard] Filtered out input field.")
|
||||
else:
|
||||
filtered_candidates.append(node)
|
||||
candidates = filtered_candidates
|
||||
|
||||
try:
|
||||
annotated_b64, box_map = self._annotate_screenshot_with_candidates(device, candidates)
|
||||
except Exception as e:
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
logger.warning(f"⚠️ [Visual Discovery] Screenshot annotation failed: {e}")
|
||||
return None
|
||||
|
||||
if not box_map:
|
||||
return None
|
||||
|
||||
self.last_box_map = box_map
|
||||
|
||||
cfg = Config()
|
||||
model = getattr(cfg.args, "ai_telepathic_model", "llava:latest")
|
||||
url = getattr(cfg.args, "ai_telepathic_url", "http://localhost:11434/api/generate")
|
||||
|
||||
# Build a compact legend of what each box contains
|
||||
box_legend_lines = []
|
||||
for idx in sorted(box_map.keys()):
|
||||
node = box_map[idx]
|
||||
label_parts = []
|
||||
if node.content_desc:
|
||||
desc = _humanize_desc(node.content_desc)
|
||||
label_parts.append(f"desc='{desc[:50]}'")
|
||||
if node.text and node.text != node.content_desc:
|
||||
text = _humanize_desc(node.text)
|
||||
label_parts.append(f"text='{text[:50]}'")
|
||||
if node.class_name:
|
||||
cls_short = node.class_name.split(".")[-1]
|
||||
label_parts.append(f"class='{cls_short}'")
|
||||
if node.long_clickable:
|
||||
label_parts.append("long_clickable=True")
|
||||
if not label_parts:
|
||||
label_parts.append("(no visible text)")
|
||||
box_legend_lines.append(f" [{idx}] {', '.join(label_parts)}")
|
||||
box_legend = "\n".join(box_legend_lines)
|
||||
logger.debug(f"BOX LEGEND:\n{box_legend}")
|
||||
|
||||
prompt = (
|
||||
f"You are looking at a mobile app screenshot with numbered bounding boxes drawn around interactive UI elements.\n"
|
||||
f"Each box has a number label in a colored rectangle.\n\n"
|
||||
f"Box legend (what each box contains):\n{box_legend}\n\n"
|
||||
f"Your task: Find the exact box number that corresponds to this intent: '{intent_description}'\n\n"
|
||||
f"CRITICAL RULES:\n"
|
||||
f"0. MULTILINGUAL UI AWARENESS: The UI might be in any language (English, German, etc.). You MUST translate the intent conceptually. If looking for 'Search', also accept 'Suche'. If looking for 'Following', also accept 'Abonniert'. If looking for 'Message', also accept 'Nachricht'.\n"
|
||||
f"1. If the intent contains a word in quotes (e.g., 'Search', 'New Message'), look at the Box legend and pick the box that contains that word or its localized equivalent.\n"
|
||||
f"2. For icons without text:\n"
|
||||
f" - 'like button' = HEART-SHAPED ICON (♡/❤).\n"
|
||||
f" - 'comment button' = SPEECH BUBBLE ICON.\n"
|
||||
f"3. Do NOT select text, captions, or view counts if looking for an icon.\n"
|
||||
f"4. Ignore numbers inside the text itself. Do not confuse the text '19' with Box [19].\n"
|
||||
f"5. If the intent is to tap a 'post', 'first post', or 'grid item':\n"
|
||||
f" - Look for boxes indicating a photo or video by a user (e.g., 'photos by', 'Foto von', or grid coordinates like 'row 1' / 'Reihe 1').\n"
|
||||
f" - Pick the FIRST matching box index.\n"
|
||||
f" - Do NOT pick navigation buttons like 'Search'.\n"
|
||||
f"6. If the intent is a bottom navigation tab (e.g. 'profile tab', 'home tab'):\n"
|
||||
f" - These are always at the BOTTOM edge of the screen.\n"
|
||||
f" - 'profile tab' is usually the furthest right icon (your avatar).\n"
|
||||
f" - 'home tab' is the furthest left icon (house).\n"
|
||||
f" - 'explore tab' is the magnifying glass.\n"
|
||||
f" - 'reels tab' is the video clapperboard.\n"
|
||||
f"7. If the intent involves 'author username' or 'author profile':\n"
|
||||
f" - Pick the profile picture or the username text.\n"
|
||||
f" - NEVER pick a 'Follow' button.\n"
|
||||
f"8. If the intent is 'save post':\n"
|
||||
f" - The save icon is the bookmark icon on the bottom right of the post image/video.\n"
|
||||
f"9. DISTINGUISHING BOTTOM TABS vs CONTENT BUTTONS:\n"
|
||||
f" - Bottom Navigation Tabs (Home, Search, Reels, Profile) are ALWAYS at the very bottom (y > 2100).\n"
|
||||
f" - Content Interaction Buttons (Like, Comment, Share, Reactions, Message Input) are attached to posts or threads, NOT the bottom nav bar.\n"
|
||||
f"10. If the intent is 'feed post content' or 'post media content':\n"
|
||||
f" - Pick the largest box that contains the actual image or video.\n"
|
||||
f"11. DO NOT HALLUCINATE. If you are on the wrong screen, or if the exact target is simply NOT visible, you MUST return null.\n"
|
||||
f"12. EXTREME GUARD: NEVER pick an input field (class='EditText') unless the intent EXPLICITLY asks you to type or reply.\n"
|
||||
f"13. EXTREME GUARD: Do NOT pick items that are 'long_clickable=True' if your intent is just a simple navigation click.\n"
|
||||
f"14. If the intent is 'tap post username', DO NOT pick random gallery folders like 'Recents' or 'Select album'. Return null.\n"
|
||||
f"15. EXTREME GUARD: If the intent is to tap 'following' or 'followers' list, NEVER pick a 'Follow' or 'Follow back' button.\n\n"
|
||||
f"VALID BOX NUMBERS: {list(box_map.keys())}\n"
|
||||
f'Reply ONLY with a valid JSON object: {{"box": <number from VALID BOX NUMBERS>}} or {{"box": null}}'
|
||||
)
|
||||
|
||||
try:
|
||||
res = query_telepathic_llm(
|
||||
model=model,
|
||||
url=url,
|
||||
system_prompt="Strict visual JSON box selector. Respond only with JSON.",
|
||||
user_prompt=prompt,
|
||||
use_local_edge=True,
|
||||
images_b64=[annotated_b64],
|
||||
)
|
||||
data = json.loads(res)
|
||||
box_idx = self._parse_box_index(data)
|
||||
|
||||
# Additional safety check to prevent hallucinated numbers
|
||||
if box_idx not in box_map:
|
||||
logger.warning(f"👁️ [Visual Discovery] VLM hallucinated invalid box number: {box_idx}")
|
||||
box_idx = None
|
||||
|
||||
selected = self._validate_and_get_node(box_idx, box_map)
|
||||
|
||||
if selected:
|
||||
logger.info(
|
||||
f"👁️ [Visual Discovery] VLM selected box [{box_idx}] → "
|
||||
f"id='{selected.resource_id}', desc='{selected.content_desc}'"
|
||||
)
|
||||
return selected
|
||||
else:
|
||||
logger.warning(f"👁️ [Visual Discovery] VLM returned invalid box={box_idx} (OOB or unparsable).")
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ [Visual Discovery] VLM call failed: {e}")
|
||||
|
||||
return None
|
||||
|
||||
def _parse_box_index(self, data: Dict) -> Optional[int]:
|
||||
"""Parses the box index from VLM JSON response, handling common hallucination formats."""
|
||||
if not isinstance(data, dict):
|
||||
return None
|
||||
|
||||
# Check common keys
|
||||
val = None
|
||||
for key in ["box", "selected_index", "box_index", "index", "target"]:
|
||||
if key in data:
|
||||
val = data[key]
|
||||
break
|
||||
|
||||
if val is None:
|
||||
return None
|
||||
|
||||
# Handle null/None
|
||||
if val in [None, "null", "None", "None ", "null "]:
|
||||
return None
|
||||
|
||||
# Convert to string and clean up common VLM prefixes
|
||||
val_str = str(val).strip().lower()
|
||||
|
||||
# Remove "box " prefix if present (e.g. "Box 5")
|
||||
if val_str.startswith("box"):
|
||||
val_str = val_str.replace("box", "").strip()
|
||||
|
||||
# Attempt integer conversion
|
||||
try:
|
||||
# Extract first number if it's a messy string
|
||||
import re
|
||||
|
||||
match = re.search(r"(\d+)", val_str)
|
||||
if match:
|
||||
return int(match.group(1))
|
||||
return None
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
|
||||
def _validate_and_get_node(self, box_idx: Optional[int], box_map: Dict[int, SpatialNode]) -> Optional[SpatialNode]:
|
||||
"""Validates that the box index exists within the current SoM box_map."""
|
||||
if box_idx is None:
|
||||
return None
|
||||
|
||||
if box_idx in box_map:
|
||||
return box_map[box_idx]
|
||||
|
||||
return None
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# Text-based Fallback (no device/screenshot)
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def _text_based_resolve(
|
||||
self, intent_description: str, candidates: List[SpatialNode], device=None, screen_height: int = 2400
|
||||
) -> Optional[SpatialNode]:
|
||||
"""
|
||||
Fallback resolution via text descriptions of XML nodes.
|
||||
Used only when no device is available for screenshots.
|
||||
"""
|
||||
from GramAddict.core.config import Config
|
||||
from GramAddict.core.llm_provider import query_telepathic_llm
|
||||
|
||||
intent_lower = intent_description.lower()
|
||||
|
||||
filtered_candidates = [n for n in candidates if n.area < 500000]
|
||||
filtered_candidates = self.filter_navigation_conflicts(
|
||||
filtered_candidates, intent_description, screen_height=screen_height
|
||||
)
|
||||
if "profile" in intent_lower:
|
||||
filtered_candidates = [
|
||||
n
|
||||
for n in filtered_candidates
|
||||
if not any(kw in (n.resource_id or "").lower() for kw in ("tab", "navigation", "action_bar"))
|
||||
]
|
||||
if not filtered_candidates:
|
||||
filtered_candidates = [n for n in candidates if n.area < 500000]
|
||||
|
||||
cfg = Config()
|
||||
model = getattr(cfg.args, "ai_telepathic_model", "qwen3.5:latest")
|
||||
url = getattr(cfg.args, "ai_telepathic_url", "http://localhost:11434/api/generate")
|
||||
|
||||
node_context = []
|
||||
for i, node in enumerate(filtered_candidates):
|
||||
text = _humanize_desc(node.text or "")
|
||||
desc = _humanize_desc(node.content_desc or "")
|
||||
res_id = node.resource_id or ""
|
||||
node_context.append(f"[{i}] text='{text}', desc='{desc}', id='{res_id}', bounds=[{node.y1},{node.y2}]")
|
||||
|
||||
prompt = (
|
||||
f"You are a Spatial UI Intent Resolver.\n"
|
||||
f"Goal: Find the single best UI element to interact with to satisfy the intent: '{intent_description}'.\n"
|
||||
f"Candidates:\n" + "\n".join(node_context) + "\n\n"
|
||||
"CRITICAL RULES:\n"
|
||||
"0. MULTILINGUAL UI AWARENESS: The UI might be in any language (English, German, etc.). You MUST translate the intent conceptually and find the corresponding localized element.\n"
|
||||
"1. If the intent is a bottom navigation tab (e.g. 'profile tab', 'home tab'):\n"
|
||||
" - These are always at the BOTTOM of the screen (typically y > 2100).\n"
|
||||
" - 'profile tab' is usually the furthest right.\n"
|
||||
" - 'home tab' is the furthest left.\n"
|
||||
" - Do NOT select 'Go to <user>'s profile' or other header text.\n"
|
||||
"2. EXTREME GUARD: NEVER pick an input field (EditText) unless explicitly asked to type.\n"
|
||||
"3. EXTREME GUARD: If you are on the wrong screen entirely (e.g. 'Select album' gallery) instead of a profile, return null.\n"
|
||||
"4. If none of the candidates clearly and safely match the intent, return null. DO NOT guess.\n\n"
|
||||
"Reply ONLY with a valid JSON object strictly matching this schema:\n"
|
||||
'{"selected_index": <integer or null>}\n'
|
||||
)
|
||||
|
||||
try:
|
||||
res = query_telepathic_llm(
|
||||
model=model,
|
||||
url=url,
|
||||
system_prompt="Strict JSON intent resolver.",
|
||||
user_prompt=prompt,
|
||||
use_local_edge=True,
|
||||
)
|
||||
data = json.loads(res)
|
||||
idx = data.get("selected_index")
|
||||
if idx is not None and 0 <= idx < len(filtered_candidates):
|
||||
return filtered_candidates[idx]
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ [IntentResolver] Text-based VLM resolution failed ({e}).")
|
||||
|
||||
return None
|
||||
427
GramAddict/core/perception/screen_identity.py
Normal file
427
GramAddict/core/perception/screen_identity.py
Normal file
@@ -0,0 +1,427 @@
|
||||
import hashlib
|
||||
import logging
|
||||
import re
|
||||
import xml.etree.ElementTree as ET
|
||||
from enum import Enum
|
||||
from typing import Any, Dict
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ScreenType(Enum):
|
||||
HOME_FEED = "home_feed"
|
||||
EXPLORE_GRID = "explore_grid"
|
||||
REELS_FEED = "reels_feed"
|
||||
AUDIO_PAGE = "audio_page"
|
||||
OWN_PROFILE = "own_profile"
|
||||
OTHER_PROFILE = "other_profile"
|
||||
POST_DETAIL = "post_detail"
|
||||
STORY_VIEW = "story_view"
|
||||
DM_INBOX = "dm_inbox"
|
||||
DM_THREAD = "dm_thread"
|
||||
SEARCH_RESULTS = "search_results"
|
||||
FOLLOW_LIST = "follow_list"
|
||||
COMMENTS = "comments"
|
||||
MODAL = "modal"
|
||||
FOREIGN_APP = "foreign_app"
|
||||
NOTIFICATIONS = "notifications"
|
||||
UNKNOWN = "unknown"
|
||||
|
||||
|
||||
class ScreenIdentity:
|
||||
"""
|
||||
Understands what screen the bot is on by analyzing the XML dump.
|
||||
NO hardcoded states — purely structural analysis.
|
||||
|
||||
This is the bot's EYES. It answers: "What do I see right now?"
|
||||
"""
|
||||
|
||||
def __init__(self, bot_username: str):
|
||||
self.bot_username = bot_username.lower()
|
||||
try:
|
||||
from GramAddict.core.qdrant_memory import ScreenMemoryDB
|
||||
|
||||
self.screen_memory = ScreenMemoryDB()
|
||||
if self.screen_memory:
|
||||
self.screen_memory.purge_stale_screens()
|
||||
except ImportError:
|
||||
self.screen_memory = None
|
||||
|
||||
def identify(self, xml_dump: str, screenshot_b64: str = None) -> Dict[str, Any]:
|
||||
"""
|
||||
Analyzes an XML dump and returns a complete screen description.
|
||||
|
||||
Returns:
|
||||
{
|
||||
'screen_type': ScreenType,
|
||||
'available_actions': ['tap like button', 'tap explore tab', ...],
|
||||
'selected_tab': 'feed_tab' | 'search_tab' | ...,
|
||||
'context': {'username': '...', 'post_count': '...', ...}
|
||||
}
|
||||
"""
|
||||
if not xml_dump or not isinstance(xml_dump, str):
|
||||
return self._empty_screen()
|
||||
|
||||
try:
|
||||
clean = re.sub(r"<\?xml.*?\?>", "", xml_dump).strip()
|
||||
root = ET.fromstring(clean)
|
||||
except Exception:
|
||||
return self._empty_screen()
|
||||
|
||||
# Extract structural signals
|
||||
packages = set()
|
||||
resource_ids = set()
|
||||
content_descs = []
|
||||
texts = []
|
||||
selected_tab = None
|
||||
clickable_elements = []
|
||||
|
||||
app_id = "com.instagram.android"
|
||||
|
||||
for elem in root.iter("node"):
|
||||
pkg = elem.get("package", "")
|
||||
if pkg:
|
||||
packages.add(pkg)
|
||||
|
||||
rid = elem.get("resource-id", "").strip()
|
||||
text = elem.get("text", "").strip()
|
||||
desc = elem.get("content-desc", "").strip()
|
||||
clickable = elem.get("clickable", "false") == "true"
|
||||
selected = elem.get("selected", "false") == "true"
|
||||
long_clickable = elem.get("long-clickable", "false") == "true"
|
||||
class_name = elem.get("class", "").strip()
|
||||
bounds = elem.get("bounds", "")
|
||||
|
||||
if rid:
|
||||
# Normalize: "com.instagram.android:id/feed_tab" → "feed_tab"
|
||||
short_id = rid.split("/")[-1] if "/" in rid else rid
|
||||
resource_ids.add(short_id)
|
||||
|
||||
# Track which tab is selected
|
||||
if selected and short_id in (
|
||||
"feed_tab",
|
||||
"search_tab",
|
||||
"clips_tab",
|
||||
"profile_tab",
|
||||
"direct_tab",
|
||||
"news_tab",
|
||||
):
|
||||
selected_tab = short_id
|
||||
|
||||
if text:
|
||||
texts.append(text)
|
||||
if desc:
|
||||
content_descs.append(desc)
|
||||
|
||||
if clickable and bounds:
|
||||
match = re.match(r"\[(\d+),(\d+)\]\[(\d+),(\d+)\]", bounds)
|
||||
if match:
|
||||
left, t, r, b = map(int, match.groups())
|
||||
cx, cy = (left + r) // 2, (t + b) // 2
|
||||
clickable_elements.append(
|
||||
{
|
||||
"text": text,
|
||||
"desc": desc,
|
||||
"id": rid.split("/")[-1] if "/" in rid else rid,
|
||||
"class": class_name,
|
||||
"long_clickable": long_clickable,
|
||||
"x": cx,
|
||||
"y": cy,
|
||||
"bounds": bounds,
|
||||
"bottom": b,
|
||||
}
|
||||
)
|
||||
|
||||
from GramAddict.core.situational_awareness import SituationalAwarenessEngine
|
||||
|
||||
sae = SituationalAwarenessEngine.get_instance()
|
||||
signature = sae._compress_xml(xml_dump) if sae else self._compute_signature(resource_ids, content_descs, texts)
|
||||
|
||||
# ── Foreign app check ──
|
||||
if app_id not in packages:
|
||||
return {
|
||||
"screen_type": ScreenType.FOREIGN_APP,
|
||||
"available_actions": ["press back", "force start instagram"],
|
||||
"selected_tab": None,
|
||||
"context": {"packages": list(packages)},
|
||||
"signature": signature,
|
||||
}
|
||||
|
||||
desc_lower = " ".join(content_descs).lower()
|
||||
text_lower = " ".join(texts).lower()
|
||||
ids_str = " ".join(resource_ids).lower()
|
||||
|
||||
# ── Identify screen type from structural signals ──
|
||||
screen_type = self._classify_screen(
|
||||
resource_ids, content_descs, texts, selected_tab, desc_lower, text_lower, ids_str, signature, screenshot_b64
|
||||
)
|
||||
|
||||
# ── Extract available actions from clickable elements ──
|
||||
available_actions = self._extract_available_actions(
|
||||
clickable_elements, resource_ids, content_descs, texts, screen_type
|
||||
)
|
||||
|
||||
# ── Extract context ──
|
||||
context = self._extract_context(content_descs, texts, resource_ids, screen_type)
|
||||
|
||||
return {
|
||||
"screen_type": screen_type,
|
||||
"available_actions": available_actions,
|
||||
"selected_tab": selected_tab,
|
||||
"context": context,
|
||||
"signature": signature,
|
||||
"resource_ids": resource_ids,
|
||||
}
|
||||
|
||||
def _classify_screen(
|
||||
self, ids, descs, texts, selected_tab, desc_lower, text_lower, ids_str, signature=None, screenshot_b64=None
|
||||
):
|
||||
"""
|
||||
Classify screen type using Semantic Memory with LLM fallback — NO hardcoded states."""
|
||||
|
||||
# Priority 0: Fetch Qdrant Semantic Cache
|
||||
# We fetch this early to see if there is a 'NORMAL' override for the MODAL check.
|
||||
# We DO NOT let this override deterministic structural heuristics! Fuzzy vector matching
|
||||
# can easily confuse HOME_FEED and OWN_PROFILE if the bottom navigation bar is identical.
|
||||
cached_type_str = None
|
||||
if signature and self.screen_memory and self.screen_memory.is_connected:
|
||||
cached_type_str = self.screen_memory.get_screen_type(signature, similarity_threshold=0.98)
|
||||
|
||||
is_normal_override = cached_type_str == "NORMAL"
|
||||
|
||||
# Priority 1: High-Confidence Structural Fast-Paths (resource-id invariants)
|
||||
# Resource IDs are architectural constants of the APK, not localized text.
|
||||
# This complies with the Zero Maintenance rule while preventing VLM hallucinations.
|
||||
|
||||
# Story View
|
||||
if "reel_viewer_root" in ids_str or "story_viewer_root" in ids_str or "reel_viewer_media_container" in ids_str:
|
||||
return ScreenType.STORY_VIEW
|
||||
|
||||
# Reels Feed
|
||||
if selected_tab == "clips_tab" or "clips_video_container" in ids_str or "clips_slider" in ids_str:
|
||||
return ScreenType.REELS_FEED
|
||||
|
||||
# Direct Messages
|
||||
if "direct_inbox_action_bar" in ids_str or "inbox_refreshable_thread_list_recyclerview" in ids_str:
|
||||
return ScreenType.DM_INBOX
|
||||
if (
|
||||
"direct_text_message_text_view" in ids_str
|
||||
or "message_content" in ids_str
|
||||
or "thread_title" in ids_str
|
||||
or "message_list" in ids_str
|
||||
):
|
||||
return ScreenType.DM_THREAD
|
||||
|
||||
# Notifications
|
||||
if selected_tab == "news_tab" or "notifications_list" in ids_str or "activity_feed_root" in ids_str:
|
||||
return ScreenType.NOTIFICATIONS
|
||||
|
||||
# ── PROFILE DETECTION (must happen BEFORE tab-based HOME/EXPLORE) ──
|
||||
# WHY: OWN_PROFILE and OTHER_PROFILE have profile_tab visible but NOT selected.
|
||||
# If we check selected_tab == "feed_tab" first, profiles without a selected tab
|
||||
# fall through to the Qdrant cache, which can be poisoned.
|
||||
#
|
||||
# Structural Anchor: profile_tab selected=true → OWN_PROFILE (absolute invariant)
|
||||
# When viewing someone else's profile, profile_tab is NEVER selected.
|
||||
if selected_tab == "profile_tab":
|
||||
return ScreenType.OWN_PROFILE
|
||||
|
||||
# Profile screens with explicit header markers
|
||||
if "profile_header" in ids_str or "profile_tab_layout" in ids_str:
|
||||
# OWN_PROFILE has "Edit Profile" or the tab switcher (Posts/Reels/Tagged)
|
||||
if "profile_header_edit_profile_button" in ids_str or "layout_button_group_view_switcher" in ids_str:
|
||||
return ScreenType.OWN_PROFILE
|
||||
# OTHER_PROFILE has "Message" or "Follow" buttons
|
||||
if (
|
||||
"profile_header_message_button" in ids_str
|
||||
or "profile_header_follow_button" in ids_str
|
||||
or "button_message" in ids_str
|
||||
):
|
||||
return ScreenType.OTHER_PROFILE
|
||||
# Fallback: If profile_header exists but neither edit-profile nor follow-button,
|
||||
# it's likely OWN_PROFILE in a non-standard state (e.g. professional dashboard).
|
||||
# Better to guess OWN_PROFILE than to let Qdrant poison us with OTHER_PROFILE.
|
||||
logger.debug(
|
||||
"📐 [ScreenIdentity] Profile header detected but no edit/follow button. Defaulting to OWN_PROFILE."
|
||||
)
|
||||
return ScreenType.OWN_PROFILE
|
||||
|
||||
# Explore Grid
|
||||
if selected_tab == "search_tab":
|
||||
return ScreenType.EXPLORE_GRID
|
||||
|
||||
# Home Feed
|
||||
if selected_tab == "feed_tab":
|
||||
return ScreenType.HOME_FEED
|
||||
|
||||
# Post Detail: Typically has comment box or media note view but NO bottom tab layout
|
||||
if "media_note_view" in ids_str or "comment" in ids_str or "row_feed_button_comment" in ids_str:
|
||||
if "feed_tab" not in ids_str and "profile_tab" not in ids_str:
|
||||
return ScreenType.POST_DETAIL
|
||||
|
||||
# Comments
|
||||
if "layout_comment_thread" in ids_str or "comment_thread_recyclerview" in ids_str:
|
||||
return ScreenType.COMMENTS
|
||||
|
||||
# Follow List
|
||||
if (
|
||||
"follow_list_container" in ids_str
|
||||
or "layout_user_list" in ids_str
|
||||
or "layout_user_row" in ids_str
|
||||
or "follow_list_username" in ids_str
|
||||
):
|
||||
return ScreenType.FOLLOW_LIST
|
||||
|
||||
# Priority 2: Modal / Overlay Overrides
|
||||
if not is_normal_override:
|
||||
if "bottom_sheet_container" in ids_str or "action_sheet" in ids_str:
|
||||
return ScreenType.MODAL
|
||||
|
||||
# Priority 3: Cached Semantic Type (If deterministic heuristics failed)
|
||||
# GUARD: Profile types MUST be resolved by structural fast-paths above.
|
||||
# If they weren't, the cache is poisoned. Reject profile cache hits.
|
||||
_STRUCTURALLY_GATED_TYPES = (
|
||||
ScreenType.STORY_VIEW,
|
||||
ScreenType.REELS_FEED,
|
||||
ScreenType.OWN_PROFILE,
|
||||
ScreenType.OTHER_PROFILE,
|
||||
)
|
||||
if cached_type_str and cached_type_str != "NORMAL":
|
||||
try:
|
||||
cached_type = ScreenType[cached_type_str]
|
||||
if cached_type in _STRUCTURALLY_GATED_TYPES:
|
||||
logger.warning(
|
||||
f"⚠️ [ScreenIdentity] Rejecting cached {cached_type.name} — "
|
||||
f"this type MUST be resolved structurally. Cache is unreliable."
|
||||
)
|
||||
else:
|
||||
return cached_type
|
||||
except KeyError:
|
||||
pass
|
||||
|
||||
# Priority 4: Semantic VLM Classification Fallback
|
||||
if not screenshot_b64 and getattr(self, "device", None) is not None:
|
||||
screenshot_b64 = self.device.get_screenshot_b64()
|
||||
|
||||
from GramAddict.core.config import Config
|
||||
from GramAddict.core.llm_provider import query_telepathic_llm
|
||||
|
||||
cfg = Config()
|
||||
url = (
|
||||
getattr(cfg.args, "ai_telepathic_url", "http://localhost:11434/api/generate")
|
||||
if hasattr(cfg, "args")
|
||||
else "http://localhost:11434/api/generate"
|
||||
)
|
||||
model = getattr(cfg.args, "ai_telepathic_model", "llava:latest") if hasattr(cfg, "args") else "llava:latest"
|
||||
|
||||
layout_context = (
|
||||
f"Selected Tab: {selected_tab}\nResource IDs: {list(ids)}\nVisible Texts context: {texts[:10]}\n"
|
||||
)
|
||||
prompt = (
|
||||
f"Identify the Instagram screen layout type based on the provided screenshot and structural signals.\n"
|
||||
f"Valid types: {[t.name for t in ScreenType]}\n"
|
||||
f"Context:\n{layout_context}\n"
|
||||
f"Reply ONLY with the exact matching enum Type Name string, or 'UNKNOWN' if no type matches."
|
||||
)
|
||||
|
||||
try:
|
||||
response = query_telepathic_llm(
|
||||
model=model,
|
||||
url=url,
|
||||
system_prompt=prompt,
|
||||
user_prompt="Classify this screen layout.",
|
||||
images_b64=[screenshot_b64] if screenshot_b64 else None,
|
||||
temperature=0.0,
|
||||
use_local_edge=True,
|
||||
)
|
||||
|
||||
result = response.strip().upper() if response else "UNKNOWN"
|
||||
|
||||
for t in ScreenType:
|
||||
if t.name in result:
|
||||
if is_normal_override and t == ScreenType.MODAL:
|
||||
# Prevent the LLM from hallucinating an obstacle if explicitly verified as NORMAL
|
||||
return ScreenType.UNKNOWN
|
||||
|
||||
# Enforce absolute structural parity: These types MUST be resolved
|
||||
# by resource-id fast-paths. VLM guessing them poisons the Qdrant cache.
|
||||
if t in _STRUCTURALLY_GATED_TYPES:
|
||||
logger.warning(
|
||||
f"⚠️ [ScreenIdentity] Rejecting VLM hallucinated {t.name} — "
|
||||
f"this type MUST be resolved structurally to prevent cache poisoning."
|
||||
)
|
||||
return ScreenType.UNKNOWN
|
||||
|
||||
if signature and self.screen_memory:
|
||||
self.screen_memory.store_screen(signature, t.name)
|
||||
return t
|
||||
except Exception as e:
|
||||
import logging
|
||||
|
||||
logging.getLogger(__name__).debug(f"LLM Classification failed: {e}")
|
||||
|
||||
return ScreenType.UNKNOWN
|
||||
|
||||
def _extract_available_actions(self, clickable_elements, resource_ids, content_descs, texts, screen_type):
|
||||
"""Discover what actions are possible on this screen using structural fast-paths.
|
||||
|
||||
PRIMARY: resource-id matching (architectural constants of the Instagram APK).
|
||||
Resource IDs like feed_tab, profile_tab, clips_tab are compile-time Android
|
||||
identifiers that NEVER change across locales. This is NOT hardcoded text matching.
|
||||
"""
|
||||
actions = []
|
||||
|
||||
# --- Structural Tab Detection (resource-id based) ---
|
||||
# These are architectural constants of the APK, not localized strings.
|
||||
tab_map = {
|
||||
"feed_tab": "tap home tab",
|
||||
"search_tab": "tap explore tab",
|
||||
"clips_tab": "tap reels tab",
|
||||
"profile_tab": "tap profile tab",
|
||||
"direct_tab": "tap messages tab",
|
||||
"news_tab": "tap activity heart icon notifications",
|
||||
}
|
||||
for tab_id, action in tab_map.items():
|
||||
if tab_id in resource_ids:
|
||||
actions.append(action)
|
||||
|
||||
ids_str = " ".join(resource_ids).lower()
|
||||
if screen_type == ScreenType.OWN_PROFILE or screen_type == ScreenType.OTHER_PROFILE:
|
||||
if "button_message" in ids_str or "profile_header_message_button" in ids_str:
|
||||
actions.append("tap message button")
|
||||
if "profile_header_following" in ids_str or "profile_header_follow_button" in ids_str:
|
||||
actions.append("tap following list")
|
||||
|
||||
# Grid items
|
||||
if screen_type == ScreenType.EXPLORE_GRID:
|
||||
actions.append("tap first post")
|
||||
|
||||
# Scroll
|
||||
actions.append("scroll down")
|
||||
actions.append("scroll up")
|
||||
actions.append("press back")
|
||||
|
||||
return list(set(actions)) # Deduplicate
|
||||
|
||||
def _extract_context(self, content_descs, texts, resource_ids, screen_type):
|
||||
"""Extract meaningful context from the screen using structural heuristics."""
|
||||
context = {}
|
||||
# Zero Maintenance: Do not rely on localized regex strings like "followers" or "liked"
|
||||
return context
|
||||
|
||||
def _compute_signature(self, resource_ids, content_descs, texts):
|
||||
"""Compute a stable hash for this screen state (for Qdrant lookup)."""
|
||||
# Use sorted IDs + key content for stability
|
||||
sig_parts = sorted(resource_ids)[:20]
|
||||
sig_parts.extend(sorted(set(d.lower()[:30] for d in content_descs if len(d) > 2))[:10])
|
||||
sig = "|".join(sig_parts)
|
||||
return hashlib.sha256(sig.encode()).hexdigest()[:24]
|
||||
|
||||
def _empty_screen(self):
|
||||
return {
|
||||
"screen_type": ScreenType.FOREIGN_APP,
|
||||
"available_actions": ["press back", "force start instagram"],
|
||||
"selected_tab": None,
|
||||
"context": {},
|
||||
"signature": "empty",
|
||||
}
|
||||
229
GramAddict/core/perception/semantic_evaluator.py
Normal file
229
GramAddict/core/perception/semantic_evaluator.py
Normal file
@@ -0,0 +1,229 @@
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from typing import List, Optional
|
||||
|
||||
from GramAddict.core.llm_provider import query_telepathic_llm
|
||||
from GramAddict.core.perception.spatial_parser import SpatialNode
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class SemanticEvaluator:
|
||||
"""
|
||||
Handles LLM/VLM interaction for high-level semantic analysis of the UI.
|
||||
Delegates vision processing and prompt engineering out of the core routing engine.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
from GramAddict.core.config import Config
|
||||
|
||||
try:
|
||||
self.args = Config().args
|
||||
except Exception:
|
||||
self.args = None
|
||||
|
||||
def _query_vlm(self, prompt: str, screenshot_b64: str) -> Optional[str]:
|
||||
if not self.args:
|
||||
logger.warning("👁️ [Vision Core] No config available. Cannot query VLM.")
|
||||
return None
|
||||
|
||||
model = getattr(self.args, "ai_telepathic_model", "llama3.2-vision")
|
||||
url = getattr(self.args, "ai_telepathic_url", "http://localhost:11434/api/generate")
|
||||
|
||||
try:
|
||||
res = query_telepathic_llm(
|
||||
model=model,
|
||||
url=url,
|
||||
system_prompt="You are an expert Instagram assistant.",
|
||||
user_prompt=prompt,
|
||||
images_b64=[screenshot_b64],
|
||||
)
|
||||
return res
|
||||
except Exception as e:
|
||||
logger.error(f"👁️ [Vision Core] LLM query failed: {e}")
|
||||
return None
|
||||
|
||||
def evaluate_grid_visuals(
|
||||
self, device, persona_interests: list[str], grid_nodes: List[SpatialNode]
|
||||
) -> Optional[SpatialNode]:
|
||||
"""
|
||||
Takes the spatial grid nodes and asks the VLM which one best matches the persona.
|
||||
"""
|
||||
logger.info(f"👁️ [Vision Core] Analyzing grid aesthetics against niche interests: {persona_interests}...")
|
||||
|
||||
if not grid_nodes:
|
||||
return None
|
||||
|
||||
# Take a screenshot
|
||||
try:
|
||||
screenshot_b64 = device.get_screenshot_b64()
|
||||
except Exception as e:
|
||||
logger.error(f"👁️ [Vision Core] Failed to capture screenshot: {e}")
|
||||
return None
|
||||
|
||||
simplified_nodes = []
|
||||
for i, node in enumerate(grid_nodes[:9]): # Limit to 9 to save tokens
|
||||
simplified_nodes.append({"index": i, "bounds": node.bounds})
|
||||
|
||||
prompt = f"""
|
||||
You are a highly perceptive Instagram user with the following interests: {', '.join(persona_interests)}.
|
||||
Look at the provided screenshot of the Instagram Explore/Profile grid.
|
||||
Below are the bounding boxes for the top grid posts currently visible.
|
||||
|
||||
{simplified_nodes}
|
||||
|
||||
Your task:
|
||||
1. Identify which of these posts visually aligns BEST with your interests.
|
||||
2. Reply ONLY in JSON format: {{"index": <int>}}
|
||||
3. If absolutely none of them are relevant, reply with {{"index": -1}}.
|
||||
"""
|
||||
|
||||
try:
|
||||
response = self._query_vlm(prompt, screenshot_b64)
|
||||
if not response:
|
||||
return None
|
||||
|
||||
try:
|
||||
data = json.loads(response)
|
||||
idx = data.get("index", -1)
|
||||
if idx == -1:
|
||||
logger.info("👁️ [Vision Core] VLM rejected all grid items. Will scroll down.")
|
||||
return None
|
||||
|
||||
if 0 <= idx < len(grid_nodes):
|
||||
logger.info(f"👁️ [Vision Core] VLM selected grid item index [{idx}] as the best match.")
|
||||
return grid_nodes[idx]
|
||||
except json.JSONDecodeError:
|
||||
# Fallback to fuzzy
|
||||
clean_res = response.strip().upper()
|
||||
match = re.search(r"\d+", clean_res)
|
||||
if match:
|
||||
idx = int(match.group())
|
||||
if 0 <= idx < len(grid_nodes):
|
||||
logger.info(f"👁️ [Vision Core] VLM selected grid item index [{idx}] as the best match.")
|
||||
return grid_nodes[idx]
|
||||
except Exception as e:
|
||||
logger.warning(f"👁️ [Vision Core] Exception during grid evaluation: {e}")
|
||||
|
||||
return None
|
||||
|
||||
def evaluate_post_vibe(self, device, persona_interests: list[str]) -> Optional[dict]:
|
||||
"""Evaluates whether the currently viewed post aligns with persona interests."""
|
||||
logger.info(f"👁️ [Vision Core] Evaluating post vibe against: {persona_interests}")
|
||||
try:
|
||||
screenshot_b64 = device.get_screenshot_b64()
|
||||
prompt = f"""
|
||||
You are a user with the following interests: {', '.join(persona_interests)}.
|
||||
You are looking at an Instagram post.
|
||||
Evaluate if this post is highly relevant to your interests and if you should like/comment on it.
|
||||
CRITICAL: Check if this post is an advertisement or sponsored content (look for "Sponsored", "Ad", or promotional product placement).
|
||||
|
||||
Reply ONLY in valid JSON format:
|
||||
{{
|
||||
"should_like": true/false,
|
||||
"should_comment": true/false,
|
||||
"is_ad": true/false
|
||||
}}
|
||||
"""
|
||||
response = self._query_vlm(prompt, screenshot_b64)
|
||||
if response:
|
||||
if "```json" in response:
|
||||
json_str = response.split("```json")[1].split("```")[0].strip()
|
||||
else:
|
||||
json_str = response.strip()
|
||||
try:
|
||||
return json.loads(json_str)
|
||||
except json.JSONDecodeError:
|
||||
# Try to close potential unclosed JSON strings
|
||||
if not json_str.endswith("}"):
|
||||
json_str += "}"
|
||||
try:
|
||||
return json.loads(json_str)
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
logger.warning(f"👁️ [Vision Core] VLM returned malformed JSON: {response}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to evaluate post vibe: {e}")
|
||||
return None
|
||||
|
||||
def evaluate_profile_vibe(self, device, persona_interests: list[str]) -> Optional[dict]:
|
||||
"""Evaluates if a profile is worth following."""
|
||||
pass
|
||||
|
||||
def classify_screen_content(self, xml_hierarchy: str, target_class: str) -> Optional[str]:
|
||||
"""
|
||||
Fast-Path Structural Analysis.
|
||||
Replaces VLM calls for basic structural states.
|
||||
"""
|
||||
if not xml_hierarchy:
|
||||
return None
|
||||
|
||||
xml_lower = xml_hierarchy.lower()
|
||||
|
||||
if target_class == "carousel_or_single_post":
|
||||
if 'resource-id="com.instagram.android:id/carousel_image"' in xml_lower or (
|
||||
'scrollable="true"' in xml_lower and "viewpager" in xml_lower
|
||||
):
|
||||
return "carousel"
|
||||
return "single"
|
||||
|
||||
elif target_class == "post_has_comments":
|
||||
if 'resource-id="com.instagram.android:id/row_feed_button_comment"' in xml_lower:
|
||||
return "has_comments"
|
||||
return "no_comments"
|
||||
|
||||
elif target_class == "sponsored_content":
|
||||
# Heuristic for sponsored content
|
||||
if "sponsored" in xml_lower or "gesponsert" in xml_lower:
|
||||
return "sponsored"
|
||||
return "organic"
|
||||
|
||||
elif target_class == "story_ring_presence":
|
||||
if "reel_ring" in xml_lower or "story_ring" in xml_lower:
|
||||
return "has_unseen_story"
|
||||
return "no_unseen_story"
|
||||
|
||||
elif target_class == "main_feed_presence":
|
||||
if 'content-desc="home"' in xml_lower and 'selected="true"' in xml_lower:
|
||||
return "main_feed"
|
||||
if "feed_tab" in xml_lower and 'selected="true"' in xml_lower:
|
||||
return "main_feed"
|
||||
return "other"
|
||||
|
||||
elif target_class == "private_account":
|
||||
if (
|
||||
'resource-id="com.instagram.android:id/row_profile_header_empty_profile_notice_title"' in xml_lower
|
||||
and "private" in xml_lower
|
||||
):
|
||||
return "private"
|
||||
return "public"
|
||||
|
||||
elif target_class == "empty_account":
|
||||
if 'resource-id="com.instagram.android:id/row_profile_header_empty_profile_notice_title"' in xml_lower and (
|
||||
"no posts" in xml_lower or "noch keine" in xml_lower
|
||||
):
|
||||
return "empty"
|
||||
return "not_empty"
|
||||
|
||||
elif target_class == "close_friends_content":
|
||||
if "close_friends" in xml_lower or "close friends" in xml_lower or "enge freunde" in xml_lower:
|
||||
return "close_friends"
|
||||
return "normal_content"
|
||||
|
||||
elif target_class in ("profile_follow_status", "follow_button_state"):
|
||||
if 'resource-id="com.instagram.android:id/profile_header_follow_button"' in xml_lower:
|
||||
return "not_following"
|
||||
if 'resource-id="com.instagram.android:id/profile_header_following_button"' in xml_lower:
|
||||
return "already_following"
|
||||
# Fallback text check for requested state
|
||||
if "requested" in xml_lower or "angefragt" in xml_lower:
|
||||
return "requested"
|
||||
return "unknown"
|
||||
|
||||
elif target_class == "unfollow_bottom_sheet_presence":
|
||||
if 'resource-id="com.instagram.android:id/follow_sheet_unfollow_row"' in xml_lower:
|
||||
return "unfollow_sheet"
|
||||
return "other"
|
||||
|
||||
return None
|
||||
201
GramAddict/core/perception/spatial_parser.py
Normal file
201
GramAddict/core/perception/spatial_parser.py
Normal file
@@ -0,0 +1,201 @@
|
||||
import re
|
||||
import xml.etree.ElementTree as ET
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
|
||||
|
||||
@dataclass
|
||||
class SpatialNode:
|
||||
"""A single node in the Spatial Graph, representing a UI element and its geometry."""
|
||||
|
||||
bounds: Tuple[int, int, int, int] # (x1, y1, x2, y2)
|
||||
node_id: str = ""
|
||||
class_name: str = ""
|
||||
text: str = ""
|
||||
content_desc: str = ""
|
||||
resource_id: str = ""
|
||||
clickable: bool = False
|
||||
long_clickable: bool = False
|
||||
scrollable: bool = False
|
||||
|
||||
# Spatial Properties
|
||||
children: List["SpatialNode"] = field(default_factory=list)
|
||||
parent: Optional["SpatialNode"] = None
|
||||
|
||||
@property
|
||||
def x1(self) -> int:
|
||||
return self.bounds[0]
|
||||
|
||||
@property
|
||||
def y1(self) -> int:
|
||||
return self.bounds[1]
|
||||
|
||||
@property
|
||||
def x2(self) -> int:
|
||||
return self.bounds[2]
|
||||
|
||||
@property
|
||||
def y2(self) -> int:
|
||||
return self.bounds[3]
|
||||
|
||||
@property
|
||||
def width(self) -> int:
|
||||
return self.x2 - self.x1
|
||||
|
||||
@property
|
||||
def height(self) -> int:
|
||||
return self.y2 - self.y1
|
||||
|
||||
@property
|
||||
def center_x(self) -> int:
|
||||
return self.x1 + (self.width // 2)
|
||||
|
||||
@property
|
||||
def center_y(self) -> int:
|
||||
return self.y1 + (self.height // 2)
|
||||
|
||||
@property
|
||||
def area(self) -> int:
|
||||
return self.width * self.height
|
||||
|
||||
def contains(self, other: "SpatialNode") -> bool:
|
||||
"""Returns True if this node completely encompasses the other node geometrically."""
|
||||
return self.x1 <= other.x1 and self.y1 <= other.y1 and self.x2 >= other.x2 and self.y2 >= other.y2
|
||||
|
||||
def intersects(self, other: "SpatialNode") -> bool:
|
||||
"""Returns True if this node's bounding box overlaps with the other's bounding box."""
|
||||
if self.x1 >= other.x2 or other.x1 >= self.x2:
|
||||
return False
|
||||
if self.y1 >= other.y2 or other.y1 >= self.y2:
|
||||
return False
|
||||
return True
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"id": self.node_id,
|
||||
"class": self.class_name,
|
||||
"text": self.text,
|
||||
"content_desc": self.content_desc,
|
||||
"resource_id": self.resource_id,
|
||||
"bounds": self.bounds,
|
||||
"clickable": self.clickable,
|
||||
"scrollable": self.scrollable,
|
||||
"center": (self.center_x, self.center_y),
|
||||
}
|
||||
|
||||
|
||||
class SpatialParser:
|
||||
"""
|
||||
Parses Android UI XML into a structured 2D Spatial Tree.
|
||||
Calculates parent-child relationships structurally, not just based on XML nesting.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self._node_counter = 0
|
||||
|
||||
def parse(self, xml_string: str) -> Optional[SpatialNode]:
|
||||
"""Parses the raw XML dump into a Spatial Graph."""
|
||||
try:
|
||||
clean_xml = re.sub(r"<\?xml.*?\?>", "", xml_string).strip()
|
||||
if not clean_xml:
|
||||
return None
|
||||
root_elem = ET.fromstring(clean_xml)
|
||||
|
||||
# 1. First Pass: Create flat list of spatial nodes
|
||||
all_nodes = []
|
||||
self._flatten_xml(root_elem, all_nodes)
|
||||
|
||||
if not all_nodes:
|
||||
return None
|
||||
|
||||
# 2. Second Pass: Reconstruct tree based on strict spatial containment
|
||||
# Sort nodes by area descending (largest first)
|
||||
all_nodes.sort(key=lambda n: n.area, reverse=True)
|
||||
|
||||
root_node = all_nodes[0]
|
||||
|
||||
for i in range(1, len(all_nodes)):
|
||||
child = all_nodes[i]
|
||||
# Find the smallest node that contains this child
|
||||
# Since we sorted by area descending, we search backwards to find the tightest fit
|
||||
parent_found = False
|
||||
for j in range(i - 1, -1, -1):
|
||||
potential_parent = all_nodes[j]
|
||||
if potential_parent.contains(child):
|
||||
potential_parent.children.append(child)
|
||||
child.parent = potential_parent
|
||||
parent_found = True
|
||||
break
|
||||
|
||||
# Fallback to root if no parent found (floating node)
|
||||
if not parent_found and child != root_node:
|
||||
root_node.children.append(child)
|
||||
child.parent = root_node
|
||||
|
||||
return root_node
|
||||
|
||||
except ET.ParseError:
|
||||
return None
|
||||
|
||||
def _flatten_xml(self, element: ET.Element, nodes_list: List[SpatialNode]):
|
||||
"""Recursively traverses the XML and creates a flat list of SpatialNodes."""
|
||||
attrib = element.attrib
|
||||
|
||||
bounds_str = attrib.get("bounds", "")
|
||||
match = re.match(r"\[(\d+),(\d+)\]\[(\d+),(\d+)\]", bounds_str)
|
||||
|
||||
if match:
|
||||
left, top, right, bottom = map(int, match.groups())
|
||||
|
||||
# Filter zero-area nodes early
|
||||
if right > left and bottom > top:
|
||||
self._node_counter += 1
|
||||
text_val = attrib.get("text", "").strip()
|
||||
hint_val = attrib.get("hint", "").strip()
|
||||
if not text_val and hint_val:
|
||||
text_val = hint_val
|
||||
|
||||
node = SpatialNode(
|
||||
node_id=f"n_{self._node_counter}",
|
||||
class_name=attrib.get("class", ""),
|
||||
text=text_val,
|
||||
content_desc=attrib.get("content-desc", "").strip(),
|
||||
resource_id=attrib.get("resource-id", "").strip(),
|
||||
bounds=(left, top, right, bottom),
|
||||
clickable=attrib.get("clickable", "false") == "true",
|
||||
long_clickable=attrib.get("long-clickable", "false") == "true",
|
||||
scrollable=attrib.get("scrollable", "false") == "true",
|
||||
)
|
||||
nodes_list.append(node)
|
||||
|
||||
for child in element:
|
||||
self._flatten_xml(child, nodes_list)
|
||||
|
||||
def get_all_nodes(self, root: SpatialNode) -> List[SpatialNode]:
|
||||
"""Flattens the Spatial Tree into a list for easy filtering."""
|
||||
result = [root]
|
||||
for child in root.children:
|
||||
result.extend(self.get_all_nodes(child))
|
||||
return result
|
||||
|
||||
def get_clickable_nodes(self, root: SpatialNode) -> List[SpatialNode]:
|
||||
"""Returns all nodes that are clickable or have strong semantic meaning."""
|
||||
all_nodes = self.get_all_nodes(root)
|
||||
clickables = []
|
||||
|
||||
for n in all_nodes:
|
||||
has_semantic = bool(n.text or n.content_desc)
|
||||
semantic_res = n.resource_id and any(
|
||||
x in n.resource_id.lower() for x in ["button", "tab", "icon", "action", "menu", "imageview"]
|
||||
)
|
||||
|
||||
if n.clickable or n.scrollable or semantic_res or (has_semantic and n.area < 500000 and n.area > 0):
|
||||
# Filter out pure massive containers (like whole screen) if they aren't explicitly clickable
|
||||
if not n.clickable and not n.scrollable and n.area > 2000000:
|
||||
continue
|
||||
# Also exclude if it's just a ViewGroup with a description but no action
|
||||
if not n.clickable and n.class_name == "android.view.ViewGroup":
|
||||
continue
|
||||
clickables.append(n)
|
||||
|
||||
return clickables
|
||||
@@ -1,9 +1,10 @@
|
||||
import json
|
||||
import os
|
||||
import logging
|
||||
import os
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PersistentList(list):
|
||||
def __init__(self, filename, encoder=None):
|
||||
super().__init__()
|
||||
@@ -12,7 +13,8 @@ class PersistentList(list):
|
||||
self.load()
|
||||
|
||||
def load(self):
|
||||
path = f"accounts/{self.filename}.json"
|
||||
base_dir = os.environ.get("GRAMADDICT_ACCOUNTS_DIR", "accounts")
|
||||
path = f"{base_dir}/{self.filename}.json"
|
||||
if os.path.exists(path):
|
||||
try:
|
||||
with open(path, "r") as f:
|
||||
@@ -26,9 +28,8 @@ class PersistentList(list):
|
||||
self.persist()
|
||||
|
||||
def persist(self, directory=None):
|
||||
if os.environ.get("PYTEST_CURRENT_TEST"):
|
||||
return
|
||||
folder = f"accounts/{directory}" if directory else "accounts"
|
||||
base_dir = os.environ.get("GRAMADDICT_ACCOUNTS_DIR", "accounts")
|
||||
folder = f"{base_dir}/{directory}" if directory else base_dir
|
||||
os.makedirs(folder, exist_ok=True)
|
||||
path = f"{folder}/{self.filename}.json"
|
||||
try:
|
||||
|
||||
29
GramAddict/core/physics/__init__.py
Normal file
29
GramAddict/core/physics/__init__.py
Normal file
@@ -0,0 +1,29 @@
|
||||
"""Physics — Humanized Input Simulation, Biomechanics & UI Timing."""
|
||||
|
||||
from GramAddict.core.physics.biomechanics import (
|
||||
BezierGesture,
|
||||
PhysicsBody,
|
||||
)
|
||||
from GramAddict.core.physics.humanized_input import (
|
||||
humanized_click,
|
||||
humanized_horizontal_swipe,
|
||||
humanized_scroll,
|
||||
)
|
||||
from GramAddict.core.physics.sendevent_injector import SendEventInjector
|
||||
from GramAddict.core.physics.timing import (
|
||||
align_active_post,
|
||||
wait_for_post_loaded,
|
||||
wait_for_story_loaded,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"humanized_scroll",
|
||||
"humanized_click",
|
||||
"humanized_horizontal_swipe",
|
||||
"wait_for_post_loaded",
|
||||
"wait_for_story_loaded",
|
||||
"align_active_post",
|
||||
"PhysicsBody",
|
||||
"BezierGesture",
|
||||
"SendEventInjector",
|
||||
]
|
||||
414
GramAddict/core/physics/biomechanics.py
Normal file
414
GramAddict/core/physics/biomechanics.py
Normal file
@@ -0,0 +1,414 @@
|
||||
"""
|
||||
Biomechanics — Organic Thumb Kinematics & Bézier Gesture Synthesis.
|
||||
|
||||
Simulates the physical behavior of a human thumb across an entire bot session:
|
||||
- Spatial drift (posture changes)
|
||||
- Fatigue (slower, less accurate over time)
|
||||
- Handedness bias (right-handers arc right)
|
||||
- Non-linear Bézier touch paths with sigmoid velocity and Gaussian pressure
|
||||
|
||||
This module produces gesture data (point sequences) that are then injected
|
||||
via SendEventInjector or fall back to adb `input swipe`.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import math
|
||||
import random
|
||||
import time
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PhysicsBody:
|
||||
"""
|
||||
Kinematic model of a human thumb over a session.
|
||||
|
||||
Tracks anchor position (where the thumb naturally rests), session-level
|
||||
spatial drift (simulating posture changes), and fatigue (affecting speed
|
||||
and accuracy). Provides biomechanically plausible start/end positions
|
||||
for all gestures.
|
||||
"""
|
||||
|
||||
_session_instance = None
|
||||
|
||||
def __init__(self, handedness="right", device_info=None):
|
||||
self.handedness = handedness
|
||||
|
||||
# Defensive parsing — device_info may contain MagicMock objects in tests
|
||||
try:
|
||||
self.w = int(device_info.get("displayWidth", 1080)) if device_info else 1080
|
||||
except (TypeError, ValueError):
|
||||
self.w = 1080
|
||||
try:
|
||||
self.h = int(device_info.get("displayHeight", 2400)) if device_info else 2400
|
||||
except (TypeError, ValueError):
|
||||
self.h = 2400
|
||||
|
||||
# Anchor Point: natural thumb rest position
|
||||
# Right-handers: lower-right quadrant; Left-handers: lower-left
|
||||
self.anchor_x = self.w * (0.75 if handedness == "right" else 0.25)
|
||||
self.anchor_y = self.h * 0.82
|
||||
|
||||
# Session Drift: simulates posture shifts over time
|
||||
self.drift_x = 0.0
|
||||
self.drift_y = 0.0
|
||||
self.gesture_count = 0
|
||||
|
||||
# Fatigue Model: 0.0 = fresh, 1.0 = exhausted
|
||||
self.fatigue = 0.0
|
||||
self.last_gesture_time = time.time()
|
||||
|
||||
logger.debug(
|
||||
f"🦴 [PhysicsBody] Initialized: {handedness}-handed, "
|
||||
f"anchor=({self.anchor_x:.0f}, {self.anchor_y:.0f}), "
|
||||
f"display={self.w}x{self.h}"
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def get_session_instance(cls, device=None, handedness="right"):
|
||||
"""
|
||||
Returns a session-persistent PhysicsBody.
|
||||
The body persists across all gestures within a single bot session,
|
||||
accumulating drift and fatigue realistically.
|
||||
"""
|
||||
if cls._session_instance is None:
|
||||
device_info = {}
|
||||
if device:
|
||||
try:
|
||||
raw = device.get_info()
|
||||
# Defensive: convert to plain dict to handle MagicMock returns
|
||||
if isinstance(raw, dict):
|
||||
device_info = raw
|
||||
else:
|
||||
device_info = {}
|
||||
except Exception:
|
||||
pass
|
||||
cls._session_instance = cls(handedness=handedness, device_info=device_info)
|
||||
return cls._session_instance
|
||||
|
||||
@classmethod
|
||||
def reset(cls):
|
||||
"""Reset for testing / new session."""
|
||||
cls._session_instance = None
|
||||
|
||||
def get_scroll_start(self):
|
||||
"""
|
||||
Returns a biomechanically plausible scroll start position.
|
||||
Right-handers start scrolls on the right side of the screen,
|
||||
with Gaussian jitter and session drift applied.
|
||||
"""
|
||||
self._apply_session_drift()
|
||||
self._update_fatigue()
|
||||
|
||||
# Base position: right side for right-handers, avoiding edges
|
||||
base_x = self.anchor_x + self.drift_x
|
||||
# Scroll starts in the lower 70-85% of the screen
|
||||
base_y = self.h * random.uniform(0.70, 0.85) + self.drift_y
|
||||
|
||||
# Gaussian jitter (natural inaccuracy, increases with fatigue)
|
||||
fatigue_mult = 1.0 + self.fatigue * 0.5
|
||||
jitter_x = random.gauss(0, self.w * 0.02 * fatigue_mult)
|
||||
jitter_y = random.gauss(0, self.h * 0.015 * fatigue_mult)
|
||||
|
||||
x = int(max(50, min(self.w - 50, base_x + jitter_x)))
|
||||
y = int(max(200, min(self.h - 200, base_y + jitter_y)))
|
||||
self.gesture_count += 1
|
||||
return x, y
|
||||
|
||||
def get_tap_position(self, target_x, target_y):
|
||||
"""
|
||||
Returns a biomechanically plausible tap position near the target.
|
||||
Applies thumb bias (right-handers land slightly left-down of center)
|
||||
and Gaussian jitter.
|
||||
"""
|
||||
self._update_fatigue()
|
||||
|
||||
# Thumb bias: right-handers hit slightly left and below center
|
||||
bias_x = -3 if self.handedness == "right" else 3
|
||||
bias_y = 4 # Thumb pad is below the actual contact center
|
||||
|
||||
fatigue_mult = 1.0 + self.fatigue * 0.3
|
||||
jitter_x = random.gauss(bias_x, 5 * fatigue_mult)
|
||||
jitter_y = random.gauss(bias_y, 5 * fatigue_mult)
|
||||
|
||||
x = int(max(5, min(self.w - 5, target_x + jitter_x)))
|
||||
y = int(max(5, min(self.h - 5, target_y + jitter_y)))
|
||||
self.gesture_count += 1
|
||||
return x, y
|
||||
|
||||
def get_thumb_arc_bias(self):
|
||||
"""
|
||||
Returns the horizontal arc bias for scroll curves.
|
||||
Right-handers naturally arc their thumb to the right during
|
||||
vertical swipes; left-handers arc left.
|
||||
"""
|
||||
base_arc = self.w * 0.04
|
||||
if self.handedness == "right":
|
||||
return base_arc + random.uniform(-self.w * 0.01, self.w * 0.02)
|
||||
else:
|
||||
return -(base_arc + random.uniform(-self.w * 0.01, self.w * 0.02))
|
||||
|
||||
def get_pressure_baseline(self):
|
||||
"""
|
||||
Returns the baseline pressure for touch events.
|
||||
Fatigued thumbs press harder (compensating for reduced precision).
|
||||
"""
|
||||
baseline = 0.35 + self.fatigue * 0.15
|
||||
return min(0.85, baseline + random.uniform(-0.05, 0.05))
|
||||
|
||||
def get_touch_major(self):
|
||||
"""
|
||||
Returns the touch contact area (touch_major) in device units.
|
||||
Fatigued thumbs have a larger contact patch (flatter press).
|
||||
"""
|
||||
base = 6 + int(self.fatigue * 4)
|
||||
return max(4, base + random.randint(-2, 2))
|
||||
|
||||
def _apply_session_drift(self):
|
||||
"""
|
||||
Every ~15-25 gestures, apply a small posture shift.
|
||||
Simulates the user adjusting their grip on the phone.
|
||||
"""
|
||||
drift_interval = random.randint(15, 25)
|
||||
if self.gesture_count > 0 and self.gesture_count % drift_interval == 0:
|
||||
old_dx, old_dy = self.drift_x, self.drift_y
|
||||
self.drift_x += random.gauss(0, self.w * 0.025)
|
||||
self.drift_y += random.gauss(0, self.h * 0.015)
|
||||
|
||||
# Clamp drift so we don't wander off the screen
|
||||
self.drift_x = max(-self.w * 0.1, min(self.w * 0.1, self.drift_x))
|
||||
self.drift_y = max(-self.h * 0.06, min(self.h * 0.06, self.drift_y))
|
||||
|
||||
if abs(self.drift_x - old_dx) > 5 or abs(self.drift_y - old_dy) > 5:
|
||||
logger.debug(
|
||||
f"🦴 [PhysicsBody] Posture drift: "
|
||||
f"Δx={self.drift_x - old_dx:+.0f}, Δy={self.drift_y - old_dy:+.0f} "
|
||||
f"(gesture #{self.gesture_count})"
|
||||
)
|
||||
|
||||
def _update_fatigue(self):
|
||||
"""
|
||||
Update fatigue based on gesture frequency.
|
||||
Rapid gestures increase fatigue; idle periods recover it.
|
||||
"""
|
||||
elapsed = time.time() - self.last_gesture_time
|
||||
if elapsed < 0.5:
|
||||
# Rapid-fire: fatigue increases
|
||||
self.fatigue = min(1.0, self.fatigue + 0.015)
|
||||
elif elapsed > 8.0:
|
||||
# Long pause: recovery
|
||||
self.fatigue = max(0.0, self.fatigue - 0.08)
|
||||
elif elapsed > 3.0:
|
||||
# Moderate pause: slight recovery
|
||||
self.fatigue = max(0.0, self.fatigue - 0.02)
|
||||
self.last_gesture_time = time.time()
|
||||
|
||||
|
||||
class BezierGesture:
|
||||
"""
|
||||
Generates multi-point cubic Bézier curves for organic touch paths.
|
||||
|
||||
Replaces the linear A→B interpolation of `adb shell input swipe`
|
||||
with biomechanically accurate gesture trajectories including:
|
||||
- Thumb arc curvature (handedness-dependent)
|
||||
- Sigmoid velocity profile (slow start → fast middle → slow end)
|
||||
- Gaussian pressure curve (light touch → firm contact → light lift)
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def scroll_curve(start, end, body: PhysicsBody, n_points=None):
|
||||
"""
|
||||
Generates a vertical scroll gesture curve.
|
||||
|
||||
Args:
|
||||
start: (x, y) start position
|
||||
end: (x, y) end position
|
||||
body: PhysicsBody for handedness/fatigue context
|
||||
n_points: Override for number of intermediate points
|
||||
|
||||
Returns:
|
||||
List of (x, y, pressure) tuples along the Bézier curve
|
||||
"""
|
||||
sx, sy = start
|
||||
ex, ey = end
|
||||
|
||||
if n_points is None:
|
||||
n_points = random.randint(10, 18)
|
||||
|
||||
# Thumb arc: the control points bias the curve sideways
|
||||
arc_bias = body.get_thumb_arc_bias()
|
||||
|
||||
# Two control points for cubic Bézier
|
||||
# CP1: early in the gesture, slight arc
|
||||
cp1_x = sx + arc_bias * random.uniform(0.2, 0.4)
|
||||
cp1_y = sy + (ey - sy) * random.uniform(0.2, 0.35)
|
||||
# CP2: later in the gesture, peak arc
|
||||
cp2_x = sx + arc_bias * random.uniform(0.5, 0.8)
|
||||
cp2_y = sy + (ey - sy) * random.uniform(0.65, 0.8)
|
||||
|
||||
pressure_baseline = body.get_pressure_baseline()
|
||||
|
||||
points = []
|
||||
for i in range(n_points + 1):
|
||||
t = i / n_points
|
||||
|
||||
# Cubic Bézier interpolation
|
||||
x = (1 - t) ** 3 * sx + 3 * (1 - t) ** 2 * t * cp1_x + 3 * (1 - t) * t**2 * cp2_x + t**3 * ex
|
||||
y = (1 - t) ** 3 * sy + 3 * (1 - t) ** 2 * t * cp1_y + 3 * (1 - t) * t**2 * cp2_y + t**3 * ey
|
||||
|
||||
# Micro-noise on each point (finger vibration)
|
||||
x += random.gauss(0, 1.5)
|
||||
y += random.gauss(0, 1.5)
|
||||
|
||||
# Pressure curve: Gaussian peak around t=0.4 (peak contact mid-gesture)
|
||||
pressure = pressure_baseline + 0.3 * math.exp(-((t - 0.4) ** 2) / 0.1)
|
||||
pressure += random.uniform(-0.04, 0.04)
|
||||
pressure = max(0.08, min(0.92, pressure))
|
||||
|
||||
points.append((int(x), int(y), round(pressure, 3)))
|
||||
|
||||
return points
|
||||
|
||||
@staticmethod
|
||||
def tap_curve(target_x, target_y, body: PhysicsBody):
|
||||
"""
|
||||
Generates a tap gesture (touch-down → micro-drift → touch-up).
|
||||
|
||||
Returns:
|
||||
List of (x, y, pressure) tuples (typically 3-5 points)
|
||||
"""
|
||||
tx, ty = body.get_tap_position(target_x, target_y)
|
||||
pressure_base = body.get_pressure_baseline()
|
||||
|
||||
# Touch-down (initial light contact)
|
||||
p_down = max(0.1, pressure_base * 0.6 + random.uniform(-0.05, 0.05))
|
||||
# Full contact
|
||||
p_full = min(0.9, pressure_base + random.uniform(-0.05, 0.1))
|
||||
# Release
|
||||
p_up = max(0.05, pressure_base * 0.3 + random.uniform(-0.03, 0.03))
|
||||
|
||||
# Micro-drift: finger slides ~2-6px during contact
|
||||
drift_x = random.randint(-4, 4)
|
||||
drift_y = random.randint(-4, 4)
|
||||
|
||||
points = [
|
||||
(tx, ty, round(p_down, 3)),
|
||||
(tx + drift_x // 2, ty + drift_y // 2, round(p_full, 3)),
|
||||
(tx + drift_x, ty + drift_y, round(p_up, 3)),
|
||||
]
|
||||
return points
|
||||
|
||||
@staticmethod
|
||||
def horizontal_swipe_curve(start, end, body: PhysicsBody, n_points=None):
|
||||
"""
|
||||
Generates a horizontal swipe curve (e.g., carousel browsing).
|
||||
|
||||
Includes vertical arc (thumb drops downward when swiping left-to-right
|
||||
for right-handers) and sigmoid velocity.
|
||||
"""
|
||||
sx, sy = start
|
||||
ex, ey = end
|
||||
|
||||
if n_points is None:
|
||||
n_points = random.randint(8, 14)
|
||||
|
||||
# Vertical arc for horizontal swipes
|
||||
# Right-handers swiping left: thumb drops 30-90px
|
||||
direction = 1 if ex < sx else -1 # 1 = swiping left
|
||||
if body.handedness == "right":
|
||||
y_arc = direction * random.uniform(25, 70)
|
||||
else:
|
||||
y_arc = -direction * random.uniform(25, 70)
|
||||
|
||||
# Control points
|
||||
cp1_x = sx + (ex - sx) * random.uniform(0.25, 0.35)
|
||||
cp1_y = sy + y_arc * 0.4
|
||||
cp2_x = sx + (ex - sx) * random.uniform(0.65, 0.75)
|
||||
cp2_y = sy + y_arc * 0.9
|
||||
|
||||
pressure_baseline = body.get_pressure_baseline()
|
||||
points = []
|
||||
|
||||
for i in range(n_points + 1):
|
||||
t = i / n_points
|
||||
x = (1 - t) ** 3 * sx + 3 * (1 - t) ** 2 * t * cp1_x + 3 * (1 - t) * t**2 * cp2_x + t**3 * ex
|
||||
y = (1 - t) ** 3 * sy + 3 * (1 - t) ** 2 * t * cp1_y + 3 * (1 - t) * t**2 * cp2_y + t**3 * ey
|
||||
x += random.gauss(0, 2)
|
||||
y += random.gauss(0, 2)
|
||||
|
||||
pressure = pressure_baseline + 0.25 * math.exp(-((t - 0.45) ** 2) / 0.12)
|
||||
pressure += random.uniform(-0.04, 0.04)
|
||||
pressure = max(0.08, min(0.92, pressure))
|
||||
|
||||
points.append((int(x), int(y), round(pressure, 3)))
|
||||
|
||||
return points
|
||||
|
||||
@staticmethod
|
||||
def compute_sigmoid_timing(n_points, total_duration_ms):
|
||||
"""
|
||||
Generates a sigmoid-based timing schedule for gesture points.
|
||||
|
||||
Produces intervals that are longer at the start and end
|
||||
(slow acceleration/deceleration) and shorter in the middle
|
||||
(peak velocity). This matches real human swipe kinematics.
|
||||
|
||||
Returns:
|
||||
List of inter-point delay times in seconds (length n_points)
|
||||
"""
|
||||
if n_points <= 1:
|
||||
return [total_duration_ms / 1000.0]
|
||||
|
||||
# Generate sigmoid-spaced t values
|
||||
raw_intervals = []
|
||||
for i in range(n_points):
|
||||
# Normalized position
|
||||
t = i / (n_points - 1) if n_points > 1 else 0.5
|
||||
# Inverted sigmoid: fast in middle, slow at edges
|
||||
# Higher value = longer delay = slower movement
|
||||
1.0 / (1.0 + math.exp(-8 * (t - 0.5)))
|
||||
# U-shaped: slow at start & end, fast in middle
|
||||
speed_factor = 0.4 + 1.2 * (4 * (t - 0.5) ** 2)
|
||||
raw_intervals.append(speed_factor)
|
||||
|
||||
# Normalize to total duration
|
||||
total_raw = sum(raw_intervals)
|
||||
total_sec = total_duration_ms / 1000.0
|
||||
intervals = [(r / total_raw) * total_sec for r in raw_intervals]
|
||||
|
||||
# Add micro-jitter to timing (humans are never perfectly rhythmic)
|
||||
intervals = [max(0.002, i + random.uniform(-0.003, 0.003)) for i in intervals]
|
||||
|
||||
return intervals
|
||||
|
||||
@staticmethod
|
||||
def compute_fling_timing(n_points, total_duration_ms):
|
||||
"""
|
||||
Generates a J-curve timing schedule for flick/swipe gestures.
|
||||
|
||||
Unlike the sigmoid (which slows down at the end), this curve
|
||||
accelerates through the middle and maintains high velocity
|
||||
until the very last point to simulate a sudden 'liftoff' flick.
|
||||
This allows Android's ScrollView to register a high fling velocity.
|
||||
|
||||
Returns:
|
||||
List of inter-point delay times in seconds (length n_points)
|
||||
"""
|
||||
if n_points <= 1:
|
||||
return [total_duration_ms / 1000.0]
|
||||
|
||||
raw_intervals = []
|
||||
for i in range(n_points):
|
||||
t = i / (n_points - 1)
|
||||
# Starts slow (larger delay), speeds up continuously (smaller delay)
|
||||
speed_factor = 1.0 - (0.8 * t)
|
||||
raw_intervals.append(speed_factor)
|
||||
|
||||
total_raw = sum(raw_intervals)
|
||||
total_sec = total_duration_ms / 1000.0
|
||||
intervals = [(r / total_raw) * total_sec for r in raw_intervals]
|
||||
|
||||
# Add micro-jitter to timing
|
||||
intervals = [max(0.002, i + random.uniform(-0.003, 0.003)) for i in intervals]
|
||||
|
||||
return intervals
|
||||
207
GramAddict/core/physics/humanized_input.py
Normal file
207
GramAddict/core/physics/humanized_input.py
Normal file
@@ -0,0 +1,207 @@
|
||||
"""
|
||||
Physics — Humanized Input Simulation.
|
||||
|
||||
All low-level device interaction functions that simulate human touch behavior:
|
||||
scroll, click, swipe, horizontal swipe.
|
||||
|
||||
Uses Biomechanical Bézier curve generation for organic, non-linear touch paths
|
||||
with sigmoid velocity profiles and Gaussian pressure variation.
|
||||
Falls back to linear `input swipe` when sendevent is unavailable.
|
||||
|
||||
Extracted from bot_flow.py to enable isolated testing and reuse.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import random
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.physics.biomechanics import BezierGesture, PhysicsBody
|
||||
from GramAddict.core.physics.sendevent_injector import SendEventInjector
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def humanized_scroll(device, is_skip=False, resonance_score=None):
|
||||
"""
|
||||
Simulates a human thumb flick to trigger native scroll-snapping.
|
||||
|
||||
Uses Bézier curves for non-linear path generation and sigmoid timing
|
||||
for organic acceleration/deceleration. The PhysicsBody provides
|
||||
session-persistent anchor drift and fatigue modeling.
|
||||
|
||||
resonance_score: Optional. If high, increases chance of 'Correction' (Reverse scroll).
|
||||
"""
|
||||
info = device.get_info()
|
||||
w, h = info.get("displayWidth", 1080), info.get("displayHeight", 2400)
|
||||
body = PhysicsBody.get_session_instance(device)
|
||||
injector = SendEventInjector.get_instance(device)
|
||||
|
||||
# 1. Calculate Base Probability for Correction (Reverse Flick)
|
||||
# Default 15% for doomscroll corrections.
|
||||
# If resonance is high, we scale up to 45% chance to "Look back" at what we just passed.
|
||||
correction_prob = 0.15
|
||||
if resonance_score is not None and resonance_score > 0.7:
|
||||
correction_prob = 0.15 + (resonance_score - 0.7) * 1.0 # 0.7=0.15, 1.0=0.45
|
||||
|
||||
# Start position from PhysicsBody (session-aware, drifting)
|
||||
start_x, start_y = body.get_scroll_start()
|
||||
end_x = start_x + random.gauss(0, w * 0.008) # Slight horizontal drift
|
||||
|
||||
if is_skip:
|
||||
# Aggressive fast fling to skip quickly. NO CORRECTIONS.
|
||||
distance = int(h * random.uniform(0.6, 0.75))
|
||||
duration = random.uniform(150, 250) # slightly longer to ensure smooth fling registration
|
||||
end_y = start_y - distance
|
||||
else:
|
||||
# Playful, organic human scrolling
|
||||
play_choice = random.random()
|
||||
|
||||
if play_choice > (1.0 - (correction_prob / 3.0)) or play_choice > 0.95:
|
||||
# "Go back" / Scroll UP
|
||||
start_y = int(h * random.uniform(0.20, 0.40))
|
||||
distance = int(h * random.uniform(0.30, 0.50))
|
||||
duration = random.uniform(100, 180)
|
||||
end_y = min(start_y + distance, h - 10)
|
||||
logger.info(f"🪀 [Playful Scroll] Correction (Prob: {correction_prob:.2f}) — Flicking back up...")
|
||||
|
||||
elif play_choice > 0.85:
|
||||
# "Reading Jitter" / Playing around (10% chance)
|
||||
distance = int(h * random.uniform(0.05, 0.15))
|
||||
duration = random.uniform(300, 600)
|
||||
if random.random() > 0.5:
|
||||
end_y = start_y - distance
|
||||
else:
|
||||
start_y = int(h * random.uniform(0.30, 0.50))
|
||||
end_y = start_y + distance
|
||||
logger.info("🪀 [Playful Scroll] Micro-jitter...")
|
||||
|
||||
elif play_choice > 0.25:
|
||||
# "Lazy Flick" - Post to Post Snap (60% chance)
|
||||
distance = int(h * random.uniform(0.15, 0.25))
|
||||
duration = random.uniform(150, 350)
|
||||
end_y = start_y - distance
|
||||
|
||||
else:
|
||||
# Medium classic swipe (25% chance)
|
||||
distance = int(h * random.uniform(0.30, 0.45))
|
||||
duration = random.uniform(250, 500)
|
||||
end_y = start_y - distance
|
||||
|
||||
# --- Behavioral Micro-Patterns (new human behaviors) ---
|
||||
behavior = None if is_skip else _select_scroll_behavior()
|
||||
|
||||
if behavior == "pre_touch_dwell":
|
||||
# Finger lands on glass before swiping (50-200ms dwell)
|
||||
logger.debug("🦴 [Biomechanics] Pre-touch dwell...")
|
||||
|
||||
if behavior == "overshoot_correction":
|
||||
# Scroll too far, then micro-correct back
|
||||
logger.debug("🦴 [Biomechanics] Overshoot + Correction pattern")
|
||||
# Extend original distance, then we'll add a correction swipe after
|
||||
original_end_y = end_y
|
||||
overshoot = int(h * random.uniform(0.08, 0.15))
|
||||
if end_y < start_y:
|
||||
end_y -= overshoot # Scroll further down
|
||||
else:
|
||||
end_y += overshoot # Scroll further up
|
||||
|
||||
if behavior == "reading_pause":
|
||||
logger.debug("🦴 [Biomechanics] Mid-scroll reading pause")
|
||||
|
||||
# --- Generate Bézier Curve ---
|
||||
points = BezierGesture.scroll_curve((start_x, start_y), (int(end_x), end_y), body)
|
||||
timing = BezierGesture.compute_sigmoid_timing(len(points), duration)
|
||||
|
||||
# Pre-touch dwell: hold finger on glass before moving
|
||||
if behavior == "pre_touch_dwell":
|
||||
pre_dwell_ms = random.uniform(0.05, 0.2)
|
||||
# Insert a stationary point at the beginning
|
||||
points.insert(0, points[0])
|
||||
timing.insert(0, pre_dwell_ms)
|
||||
|
||||
# Reading pause: insert a long dwell mid-gesture
|
||||
if behavior == "reading_pause":
|
||||
mid = len(points) // 2
|
||||
pause_point = points[mid]
|
||||
pause_duration = random.uniform(0.5, 2.0)
|
||||
points.insert(mid + 1, pause_point)
|
||||
timing.insert(mid, pause_duration)
|
||||
|
||||
# --- Inject Gesture ---
|
||||
injector.inject_gesture(points, timing, touch_major=body.get_touch_major())
|
||||
|
||||
# Post-gesture: overshoot correction
|
||||
if behavior == "overshoot_correction":
|
||||
sleep(random.uniform(0.3, 0.6))
|
||||
# Small corrective scroll back
|
||||
corr_start_x, corr_start_y = body.get_scroll_start()
|
||||
corr_distance = int(h * random.uniform(0.05, 0.1))
|
||||
if original_end_y < start_y:
|
||||
corr_end_y = corr_start_y + corr_distance # Scroll back up
|
||||
else:
|
||||
corr_end_y = corr_start_y - corr_distance # Scroll back down
|
||||
|
||||
corr_points = BezierGesture.scroll_curve(
|
||||
(corr_start_x, corr_start_y), (corr_start_x, corr_end_y), body, n_points=6
|
||||
)
|
||||
corr_timing = BezierGesture.compute_sigmoid_timing(len(corr_points), 200)
|
||||
|
||||
injector.inject_gesture(corr_points, corr_timing, touch_major=body.get_touch_major())
|
||||
|
||||
|
||||
def humanized_click(device, x, y, double=False, sleep_mod=1.0):
|
||||
"""Simulates a human tap with biomechanical jitter and micro-drift."""
|
||||
|
||||
def single_tap():
|
||||
# Apply biomechanical jitter
|
||||
jx = int(x + random.gauss(0, 5))
|
||||
jy = int(y + random.gauss(0, 5))
|
||||
device.shell(f"input tap {jx} {jy}")
|
||||
|
||||
if double:
|
||||
# For double tap, the timing is extremely critical (<300ms between taps).
|
||||
# We bypass sendevent overhead and batch two input taps directly in the shell.
|
||||
device.shell(f"input tap {int(x)} {int(y)} && input tap {int(x)} {int(y)}")
|
||||
|
||||
else:
|
||||
single_tap()
|
||||
|
||||
|
||||
def humanized_horizontal_swipe(device, start_x, end_x, y, duration_ms):
|
||||
"""Simulates a human horizontal swipe with thumb arc simulation."""
|
||||
body = PhysicsBody.get_session_instance(device)
|
||||
injector = SendEventInjector.get_instance(device)
|
||||
|
||||
# Apply jitter to start/end positions
|
||||
noise_y = random.randint(-15, 15)
|
||||
actual_start_x = int(start_x) + random.randint(-10, 10)
|
||||
actual_end_x = int(end_x) + random.randint(-20, 20)
|
||||
actual_y = int(y) + noise_y
|
||||
|
||||
# Timing wobble (+/- 30%)
|
||||
actual_duration = int(duration_ms * random.uniform(0.7, 1.3))
|
||||
|
||||
points = BezierGesture.horizontal_swipe_curve((actual_start_x, actual_y), (actual_end_x, actual_y), body)
|
||||
timing = BezierGesture.compute_sigmoid_timing(len(points), actual_duration)
|
||||
|
||||
injector.inject_gesture(points, timing, touch_major=body.get_touch_major())
|
||||
|
||||
|
||||
def _select_scroll_behavior():
|
||||
"""
|
||||
Selects a micro-behavior pattern for the current scroll gesture.
|
||||
|
||||
Returns one of:
|
||||
- None: standard scroll (most common)
|
||||
- "pre_touch_dwell": finger lands on glass before swiping
|
||||
- "overshoot_correction": scrolls too far, then corrects back
|
||||
- "reading_pause": finger pauses mid-scroll
|
||||
"""
|
||||
roll = random.random()
|
||||
if roll < 0.08:
|
||||
return "pre_touch_dwell"
|
||||
elif roll < 0.20:
|
||||
return "overshoot_correction"
|
||||
elif roll < 0.35:
|
||||
return "reading_pause"
|
||||
return None
|
||||
263
GramAddict/core/physics/sendevent_injector.py
Normal file
263
GramAddict/core/physics/sendevent_injector.py
Normal file
@@ -0,0 +1,263 @@
|
||||
"""
|
||||
SendEvent Injector — Kernel-Level Touch Event Injection via ADB.
|
||||
|
||||
Injects raw MotionEvent sequences through `adb shell sendevent` to produce
|
||||
touch events that are indistinguishable from real finger input at the kernel level.
|
||||
|
||||
Key advantages over `input swipe`:
|
||||
- Supports pressure variation (ABS_MT_PRESSURE)
|
||||
- Supports touch contact area (ABS_MT_TOUCH_MAJOR)
|
||||
- Produces SOURCE_TOUCHSCREEN events (not SOURCE_UNKNOWN)
|
||||
- Multi-point non-linear paths
|
||||
|
||||
Falls back to `input swipe` if sendevent device detection fails.
|
||||
|
||||
Note: sendevent codes are device-specific. This module auto-detects the
|
||||
correct /dev/input/eventX and the axis ranges on first use.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import re
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class SendEventInjector:
|
||||
"""
|
||||
Injects touch events via adb shell sendevent for organic gesture simulation.
|
||||
|
||||
Uses a batched shell command approach: all events for one gesture are piped
|
||||
into a single `adb shell` invocation to minimize latency.
|
||||
"""
|
||||
|
||||
_instance = None
|
||||
|
||||
# Standard Linux input event types/codes
|
||||
EV_ABS = 3
|
||||
EV_SYN = 0
|
||||
EV_KEY = 1
|
||||
|
||||
# Multitouch protocol B codes (most modern Android devices)
|
||||
ABS_MT_TRACKING_ID = 0x39 # 57
|
||||
ABS_MT_POSITION_X = 0x35 # 53
|
||||
ABS_MT_POSITION_Y = 0x36 # 54
|
||||
ABS_MT_PRESSURE = 0x3A # 58
|
||||
ABS_MT_TOUCH_MAJOR = 0x30 # 48
|
||||
|
||||
SYN_REPORT = 0
|
||||
BTN_TOUCH = 0x14A # 330
|
||||
|
||||
def __init__(self, device):
|
||||
self.device = device
|
||||
self.event_device = None
|
||||
self.x_max = 1080
|
||||
self.y_max = 2400
|
||||
self.pressure_max = 255
|
||||
self.touch_major_max = 30
|
||||
self._fallback_mode = False
|
||||
self._detected = False
|
||||
|
||||
@classmethod
|
||||
def get_instance(cls, device):
|
||||
"""Returns a singleton injector for the device."""
|
||||
if cls._instance is None:
|
||||
cls._instance = cls(device)
|
||||
cls._instance._detect_touch_device()
|
||||
return cls._instance
|
||||
|
||||
@classmethod
|
||||
def reset(cls):
|
||||
"""Reset for testing / device change."""
|
||||
cls._instance = None
|
||||
|
||||
def _detect_touch_device(self):
|
||||
"""
|
||||
Auto-detects the touchscreen input device and its axis ranges
|
||||
by parsing `getevent -pl` output.
|
||||
"""
|
||||
try:
|
||||
result = self.device.shell("getevent -pl")
|
||||
if not isinstance(result, str):
|
||||
result = str(result)
|
||||
|
||||
# Find device with ABS_MT_POSITION_X
|
||||
current_device = None
|
||||
for line in result.split("\n"):
|
||||
line = line.strip()
|
||||
|
||||
# Device header: /dev/input/eventX
|
||||
dev_match = re.match(r"add device \d+:\s*(/dev/input/event\d+)", line)
|
||||
if dev_match:
|
||||
current_device = dev_match.group(1)
|
||||
|
||||
# Check for multitouch capability
|
||||
if current_device and "ABS_MT_POSITION_X" in line:
|
||||
self.event_device = current_device
|
||||
logger.info(f"🖐️ [SendEvent] Touch device detected: {self.event_device}")
|
||||
|
||||
# Parse axis ranges from the same section
|
||||
self._parse_axis_ranges(result, current_device)
|
||||
self._detected = True
|
||||
return
|
||||
|
||||
# If no MT device found, try fallback pattern
|
||||
logger.debug("⚠️ [SendEvent] No multitouch device found. " "Falling back to `input swipe` mode.")
|
||||
self._fallback_mode = True
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ [SendEvent] Device detection failed: {e}. " f"Falling back to `input swipe` mode.")
|
||||
self._fallback_mode = True
|
||||
|
||||
def _parse_axis_ranges(self, getevent_output, device_path):
|
||||
"""
|
||||
Parses axis max values from getevent output.
|
||||
Lines look like: ABS_MT_POSITION_X : value 0, min 0, max 1079, ...
|
||||
"""
|
||||
try:
|
||||
in_device = False
|
||||
for line in getevent_output.split("\n"):
|
||||
if device_path in line:
|
||||
in_device = True
|
||||
continue
|
||||
if in_device and line.strip().startswith("add device"):
|
||||
break # Next device
|
||||
|
||||
if in_device:
|
||||
if "ABS_MT_POSITION_X" in line:
|
||||
m = re.search(r"max\s+(\d+)", line)
|
||||
if m:
|
||||
self.x_max = int(m.group(1))
|
||||
elif "ABS_MT_POSITION_Y" in line:
|
||||
m = re.search(r"max\s+(\d+)", line)
|
||||
if m:
|
||||
self.y_max = int(m.group(1))
|
||||
elif "ABS_MT_PRESSURE" in line:
|
||||
m = re.search(r"max\s+(\d+)", line)
|
||||
if m:
|
||||
self.pressure_max = int(m.group(1))
|
||||
elif "ABS_MT_TOUCH_MAJOR" in line:
|
||||
m = re.search(r"max\s+(\d+)", line)
|
||||
if m:
|
||||
self.touch_major_max = int(m.group(1))
|
||||
|
||||
logger.debug(
|
||||
f"🖐️ [SendEvent] Axis ranges: X=0-{self.x_max}, "
|
||||
f"Y=0-{self.y_max}, P=0-{self.pressure_max}, "
|
||||
f"TM=0-{self.touch_major_max}"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"[SendEvent] Axis parsing error: {e}")
|
||||
|
||||
def inject_gesture(self, points, timing_intervals, touch_major=6):
|
||||
"""
|
||||
Injects a complete gesture (touch-down → move → touch-up) using sendevent.
|
||||
|
||||
Args:
|
||||
points: List of (x, y, pressure) tuples from BezierGesture
|
||||
timing_intervals: List of inter-point delays in seconds
|
||||
touch_major: Contact area size
|
||||
|
||||
Falls back to `input swipe` if sendevent is unavailable.
|
||||
"""
|
||||
if self._fallback_mode or not self.event_device:
|
||||
return self._fallback_input_swipe(points, timing_intervals)
|
||||
|
||||
if len(points) < 2:
|
||||
return
|
||||
|
||||
try:
|
||||
dev = self.event_device
|
||||
|
||||
# Scale coordinates from display space to input device space
|
||||
try:
|
||||
info = self.device.get_info()
|
||||
display_w = int(info.get("displayWidth", 1080)) if isinstance(info, dict) else 1080
|
||||
display_h = int(info.get("displayHeight", 2400)) if isinstance(info, dict) else 2400
|
||||
except (TypeError, ValueError):
|
||||
display_w, display_h = 1080, 2400
|
||||
scale_x = self.x_max / display_w
|
||||
scale_y = self.y_max / display_h
|
||||
|
||||
# Build batch command list
|
||||
cmds = []
|
||||
|
||||
# --- Touch Down (first point) ---
|
||||
x, y, pressure = points[0]
|
||||
ix = int(x * scale_x)
|
||||
iy = int(y * scale_y)
|
||||
ip = int(pressure * self.pressure_max)
|
||||
itm = min(touch_major, self.touch_major_max)
|
||||
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_TRACKING_ID} 0")
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_POSITION_X} {ix}")
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_POSITION_Y} {iy}")
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_PRESSURE} {ip}")
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_TOUCH_MAJOR} {itm}")
|
||||
cmds.append(f"sendevent {dev} {self.EV_KEY} {self.BTN_TOUCH} 1")
|
||||
cmds.append(f"sendevent {dev} {self.EV_SYN} {self.SYN_REPORT} 0")
|
||||
|
||||
# --- Move through intermediate points ---
|
||||
for i in range(1, len(points) - 1):
|
||||
if i - 1 < len(timing_intervals):
|
||||
delay = timing_intervals[i - 1]
|
||||
if delay > 0.001:
|
||||
cmds.append(f"sleep {delay:.3f}")
|
||||
|
||||
x, y, pressure = points[i]
|
||||
ix = int(x * scale_x)
|
||||
iy = int(y * scale_y)
|
||||
ip = int(pressure * self.pressure_max)
|
||||
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_POSITION_X} {ix}")
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_POSITION_Y} {iy}")
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_PRESSURE} {ip}")
|
||||
cmds.append(f"sendevent {dev} {self.EV_SYN} {self.SYN_REPORT} 0")
|
||||
|
||||
# --- Touch Up (last point) ---
|
||||
if len(timing_intervals) >= len(points) - 1:
|
||||
delay = timing_intervals[-1]
|
||||
else:
|
||||
delay = 0.01
|
||||
|
||||
if delay > 0.001:
|
||||
cmds.append(f"sleep {delay:.3f}")
|
||||
|
||||
x, y, pressure = points[-1]
|
||||
ix = int(x * scale_x)
|
||||
iy = int(y * scale_y)
|
||||
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_POSITION_X} {ix}")
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_POSITION_Y} {iy}")
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_PRESSURE} 0")
|
||||
cmds.append(f"sendevent {dev} {self.EV_ABS} {self.ABS_MT_TRACKING_ID} -1")
|
||||
cmds.append(f"sendevent {dev} {self.EV_KEY} {self.BTN_TOUCH} 0")
|
||||
cmds.append(f"sendevent {dev} {self.EV_SYN} {self.SYN_REPORT} 0")
|
||||
|
||||
# Execute ALL events in one atomic batch to eliminate ADB latency
|
||||
self.device.shell(" && ".join(cmds))
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ [SendEvent] Injection failed: {e}. Falling back.")
|
||||
self._fallback_input_swipe(points, timing_intervals)
|
||||
|
||||
def _fallback_input_swipe(self, points, timing_intervals):
|
||||
"""
|
||||
Fallback: Uses adb `input swipe` with first and last point.
|
||||
Loses pressure and curvature but maintains timing.
|
||||
"""
|
||||
if len(points) < 2:
|
||||
return
|
||||
|
||||
sx, sy, _ = points[0]
|
||||
ex, ey, _ = points[-1]
|
||||
total_ms = int(sum(timing_intervals) * 1000) if timing_intervals else 300
|
||||
|
||||
dist_x = abs(ex - sx)
|
||||
dist_y = abs(ey - sy)
|
||||
|
||||
# Android sometimes interprets a low-duration swipe with minimal movement as a long press or cancels it.
|
||||
# If it's physically a tap (minimal movement, short duration), use native input tap.
|
||||
if dist_x < 15 and dist_y < 15 and total_ms < 150:
|
||||
self.device.shell(f"input tap {int(sx)} {int(sy)}")
|
||||
else:
|
||||
self.device.shell(f"input swipe {int(sx)} {int(sy)} {int(ex)} {int(ey)} {total_ms}")
|
||||
273
GramAddict/core/physics/timing.py
Normal file
273
GramAddict/core/physics/timing.py
Normal file
@@ -0,0 +1,273 @@
|
||||
"""
|
||||
Physics — Timing & Wait Utilities.
|
||||
|
||||
UI readiness polling, post alignment, and adaptive snap recovery.
|
||||
These functions wait for the Android UI to reach a known state before
|
||||
the bot proceeds with interactions.
|
||||
|
||||
Extracted from bot_flow.py to enable isolated testing.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
from time import sleep
|
||||
|
||||
from GramAddict.core.diagnostic_dump import dump_ui_state
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def wait_for_post_loaded(device, timeout=5, nav_graph=None):
|
||||
"""
|
||||
Polls the UI hierarchy until feed markers appear, confirming a post is on screen.
|
||||
|
||||
If timeout is reached, attempts Adaptive Snap recovery:
|
||||
1. Detects trap states (Story/Reel viewer, Profile)
|
||||
2. Presses BACK to escape
|
||||
3. Micro-wobbles to force render
|
||||
"""
|
||||
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity, ScreenType
|
||||
|
||||
identity = ScreenIdentity("")
|
||||
|
||||
start = time.time()
|
||||
xml = ""
|
||||
while time.time() - start < timeout:
|
||||
try:
|
||||
xml = device.dump_hierarchy()
|
||||
state = identity.identify(xml)
|
||||
if state["screen_type"] in (ScreenType.POST_DETAIL, ScreenType.HOME_FEED, ScreenType.REELS_FEED):
|
||||
logger.debug("📱 Post loaded successfully.")
|
||||
return True
|
||||
|
||||
# Handle high-latency loads
|
||||
if "android.widget.ProgressBar" in xml or "loading_spinner" in xml.lower():
|
||||
# Extend timeout by 5 seconds if we're about to time out and still loading
|
||||
if time.time() - start > timeout - 1.0:
|
||||
timeout += 5.0
|
||||
logger.debug("⏳ Detected high-latency load (spinner active), extending timeout.")
|
||||
except Exception:
|
||||
pass
|
||||
sleep(0.5)
|
||||
|
||||
logger.warning("⚠️ Post did not load within timeout. Attempting Adaptive Snap.")
|
||||
dump_ui_state(device, "post_load_timeout", {"timeout_sec": timeout})
|
||||
|
||||
try:
|
||||
xml = device.dump_hierarchy()
|
||||
state = identity.identify(xml)
|
||||
|
||||
# 1. Trapped in a Story viewer? Press back.
|
||||
if state["screen_type"] == ScreenType.STORY_VIEW:
|
||||
logger.warning("🧗 [Adaptive Snap] Trapped in Story viewer. Pressing BACK.")
|
||||
device.press("back")
|
||||
sleep(1.5)
|
||||
# Give it one more chance to load the feed
|
||||
xml = device.dump_hierarchy()
|
||||
state = identity.identify(xml)
|
||||
if state["screen_type"] in (ScreenType.POST_DETAIL, ScreenType.HOME_FEED, ScreenType.REELS_FEED):
|
||||
logger.info("✅ Recovered to Feed.")
|
||||
return True
|
||||
|
||||
# 2. Trapped in Profile?
|
||||
# Only press back if we did NOT intend to be on a profile!
|
||||
expected_state = nav_graph.current_state if nav_graph else ""
|
||||
if expected_state != "ProfileView" and state["screen_type"] in (
|
||||
ScreenType.OWN_PROFILE,
|
||||
ScreenType.OTHER_PROFILE,
|
||||
):
|
||||
logger.warning("🧗 [Adaptive Snap] Trapped in Profile. Pressing BACK.")
|
||||
device.press("back")
|
||||
sleep(1.5)
|
||||
xml = device.dump_hierarchy()
|
||||
state = identity.identify(xml)
|
||||
|
||||
# 3. Stuck on Grid? The tap didn't register. Do not wobble.
|
||||
if state["screen_type"] in (ScreenType.EXPLORE_GRID, ScreenType.OWN_PROFILE, ScreenType.OTHER_PROFILE):
|
||||
logger.warning(
|
||||
"🧗 [Adaptive Snap] Detected bot is STILL on the Grid/Profile. Tap likely missed. Aborting snap."
|
||||
)
|
||||
return False
|
||||
|
||||
# 4. Stuck between posts (Feed markers not fully visible)? Micro-wobble.
|
||||
info = device.get_info()
|
||||
w, h = info.get("displayWidth", 1080), info.get("displayHeight", 2400)
|
||||
logger.warning("🧗 [Adaptive Snap] Wobbling to force render.")
|
||||
device.swipe(int(w / 2), int(h / 2), int(w / 2), int(h / 2) - 100, 0.1)
|
||||
sleep(0.5)
|
||||
device.swipe(int(w / 2), int(h / 2) - 100, int(w / 2), int(h / 2), 0.1)
|
||||
except Exception as e:
|
||||
logger.error(f"❌ [Adaptive Snap] Failed: {e}")
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def wait_for_story_loaded(device, timeout=5):
|
||||
"""Polls the UI hierarchy until story screen is identified via autonomous VLM classification."""
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity, ScreenType
|
||||
|
||||
start = time.time()
|
||||
identity = ScreenIdentity("")
|
||||
|
||||
while time.time() - start < timeout:
|
||||
try:
|
||||
xml = device.dump_hierarchy()
|
||||
state = identity.identify(xml)
|
||||
if state["screen_type"] == ScreenType.STORY_VIEW:
|
||||
logger.debug("📱 Story loaded successfully.")
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
sleep(0.5)
|
||||
|
||||
logger.warning("⚠️ Story did not load within timeout.")
|
||||
return False
|
||||
|
||||
|
||||
def wait_for_profile_loaded(device, timeout=5):
|
||||
"""Polls the UI hierarchy until the profile screen is identified via autonomous VLM classification."""
|
||||
import time
|
||||
|
||||
from GramAddict.core.perception.screen_identity import ScreenIdentity, ScreenType
|
||||
|
||||
start = time.time()
|
||||
identity = ScreenIdentity("")
|
||||
|
||||
while time.time() - start < timeout:
|
||||
try:
|
||||
xml = device.dump_hierarchy()
|
||||
state = identity.identify(xml)
|
||||
if state["screen_type"] in (ScreenType.OWN_PROFILE, ScreenType.OTHER_PROFILE):
|
||||
logger.debug("📱 Profile loaded successfully.")
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
sleep(0.5)
|
||||
|
||||
logger.warning("⚠️ Profile did not load within timeout.")
|
||||
return False
|
||||
|
||||
|
||||
def align_active_post(device):
|
||||
"""
|
||||
Programmatic snapping correction. Finds the nearest post header and perfectly
|
||||
snaps it to the top margin. Fixes inverted scroll mapping that pushed content away.
|
||||
Loops to ensure absolute alignment if stuck deeply between posts.
|
||||
"""
|
||||
aligned = False
|
||||
attempts = 0
|
||||
max_attempts = 5 # Increased for structural retry loop
|
||||
failed_bounds = set()
|
||||
|
||||
# Intents for structural discovery
|
||||
intents = [
|
||||
"post author username text (exclude follow buttons)",
|
||||
"post author header profile",
|
||||
"row_feed_photo_profile_name", # ID fallback
|
||||
"clips_viewer_author_container", # Reels fallback
|
||||
"feed post content", # Final desperation
|
||||
]
|
||||
|
||||
while not aligned and attempts < max_attempts:
|
||||
attempts += 1
|
||||
try:
|
||||
xml = device.dump_hierarchy()
|
||||
if "clips_video_container" in xml or "clips_viewer_container" in xml:
|
||||
logger.info("🎯 [Alignment] Reels view detected. Auto-snapping is native.")
|
||||
return True
|
||||
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
telepath = TelepathicEngine.get_instance()
|
||||
|
||||
target_node = None
|
||||
for intent in intents:
|
||||
target_node = telepath.find_best_node(
|
||||
xml, intent, min_confidence=0.35, device=device, track=False, exclude_bounds=list(failed_bounds)
|
||||
)
|
||||
if target_node:
|
||||
break
|
||||
|
||||
if target_node:
|
||||
original_attribs = target_node.get("original_attribs", {})
|
||||
bounds = original_attribs.get("bounds")
|
||||
|
||||
bounds_str = ""
|
||||
# If bounds is a tuple from SpatialNode.to_dict()
|
||||
if isinstance(bounds, (tuple, list)) and len(bounds) == 4:
|
||||
left, t, r, b = bounds
|
||||
bounds_str = f"[{left},{t}][{r},{b}]"
|
||||
else:
|
||||
# Fallback to string parsing
|
||||
if not bounds:
|
||||
bounds = target_node.get("bounds", "")
|
||||
m = re.match(r"\[(\d+),(\d+)\]\[(\d+),(\d+)\]", str(bounds))
|
||||
if m:
|
||||
left, t, r, b = map(int, m.groups())
|
||||
bounds_str = f"[{left},{t}][{r},{b}]"
|
||||
else:
|
||||
logger.warning(f"📐 [Alignment] Could not parse bounds: {bounds}")
|
||||
continue
|
||||
|
||||
# Check if this is a false positive (e.g. bottom bar item misclassified)
|
||||
# Post headers should be in the top half usually, or at least not at the very bottom
|
||||
info = device.get_info()
|
||||
h = info.get("displayHeight", 2400)
|
||||
if t > h * 0.85:
|
||||
logger.debug(f"📐 [Alignment] Rejecting node at y={t} (too low, likely bottom bar)")
|
||||
failed_bounds.add(bounds_str)
|
||||
continue
|
||||
|
||||
header_y = (t + b) // 2
|
||||
target_y = 250 # Top margin for headers
|
||||
diff = header_y - target_y
|
||||
|
||||
# If target is off-center (> 50px for higher precision), execute precise correction swipe
|
||||
if abs(diff) > 50:
|
||||
info = device.get_info()
|
||||
w = info.get("displayWidth", 1080)
|
||||
cx = w // 2
|
||||
|
||||
max_safe_swipe = int(h * 0.4)
|
||||
|
||||
# Calculate movement
|
||||
dist = min(abs(diff), max_safe_swipe)
|
||||
if diff > 0:
|
||||
# Content is too LOW. Move it UP (Swipe UP).
|
||||
start_y = int(h * 0.7)
|
||||
end_y = start_y - dist
|
||||
else:
|
||||
# Content is too HIGH. Move it DOWN (Swipe DOWN).
|
||||
start_y = int(h * 0.3)
|
||||
end_y = start_y + dist
|
||||
|
||||
logger.debug(f"📐 [Alignment] Attempt {attempts}: Snapping {diff}px (Swipe {start_y} -> {end_y})")
|
||||
# Duration 1.5s = ultra-precise mechanical drag with ZERO momentum
|
||||
device.swipe(cx, start_y, cx, end_y, duration=1.5)
|
||||
sleep(1.0)
|
||||
|
||||
# Refresh XML for next iteration check
|
||||
continue
|
||||
else:
|
||||
logger.info(f"🎯 [Alignment] Perfect snap achieved after {attempts} attempts.")
|
||||
aligned = True
|
||||
else:
|
||||
logger.debug(f"📐 [Alignment] No structural markers found on attempt {attempts}.")
|
||||
# If we can't find any markers, maybe we are stuck in a transition.
|
||||
# Micro-wobble to force a layout update.
|
||||
if attempts < 3:
|
||||
info = device.get_info()
|
||||
w, h = info.get("displayWidth", 1080), info.get("displayHeight", 2400)
|
||||
device.swipe(w // 2, h // 2, w // 2, h // 2 - 20, duration=0.2)
|
||||
sleep(0.5)
|
||||
device.swipe(w // 2, h // 2 - 20, w // 2, h // 2, duration=0.2)
|
||||
sleep(1.0)
|
||||
else:
|
||||
break
|
||||
except Exception as e:
|
||||
logger.debug(f"📐 [Alignment] Snapping correction failed: {e}")
|
||||
break
|
||||
|
||||
return aligned
|
||||
@@ -1,50 +1,65 @@
|
||||
import logging
|
||||
import json
|
||||
import os
|
||||
import uuid
|
||||
import time
|
||||
import random
|
||||
from GramAddict.core.utils import random_sleep
|
||||
from GramAddict.core.compiler_engine import VLMCompilerEngine
|
||||
from GramAddict.core.qdrant_memory import NavigationMemoryDB
|
||||
import time
|
||||
|
||||
from GramAddict.core.compiler_engine import VLMCompilerEngine
|
||||
from GramAddict.core.goap import GoalExecutor, ScreenType
|
||||
from GramAddict.core.qdrant_memory import NavigationMemoryDB
|
||||
from GramAddict.core.screen_topology import ScreenTopology
|
||||
from GramAddict.core.situational_awareness import SituationalAwarenessEngine, SituationType
|
||||
from GramAddict.core.utils import random_sleep
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Node:
|
||||
def __init__(self, name: str):
|
||||
self.name = name
|
||||
self.transitions = {} # Action (e.g. "tap_search") -> Node
|
||||
self.transitions = {} # Action (e.g. "tap_search") -> Node
|
||||
|
||||
|
||||
class QNavGraph:
|
||||
"""
|
||||
Project Singularity V7: Topological Navigation Map
|
||||
Maintains a directed graph of UI states. Instead of hardcoded navigation scripts,
|
||||
Topological Navigation Map
|
||||
Maintains a directed graph of UI states. Instead of hardcoded navigation scripts,
|
||||
the bot traverses this graph. If a path fails, it invokes the VLMCompilerEngine to repair it.
|
||||
"""
|
||||
|
||||
def __init__(self, device):
|
||||
self.device = device
|
||||
self.nodes = {}
|
||||
self.current_state = "UNKNOWN"
|
||||
self.nav_memory = NavigationMemoryDB()
|
||||
|
||||
self.sae = SituationalAwarenessEngine.get_instance(device)
|
||||
self.goap = GoalExecutor.get_instance(device)
|
||||
|
||||
self.compiler = VLMCompilerEngine(device)
|
||||
self._load_graph()
|
||||
|
||||
|
||||
def _load_graph(self):
|
||||
"""Loads the topological map from Qdrant. Merges with core seeds to guarantee baseline navigation."""
|
||||
"""Loads the topological map from Qdrant. Merges with core seeds from ScreenTopology (SSOT)."""
|
||||
logger.debug("🌐 [NavGraph] Syncing topological map with Qdrant...")
|
||||
self.nodes = self.nav_memory.get_all_transitions()
|
||||
|
||||
core_nodes = {
|
||||
"HomeFeed": {"transitions": {"tap_explore_tab": "ExploreFeed", "tap_profile_tab": "OwnProfile", "tap_message_icon": "MessageInbox"}},
|
||||
"ExploreFeed": {"transitions": {"tap_home_tab": "HomeFeed"}},
|
||||
"OwnProfile": {"transitions": {"tap_home_tab": "HomeFeed", "tap_following_list": "FollowingList"}},
|
||||
"MessageInbox": {"transitions": {"tap_back": "HomeFeed"}},
|
||||
"FollowingList": {"transitions": {"tap_back": "OwnProfile"}},
|
||||
"UNKNOWN": {"transitions": {"tap_home_tab": "HomeFeed"}}
|
||||
}
|
||||
|
||||
# Generate core_nodes from ScreenTopology (single source of truth)
|
||||
core_nodes = {}
|
||||
for screen_type, transitions in ScreenTopology.TRANSITIONS.items():
|
||||
# Reverse lookup: ScreenType → QNavGraph string name from SSOT
|
||||
screen_name_map = {
|
||||
v: k
|
||||
for k, v in ScreenTopology.SCREEN_NAME_MAP.items()
|
||||
if v not in (ScreenType.HOME_FEED, ScreenType.EXPLORE_GRID) or k not in ("StoriesFeed", "SearchFeed")
|
||||
}
|
||||
node_name = screen_name_map.get(screen_type)
|
||||
if not node_name:
|
||||
continue
|
||||
node_transitions = {}
|
||||
for action, target_screen in transitions.items():
|
||||
# Convert action format: "tap profile tab" → "tap_profile_tab"
|
||||
action_key = action.replace(" ", "_")
|
||||
target_name = screen_name_map.get(target_screen, target_screen.name)
|
||||
node_transitions[action_key] = target_name
|
||||
core_nodes[node_name] = {"transitions": node_transitions}
|
||||
|
||||
# Merge core nodes into loaded nodes
|
||||
for node, data in core_nodes.items():
|
||||
@@ -59,251 +74,103 @@ class QNavGraph:
|
||||
"""Deprecated: Navigation state is now persisted per-transition in Qdrant."""
|
||||
pass
|
||||
|
||||
|
||||
def navigate_to(self, target_state: str, zero_engine, recovery_attempts: int = 0):
|
||||
"""
|
||||
Attempts to navigate from current_state to target_state using the Graph.
|
||||
GOAP-powered autonomous navigation.
|
||||
Delegates to the Goal-Oriented Action Planner instead of
|
||||
using hardcoded state machines and BFS pathfinding.
|
||||
"""
|
||||
logger.info(f"📍 Navigating autonomously to: {target_state}")
|
||||
|
||||
if recovery_attempts > 2:
|
||||
logger.error(f"FATAL: Context recovery failed after {recovery_attempts} attempts. Bailing out of navigation loop.")
|
||||
return False
|
||||
|
||||
# Stories are viewed from the HomeFeed natively. There is no separate StoriesFeed node.
|
||||
# We navigate to HomeFeed dynamically, and let bot_flow handle the interaction.
|
||||
logical_target = "HomeFeed" if target_state == "StoriesFeed" else target_state
|
||||
|
||||
# Simple BFS to find sequence of actions
|
||||
path = self._find_path(self.current_state, logical_target)
|
||||
|
||||
if path is None:
|
||||
logger.warning(f"No known path from {self.current_state} to {target_state}. Attempting semantic recovery via Global Navigation Bar...")
|
||||
|
||||
# The global bottom navigation often gives us direct access from most positions
|
||||
# Map target_state to its global tab action
|
||||
target_to_action = {
|
||||
"ExploreFeed": "tap_explore_tab",
|
||||
"HomeFeed": "tap_home_tab",
|
||||
"OwnProfile": "tap_profile_tab",
|
||||
"ReelsFeed": "tap_reels_tab",
|
||||
"StoriesFeed": "tap_home_tab",
|
||||
}
|
||||
|
||||
direct_action = target_to_action.get(target_state, "tap_home_tab")
|
||||
target_anchor = target_state if direct_action != "tap_home_tab" else "HomeFeed"
|
||||
|
||||
success = self._execute_transition(direct_action)
|
||||
if success is True:
|
||||
logger.info(f"Successfully anchored! Learned new global edge: {self.current_state} -> {target_anchor} via {direct_action}")
|
||||
if self.current_state not in self.nodes:
|
||||
self.nodes[self.current_state] = {"transitions": {}}
|
||||
self.nodes[self.current_state]["transitions"][direct_action] = target_anchor
|
||||
self.nav_memory.store_transition(self.current_state, direct_action, target_anchor)
|
||||
|
||||
self.current_state = target_anchor
|
||||
path = self._find_path(self.current_state, logical_target)
|
||||
elif success == "CONTEXT_LOST":
|
||||
logger.warning(f"⚠️ Context was lost during direct action '{direct_action}'. Forcing app focus and resetting path.")
|
||||
self.device.deviceV2.app_start(self.device.app_id, use_monkey=True)
|
||||
random_sleep(2.5, 4.0)
|
||||
logger.info(f"📍 [GOAP] Navigating autonomously to: {target_state}")
|
||||
|
||||
# Set bot username for screen identity
|
||||
try:
|
||||
from GramAddict.core.config import Config
|
||||
|
||||
args = getattr(Config(), "args", None)
|
||||
if args and hasattr(args, "username"):
|
||||
self.goap.screen_id.bot_username = args.username.lower()
|
||||
except Exception as e:
|
||||
logger.debug(f"⚠️ [GOAP] Skipping username sync: {e}")
|
||||
|
||||
success = self.goap.navigate_to_screen(target_state)
|
||||
|
||||
if success:
|
||||
self.current_state = target_state
|
||||
logger.info(f"✅ [GOAP] Reached {target_state}")
|
||||
else:
|
||||
logger.error(f"❌ [GOAP] Failed to reach {target_state}")
|
||||
# Final fallback: force app start and reset
|
||||
if recovery_attempts < 2:
|
||||
logger.warning(
|
||||
f"🔄 [GOAP Recovery] Step {recovery_attempts + 1}: Attempting app restart to escape softlock..."
|
||||
)
|
||||
self.device.app_start(self.device.app_id, use_monkey=True)
|
||||
random_sleep(3.0, 4.5)
|
||||
self.current_state = "HomeFeed"
|
||||
return self.navigate_to(target_state, zero_engine, recovery_attempts=recovery_attempts + 1)
|
||||
# Clear GOAP status for fresh attempt
|
||||
return self.navigate_to(target_state, zero_engine, recovery_attempts + 1)
|
||||
else:
|
||||
# NEW: Attempt Back-out recovery if we are in UNKNOWN and direct tap failed
|
||||
if self.current_state == "UNKNOWN":
|
||||
logger.warning(f"📍 [Recovery] Semantic tap failed from UNKNOWN. Attempting to back out of sub-view...")
|
||||
self.device.deviceV2.press("back")
|
||||
random_sleep(1.5, 3.0)
|
||||
# We stay in UNKNOWN, but next attempt might see the nav bar
|
||||
return self.navigate_to(target_state, zero_engine, recovery_attempts=recovery_attempts + 0.5)
|
||||
path = None
|
||||
|
||||
if path is None:
|
||||
# Absolute last resort fallback: force app to main activity
|
||||
logger.warning("Semantic recovery failed. Forcing main activity intent...")
|
||||
self.device.deviceV2.app_start(self.device.app_id)
|
||||
random_sleep(2.5, 4.0)
|
||||
self.current_state = "HomeFeed"
|
||||
path = self._find_path(self.current_state, logical_target)
|
||||
|
||||
if path is None:
|
||||
logger.error(f"FATAL: Cannot find any path to {target_state} even after forcing main activity.")
|
||||
return False
|
||||
logger.critical(
|
||||
f"🛑 [GOAP Recovery] Max recovery attempts reached. Navigation to {target_state} aborted."
|
||||
)
|
||||
|
||||
for action in path:
|
||||
result = self._execute_transition(action)
|
||||
|
||||
if result == "CONTEXT_LOST":
|
||||
logger.warning(f"⚠️ Context was lost during '{action}'. Forcing app focus and resetting path.")
|
||||
self.device.deviceV2.app_start(self.device.app_id, use_monkey=True)
|
||||
random_sleep(2.5, 4.0)
|
||||
# After app start, we are at HomeFeed (usually)
|
||||
self.current_state = "HomeFeed"
|
||||
# Recursively call navigate_to from the new anchor
|
||||
return self.navigate_to(target_state, zero_engine, recovery_attempts=recovery_attempts + 1)
|
||||
return success
|
||||
|
||||
if not result:
|
||||
logger.error(f"Nav transition '{action}' failed! Initiating self-repair...")
|
||||
self._repair_transition(action)
|
||||
# Retry after repair
|
||||
success = self._execute_transition(action)
|
||||
if not success or success == "CONTEXT_LOST":
|
||||
logger.error(f"FATAL: Auto-repair failed for transition: {action}")
|
||||
return False
|
||||
|
||||
self.current_state = logical_target
|
||||
return True
|
||||
def do(self, goal: str) -> bool:
|
||||
"""
|
||||
GOAP-powered action execution.
|
||||
Replaces _execute_transition() for post interactions.
|
||||
|
||||
Usage:
|
||||
nav_graph.do("like this post") # instead of _execute_transition("tap_like_button")
|
||||
nav_graph.do("follow this user") # instead of _execute_transition("tap_follow_button")
|
||||
nav_graph.do("tap first grid item") # instead of _execute_transition("tap_explore_grid_item")
|
||||
"""
|
||||
|
||||
screen = self.goap.perceive()
|
||||
return self.goap._execute_action(goal, screen_state=screen)
|
||||
|
||||
def _find_path(self, start: str, end: str):
|
||||
if start == end: return []
|
||||
if start not in self.nodes: return None
|
||||
|
||||
queue = [(start, [])]
|
||||
visited = set()
|
||||
|
||||
while queue:
|
||||
current, path = queue.pop(0)
|
||||
if current == end:
|
||||
return path
|
||||
|
||||
visited.add(current)
|
||||
transitions = self.nodes.get(current, {}).get("transitions", {})
|
||||
|
||||
for action, next_state in transitions.items():
|
||||
if next_state not in visited:
|
||||
queue.append((next_state, path + [action]))
|
||||
|
||||
return None
|
||||
"""Delegates to ScreenTopology for BFS pathfinding (SSOT)."""
|
||||
from_screen = ScreenTopology.SCREEN_NAME_MAP.get(start)
|
||||
to_screen = ScreenTopology.SCREEN_NAME_MAP.get(end)
|
||||
if not from_screen or not to_screen:
|
||||
return None
|
||||
|
||||
def _clear_anomaly_obstacles(self, max_attempts=2) -> bool:
|
||||
route = ScreenTopology.find_route(from_screen, to_screen)
|
||||
if route is None:
|
||||
return None
|
||||
|
||||
# Convert back to QNavGraph action format: "tap profile tab" → "tap_profile_tab"
|
||||
return [action.replace(" ", "_") for action, _ in route]
|
||||
|
||||
def _clear_anomaly_obstacles(self, max_attempts=2, xml_dump: str = None) -> bool:
|
||||
"""
|
||||
Actively hunts down and dismisses known edge-case overlays (OS Permissions, Surveys)
|
||||
that block navigation. If an unknown modal is detected, falls back to pressing BACK.
|
||||
Returns True if an obstacle was detected and handled, False if the UI is clear.
|
||||
Delegates ALL obstacle detection to the Situational Awareness Engine.
|
||||
Returns True if an obstacle was cleared, False otherwise.
|
||||
"""
|
||||
import xml.etree.ElementTree as ET
|
||||
import re
|
||||
import time
|
||||
from GramAddict.core.exceptions import ActionBlockedError
|
||||
success = self.sae.ensure_clear_screen(max_attempts=max_attempts + 5, initial_xml=xml_dump)
|
||||
return success
|
||||
|
||||
for attempt in range(max_attempts):
|
||||
xml_dump = self.device.dump_hierarchy()
|
||||
if not isinstance(xml_dump, str):
|
||||
return False
|
||||
|
||||
xml_dump_lower = xml_dump.lower()
|
||||
|
||||
# --- 0. FATAL: Action Blocked Guard ---
|
||||
# If Instagram explicitly restricts our activity, we must hard crash to prevent permanent account bans.
|
||||
is_action_blocked = (
|
||||
"try again later" in xml_dump_lower or
|
||||
"action blocked" in xml_dump_lower or
|
||||
"restrict certain activity" in xml_dump_lower or
|
||||
"help us confirm you own" in xml_dump_lower or
|
||||
"confirm it's you" in xml_dump_lower or
|
||||
"später erneut versuchen" in xml_dump_lower or
|
||||
"bestätige, dass du es bist" in xml_dump_lower or
|
||||
"handlung blockiert" in xml_dump_lower or
|
||||
"eingeschränkt" in xml_dump_lower
|
||||
)
|
||||
|
||||
if is_action_blocked:
|
||||
logger.error("🚫 [CRITICAL GUARD] Instagram Action Block Dialog Detected! Aborting run to protect account.")
|
||||
raise ActionBlockedError("Instagram soft-banned the account. We hit a rate limit or restriction. Halting all activities.")
|
||||
|
||||
try:
|
||||
tree = ET.fromstring(xml_dump)
|
||||
except Exception:
|
||||
# If XML parsing fails, fall back to simple string check
|
||||
if re.search(r'bottom_sheet_container|dialog_container|dialog_root|bottom_sheet_drag|action_sheet_container', xml_dump):
|
||||
logger.warning("🛡️ [Z-Depth Guard] Generic obstacle detected. Pressing BACK to clear...")
|
||||
self.device.deviceV2.press("back")
|
||||
random_sleep(1.0, 2.5)
|
||||
return True
|
||||
return False
|
||||
|
||||
handled = False
|
||||
|
||||
# --- 1. OS Permission Dialogs (Android) ---
|
||||
grant_dialog = tree.find(".//node[@resource-id='com.android.permissioncontroller:id/grant_dialog']")
|
||||
if grant_dialog is not None:
|
||||
logger.warning("🛡️ [Z-Depth Guard] OS Permission Dialog detected! Searching for Deny button...")
|
||||
deny_btn = grant_dialog.find(".//node[@resource-id='com.android.permissioncontroller:id/permission_deny_button']")
|
||||
if deny_btn is not None and deny_btn.get("bounds"):
|
||||
bounds = re.findall(r'\d+', deny_btn.get("bounds"))
|
||||
if len(bounds) == 4:
|
||||
x = (int(bounds[0]) + int(bounds[2])) // 2
|
||||
y = (int(bounds[1]) + int(bounds[3])) // 2
|
||||
logger.info(f"👆 Clicking 'Deny' at ({x}, {y})")
|
||||
from GramAddict.core.bot_flow import _humanized_click
|
||||
_humanized_click(self.device, x, y)
|
||||
random_sleep(1.0, 2.5)
|
||||
handled = True
|
||||
|
||||
# --- 2. Instagram Surveys & Interstitials ---
|
||||
if not handled:
|
||||
survey_cont = tree.find(".//node[@resource-id='com.instagram.android:id/survey_container']")
|
||||
if survey_cont is not None:
|
||||
logger.warning("🛡️ [Z-Depth Guard] Instagram Survey detected! Searching for Dismiss/Not Now button...")
|
||||
|
||||
# Usually the negative button has an explicit ID
|
||||
neg_btn = survey_cont.find(".//node[@resource-id='com.instagram.android:id/button_negative']")
|
||||
|
||||
# Fallback to semantic text search if ID fails
|
||||
if neg_btn is None:
|
||||
for n in survey_cont.iter('node'):
|
||||
txt = n.get("text", "").lower() + " " + n.get("content-desc", "").lower()
|
||||
if "not now" in txt or "cancel" in txt or "dismiss" in txt or "skip" in txt:
|
||||
neg_btn = n
|
||||
break
|
||||
|
||||
if neg_btn is not None and neg_btn.get("bounds"):
|
||||
bounds = re.findall(r'\d+', neg_btn.get("bounds"))
|
||||
if len(bounds) == 4:
|
||||
x = (int(bounds[0]) + int(bounds[2])) // 2
|
||||
y = (int(bounds[1]) + int(bounds[3])) // 2
|
||||
logger.info(f"👆 Clicking Survey Dismiss at ({x}, {y})")
|
||||
from GramAddict.core.bot_flow import _humanized_click
|
||||
_humanized_click(self.device, x, y)
|
||||
random_sleep(1.0, 2.5)
|
||||
handled = True
|
||||
|
||||
# --- 3. Intrusive Bottom / Action Sheets ---
|
||||
if not handled:
|
||||
if re.search(r'bottom_sheet_container|dialog_container|dialog_root|bottom_sheet_drag|action_sheet_container', xml_dump):
|
||||
logger.warning("🛡️ [Z-Depth Guard] Generic obstacle or Action Sheet detected. Pressing BACK to clear...")
|
||||
self.device.deviceV2.press("back")
|
||||
random_sleep(1.0, 2.5)
|
||||
handled = True
|
||||
|
||||
if handled:
|
||||
# Loop around: could be multiple stacked dialogs
|
||||
continue
|
||||
else:
|
||||
# No known anomaly obstacles detected
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def _execute_transition(self, action: str, mock_semantic_engine=None, max_retries: int = 2) -> bool:
|
||||
def _execute_transition(self, action: str, max_retries: int = 2) -> bool:
|
||||
"""
|
||||
Executes a transition (e.g. 'tap_explore_tab') using the Telepathic Semantic Engine.
|
||||
"""
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
engine = mock_semantic_engine or TelepathicEngine.get_instance()
|
||||
|
||||
|
||||
engine = TelepathicEngine.get_instance()
|
||||
|
||||
failed_positions = set() # Track (x, y) of clicks that failed, for grid retry diversity
|
||||
|
||||
|
||||
for attempt in range(max_retries + 1):
|
||||
context_xml = self.device.dump_hierarchy()
|
||||
|
||||
|
||||
# ── Z-Depth Guard / Anomaly Obstacle Clearance ──
|
||||
cleared_something = self._clear_anomaly_obstacles()
|
||||
cleared_something = self._clear_anomaly_obstacles(xml_dump=context_xml)
|
||||
if cleared_something:
|
||||
# Re-acquire context after clearing obstacle
|
||||
context_xml = self.device.dump_hierarchy()
|
||||
|
||||
|
||||
# We phrase the action as an intent for the semantic engine
|
||||
# e.g. "tap_explore_tab" -> "tap explore tab"
|
||||
# We add some common synonyms for Instagram to help the vector engine
|
||||
@@ -323,28 +190,38 @@ class QNavGraph:
|
||||
# Grid & Profile
|
||||
"tap_explore_grid_item": "first image in explore grid",
|
||||
"tap_story_tray_item": "profile picture avatar story ring",
|
||||
"tap_follow_button": "tap follow button on profile",
|
||||
"tap_follow_button": "tap 'Follow' button on profile",
|
||||
"tap_grid_first_post": "first image post in profile grid",
|
||||
"tap_back": "tap back button icon arrow",
|
||||
"tap_message_icon": "tap direct message icon inbox",
|
||||
"tap_newsfeed_tab": "tap activity heart icon notifications",
|
||||
}
|
||||
intent_description = intent_map.get(action, action.replace("_", " "))
|
||||
|
||||
|
||||
# Use TelepathicEngine to find the most likely node for this intent
|
||||
# If vector score < 0.82, it will trigger the Vision Cortex Fallback (VLM)
|
||||
# Pass failed_positions so grid fast-path picks a different item on retry
|
||||
best_node = engine.find_best_node(context_xml, intent_description, min_confidence=0.82, device=self.device, skip_positions=failed_positions)
|
||||
|
||||
best_node = engine.find_best_node(
|
||||
context_xml,
|
||||
intent_description,
|
||||
min_confidence=0.82,
|
||||
device=self.device,
|
||||
skip_positions=failed_positions,
|
||||
)
|
||||
|
||||
# ── Blocked by Modal Recovery ──
|
||||
if best_node and best_node.get("blocked_by_modal"):
|
||||
logger.warning(f"🛡️ [Modal Recovery] Navigation '{action}' is blocked by a modal. Attempting anomaly clearance...")
|
||||
logger.warning(
|
||||
f"🛡️ [Modal Recovery] Navigation '{action}' is blocked by a modal. Attempting anomaly clearance..."
|
||||
)
|
||||
self._clear_anomaly_obstacles()
|
||||
if attempt < max_retries:
|
||||
context_xml = self.device.dump_hierarchy()
|
||||
continue
|
||||
else:
|
||||
logger.error(f"❌ [Modal Recovery] Persistent blockage for '{action}'. Escalating to Context Lost (App Restart).")
|
||||
logger.error(
|
||||
f"❌ [Modal Recovery] Persistent blockage for '{action}'. Escalating to Context Lost (App Restart)."
|
||||
)
|
||||
return "CONTEXT_LOST"
|
||||
|
||||
if not best_node:
|
||||
@@ -352,60 +229,76 @@ class QNavGraph:
|
||||
# Check if we are even in the right app
|
||||
current_app = self.device._get_current_app()
|
||||
if current_app != self.device.app_id:
|
||||
logger.warning(f"⚠️ [Context Lost] Currently in '{current_app}', expected '{self.device.app_id}'. Transition '{action}' aborted.")
|
||||
logger.warning(
|
||||
f"⚠️ [Context Lost] Currently in '{current_app}', expected '{self.device.app_id}'. Transition '{action}' aborted."
|
||||
)
|
||||
return "CONTEXT_LOST"
|
||||
|
||||
|
||||
# Try again if within retries, UI might be animating
|
||||
if attempt < max_retries:
|
||||
time.sleep(1.0)
|
||||
continue
|
||||
|
||||
# FINAL ATTEMPT ESCAPE:
|
||||
# If we are looking for the 'Home' tab (our baseline) and everything failed,
|
||||
|
||||
# FINAL ATTEMPT ESCAPE:
|
||||
# If we are looking for the 'Home' tab (our baseline) and everything failed,
|
||||
# we might be in an unknown sub-view. Try one last 'BACK' press.
|
||||
if action == "tap_home_tab":
|
||||
logger.warning("📍 [Escape] Home tab not found after all retries. Attempting final BACK press to escape sub-view...")
|
||||
self.device.deviceV2.press("back")
|
||||
logger.warning(
|
||||
"📍 [Escape] Home tab not found after all retries. Attempting final BACK press to escape sub-view..."
|
||||
)
|
||||
self.device.press("back")
|
||||
time.sleep(2.0)
|
||||
|
||||
|
||||
return False
|
||||
|
||||
|
||||
if best_node.get("skip") or (best_node.get("selected") and "tab" in action):
|
||||
logger.info(f"⏭️ Skipping physical tap for '{action}' (Semantic Fast-Path indicated state already fulfilled)")
|
||||
logger.info(
|
||||
f"⏭️ Skipping physical tap for '{action}' (Semantic Fast-Path indicated state already fulfilled)"
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
source_tag = best_node.get("source", "telepathic").replace("_", " ").title()
|
||||
logger.info(f"QNavGraph executing transition '{action}' via [{source_tag}] (Score: {best_node.get('score', 1.0):.3f})")
|
||||
|
||||
logger.info(
|
||||
f"QNavGraph executing transition '{action}' via [{source_tag}] (Score: {best_node.get('score', 1.0):.3f})"
|
||||
)
|
||||
|
||||
# Execute click
|
||||
self.device.click(obj=best_node)
|
||||
time.sleep(random.uniform(1.2, 2.5))
|
||||
|
||||
time.sleep(random.uniform(1.6, 2.8))
|
||||
|
||||
# ── Post-Click Verification: Did it work? ──
|
||||
post_click_xml = self.device.dump_hierarchy()
|
||||
|
||||
# ── App Perimeter Guard ──
|
||||
current_app = self.device._get_current_app()
|
||||
if current_app != self.device.app_id:
|
||||
logger.error(f"🚨 [Perimeter Guard] FATAL: Transition '{action}' caused app to drift to '{current_app}'! Rejecting VLM snippet.")
|
||||
|
||||
# ── App Perimeter Guard (SAE-powered) ──
|
||||
post_situation = self.sae.perceive(post_click_xml)
|
||||
if post_situation in (
|
||||
SituationType.OBSTACLE_FOREIGN_APP,
|
||||
SituationType.OBSTACLE_SYSTEM,
|
||||
SituationType.OBSTACLE_MODAL,
|
||||
):
|
||||
logger.warning(
|
||||
f"🚨 [SAE Perimeter] Transition '{action}' caused drift ({post_situation.value}). Initiating autonomous recovery..."
|
||||
)
|
||||
failed_positions.add((best_node["x"], best_node["y"]))
|
||||
engine.reject_click(intent_description)
|
||||
|
||||
# Attempt immediate recovery to main app
|
||||
self.device.deviceV2.press("back")
|
||||
random_sleep(1.0, 2.0)
|
||||
if self.device._get_current_app() != self.device.app_id:
|
||||
self.device.deviceV2.app_start(self.device.app_id, use_monkey=True)
|
||||
|
||||
# Return CONTEXT_LOST immediately to prevent memory poisoning
|
||||
|
||||
# Let SAE handle recovery autonomously
|
||||
recovered = self.sae.ensure_clear_screen(max_attempts=5)
|
||||
if not recovered:
|
||||
return "CONTEXT_LOST"
|
||||
|
||||
# Screen is clear but the transition itself failed — retry
|
||||
if attempt < max_retries:
|
||||
logger.info(f"🔄 [SAE Recovery] Screen recovered. Retrying transition '{action}'...")
|
||||
continue
|
||||
return "CONTEXT_LOST"
|
||||
|
||||
|
||||
# 1. Semantic Verification (Hardened)
|
||||
is_verified = engine.verify_success(intent_description, post_click_xml)
|
||||
|
||||
|
||||
# 2. UI Change Verification (Fallback/Navigation)
|
||||
ui_changed = post_click_xml != context_xml
|
||||
|
||||
|
||||
if is_verified and ui_changed:
|
||||
engine.confirm_click(intent_description)
|
||||
return True
|
||||
@@ -414,27 +307,33 @@ class QNavGraph:
|
||||
failed_positions.add((best_node["x"], best_node["y"]))
|
||||
engine.reject_click(intent_description)
|
||||
if attempt < max_retries:
|
||||
logger.info(f"🔄 [Autonomy] UI unchanged. Retrying transition '{action}' ({attempt + 1}/{max_retries})...")
|
||||
logger.info(
|
||||
f"🔄 [Autonomy] UI unchanged. Retrying transition '{action}' ({attempt + 1}/{max_retries})..."
|
||||
)
|
||||
continue
|
||||
else:
|
||||
return False
|
||||
else:
|
||||
# UI changed but semantic verification failed (accidental click or false positive)
|
||||
logger.warning(f"❌ [Ambiguity Guard] UI changed after '{action}', but semantic verification FAILED. Rejecting mapping.")
|
||||
logger.warning(
|
||||
f"❌ [Ambiguity Guard] UI changed after '{action}', but semantic verification FAILED. Rejecting mapping."
|
||||
)
|
||||
failed_positions.add((best_node["x"], best_node["y"]))
|
||||
engine.reject_click(intent_description)
|
||||
|
||||
|
||||
# Safety: If we're not where we expect to be, try to back out to clear any accidentally opened menus
|
||||
logger.info("🛡️ [Safety Reset] Pressing BACK to clear potential accidental menu/sub-view.")
|
||||
self.device.deviceV2.press("back")
|
||||
self.device.press("back")
|
||||
time.sleep(1.0)
|
||||
|
||||
|
||||
if attempt < max_retries:
|
||||
logger.info(f"🔄 [Autonomy] Negative learning acquired. Retrying transition '{action}' ({attempt + 1}/{max_retries})...")
|
||||
logger.info(
|
||||
f"🔄 [Autonomy] Negative learning acquired. Retrying transition '{action}' ({attempt + 1}/{max_retries})..."
|
||||
)
|
||||
continue
|
||||
else:
|
||||
return False
|
||||
|
||||
|
||||
return False
|
||||
|
||||
def _repair_transition(self, action: str):
|
||||
@@ -443,13 +342,14 @@ class QNavGraph:
|
||||
and write a new rule for `action`.
|
||||
"""
|
||||
from GramAddict.core.dojo_engine import DojoEngine
|
||||
|
||||
dojo = DojoEngine.get_instance(self.device)
|
||||
|
||||
logger.warning(f"⛩️ [Dojo] Enqueuing auto-labeling job for missing '{action}'.", extra={"color": f"\x1b[36m"})
|
||||
|
||||
logger.warning(f"⛩️ [Dojo] Enqueuing auto-labeling job for missing '{action}'.", extra={"color": "\x1b[36m"})
|
||||
context_xml = self.device.dump_hierarchy()
|
||||
|
||||
|
||||
dojo.submit_snapshot(
|
||||
heuristic_name=action,
|
||||
context_xml=context_xml,
|
||||
intent_prompt=f"Find the button that performs: {action}. Be extremely robust against structural UI changes."
|
||||
intent_prompt=f"Find the button that performs: {action}. Be extremely robust against structural UI changes.",
|
||||
)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,20 +1,24 @@
|
||||
import logging
|
||||
import math
|
||||
import random
|
||||
import re
|
||||
from typing import Optional
|
||||
|
||||
from colorama import Fore
|
||||
|
||||
from GramAddict.core.qdrant_memory import ContentMemoryDB, PersonaMemoryDB, ParasocialCRMDB, CommentMemoryDB
|
||||
from GramAddict.core.llm_provider import query_llm
|
||||
from GramAddict.core.qdrant_memory import CommentMemoryDB, ContentMemoryDB, ParasocialCRMDB, PersonaMemoryDB
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ResonanceEngine:
|
||||
"""
|
||||
The Aesthetic Oracle — Real AI Content Evaluation.
|
||||
|
||||
|
||||
Calculates semantic alignment (Resonance Score) between the bot's
|
||||
configured persona interests and target content using vector embeddings.
|
||||
|
||||
|
||||
This drives ALL downstream decisions:
|
||||
- Like probability (score >= 0.35)
|
||||
- Comment probability (score >= 0.8)
|
||||
@@ -22,6 +26,7 @@ class ResonanceEngine:
|
||||
- Dopamine spike intensity
|
||||
- Darwin dwell time modulation
|
||||
"""
|
||||
|
||||
def __init__(self, my_username: str, persona_interests: list[str] = None, crm: ParasocialCRMDB = None):
|
||||
self.my_username = my_username
|
||||
self.content_memory = ContentMemoryDB()
|
||||
@@ -29,12 +34,11 @@ class ResonanceEngine:
|
||||
self.crm = crm
|
||||
self.threshold = 0.5
|
||||
|
||||
|
||||
# The persona vector is the mathematical identity of what content we care about.
|
||||
# It's generated from config's persona_interests and cached for the entire session.
|
||||
self._persona_vector: Optional[list] = None
|
||||
self._persona_interests = persona_interests or []
|
||||
|
||||
|
||||
# Bootstrap persona on init
|
||||
if self._persona_interests:
|
||||
self._bootstrap_persona()
|
||||
@@ -46,19 +50,47 @@ class ResonanceEngine:
|
||||
"""
|
||||
persona_text = f"Content about: {', '.join(self._persona_interests)}"
|
||||
self._persona_vector = self.content_memory._get_embedding(persona_text)
|
||||
|
||||
|
||||
if self._persona_vector:
|
||||
# Store in PersonaMemoryDB for persistence across sessions
|
||||
self.persona_memory.store_persona_insight(
|
||||
"interests",
|
||||
f"Core niche interests: {', '.join(self._persona_interests)}"
|
||||
"interests", f"Core niche interests: {', '.join(self._persona_interests)}"
|
||||
)
|
||||
logger.info(
|
||||
f"✨ [Resonance Oracle] Persona vector initialized from config: {self._persona_interests}",
|
||||
extra={"color": f"{Fore.MAGENTA}"}
|
||||
extra={"color": f"{Fore.MAGENTA}"},
|
||||
)
|
||||
else:
|
||||
logger.warning("✨ [Resonance Oracle] Could not generate persona embedding. Falling back to neutral scoring.")
|
||||
logger.warning(
|
||||
"✨ [Resonance Oracle] Could not generate persona embedding. Falling back to neutral scoring."
|
||||
)
|
||||
|
||||
def update_identity(self, persona: list, vibe: str):
|
||||
"""Dynamically update the core agent identity and embeddings during a session"""
|
||||
self._persona_interests = persona
|
||||
|
||||
# Build embedding for updated persona
|
||||
combined_text = " ".join(self._persona_interests)
|
||||
new_vector = self.content_memory._get_embedding(combined_text)
|
||||
|
||||
if new_vector:
|
||||
self._persona_vector = new_vector
|
||||
self.persona_memory.store_persona_insight(
|
||||
"interests", f"Dynamically updated interests: {', '.join(self._persona_interests)}"
|
||||
)
|
||||
logger.info(
|
||||
f"✨ [Resonance Oracle] Identity dynamically updated! New Persona: {self._persona_interests} | Vibe: {vibe}",
|
||||
extra={"color": f"{Fore.MAGENTA}"},
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
"✨ [Resonance Oracle] Failed to build embedding for new identity. Retaining previous state."
|
||||
)
|
||||
|
||||
def _classification_to_score(self, classification: str) -> float:
|
||||
"""Maps semantic classification labels to numerical scores."""
|
||||
mapping = {"high": 0.85, "medium": 0.5, "low": 0.2}
|
||||
return mapping.get(classification.lower(), 0.5)
|
||||
|
||||
def _cosine_similarity(self, v1: list, v2: list) -> float:
|
||||
"""Pure python cosine similarity — no numpy dependency."""
|
||||
@@ -76,71 +108,77 @@ class ResonanceEngine:
|
||||
Real AI resonance score based on embedding cosine similarity.
|
||||
"""
|
||||
username = post_content.get("username", "Unknown")
|
||||
description = post_content.get("description", "")
|
||||
|
||||
logger.info(f"✨ [Resonance Oracle] Evaluating content from @{username}...", extra={"color": f"{Fore.MAGENTA}"})
|
||||
|
||||
# Build a rich text representation of the post
|
||||
|
||||
description = post_content.get("description", "")
|
||||
caption = post_content.get("caption", "")
|
||||
username = post_content.get("username", "")
|
||||
|
||||
|
||||
logger.info(f"✨ [Resonance Oracle] Evaluating content from @{username}...", extra={"color": f"{Fore.MAGENTA}"})
|
||||
|
||||
# Build a rich text representation of the post
|
||||
content_text = " ".join(filter(None, [description, caption])).strip()
|
||||
|
||||
|
||||
if not content_text or len(content_text) < 5:
|
||||
logger.debug("✨ [Resonance] Post has no extractable content. Neutral score.")
|
||||
return 0.5 # Neutral — can't evaluate what we can't see
|
||||
|
||||
|
||||
# 0. Ads are now checked upstream structurally via `is_ad(xml)` in bot_flow.
|
||||
# This prevents false positives from users writing 'Werbung' in non-ad contexts.
|
||||
|
||||
# 1. Check ContentMemoryDB cache — have we seen nearly identical content?
|
||||
cached = self.content_memory.get_cached_evaluation(content_text)
|
||||
if cached:
|
||||
score = self._classification_to_score(cached.get("classification", "medium"))
|
||||
# P1-4: Prioritize raw continuous score from cache if available
|
||||
cached_score = cached.get("resonance_score")
|
||||
if cached_score is not None:
|
||||
score = float(cached_score)
|
||||
else:
|
||||
score = self._classification_to_score(cached.get("classification", "medium"))
|
||||
|
||||
logger.info(
|
||||
f"✨ [Resonance Cache Hit] '{content_text[:40]}...' → {score*100:.1f}%",
|
||||
extra={"color": f"{Fore.MAGENTA}"}
|
||||
extra={"color": f"{Fore.MAGENTA}"},
|
||||
)
|
||||
return score
|
||||
|
||||
|
||||
# 2. No persona vector? Can't do real evaluation.
|
||||
if not self._persona_vector:
|
||||
logger.debug("✨ [Resonance] No persona vector. Configure persona_interests in config.yml.")
|
||||
return 0.5
|
||||
|
||||
|
||||
# 3. Generate embedding of the post content
|
||||
post_vector = self.content_memory._get_embedding(content_text)
|
||||
if not post_vector:
|
||||
return 0.5
|
||||
|
||||
|
||||
# 4. Cosine similarity against persona = resonance score
|
||||
raw_score = self._cosine_similarity(post_vector, self._persona_vector)
|
||||
|
||||
|
||||
# Normalize: text-embedding-3-small cosine similarity for text embeddings typically ranges 0.15 (completely distinct) to 0.55 (very matched, but not literal identical copies)
|
||||
# Map this to a more useful 0.0-1.0 range
|
||||
score = max(0.0, min(1.0, (raw_score - 0.15) / 0.30))
|
||||
|
||||
|
||||
# ── Contextual Empathy Filter ──
|
||||
# If the content is tragic or highly controversial, we must NOT like it, regardless of interest alignment.
|
||||
score = self._apply_empathy_filter(content_text, score)
|
||||
|
||||
|
||||
# 5. Store evaluation in ContentMemoryDB for future cache hits
|
||||
classification = "high" if score > 0.7 else "medium" if score > 0.4 else "low"
|
||||
self.content_memory.store_evaluation(
|
||||
content_text[:500], # Cap length for storage
|
||||
classification,
|
||||
f"Resonance: {score:.3f} (raw cosine: {raw_score:.3f})"
|
||||
f"Resonance: {score:.3f} (raw cosine: {raw_score:.3f})",
|
||||
resonance_score=score,
|
||||
)
|
||||
|
||||
|
||||
# 6. Feed the Parasocial CRM
|
||||
if self.crm and username:
|
||||
intent = f"aesthetic_evaluation_{classification}"
|
||||
# Stage mapping: high resonance -> stage 1 (Curiosity)
|
||||
new_stage = 1 if classification == "high" else None
|
||||
self.crm.log_interaction(username, intent, new_stage=new_stage)
|
||||
|
||||
|
||||
logger.info(
|
||||
f"✨ [Resonance Oracle] '{content_text[:50]}...' → {score*100:.1f}% ({classification})",
|
||||
extra={"color": f"{Fore.MAGENTA}"}
|
||||
extra={"color": f"{Fore.MAGENTA}"},
|
||||
)
|
||||
return score
|
||||
|
||||
@@ -151,34 +189,111 @@ class ResonanceEngine:
|
||||
"""
|
||||
tragic_keywords = [
|
||||
# English
|
||||
"rip", "rest in peace", "tragedy", "died", "killed", "accident", "shooting",
|
||||
"funeral", "sad news", "memorial", "cancer", "disease", "breaking news",
|
||||
"rip",
|
||||
"rest in peace",
|
||||
"tragedy",
|
||||
"died",
|
||||
"killed",
|
||||
"accident",
|
||||
"shooting",
|
||||
"funeral",
|
||||
"sad news",
|
||||
"memorial",
|
||||
"cancer",
|
||||
"disease",
|
||||
"breaking news",
|
||||
# German
|
||||
"ruhe in frieden", "verstorben", "tragödie", "unfall", "tot", "beerdigung",
|
||||
"trauer", "krebs", "krankheit"
|
||||
"ruhe in frieden",
|
||||
"verstorben",
|
||||
"tragödie",
|
||||
"unfall",
|
||||
"tot",
|
||||
"beerdigung",
|
||||
"trauer",
|
||||
"krebs",
|
||||
"krankheit",
|
||||
]
|
||||
|
||||
|
||||
text_lower = text.lower()
|
||||
if any(f" {word} " in f" {text_lower} " for word in tragic_keywords):
|
||||
logger.warning("🛡️ [Empathy Filter] Tragic/Sensitive content detected. Suppressing resonance to prevent blind liking.")
|
||||
if any(re.search(rf"\b{re.escape(word)}\b", text_lower) for word in tragic_keywords):
|
||||
logger.warning(
|
||||
"🛡️ [Empathy Filter] Tragic/Sensitive content detected. Suppressing resonance to prevent blind liking."
|
||||
)
|
||||
# Drastically reduce score to "low resonance" zone (avoid liking)
|
||||
return min(current_score, 0.2)
|
||||
|
||||
|
||||
return current_score
|
||||
|
||||
|
||||
def _classification_to_score(self, classification: str) -> float:
|
||||
"""Converts stored classification back to a usable score."""
|
||||
return {"high": 0.85, "medium": 0.55, "low": 0.2}.get(classification, 0.5)
|
||||
|
||||
def judge_interaction(self, score: float) -> bool:
|
||||
"""Determines whether the resonance is high enough to warrant interaction."""
|
||||
if score >= self.threshold:
|
||||
logger.info("✨ [Resonance] POSITIVE ALIGNMENT. Interaction authorized.", extra={"color": f"{Fore.MAGENTA}"})
|
||||
return True
|
||||
else:
|
||||
logger.info("✨ [Resonance] NEGATIVE ALIGNMENT. Skipping profile.", extra={"color": f"{Fore.MAGENTA}"})
|
||||
"""
|
||||
Binary engagement gate.
|
||||
|
||||
Returns True if the resonance score is high enough to warrant any
|
||||
interaction (like, comment, profile visit). The threshold mirrors
|
||||
the like-gate used in the feed loop (bot_flow.py, res_score >= 0.35).
|
||||
|
||||
Args:
|
||||
score: Resonance score in [0.0, 1.0] as returned by calculate_resonance().
|
||||
|
||||
Returns:
|
||||
True → score qualifies for engagement.
|
||||
False → score is too low; skip this post.
|
||||
"""
|
||||
return score >= 0.35
|
||||
|
||||
def get_suggested_action(self, username: str, base_resonance: float) -> str:
|
||||
"""
|
||||
[Phase 2] High-fidelity relationship escalation.
|
||||
Determines the 'best' interaction based on content resonance AND
|
||||
past engagement history (CRM).
|
||||
"""
|
||||
if not self.crm or not username:
|
||||
# Default logic: Like if resonance is good enough
|
||||
if base_resonance >= 0.7:
|
||||
return "LIKE"
|
||||
return "SKIP"
|
||||
|
||||
relationship = self.crm.get_relationship_stage(username)
|
||||
stage = relationship.get("stage", 0)
|
||||
|
||||
# ── Escalation Logic ──
|
||||
# Stage 0: Awareness (Seen/Cold) -> Only Like
|
||||
# Stage 1: Curiosity (Interacted once) -> Like + Comment
|
||||
# Stage 2: Rapport (Multiple interactions) -> Like + Comment + Follow
|
||||
# Stage 3: Conversion (Max relationship) -> High-frequency engagement
|
||||
|
||||
if stage == 0:
|
||||
if base_resonance >= 0.85:
|
||||
return "COMMENT" # Instant hook if amazing
|
||||
if base_resonance >= 0.60:
|
||||
return "LIKE"
|
||||
elif stage == 1:
|
||||
if base_resonance >= 0.70:
|
||||
return "COMMENT"
|
||||
if base_resonance >= 0.40:
|
||||
return "LIKE"
|
||||
elif stage >= 2:
|
||||
if base_resonance >= 0.60:
|
||||
return "COMMENT"
|
||||
if base_resonance >= 0.30:
|
||||
return "LIKE"
|
||||
|
||||
return "SKIP"
|
||||
|
||||
# ── [Phase 3] Engagement Decision Logic ──
|
||||
|
||||
def wants_to_reply(self, base_resonance: float) -> bool:
|
||||
"""Decides if the bot should reply to a comment."""
|
||||
if base_resonance < 0.75:
|
||||
return False
|
||||
# CRM stage 1+ increases reply chance
|
||||
return random.random() < 0.35
|
||||
|
||||
def wants_to_deep_engage(self, base_resonance: float) -> bool:
|
||||
"""Decides if the bot should click through to a commenter profile."""
|
||||
if base_resonance < 0.8:
|
||||
return False
|
||||
return random.random() < 0.25
|
||||
|
||||
def extract_and_learn_comments(self, xml_hierarchy: str, configs, author: str = "unknown", images_b64: list = None):
|
||||
"""
|
||||
@@ -189,74 +304,119 @@ class ResonanceEngine:
|
||||
"""
|
||||
if not configs or not getattr(configs.args, "ai_learn_comments", False):
|
||||
return
|
||||
|
||||
|
||||
vibe = getattr(configs.args, "ai_vibe", "")
|
||||
blacklist = getattr(configs.args, "ai_blacklist_topics", "")
|
||||
if not vibe:
|
||||
return # No vibe to learn
|
||||
|
||||
logger.info(f"🧠 [Comment Learning] Extracting comments matching vibe: '{vibe}'...", extra={"color": f"{Fore.CYAN}"})
|
||||
|
||||
|
||||
logger.info(
|
||||
f"🧠 [Comment Learning] Extracting comments matching vibe: '{vibe}'...", extra={"color": f"{Fore.CYAN}"}
|
||||
)
|
||||
|
||||
# 1. Very basic semantic extraction (grab text nodes that look like comments)
|
||||
raw_comments = []
|
||||
|
||||
|
||||
try:
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
root = ET.fromstring(xml_hierarchy)
|
||||
for node in root.iter('node'):
|
||||
for node in root.iter("node"):
|
||||
# 1. Block System UI (Notifications, WiFi, etc)
|
||||
pkg = node.get("package", "").lower()
|
||||
if pkg != "com.instagram.android":
|
||||
continue
|
||||
|
||||
text = node.get("text", "")
|
||||
content_desc = node.get("content-desc", "")
|
||||
val = text if text else content_desc
|
||||
if val and len(val) > 15:
|
||||
if val.lower() not in ["reply", "like", "view replies", "see translation", "hide replies"]:
|
||||
val = (text if text else content_desc).strip()
|
||||
res_id = node.get("resource-id", "").lower()
|
||||
|
||||
# 2. Heuristics: Only target comment text views
|
||||
is_comment_node = "comment" in res_id or "textview" in res_id
|
||||
|
||||
# 3. Block accessibility garbage & UI labels
|
||||
# Zero-Maintenance: Only structural patterns. Short strings
|
||||
# (< 5 chars) from UI buttons are blocked by length, not by
|
||||
# translating every possible language.
|
||||
is_ui_junk = (
|
||||
val.lower().startswith("go to")
|
||||
or val.lower().startswith("tap to")
|
||||
or "actions for this post" in val.lower()
|
||||
or len(val.strip()) < 3
|
||||
)
|
||||
|
||||
# Block known English UI action labels.
|
||||
# We intentionally do NOT add German/Spanish/etc translations.
|
||||
# Instead, we rely on the structural `is_comment_node` filter
|
||||
# above + length heuristic to catch non-comment UI elements.
|
||||
blocked_exact = [
|
||||
"reply",
|
||||
"like",
|
||||
"view replies",
|
||||
"see translation",
|
||||
"hide replies",
|
||||
"view all comments",
|
||||
"send",
|
||||
]
|
||||
|
||||
if val and len(val) > 2 and is_comment_node and not is_ui_junk:
|
||||
if val.lower() not in blocked_exact:
|
||||
raw_comments.append(val)
|
||||
except Exception as e:
|
||||
logger.error(f"🧠 [Comment Learning] Failed to parse XML: {e}")
|
||||
return
|
||||
|
||||
|
||||
if not raw_comments:
|
||||
logger.debug("🧠 [Comment Learning] No legible comments found in UI.")
|
||||
return
|
||||
|
||||
|
||||
# Deduplicate and limit
|
||||
raw_comments = list(set(raw_comments))[:10]
|
||||
logger.debug(f"🧠 [Comment Learning] Scraped {len(raw_comments)} potential comment nodes. Passing to Condenser...")
|
||||
|
||||
logger.debug(
|
||||
f"🧠 [Comment Learning] Scraped {len(raw_comments)} potential comment nodes. Passing to Condenser..."
|
||||
)
|
||||
|
||||
logger.debug(f"🧠 [Comment Learning] Raw texts passed to Condenser:\n{chr(10).join(raw_comments)}")
|
||||
|
||||
|
||||
# 2. Filter via VLM Condenser
|
||||
prompt = (
|
||||
f"Evaluate Instagram comments for SPAM. Your only goal is blocking bad topics.\n"
|
||||
f"Evaluate these Instagram comments. Your goal is to identify comments that generally match this vibe while blocking SPAM, UI junk, and harmful topics.\n"
|
||||
f"VIBE = '{vibe}'\n"
|
||||
f"BLACKLIST = {blacklist}\n\n"
|
||||
f"Comments:\n{chr(10).join(['- ' + c for c in raw_comments])}\n\n"
|
||||
"Return a JSON formatting exactly like this example:\n"
|
||||
"{\n"
|
||||
" \"evaluations\": [\n"
|
||||
" {\"text\": \"love it!\", \"has_blacklist_words\": false, \"keep\": true},\n"
|
||||
" {\"text\": \"dm me for bitcoin\", \"has_blacklist_words\": true, \"keep\": false}\n"
|
||||
" ]\n"
|
||||
"}"
|
||||
f"Comments to evaluate:\n{chr(10).join(['- ' + c for c in raw_comments])}\n\n"
|
||||
"Return a JSON object with 'evaluations' array. Each item must have 'text', 'has_blacklist_words' (bool), and 'keep' (bool).\n"
|
||||
"Set 'keep' to true if the comment feels authentic and matches the vibe.\n"
|
||||
"Set 'keep' to false only for clear spam, bots, UI buttons, or blacklist violations.\n"
|
||||
)
|
||||
|
||||
|
||||
model = getattr(configs.args, "ai_condenser_model", "llama3.2:1b")
|
||||
url = getattr(configs.args, "ai_condenser_url", "http://localhost:11434/api/generate")
|
||||
|
||||
|
||||
try:
|
||||
import json
|
||||
|
||||
system = "You are a precise JSON filtering agent."
|
||||
# Fix: kwargs match query_llm signature EXACTLY to evade TypeError
|
||||
response_dict = query_llm(url=url, model=model, prompt=prompt, system=system, format_json=True, images_b64=images_b64)
|
||||
response_dict = query_llm(
|
||||
url=url,
|
||||
model=model,
|
||||
prompt=prompt,
|
||||
system=system,
|
||||
format_json=True,
|
||||
images_b64=images_b64,
|
||||
max_tokens=600,
|
||||
temperature=0.1,
|
||||
)
|
||||
if not response_dict or "response" not in response_dict:
|
||||
return
|
||||
|
||||
|
||||
response_text = response_dict["response"]
|
||||
# DEBUG
|
||||
logger.debug(f"DEBUG CONDENSER RAW: {response_text}")
|
||||
print(f"DEBUG CONDENSER RAW: {response_text}")
|
||||
|
||||
|
||||
# Parse json gracefully
|
||||
if type(response_text) is str:
|
||||
if isinstance(response_text, str):
|
||||
clean_json = response_text.strip()
|
||||
if clean_json.startswith("```json"):
|
||||
clean_json = clean_json[7:]
|
||||
@@ -270,7 +430,7 @@ class ResonanceEngine:
|
||||
else:
|
||||
# In case expect_json already returned a parsed list somehow, though extract_json returns str
|
||||
learned_comments = response_text
|
||||
|
||||
|
||||
# Filter the dict based on evaluations array
|
||||
if isinstance(learned_comments, dict):
|
||||
valid_list = []
|
||||
@@ -278,20 +438,29 @@ class ResonanceEngine:
|
||||
for ev in evals:
|
||||
# Qwen 3.5 correctly identifies 'has_blacklist_words' but hallucinates 'keep': true
|
||||
has_spam = ev.get("has_blacklist_words", False)
|
||||
if not has_spam:
|
||||
keep = ev.get("keep", True)
|
||||
if not has_spam and keep:
|
||||
valid_list.append(ev.get("text"))
|
||||
learned_comments = valid_list
|
||||
|
||||
|
||||
if not isinstance(learned_comments, list):
|
||||
logger.error(f"🧠 [Comment Learning] Condenser failed to return a valid JSON structure: {learned_comments}")
|
||||
logger.error(
|
||||
f"🧠 [Comment Learning] Condenser failed to return a valid JSON structure: {learned_comments}"
|
||||
)
|
||||
return
|
||||
|
||||
|
||||
if not learned_comments:
|
||||
logger.info("🧠 [Comment Learning] Condenser rejected all scraped comments (did not align with vibe or hit blacklist).", extra={"color": f"{Fore.YELLOW}"})
|
||||
logger.info(
|
||||
"🧠 [Comment Learning] Condenser rejected all scraped comments (did not align with vibe or hit blacklist).",
|
||||
extra={"color": f"{Fore.YELLOW}"},
|
||||
)
|
||||
return
|
||||
|
||||
logger.info(f"🧠 [Comment Learning] Condenser approved {len(learned_comments)} comments. Persisting to Qdrant...", extra={"color": f"{Fore.GREEN}"})
|
||||
|
||||
|
||||
logger.info(
|
||||
f"🧠 [Comment Learning] Condenser approved {len(learned_comments)} comments. Persisting to Qdrant...",
|
||||
extra={"color": f"{Fore.GREEN}"},
|
||||
)
|
||||
|
||||
# 3. Store the passing comments into Qdrant
|
||||
comment_db = CommentMemoryDB()
|
||||
stored = 0
|
||||
@@ -300,9 +469,12 @@ class ResonanceEngine:
|
||||
logger.debug(f" 👉 Storing: '{c}'")
|
||||
comment_db.store_comment(text=c, vibe=vibe, author=author)
|
||||
stored += 1
|
||||
|
||||
|
||||
if stored > 0:
|
||||
logger.info(f"✅ [Comment Vector Sync] Successfully embedded {stored} high-vibe comments into memory.", extra={"color": f"{Fore.GREEN}"})
|
||||
|
||||
logger.info(
|
||||
f"✅ [Comment Vector Sync] Successfully embedded {stored} high-vibe comments into memory.",
|
||||
extra={"color": f"{Fore.GREEN}"},
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"🧠 [Comment Learning] Condenser failed: {e}")
|
||||
|
||||
249
GramAddict/core/screen_topology.py
Normal file
249
GramAddict/core/screen_topology.py
Normal file
@@ -0,0 +1,249 @@
|
||||
"""
|
||||
ScreenTopology — The Instagram HD Map
|
||||
|
||||
Pure-data BFS pathfinding between Instagram screen states.
|
||||
Zero dependencies on device, VLM, Qdrant, or any runtime state.
|
||||
|
||||
This is the bot's GPS: it knows HOW to get from screen A to screen B
|
||||
before the bot starts moving. The GOAP planner consults this map
|
||||
as its primary routing strategy.
|
||||
"""
|
||||
|
||||
from collections import deque
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
from GramAddict.core.goap import ScreenType
|
||||
|
||||
|
||||
class ScreenTopology:
|
||||
"""
|
||||
Topological HD Map of Instagram's screen graph.
|
||||
|
||||
Provides BFS pathfinding between any two ScreenTypes.
|
||||
All transitions use the same action string format as
|
||||
the TelepathicEngine intent system — no translation needed.
|
||||
"""
|
||||
|
||||
# ── The Map: ScreenType → {action_string → ScreenType} ──
|
||||
# These are structural facts about Instagram's UI, not learned behavior.
|
||||
# They survive blank_start because they describe the app's architecture.
|
||||
TRANSITIONS: Dict[ScreenType, Dict[str, ScreenType]] = {
|
||||
ScreenType.HOME_FEED: {
|
||||
"tap explore tab": ScreenType.EXPLORE_GRID,
|
||||
"tap profile tab": ScreenType.OWN_PROFILE,
|
||||
"tap reels tab": ScreenType.REELS_FEED,
|
||||
"tap messages tab": ScreenType.DM_INBOX,
|
||||
"tap activity heart icon notifications": ScreenType.NOTIFICATIONS,
|
||||
"tap story ring avatar": ScreenType.STORY_VIEW,
|
||||
},
|
||||
ScreenType.EXPLORE_GRID: {
|
||||
"tap home tab": ScreenType.HOME_FEED,
|
||||
"tap profile tab": ScreenType.OWN_PROFILE,
|
||||
"tap reels tab": ScreenType.REELS_FEED,
|
||||
"view a post": ScreenType.POST_DETAIL,
|
||||
},
|
||||
ScreenType.REELS_FEED: {
|
||||
"tap home tab": ScreenType.HOME_FEED,
|
||||
"tap explore tab": ScreenType.EXPLORE_GRID,
|
||||
"tap profile tab": ScreenType.OWN_PROFILE,
|
||||
},
|
||||
ScreenType.OWN_PROFILE: {
|
||||
"tap home tab": ScreenType.HOME_FEED,
|
||||
"tap explore tab": ScreenType.EXPLORE_GRID,
|
||||
"tap reels tab": ScreenType.REELS_FEED,
|
||||
"tap following list": ScreenType.FOLLOW_LIST,
|
||||
},
|
||||
ScreenType.DM_INBOX: {
|
||||
"press back": ScreenType.HOME_FEED,
|
||||
},
|
||||
ScreenType.FOLLOW_LIST: {
|
||||
"press back": ScreenType.OWN_PROFILE,
|
||||
},
|
||||
ScreenType.STORY_VIEW: {
|
||||
"press back": ScreenType.HOME_FEED,
|
||||
},
|
||||
ScreenType.OTHER_PROFILE: {
|
||||
"tap home tab": ScreenType.HOME_FEED,
|
||||
"tap explore tab": ScreenType.EXPLORE_GRID,
|
||||
"tap reels tab": ScreenType.REELS_FEED,
|
||||
"tap profile tab": ScreenType.OWN_PROFILE,
|
||||
# NOTE: 'press back' intentionally omitted — destination is non-deterministic
|
||||
# (could be HOME_FEED, EXPLORE_GRID, POST_DETAIL, etc. depending on navigation history)
|
||||
},
|
||||
ScreenType.POST_DETAIL: {
|
||||
"tap view all comments": ScreenType.COMMENTS,
|
||||
"tap home tab": ScreenType.HOME_FEED,
|
||||
"tap explore tab": ScreenType.EXPLORE_GRID,
|
||||
"tap profile tab": ScreenType.OWN_PROFILE,
|
||||
# NOTE: 'press back' intentionally omitted — destination is non-deterministic
|
||||
# (could be HOME_FEED, EXPLORE_GRID, OTHER_PROFILE, etc.)
|
||||
},
|
||||
ScreenType.COMMENTS: {
|
||||
"press back": ScreenType.POST_DETAIL,
|
||||
},
|
||||
ScreenType.SEARCH_RESULTS: {
|
||||
"tap home tab": ScreenType.HOME_FEED,
|
||||
# NOTE: 'press back' intentionally omitted — destination is non-deterministic
|
||||
},
|
||||
ScreenType.NOTIFICATIONS: {
|
||||
"tap home tab": ScreenType.HOME_FEED,
|
||||
"press back": ScreenType.HOME_FEED,
|
||||
"tap profile tab": ScreenType.OWN_PROFILE,
|
||||
"tap explore tab": ScreenType.EXPLORE_GRID,
|
||||
},
|
||||
ScreenType.UNKNOWN: {
|
||||
"tap home tab": ScreenType.HOME_FEED,
|
||||
},
|
||||
}
|
||||
|
||||
# ── Goal → ScreenType mapping ──
|
||||
# Maps natural-language goals to their target screen.
|
||||
_GOAL_MAP: Dict[str, ScreenType] = {
|
||||
"open home feed": ScreenType.HOME_FEED,
|
||||
"open home": ScreenType.HOME_FEED,
|
||||
"open explore feed": ScreenType.EXPLORE_GRID,
|
||||
"open explore": ScreenType.EXPLORE_GRID,
|
||||
"open reels": ScreenType.REELS_FEED,
|
||||
"open profile": ScreenType.OWN_PROFILE,
|
||||
"learn own profile": ScreenType.OWN_PROFILE,
|
||||
"open messages": ScreenType.DM_INBOX,
|
||||
"open following list": ScreenType.FOLLOW_LIST,
|
||||
"open followers list": ScreenType.FOLLOW_LIST,
|
||||
"view a post": ScreenType.POST_DETAIL,
|
||||
"open post": ScreenType.POST_DETAIL,
|
||||
"open post author profile": ScreenType.OTHER_PROFILE,
|
||||
"view the user profile": ScreenType.OTHER_PROFILE,
|
||||
"view user profile": ScreenType.OTHER_PROFILE,
|
||||
"open user profile": ScreenType.OTHER_PROFILE,
|
||||
"open search": ScreenType.SEARCH_RESULTS,
|
||||
"view comments": ScreenType.COMMENTS,
|
||||
"open notifications": ScreenType.NOTIFICATIONS,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def find_route(
|
||||
cls, from_screen: ScreenType, to_screen: ScreenType, avoid_actions: set = None
|
||||
) -> Optional[List[Tuple[str, ScreenType]]]:
|
||||
"""
|
||||
BFS shortest path from from_screen to to_screen.
|
||||
|
||||
Returns:
|
||||
[] if already there,
|
||||
[(action, resulting_screen), ...] for a path,
|
||||
None if unreachable.
|
||||
"""
|
||||
if from_screen == to_screen:
|
||||
return []
|
||||
|
||||
avoid_actions = avoid_actions or set()
|
||||
|
||||
queue: deque = deque()
|
||||
queue.append((from_screen, []))
|
||||
visited = {from_screen}
|
||||
|
||||
while queue:
|
||||
current, path = queue.popleft()
|
||||
transitions = cls.TRANSITIONS.get(current, {})
|
||||
|
||||
for action, next_screen in transitions.items():
|
||||
if action in avoid_actions or action.replace(" ", "_") in avoid_actions:
|
||||
continue
|
||||
|
||||
if next_screen == to_screen:
|
||||
return path + [(action, next_screen)]
|
||||
|
||||
if next_screen not in visited:
|
||||
visited.add(next_screen)
|
||||
queue.append((next_screen, path + [(action, next_screen)]))
|
||||
|
||||
return None # Unreachable
|
||||
|
||||
@classmethod
|
||||
def get_transitions(cls, screen: ScreenType) -> Dict[str, ScreenType]:
|
||||
"""Get all known transitions from a screen."""
|
||||
return dict(cls.TRANSITIONS.get(screen, {}))
|
||||
|
||||
@classmethod
|
||||
def goal_to_target_screen(cls, goal: str) -> Optional[ScreenType]:
|
||||
"""Map a goal string to its target ScreenType. Returns None for non-navigation goals."""
|
||||
goal_lower = goal.lower().strip()
|
||||
|
||||
# Exact match first
|
||||
if goal_lower in cls._GOAL_MAP:
|
||||
return cls._GOAL_MAP[goal_lower]
|
||||
|
||||
# Substring match for flexibility
|
||||
for key, screen in cls._GOAL_MAP.items():
|
||||
if key in goal_lower:
|
||||
return screen
|
||||
|
||||
return None
|
||||
|
||||
# ── QNavGraph screen name ↔ ScreenType mapping (SSOT) ──
|
||||
SCREEN_NAME_MAP: Dict[str, ScreenType] = {
|
||||
"HomeFeed": ScreenType.HOME_FEED,
|
||||
"ExploreFeed": ScreenType.EXPLORE_GRID,
|
||||
"ReelsFeed": ScreenType.REELS_FEED,
|
||||
"OwnProfile": ScreenType.OWN_PROFILE,
|
||||
"MessageInbox": ScreenType.DM_INBOX,
|
||||
"FollowingList": ScreenType.FOLLOW_LIST,
|
||||
"OtherProfile": ScreenType.OTHER_PROFILE,
|
||||
"StoriesFeed": ScreenType.HOME_FEED, # Stories are on home feed
|
||||
"SearchFeed": ScreenType.EXPLORE_GRID, # Search uses explore
|
||||
"UNKNOWN": ScreenType.UNKNOWN,
|
||||
}
|
||||
|
||||
# ── Reverse map: ScreenType → canonical goal string ──
|
||||
_SCREEN_TO_GOAL: Dict[ScreenType, str] = {
|
||||
ScreenType.HOME_FEED: "open home feed",
|
||||
ScreenType.EXPLORE_GRID: "open explore feed",
|
||||
ScreenType.REELS_FEED: "open reels",
|
||||
ScreenType.OWN_PROFILE: "open profile",
|
||||
ScreenType.DM_INBOX: "open messages",
|
||||
ScreenType.FOLLOW_LIST: "open following list",
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def screen_name_to_goal(cls, screen_name: str) -> str:
|
||||
"""Convert QNavGraph screen name to GOAP goal string.
|
||||
|
||||
Returns a canonical goal string for known screens,
|
||||
or 'navigate to <name>' for unknown ones.
|
||||
"""
|
||||
screen_type = cls.SCREEN_NAME_MAP.get(screen_name)
|
||||
if screen_type and screen_type in cls._SCREEN_TO_GOAL:
|
||||
return cls._SCREEN_TO_GOAL[screen_type]
|
||||
return f"navigate to {screen_name}"
|
||||
|
||||
@classmethod
|
||||
def expected_screen_for_action(cls, action: str, from_screen: ScreenType) -> Optional[ScreenType]:
|
||||
"""What screen should we land on after this action from this screen?
|
||||
|
||||
Used by _execute_action to validate INTERMEDIATE navigation steps.
|
||||
Returns None if the action isn't a known transition from this screen.
|
||||
"""
|
||||
# Hardcode self-edges for main tabs (which are no-ops)
|
||||
if action == "tap home tab" and from_screen == ScreenType.HOME_FEED:
|
||||
return ScreenType.HOME_FEED
|
||||
if action == "tap explore tab" and from_screen == ScreenType.EXPLORE_GRID:
|
||||
return ScreenType.EXPLORE_GRID
|
||||
if action == "tap reels tab" and from_screen == ScreenType.REELS_FEED:
|
||||
return ScreenType.REELS_FEED
|
||||
if action == "tap profile tab" and from_screen == ScreenType.OWN_PROFILE:
|
||||
return ScreenType.OWN_PROFILE
|
||||
if action == "tap messages tab" and from_screen == ScreenType.DM_INBOX:
|
||||
return ScreenType.DM_INBOX
|
||||
|
||||
transitions = cls.TRANSITIONS.get(from_screen, {})
|
||||
return transitions.get(action)
|
||||
|
||||
@classmethod
|
||||
def is_structural_action(cls, screen: ScreenType, action: str) -> bool:
|
||||
"""Check if an action is a structural transition in the HD Map.
|
||||
|
||||
Structural actions must NEVER be aversively learned as traps —
|
||||
they are architectural facts about Instagram's UI.
|
||||
VLM may fail to find the element, but the route itself is valid.
|
||||
"""
|
||||
transitions = cls.TRANSITIONS.get(screen, {})
|
||||
return action in transitions
|
||||
@@ -4,18 +4,19 @@ import xml.etree.ElementTree as ET
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class HoneypotRadome:
|
||||
"""
|
||||
Project Dojo: The Anti-Test Sensor.
|
||||
Filters the Android XML Hierarchy to remove "invisible traps" and honeypots
|
||||
that Instagram uses to detect deterministic bots (e.g., 1x1 pixel buttons,
|
||||
Filters the Android XML Hierarchy to remove "invisible traps" and honeypots
|
||||
that Instagram uses to detect deterministic bots (e.g., 1x1 pixel buttons,
|
||||
off-screen elements with clickable=True).
|
||||
"""
|
||||
|
||||
|
||||
def __init__(self, display_width=1080, display_height=2400):
|
||||
self.display_width = display_width
|
||||
self.display_height = display_height
|
||||
self.bounds_pattern = re.compile(r'\[(\d+),(\d+)\]\[(\d+),(\d+)\]')
|
||||
self.bounds_pattern = re.compile(r"\[(\d+),(\d+)\]\[(\d+),(\d+)\]")
|
||||
|
||||
def sanitize_xml(self, xml_string: str) -> str:
|
||||
"""
|
||||
@@ -23,19 +24,22 @@ class HoneypotRadome:
|
||||
Returns the sanitized XML string.
|
||||
"""
|
||||
try:
|
||||
# Android XML dumps often have multiple root nodes or formatting issues,
|
||||
# Android XML dumps often have multiple root nodes or formatting issues,
|
||||
# let's try reading it safely.
|
||||
# Handle potential encoding issues from dump_hierarchy
|
||||
clean_xml = xml_string.replace(' ', '').replace(' ', '')
|
||||
|
||||
clean_xml = xml_string.replace(" ", "").replace(" ", "")
|
||||
|
||||
root = ET.fromstring(clean_xml)
|
||||
removed_count = self._filter_node(root)
|
||||
|
||||
|
||||
if removed_count > 0:
|
||||
logger.info(f"🛡️ [Honeypot Radome] Stripped {removed_count} phantom nodes from view.", extra={"color": "\x1b[33m"})
|
||||
|
||||
logger.info(
|
||||
f"🛡️ [Honeypot Radome] Stripped {removed_count} phantom nodes from view.",
|
||||
extra={"color": "\x1b[33m"},
|
||||
)
|
||||
|
||||
# Convert back to string
|
||||
return ET.tostring(root, encoding='unicode')
|
||||
return ET.tostring(root, encoding="unicode")
|
||||
except Exception as e:
|
||||
logger.warning(f"🛡️ [Honeypot Radome] XML Parse failed, returning raw. Err: {e}")
|
||||
return xml_string
|
||||
@@ -43,17 +47,17 @@ class HoneypotRadome:
|
||||
def _filter_node(self, node: ET.Element) -> int:
|
||||
removed = 0
|
||||
children_to_remove = []
|
||||
|
||||
|
||||
for child in node:
|
||||
if self._is_honeypot(child):
|
||||
children_to_remove.append(child)
|
||||
removed += 1
|
||||
else:
|
||||
removed += self._filter_node(child)
|
||||
|
||||
|
||||
for child in children_to_remove:
|
||||
node.remove(child)
|
||||
|
||||
|
||||
return removed
|
||||
|
||||
def _is_honeypot(self, node: ET.Element) -> bool:
|
||||
@@ -63,31 +67,45 @@ class HoneypotRadome:
|
||||
bounds = node.get("bounds")
|
||||
if not bounds:
|
||||
return False
|
||||
|
||||
|
||||
match = self.bounds_pattern.match(bounds)
|
||||
if not match:
|
||||
return False
|
||||
|
||||
|
||||
x1, y1, x2, y2 = map(int, match.groups())
|
||||
width = x2 - x1
|
||||
height = y2 - y1
|
||||
|
||||
|
||||
is_clickable = node.get("clickable", "false").lower() == "true"
|
||||
|
||||
|
||||
# Rule 1: The Zero-Point Trap (Element is exactly on 0,0 with no dimensions)
|
||||
if x1 == 0 and y1 == 0 and x2 == 0 and y2 == 0:
|
||||
return True
|
||||
|
||||
|
||||
# Rule 2: The Micro-Pixel Trap (Bot detectors often use 1x1 or 2x2 clickable overlay pixels)
|
||||
if is_clickable and width <= 2 and height <= 2:
|
||||
return True
|
||||
|
||||
|
||||
# Rule 3: The Off-Screen Trap (Buttons rendered wildly out of bounds to bait mindless loops)
|
||||
if x1 >= self.display_width or y1 >= self.display_height:
|
||||
return True
|
||||
|
||||
|
||||
# Rule 4: The Negative Coordinate Trap
|
||||
if x2 <= 0 or y2 <= 0:
|
||||
return True
|
||||
|
||||
|
||||
# Rule 5: The Transparent Interceptor (Giant invisible overlays capturing touches)
|
||||
# If a clickable element takes up >90% of screen but has no text, description, or id, it's a touch trap.
|
||||
has_text = bool(node.get("text", ""))
|
||||
has_desc = bool(node.get("content-desc", ""))
|
||||
has_id = bool(node.get("resource-id", ""))
|
||||
if is_clickable and width >= (self.display_width * 0.9) and height >= (self.display_height * 0.9):
|
||||
if not has_text and not has_desc and not has_id:
|
||||
return True
|
||||
|
||||
# Rule 6: Android Accessibility Trap (A node is clickable but explicitly not visible)
|
||||
# Sometimes uiautomator injects 'visible-to-user' manually, or it has bounds but isn't enabled.
|
||||
if is_clickable and node.get("visible-to-user", "true").lower() == "false":
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
@@ -80,64 +80,40 @@ class SessionState:
|
||||
self,
|
||||
):
|
||||
"""set the limits for current session"""
|
||||
self.args.current_likes_limit = get_value(
|
||||
getattr(self.args, "total_likes_limit", 300), None, 300
|
||||
)
|
||||
self.args.current_follow_limit = get_value(
|
||||
getattr(self.args, "total_follows_limit", 50), None, 50
|
||||
)
|
||||
self.args.current_unfollow_limit = get_value(
|
||||
getattr(self.args, "total_unfollows_limit", 50), None, 50
|
||||
)
|
||||
self.args.current_comments_limit = get_value(
|
||||
getattr(self.args, "total_comments_limit", 10), None, 10
|
||||
)
|
||||
self.args.current_likes_limit = get_value(getattr(self.args, "total_likes_limit", 300), None, 300)
|
||||
self.args.current_follow_limit = get_value(getattr(self.args, "total_follows_limit", 50), None, 50)
|
||||
self.args.current_unfollow_limit = get_value(getattr(self.args, "total_unfollows_limit", 50), None, 50)
|
||||
self.args.current_comments_limit = get_value(getattr(self.args, "total_comments_limit", 10), None, 10)
|
||||
self.args.current_pm_limit = get_value(getattr(self.args, "total_pm_limit", 10), None, 10)
|
||||
self.args.current_watch_limit = get_value(
|
||||
getattr(self.args, "total_watches_limit", 50), None, 50
|
||||
)
|
||||
self.args.current_watch_limit = get_value(getattr(self.args, "total_watches_limit", 50), None, 50)
|
||||
self.args.current_success_limit = get_value(
|
||||
getattr(self.args, "total_successful_interactions_limit", 100), None, 100
|
||||
)
|
||||
self.args.current_total_limit = get_value(
|
||||
getattr(self.args, "total_interactions_limit", 1000), None, 1000
|
||||
)
|
||||
self.args.current_scraped_limit = get_value(
|
||||
getattr(self.args, "total_scraped_limit", 200), None, 200
|
||||
)
|
||||
self.args.current_crashes_limit = get_value(
|
||||
getattr(self.args, "total_crashes_limit", 5), None, 5
|
||||
)
|
||||
self.args.current_total_limit = get_value(getattr(self.args, "total_interactions_limit", 1000), None, 1000)
|
||||
self.args.current_scraped_limit = get_value(getattr(self.args, "total_scraped_limit", 200), None, 200)
|
||||
self.args.current_crashes_limit = get_value(getattr(self.args, "total_crashes_limit", 5), None, 5)
|
||||
|
||||
def check_limit(self, limit_type=None, output=False):
|
||||
"""Returns True if limit reached - else False"""
|
||||
limit_type = SessionState.Limit.ALL if limit_type is None else limit_type
|
||||
# check limits
|
||||
total_likes = self.totalLikes >= int(self.args.current_likes_limit)
|
||||
total_followed = sum(self.totalFollowed.values()) >= int(
|
||||
self.args.current_follow_limit
|
||||
)
|
||||
total_followed = sum(self.totalFollowed.values()) >= int(self.args.current_follow_limit)
|
||||
total_unfollowed = self.totalUnfollowed >= int(self.args.current_unfollow_limit)
|
||||
total_comments = self.totalComments >= int(self.args.current_comments_limit)
|
||||
total_pm = self.totalPm >= int(self.args.current_pm_limit)
|
||||
total_watched = self.totalWatched >= int(self.args.current_watch_limit)
|
||||
total_successful = sum(self.successfulInteractions.values()) >= int(
|
||||
self.args.current_success_limit
|
||||
)
|
||||
total_interactions = sum(self.totalInteractions.values()) >= int(
|
||||
self.args.current_total_limit
|
||||
)
|
||||
total_successful = sum(self.successfulInteractions.values()) >= int(self.args.current_success_limit)
|
||||
total_interactions = sum(self.totalInteractions.values()) >= int(self.args.current_total_limit)
|
||||
|
||||
total_scraped = sum(self.totalScraped.values()) >= int(
|
||||
self.args.current_scraped_limit
|
||||
)
|
||||
total_scraped = sum(self.totalScraped.values()) >= int(self.args.current_scraped_limit)
|
||||
|
||||
total_crashes = self.totalCrashes >= int(self.args.current_crashes_limit)
|
||||
|
||||
session_info = [
|
||||
"Checking session limits:",
|
||||
f"- Total Likes:\t\t\t\t{'Limit Reached' if total_likes else 'OK'} ({self.totalLikes}/{self.args.current_likes_limit})",
|
||||
f"- Total Comments:\t\t\t\t{'Limit Reached' if total_comments else 'OK'} ({self.totalComments}/{self.args.current_comments_limit})",
|
||||
f"- Session Likes Given:\t\t{'Limit Reached' if total_likes else 'OK'} ({self.totalLikes}/{self.args.current_likes_limit})",
|
||||
f"- Session Comments Given:\t{'Limit Reached' if total_comments else 'OK'} ({self.totalComments}/{self.args.current_comments_limit})",
|
||||
f"- Total PM:\t\t\t\t\t{'Limit Reached' if total_pm else 'OK'} ({self.totalPm}/{self.args.current_pm_limit})",
|
||||
f"- Total Followed:\t\t\t\t{'Limit Reached' if total_followed else 'OK'} ({sum(self.totalFollowed.values())}/{self.args.current_follow_limit})",
|
||||
f"- Total Unfollowed:\t\t\t\t{'Limit Reached' if total_unfollowed else 'OK'} ({self.totalUnfollowed}/{self.args.current_unfollow_limit})",
|
||||
@@ -154,11 +130,16 @@ class SessionState:
|
||||
logger.info(line)
|
||||
|
||||
return (
|
||||
total_likes and getattr(self.args, "end_if_likes_limit_reached", False)
|
||||
or total_followed and getattr(self.args, "end_if_follows_limit_reached", False)
|
||||
or total_watched and getattr(self.args, "end_if_watches_limit_reached", False)
|
||||
or total_comments and getattr(self.args, "end_if_comments_limit_reached", False)
|
||||
or total_pm and getattr(self.args, "end_if_pm_limit_reached", False),
|
||||
total_likes
|
||||
and getattr(self.args, "end_if_likes_limit_reached", False)
|
||||
or total_followed
|
||||
and getattr(self.args, "end_if_follows_limit_reached", False)
|
||||
or total_watched
|
||||
and getattr(self.args, "end_if_watches_limit_reached", False)
|
||||
or total_comments
|
||||
and getattr(self.args, "end_if_comments_limit_reached", False)
|
||||
or total_pm
|
||||
and getattr(self.args, "end_if_pm_limit_reached", False),
|
||||
total_unfollowed,
|
||||
total_interactions or total_successful or total_scraped,
|
||||
)
|
||||
@@ -247,20 +228,20 @@ class SessionState:
|
||||
delta = timedelta(seconds=delta_sec)
|
||||
if not working_hours:
|
||||
return True, 0
|
||||
|
||||
|
||||
for n in working_hours:
|
||||
today = current_time.strftime("%Y-%m-%d")
|
||||
# 100% Autonomous: Hybrid Time Format Support (Legacy . vs Modern :)
|
||||
h_start = n.split('-')[0].replace(":", ".")
|
||||
h_end = n.split('-')[1].replace(":", ".")
|
||||
|
||||
h_start = n.split("-")[0].replace(":", ".")
|
||||
h_end = n.split("-")[1].replace(":", ".")
|
||||
|
||||
inf_value = f"{h_start} {today}"
|
||||
inf = datetime.strptime(inf_value, "%H.%M %Y-%m-%d") + delta
|
||||
sup_value = f"{h_end} {today}"
|
||||
sup = datetime.strptime(sup_value, "%H.%M %Y-%m-%d") + delta
|
||||
if sup - inf + timedelta(minutes=1) == timedelta(
|
||||
days=1
|
||||
) or sup - inf + timedelta(minutes=1) == timedelta(days=0):
|
||||
if sup - inf + timedelta(minutes=1) == timedelta(days=1) or sup - inf + timedelta(minutes=1) == timedelta(
|
||||
days=0
|
||||
):
|
||||
logger.debug("Whole day mode.")
|
||||
return True, 0
|
||||
if time_in_range(inf.time(), sup.time(), current_time.time()):
|
||||
@@ -296,13 +277,33 @@ class SessionState:
|
||||
|
||||
|
||||
class SessionStateEncoder(JSONEncoder):
|
||||
"""JSON encoder for SessionState that is crash-proof against non-serializable types."""
|
||||
|
||||
_SAFE_TYPES = (str, int, float, bool, type(None))
|
||||
|
||||
@classmethod
|
||||
def _sanitize_value(cls, value):
|
||||
"""Convert any non-JSON-serializable value to a safe string representation."""
|
||||
if isinstance(value, cls._SAFE_TYPES):
|
||||
return value
|
||||
if isinstance(value, datetime):
|
||||
return value.isoformat()
|
||||
if isinstance(value, dict):
|
||||
return {k: cls._sanitize_value(v) for k, v in value.items()}
|
||||
if isinstance(value, (list, tuple)):
|
||||
return [cls._sanitize_value(v) for v in value]
|
||||
# Last resort: stringify unknown objects to prevent json.dump mid-write crashes
|
||||
return str(value)
|
||||
|
||||
def default(self, session_state: SessionState):
|
||||
# Sanitize args dict — never trust raw __dict__, it may contain datetime or other garbage
|
||||
raw_args = session_state.args.__dict__ if hasattr(session_state.args, "__dict__") else {}
|
||||
safe_args = {k: self._sanitize_value(v) for k, v in raw_args.items()}
|
||||
|
||||
return {
|
||||
"id": session_state.id,
|
||||
"total_interactions": sum(session_state.totalInteractions.values()),
|
||||
"successful_interactions": sum(
|
||||
session_state.successfulInteractions.values()
|
||||
),
|
||||
"successful_interactions": sum(session_state.successfulInteractions.values()),
|
||||
"total_followed": sum(session_state.totalFollowed.values()),
|
||||
"total_likes": session_state.totalLikes,
|
||||
"total_comments": session_state.totalComments,
|
||||
@@ -312,7 +313,7 @@ class SessionStateEncoder(JSONEncoder):
|
||||
"total_scraped": session_state.totalScraped,
|
||||
"start_time": str(session_state.startTime),
|
||||
"finish_time": str(session_state.finishTime),
|
||||
"args": session_state.args.__dict__,
|
||||
"args": safe_args,
|
||||
"profile": {
|
||||
"posts": session_state.my_posts_count,
|
||||
"followers": session_state.my_followers_count,
|
||||
|
||||
1060
GramAddict/core/situational_awareness.py
Normal file
1060
GramAddict/core/situational_awareness.py
Normal file
File diff suppressed because it is too large
Load Diff
@@ -4,7 +4,8 @@ from time import sleep
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
def ghost_type(device, text: str):
|
||||
|
||||
def ghost_type(device, text: str, speed: str = "normal"):
|
||||
"""
|
||||
Tesla Stealth Ghost Keyboard.
|
||||
Bypasses UIAutomator virtual IME completely and sends raw Native InputEvents.
|
||||
@@ -12,61 +13,66 @@ def ghost_type(device, text: str):
|
||||
"""
|
||||
if not text:
|
||||
return
|
||||
|
||||
|
||||
logger.info(f"⌨️ [Ghost Keyboard] Initiating stealth injection ({len(text)} chars)...")
|
||||
|
||||
|
||||
# We slice text into variable-sized human bursts
|
||||
chunks = []
|
||||
i = 0
|
||||
while i < len(text):
|
||||
if random.random() < 0.15:
|
||||
chunk_size = 1 # single letter hunting
|
||||
chunk_size = 1 # single letter hunting
|
||||
else:
|
||||
chunk_size = random.randint(2, 6) # fluid typing bursts
|
||||
|
||||
chunks.append(text[i:i+chunk_size])
|
||||
chunk_size = random.randint(2, 6) # fluid typing bursts
|
||||
|
||||
chunks.append(text[i : i + chunk_size])
|
||||
i += chunk_size
|
||||
|
||||
for idx, chunk in enumerate(chunks):
|
||||
# 5% chance of a typo if it's an alphabetical chunk
|
||||
if random.random() < 0.05 and len(chunk) >= 2 and chunk[-1].isalpha():
|
||||
typo_letter = random.choice('abcdefghijklmnopqrstuvwxyz')
|
||||
typo_letter = random.choice("abcdefghijklmnopqrstuvwxyz")
|
||||
# Add typo instead of actual last letter
|
||||
typo_chunk = chunk[:-1] + typo_letter
|
||||
_adb_inject_text(device, typo_chunk)
|
||||
|
||||
|
||||
# Realize mistake
|
||||
sleep(random.uniform(0.2, 0.45))
|
||||
|
||||
|
||||
# Send Backspace (KEYCODE_DEL = 67)
|
||||
device.deviceV2.shell("input keyevent 67")
|
||||
device.shell("input keyevent 67")
|
||||
sleep(random.uniform(0.1, 0.25))
|
||||
|
||||
|
||||
# Inject the correct character
|
||||
_adb_inject_text(device, chunk[-1])
|
||||
else:
|
||||
_adb_inject_text(device, chunk)
|
||||
|
||||
|
||||
if speed == "fast":
|
||||
sleep(random.uniform(0.01, 0.05))
|
||||
continue
|
||||
|
||||
# Realistic pause between semantic bursts (humans think while typing)
|
||||
if chunk.endswith((" ", ".", ",", "!", "?")):
|
||||
sleep(random.uniform(0.2, 0.5))
|
||||
else:
|
||||
sleep(random.uniform(0.05, 0.18))
|
||||
|
||||
|
||||
logger.debug("⌨️ [Ghost Keyboard] Injection complete.")
|
||||
|
||||
|
||||
|
||||
def _adb_inject_text(device, text: str):
|
||||
if not text:
|
||||
return
|
||||
|
||||
|
||||
# For Android `input text`, spaces must be mapped to %s
|
||||
# Single quotes need to be bash escaped since we wrap the string in ''
|
||||
# Special characters like & | > < \ ( ) { } ! must be carefully handled.
|
||||
# The safest way is to let shell loop over characters or strictly replace.
|
||||
safe_text = text.replace(" ", "%s").replace("'", "\\'")
|
||||
|
||||
|
||||
# Send through Android's native InputManager
|
||||
try:
|
||||
device.deviceV2.shell(["input", "text", safe_text])
|
||||
device.shell(["input", "text", safe_text])
|
||||
except Exception as e:
|
||||
logger.debug(f"[Ghost Keyboard] Native injection failed: {e}")
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
import logging
|
||||
import os
|
||||
import hashlib
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
from colorama import Fore
|
||||
from qdrant_client.models import FieldCondition, Filter, MatchValue
|
||||
|
||||
from GramAddict.core.qdrant_memory import QdrantBase
|
||||
from qdrant_client.models import PointStruct, Filter, FieldCondition, MatchValue
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -13,18 +13,18 @@ logger = logging.getLogger(__name__)
|
||||
class SwarmProtocol(QdrantBase):
|
||||
"""
|
||||
Decentralized Markov state-channel for P2P knowledge sharing.
|
||||
|
||||
|
||||
Manages 'Pheromones' (successful UI transitions and interactions)
|
||||
and 'BannedPaths' (failed attempts) across bot sessions.
|
||||
|
||||
This creates a Fleet Learning effect: every session learns from
|
||||
|
||||
This creates a Fleet Learning effect: every session learns from
|
||||
every previous session's successes and failures.
|
||||
"""
|
||||
|
||||
def __init__(self, username: str):
|
||||
self.username = username
|
||||
super().__init__(collection_name="gramaddict_swarm_pheromones", vector_size=4)
|
||||
|
||||
|
||||
def emit_pheromone(self, path_hash: str, outcome: str):
|
||||
"""
|
||||
Broadcasting a successful UI transition or interaction to the fleet memory.
|
||||
@@ -32,7 +32,7 @@ class SwarmProtocol(QdrantBase):
|
||||
"""
|
||||
if not self.is_connected or not self.client:
|
||||
return
|
||||
|
||||
|
||||
try:
|
||||
self.upsert_point(
|
||||
seed_string=f"{path_hash}_{outcome}",
|
||||
@@ -44,12 +44,11 @@ class SwarmProtocol(QdrantBase):
|
||||
"timestamp": time.time(),
|
||||
"count": 1,
|
||||
},
|
||||
log_success=f"🌐 [Swarm] ⚡ Pheromone emitted: {path_hash[:16]} → {outcome}"
|
||||
log_success=f"🌐 [Swarm] ⚡ Pheromone emitted: {path_hash[:16]} → {outcome}",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"[Swarm] Pheromone emit failed: {e}")
|
||||
|
||||
|
||||
def query_consensus(self, path_hash: str) -> Optional[str]:
|
||||
"""
|
||||
Queries the swarm for historical outcomes of a specific path.
|
||||
@@ -57,32 +56,22 @@ class SwarmProtocol(QdrantBase):
|
||||
"""
|
||||
if not self.is_connected or not self.client:
|
||||
return None
|
||||
|
||||
|
||||
try:
|
||||
points, _ = self.client.scroll(
|
||||
collection_name=self.collection_name,
|
||||
scroll_filter=Filter(
|
||||
must=[
|
||||
FieldCondition(
|
||||
key="path_hash",
|
||||
match=MatchValue(value=path_hash)
|
||||
)
|
||||
]
|
||||
),
|
||||
scroll_filter=Filter(must=[FieldCondition(key="path_hash", match=MatchValue(value=path_hash))]),
|
||||
limit=1,
|
||||
with_payload=True,
|
||||
)
|
||||
|
||||
|
||||
if points:
|
||||
outcome = points[0].payload.get("outcome")
|
||||
logger.info(
|
||||
f"🌐 [Swarm] Consensus for {path_hash[:16]}: {outcome}",
|
||||
extra={"color": f"{Fore.CYAN}"}
|
||||
)
|
||||
logger.info(f"🌐 [Swarm] Consensus for {path_hash[:16]}: {outcome}", extra={"color": f"{Fore.CYAN}"})
|
||||
return outcome
|
||||
except Exception as e:
|
||||
logger.debug(f"[Swarm] Consensus query failed: {e}")
|
||||
|
||||
|
||||
return None
|
||||
|
||||
def sync_banned_paths(self, banned_paths_db):
|
||||
@@ -92,22 +81,15 @@ class SwarmProtocol(QdrantBase):
|
||||
"""
|
||||
if not self.is_connected or not self.client:
|
||||
return
|
||||
|
||||
|
||||
try:
|
||||
points, _ = self.client.scroll(
|
||||
collection_name=self.collection_name,
|
||||
scroll_filter=Filter(
|
||||
must=[
|
||||
FieldCondition(
|
||||
key="outcome",
|
||||
match=MatchValue(value="banned")
|
||||
)
|
||||
]
|
||||
),
|
||||
scroll_filter=Filter(must=[FieldCondition(key="outcome", match=MatchValue(value="banned"))]),
|
||||
limit=100,
|
||||
with_payload=True,
|
||||
)
|
||||
|
||||
|
||||
synced = 0
|
||||
for pt in points:
|
||||
payload = pt.payload or {}
|
||||
@@ -115,11 +97,10 @@ class SwarmProtocol(QdrantBase):
|
||||
if path and banned_paths_db:
|
||||
banned_paths_db.ban(path, "swarm_synced", reason="Synced from fleet memory")
|
||||
synced += 1
|
||||
|
||||
|
||||
if synced > 0:
|
||||
logger.info(
|
||||
f"🌐 [Swarm] Synced {synced} banned paths from fleet memory.",
|
||||
extra={"color": f"{Fore.CYAN}"}
|
||||
f"🌐 [Swarm] Synced {synced} banned paths from fleet memory.", extra={"color": f"{Fore.CYAN}"}
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"[Swarm] Banned path sync failed: {e}")
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,11 +1,13 @@
|
||||
import logging
|
||||
import random
|
||||
import time
|
||||
|
||||
from colorama import Fore, Style
|
||||
|
||||
from GramAddict.core.session_state import SessionState
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _humanized_scroll_down(device):
|
||||
# Same as bot_flow._humanized_scroll but strictly downward
|
||||
info = device.get_info()
|
||||
@@ -14,31 +16,38 @@ def _humanized_scroll_down(device):
|
||||
start_y = int(h * 0.7) + device.cm_to_pixels(random.uniform(-0.5, 0.5))
|
||||
end_y = int(h * 0.2) + device.cm_to_pixels(random.uniform(-0.5, 0.5))
|
||||
duration = random.uniform(0.08, 0.12)
|
||||
device.deviceV2.swipe(start_x, start_y, start_x, end_y, duration)
|
||||
device.swipe(start_x, start_y, start_x, end_y, duration)
|
||||
from GramAddict.core.utils import random_sleep
|
||||
|
||||
random_sleep(0.8, 1.5)
|
||||
|
||||
def _run_zero_latency_unfollow_loop(device, zero_engine, nav_graph, configs, session_state, current_target, cognitive_stack):
|
||||
|
||||
def _run_zero_latency_unfollow_loop(
|
||||
device, zero_engine, nav_graph, configs, session_state, current_target, cognitive_stack
|
||||
):
|
||||
"""
|
||||
Executes the autonomous Unfollow logic in the Zero-Latency architecture.
|
||||
Assumes the bot is already at the "FollowingList" UI state.
|
||||
"""
|
||||
logger.info(f"🧠 [Unfollow Engine] Initiating cleanup routine in {current_target}...", extra={"color": f"{Style.BRIGHT}{Fore.CYAN}"})
|
||||
|
||||
logger.info(
|
||||
f"🧠 [Unfollow Engine] Initiating cleanup routine in {current_target}...",
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.CYAN}"},
|
||||
)
|
||||
|
||||
telepathic = cognitive_stack.get("telepathic")
|
||||
dopamine = cognitive_stack.get("dopamine")
|
||||
|
||||
|
||||
unfollow_limit = int(getattr(configs.args, "total_unfollows_limit", 50))
|
||||
failed_scrolls = 0
|
||||
total_unfollowed_this_session = 0
|
||||
|
||||
from GramAddict.core.bot_flow import dump_ui_state, _humanized_click
|
||||
|
||||
from GramAddict.core.bot_flow import _humanized_click
|
||||
from GramAddict.core.utils import random_sleep
|
||||
|
||||
|
||||
# Initialize basic tuple if it's missing (helps with tests and initializations)
|
||||
if not hasattr(session_state, 'totalUnfollowed'):
|
||||
if not hasattr(session_state, "totalUnfollowed"):
|
||||
session_state.totalUnfollowed = 0
|
||||
|
||||
|
||||
while not dopamine.is_app_session_over():
|
||||
# Check global limit tuple logic
|
||||
limit_val = session_state.check_limit(SessionState.Limit.UNFOLLOWS)
|
||||
@@ -48,67 +57,154 @@ def _run_zero_latency_unfollow_loop(device, zero_engine, nav_graph, configs, ses
|
||||
elif limit_val is True:
|
||||
logger.info("🛑 Unfollow limit reached for session. Yielding control.")
|
||||
return "BOREDOM_CHANGE_FEED"
|
||||
|
||||
|
||||
if total_unfollowed_this_session >= unfollow_limit:
|
||||
logger.info("🛑 Configured unfollow limit reached. Yielding control.")
|
||||
return "BOREDOM_CHANGE_FEED"
|
||||
|
||||
logger.info("🛑 Configured unfollow limit reached. Yielding control.")
|
||||
return "BOREDOM_CHANGE_FEED"
|
||||
|
||||
try:
|
||||
xml_dump = device.dump_hierarchy()
|
||||
|
||||
# Use Telepathic Engine to explicitly locate existing "Following" buttons in lists
|
||||
nodes = telepathic._extract_semantic_nodes(xml_dump, "find 'Following' buttons next to usernames", threshold=0.7)
|
||||
|
||||
|
||||
# ── Perimeter Guard: Verify we're still inside Instagram ──
|
||||
if xml_dump:
|
||||
import re
|
||||
|
||||
unfollow_packages = set(re.findall(r'package="([^"]+)"', xml_dump))
|
||||
unfollow_app_id = getattr(device, "app_id", "com.instagram.android")
|
||||
if unfollow_packages and unfollow_app_id not in unfollow_packages:
|
||||
logger.error(
|
||||
f"🚨 [UnfollowLoop] FOREIGN APP DETECTED! Packages: {unfollow_packages}. Aborting loop."
|
||||
)
|
||||
device.press("back")
|
||||
random_sleep(1.0, 1.5)
|
||||
return "CONTEXT_LOST"
|
||||
|
||||
# Autonomously identify user rows via Semantic Extraction
|
||||
telepathic = cognitive_stack.get("telepathic")
|
||||
nodes = []
|
||||
if telepathic:
|
||||
nodes = telepathic._extract_semantic_nodes(
|
||||
xml_dump, "List item containing a user profile image, username, and following/following button"
|
||||
)
|
||||
else:
|
||||
logger.warning("No telepathic engine found, skipping semantic extraction.")
|
||||
|
||||
action_taken = False
|
||||
for node in nodes:
|
||||
# Basic validation it's an interactive button
|
||||
if node.get("skip") or not node.get("bounds"):
|
||||
continue
|
||||
|
||||
# Tap the first valid following button we see
|
||||
|
||||
# 1. Tap the profile row to navigate to their page
|
||||
_humanized_click(device, node["x"], node["y"])
|
||||
action_taken = True
|
||||
logger.debug(f"👆 Tapped following button at ({node['x']}, {node['y']})")
|
||||
|
||||
# Check for confirmation dialog ("Unfollow @username?")
|
||||
random_sleep(1.0, 2.0)
|
||||
confirm_xml = device.dump_hierarchy()
|
||||
confirm_nodes = telepathic._extract_semantic_nodes(confirm_xml, "find 'Unfollow' confirmation button", threshold=0.8)
|
||||
|
||||
if confirm_nodes and not confirm_nodes[0].get("skip"):
|
||||
c_node = confirm_nodes[0]
|
||||
_humanized_click(device, c_node["x"], c_node["y"])
|
||||
logger.debug(f"👆 Tapped profile row at ({node['x']}, {node['y']})")
|
||||
|
||||
# Wait for profile to load
|
||||
random_sleep(1.5, 2.5)
|
||||
profile_xml = device.dump_hierarchy()
|
||||
|
||||
# 2. Close Friend Guard via Autonomous Classification
|
||||
classification = telepathic.classify_screen_content(profile_xml.lower(), "close_friends_content")
|
||||
if classification == "close_friends":
|
||||
logger.info(
|
||||
"💚 [Anti-Friend] Profile is a Close Friend. Skipping unfollow.", extra={"color": Fore.GREEN}
|
||||
)
|
||||
device.back()
|
||||
random_sleep(0.8, 1.5)
|
||||
|
||||
logger.info("✅ [Unfollow Engine] Unfollowed a user in list.", extra={"color": Fore.GREEN})
|
||||
session_state.totalUnfollowed += 1
|
||||
total_unfollowed_this_session += 1
|
||||
failed_scrolls = 0
|
||||
|
||||
# Unfollow cost logic
|
||||
dopamine.boredom += random.uniform(1.0, 3.0)
|
||||
random_sleep(1.5, 3.0)
|
||||
break # Go next in loop
|
||||
|
||||
# 3. Resonance Evaluation
|
||||
resonance = cognitive_stack.get("resonance")
|
||||
res_score = 0.5
|
||||
if resonance:
|
||||
# Parse the description from the XML (rough pass, ResonanceEngine handles noise)
|
||||
res_score = resonance.calculate_resonance({"description": profile_xml})
|
||||
|
||||
# 4. Decision: If < 0.4, Unfollow. Else Keep.
|
||||
if res_score < 0.4:
|
||||
logger.info(
|
||||
f"🗑️ [Smart Cleanup] Resonance is low ({res_score:.2f}). Unfollowing.",
|
||||
extra={"color": Fore.YELLOW},
|
||||
)
|
||||
|
||||
# Find 'Following' button on their profile via VLM (Autonomous Learning)
|
||||
following_nodes = telepathic._extract_semantic_nodes(
|
||||
profile_xml, "find 'Following' button", threshold=0.7
|
||||
)
|
||||
|
||||
if following_nodes and not following_nodes[0].get("skip"):
|
||||
f_node = following_nodes[0]
|
||||
_humanized_click(device, f_node["x"], f_node["y"])
|
||||
random_sleep(1.0, 2.0)
|
||||
|
||||
# Verify the following button was actually clicked (bottom sheet should appear)
|
||||
confirm_xml = device.dump_hierarchy()
|
||||
classification = telepathic.classify_screen_content(
|
||||
confirm_xml.lower(), "unfollow_bottom_sheet_presence"
|
||||
)
|
||||
|
||||
if classification == "unfollow_sheet_present":
|
||||
telepathic.verify_success(
|
||||
"find 'Following' button", confirm_xml, device=device, confidence=0.0
|
||||
)
|
||||
else:
|
||||
telepathic.reject_click("find 'Following' button")
|
||||
logger.error("⚠️ Failed to open unfollow confirmation sheet. VLM might have hallucinated.")
|
||||
device.back()
|
||||
random_sleep(1.0, 2.0)
|
||||
continue
|
||||
|
||||
# Find 'Unfollow' confirm
|
||||
# This will now hit the structural fast-path for 'unfollow' in intent_resolver (O(1) Resource ID)
|
||||
confirm_nodes = telepathic._extract_semantic_nodes(confirm_xml, "unfollow", threshold=0.8)
|
||||
|
||||
if confirm_nodes and not confirm_nodes[0].get("skip"):
|
||||
c_node = confirm_nodes[0]
|
||||
_humanized_click(device, c_node["x"], c_node["y"])
|
||||
random_sleep(0.8, 1.5)
|
||||
|
||||
# Verify unfollow succeeded
|
||||
post_unfollow_xml = device.dump_hierarchy()
|
||||
telepathic.verify_success("unfollow", post_unfollow_xml, device=device, confidence=0.0)
|
||||
|
||||
logger.info("✅ [Unfollow Engine] Unfollowed a user.", extra={"color": Fore.GREEN})
|
||||
session_state.totalUnfollowed += 1
|
||||
total_unfollowed_this_session += 1
|
||||
failed_scrolls = 0
|
||||
dopamine.boredom += random.uniform(1.0, 3.0)
|
||||
else:
|
||||
logger.info(
|
||||
f"✨ [Smart Cleanup] Resonance is high ({res_score:.2f}). Keeping subscription.",
|
||||
extra={"color": Fore.MAGENTA},
|
||||
)
|
||||
failed_scrolls = 0
|
||||
|
||||
# 5. Always return to the Following list
|
||||
device.back()
|
||||
random_sleep(1.0, 2.0)
|
||||
break
|
||||
|
||||
|
||||
if not action_taken:
|
||||
# No following buttons in view, scroll down to find more
|
||||
_humanized_scroll_down(device)
|
||||
dopamine.boredom += 0.5
|
||||
failed_scrolls += 1
|
||||
|
||||
|
||||
if failed_scrolls > 5:
|
||||
logger.warning("⚠️ [Unfollow Engine] No 'Following' buttons found after multiple scrolls. Aborting or reaching bottom.")
|
||||
logger.warning(
|
||||
"⚠️ [Unfollow Engine] No 'Following' buttons found after multiple scrolls. Aborting or reaching bottom."
|
||||
)
|
||||
return "BOREDOM_CHANGE_FEED"
|
||||
|
||||
|
||||
if dopamine.wants_to_change_feed():
|
||||
logger.info("🧠 [Unfollow Engine] Desire to clean up following list satisfied. Navigating elsewhere.")
|
||||
return "BOREDOM_CHANGE_FEED"
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"⚠️ [FSD Anomaly Handler] Exception in Unfollow Loop: {e}")
|
||||
logger.error(f"⚠️ [Anomaly Handler] Exception in Unfollow Loop: {e}")
|
||||
_humanized_scroll_down(device)
|
||||
failed_scrolls += 1
|
||||
if failed_scrolls > 3:
|
||||
return "CONTEXT_LOST"
|
||||
|
||||
return "CONTEXT_LOST"
|
||||
|
||||
return "FEED_EXHAUSTED"
|
||||
|
||||
@@ -1,20 +1,23 @@
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import random
|
||||
import requests
|
||||
import sys
|
||||
from datetime import datetime, timedelta
|
||||
from time import sleep
|
||||
|
||||
from colorama import Fore, Style
|
||||
from packaging.version import parse as parse_version
|
||||
|
||||
from GramAddict.core.version import __version__
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def sanitize_text(text):
|
||||
return (text or "").strip()
|
||||
|
||||
|
||||
def random_sleep(inf=1.0, sup=3.0, modulable=True):
|
||||
from GramAddict.core.config import Config
|
||||
|
||||
configs = Config()
|
||||
try:
|
||||
multiplier = float(getattr(configs.args, "speed_multiplier", 1.0))
|
||||
@@ -23,57 +26,68 @@ def random_sleep(inf=1.0, sup=3.0, modulable=True):
|
||||
delay = random.uniform(inf, sup) / (multiplier if modulable else 1.0)
|
||||
sleep(max(delay, 0.2))
|
||||
|
||||
|
||||
def config_examples():
|
||||
logger.debug("Config examples handled by documentation.")
|
||||
|
||||
|
||||
def check_if_updated():
|
||||
logger.info(f"GramAddict v.{__version__}", extra={"color": f"{Style.BRIGHT}{Fore.MAGENTA}"})
|
||||
|
||||
|
||||
def get_instagram_version(device):
|
||||
try:
|
||||
output = device.deviceV2.shell(f"dumpsys package {device.app_id}").output
|
||||
output = device.shell(f"dumpsys package {device.app_id}").output
|
||||
import re
|
||||
|
||||
version_match = re.findall("versionName=(\\S+)", output)
|
||||
return version_match[0] if version_match else "unknown"
|
||||
except Exception:
|
||||
return "unknown"
|
||||
|
||||
|
||||
def close_instagram(device, force_kill=False):
|
||||
if force_kill:
|
||||
logger.info("Force-closing Instagram app to clean session state.")
|
||||
try:
|
||||
device.deviceV2.app_stop(device.app_id)
|
||||
device.app_stop(device.app_id)
|
||||
except Exception as e:
|
||||
logger.debug(f"Error closing app: {e}")
|
||||
else:
|
||||
logger.info("Backgrounding Instagram app (minimizing).")
|
||||
try:
|
||||
device.deviceV2.press("home")
|
||||
device.press("home")
|
||||
except Exception as e:
|
||||
logger.debug(f"Error pressing home: {e}")
|
||||
|
||||
|
||||
def open_instagram(device, force_restart=False):
|
||||
if force_restart:
|
||||
logger.info("Opening Instagram app (Fresh Start).")
|
||||
close_instagram(device, force_kill=True)
|
||||
device.deviceV2.app_start(device.app_id)
|
||||
device.app_start(device.app_id)
|
||||
random_sleep(3, 5, modulable=False)
|
||||
else:
|
||||
logger.info("Bringing Instagram app to foreground.")
|
||||
device.deviceV2.app_start(device.app_id)
|
||||
device.app_start(device.app_id)
|
||||
random_sleep(1, 2, modulable=False)
|
||||
return True
|
||||
|
||||
|
||||
def set_time_delta(args):
|
||||
args.time_delta_session = random.randint(-300, 300)
|
||||
|
||||
|
||||
def wait_for_next_session(time_left, session_state, sessions, device):
|
||||
logger.info(f"Waiting {time_left} until next working hours.")
|
||||
sleep(60)
|
||||
|
||||
|
||||
def get_value(count, name, default=0):
|
||||
if count is None: return default
|
||||
if isinstance(count, (int, float)): return count
|
||||
if count is None:
|
||||
return default
|
||||
if isinstance(count, (int, float)):
|
||||
return count
|
||||
try:
|
||||
if "-" in str(count):
|
||||
parts = str(count).split("-")
|
||||
@@ -81,3 +95,88 @@ def get_value(count, name, default=0):
|
||||
return int(count)
|
||||
except Exception:
|
||||
return default
|
||||
|
||||
|
||||
_LEARNED_AD_MARKERS_FILE = os.path.join(os.getcwd(), "learned_ad_markers.json")
|
||||
_LEARNED_AD_MARKERS_CACHE = None
|
||||
|
||||
|
||||
def get_learned_ad_markers() -> set:
|
||||
global _LEARNED_AD_MARKERS_CACHE
|
||||
if _LEARNED_AD_MARKERS_CACHE is not None:
|
||||
return _LEARNED_AD_MARKERS_CACHE
|
||||
|
||||
if os.path.exists(_LEARNED_AD_MARKERS_FILE):
|
||||
try:
|
||||
with open(_LEARNED_AD_MARKERS_FILE, "r") as f:
|
||||
_LEARNED_AD_MARKERS_CACHE = set(json.load(f))
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to load learned ad markers: {e}")
|
||||
_LEARNED_AD_MARKERS_CACHE = set()
|
||||
else:
|
||||
_LEARNED_AD_MARKERS_CACHE = set()
|
||||
|
||||
return _LEARNED_AD_MARKERS_CACHE
|
||||
|
||||
|
||||
def learn_ad_marker(marker: str, xml_hierarchy: str):
|
||||
global _LEARNED_AD_MARKERS_CACHE
|
||||
if not marker or len(marker) > 30:
|
||||
return
|
||||
|
||||
marker = marker.strip().lower()
|
||||
|
||||
# Structural verification: the VLM-suggested marker MUST exist as an exact node text/desc in the current UI!
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
try:
|
||||
root = ET.fromstring(xml_hierarchy)
|
||||
found_in_ui = False
|
||||
for node in root.iter("node"):
|
||||
text = node.attrib.get("text", "").strip().lower()
|
||||
desc = node.attrib.get("content-desc", "").strip().lower()
|
||||
if text == marker or desc == marker:
|
||||
found_in_ui = True
|
||||
break
|
||||
|
||||
if not found_in_ui:
|
||||
logger.debug(
|
||||
f"🧠 [Autonomous FSD] Rejected hallucinated Ad marker '{marker}' (not found as exact node match in UI)."
|
||||
)
|
||||
return
|
||||
except Exception:
|
||||
return
|
||||
|
||||
markers = get_learned_ad_markers()
|
||||
if marker not in markers and marker not in {"ad", "sponsored", "advertisement", "gesponsert", "anzeige", "werbung"}:
|
||||
markers.add(marker)
|
||||
logger.info(
|
||||
f"🧠 [Autonomous FSD] Verified and Learned new Ad marker: '{marker}'. Persisting for zero-latency detection.",
|
||||
extra={"color": f"{Style.BRIGHT}{Fore.GREEN}"},
|
||||
)
|
||||
try:
|
||||
with open(_LEARNED_AD_MARKERS_FILE, "w") as f:
|
||||
json.dump(list(markers), f)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to save learned ad markers: {e}")
|
||||
|
||||
|
||||
def is_ad(xml_hierarchy: str, cognitive_stack: dict = None) -> bool:
|
||||
"""
|
||||
Checks if the current view contains an advertisement using autonomous learning.
|
||||
Relies 100% on Telepathic Engine for semantic classification (Zero-Latency vector lookup).
|
||||
No hardcoded resource IDs or text labels allowed.
|
||||
"""
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
if cognitive_stack and "telepathic" in cognitive_stack:
|
||||
telepathic = cognitive_stack["telepathic"]
|
||||
else:
|
||||
telepathic = TelepathicEngine.get_instance()
|
||||
|
||||
# Semantic classification (ZERO hardcoded strings)
|
||||
classification = telepathic.classify_screen_content(xml_hierarchy, "sponsored_content")
|
||||
if classification == "sponsored":
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
@@ -4,16 +4,18 @@ import xml.etree.ElementTree as ET
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ZeroLatencyEngine:
|
||||
"""
|
||||
Project Singularity V7: The Zero-Latency Executor
|
||||
The Zero-Latency Executor
|
||||
This engine receives a pre-compiled heuristic (Regex/XPath) from the memory cache
|
||||
and executes it against the local XML layout in under 5ms.
|
||||
and executes it against the local XML layout in under 5ms.
|
||||
It is completely deterministic. No LLM calls happen here.
|
||||
"""
|
||||
|
||||
def __init__(self, device):
|
||||
self.device = device
|
||||
|
||||
|
||||
def evaluate_heuristic(self, rule: dict, context_xml: str):
|
||||
"""
|
||||
Executes a compiled heuristic rule against the provided XML dump.
|
||||
@@ -22,20 +24,20 @@ class ZeroLatencyEngine:
|
||||
"""
|
||||
if not rule or not context_xml:
|
||||
return None
|
||||
|
||||
|
||||
rule_type = rule.get("rule_type", "regex")
|
||||
target_attr = rule.get("target_attribute", "text")
|
||||
pattern = rule.get("pattern", "")
|
||||
|
||||
|
||||
if not pattern:
|
||||
return None
|
||||
|
||||
try:
|
||||
root = ET.fromstring(context_xml)
|
||||
|
||||
|
||||
if rule_type == "regex":
|
||||
# Remove (?i) if present because we compile with re.IGNORECASE anyway
|
||||
clean_pattern = pattern.replace('(?i)', '')
|
||||
clean_pattern = pattern.replace("(?i)", "")
|
||||
regex = re.compile(clean_pattern, re.IGNORECASE)
|
||||
for node in root.iter("node"):
|
||||
val = ""
|
||||
@@ -55,17 +57,17 @@ class ZeroLatencyEngine:
|
||||
match = regex.search(val)
|
||||
if match:
|
||||
if len(match.groups()) > 0:
|
||||
return match.group(1) # Return captured group (e.g., username)
|
||||
return True # Return boolean existence (e.g. is_ad)
|
||||
|
||||
return match.group(1) # Return captured group (e.g., username)
|
||||
return True # Return boolean existence (e.g. is_ad)
|
||||
|
||||
elif rule_type == "xpath":
|
||||
# Basic xpath parsing over ET
|
||||
nodes = root.findall(pattern)
|
||||
if nodes:
|
||||
return nodes[0].attrib.get(target_attr, "")
|
||||
|
||||
return False # Rule ran but found nothing
|
||||
|
||||
|
||||
return False # Rule ran but found nothing
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"ZeroLatencyEngine failed to evaluate rule {pattern}: {e}")
|
||||
return None
|
||||
|
||||
11
README.md
11
README.md
@@ -11,7 +11,7 @@
|
||||
|
||||
## 🏎️ What is GramPilot?
|
||||
|
||||
GramPilot is not a traditional script. Traditional bots rely on fixed UI locators (like XPaths) or external APIs, causing them to crash with every Instagram update or get banned within days.
|
||||
GramPilot is not a traditional script. Traditional bots rely on fixed UI locators (like XPaths) or external APIs, causing them to crash with every Instagram update or get banned within days.
|
||||
|
||||
GramPilot introduces a **Telepathic Full Self-Driving (FSD) approach** to UI navigation:
|
||||
It uses a 3-Stage Resolution Cascade backed by CPU Fast-Paths, Ollama Vector Similarity, and OpenRouter LLMs (Gemini/Qwen) to "read" the screen, understand context, and learn new UI layouts asynchronously.
|
||||
@@ -21,11 +21,20 @@ If Instagram updates its app and moves a button, GramPilot doesn't crash. It fal
|
||||
## ✨ Core Features
|
||||
|
||||
* 🚫 **Zero Limits Configuration**: Forget about configuring "max_likes" or "delays". GramPilot uses a **Dopamine Pacing Engine** to simulate human boredom. If the content isn't interesting, it skips it or ends the session early.
|
||||
* 🎯 **Mission-Driven Navigation**: Say goodbye to abstract goal configurations. Define a `strategy` (like `aggressive_growth` or `nurture_community`) in `config.yml`, and the **Goal Decomposer Engine** automatically orchestrates the optimal routing and task allocation using enabled plugins.
|
||||
* ⚖️ **Active Inference (Shadow Mode)**: The bot continuously predicts the outcome of its clicks. If it lands on a popup instead of a profile, it registers a "Prediction Error", presses back, and dynamically recalibrates without panicking.
|
||||
* ⛩️ **Telepathic Engine**: A strictly tiered resolution cascade (Keyword -> Vectors -> LLM) that ensures 90% of navigation happens at 0-token cost while maintaining fallback AI resilience.
|
||||
* 🧬 **Resonance Oracle**: The bot only interacts with content that matches a pre-defined persona aesthetic, completely bypassing spam or low-quality content.
|
||||
* 🛡️ **Honeypot Radome**: Instagram plants invisible, 1x1 pixel trap buttons for bots. Our *Radome Sensor* sanitizes the XML view before the agent ever sees it, mathematically guaranteeing evasion of tracker traps.
|
||||
|
||||
## 🏗️ Project Status (May 2026)
|
||||
|
||||
The engine has undergone a massive stabilization refactor to achieve **100% TDD compliance** on critical navigation paths.
|
||||
- **Structural Hardening:** Purged non-deterministic "Geometric Fallbacks" for navigation tabs. Replaced with strict Resource ID -> Semantic Content validation.
|
||||
- **Grid-Lock Recovery:** Implemented O(1) structural fast-paths for grid interactions and state-aware **Adaptive Snap** logic to prevent erroneous back-presses on profile grids.
|
||||
- **Navigation Reliability:** Resolved 'Identity Shadowing' bugs to ensure deterministic detection of `OWN_PROFILE`.
|
||||
- **Autonomous Recovery:** Hardened the `SituationalAwarenessEngine` (SAE) to handle anomaly states including system dialogs and persistent survey modals.
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
152
TESTING.md
Normal file
152
TESTING.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# 🧪 Instagram Bot Testing Standards
|
||||
|
||||
This project follows a strict **Test-Driven Development (TDD)** philosophy. We do not write features blindly; we ground our development in real-world observations and automated verification.
|
||||
|
||||
## 🔴🟢🔵 The TDD Workflow (Red-Green-Refactor)
|
||||
|
||||
Every new feature or bugfix should follow this cycle:
|
||||
|
||||
1. **RED**: Start by obtaining a **Real XML Dump** (using the Testing Toolkit) of the target UI state. Write a test that fails against this dump or a `--live` device.
|
||||
2. **GREEN**: Implement the minimum amount of code (logic in `TelepathicEngine`, `QNavGraph`, etc.) to make the test pass.
|
||||
3. **REFACTOR**: Clean up the code. Ensure it adheres to our [Best Practices](#-best-practices--no-gos), is well-documented, and doesn't introduce regressions.
|
||||
|
||||
> [!TIP]
|
||||
> TDD-specific tests, regression fixes, and edge-case hardening should be placed in the `tests/tdd/` directory.
|
||||
|
||||
---
|
||||
|
||||
|
||||
## 1. Mock Testing (Offline Mode)
|
||||
This is the **default mode** used in CI/CD pipelines and for rapid local development iterations.
|
||||
|
||||
- **Concept**: The `DeviceFacade` is fully mocked. UI states are loaded from static XML fixtures located in `tests/fixtures/`.
|
||||
- **Primary Benefit**: Extremely fast (<1s per test) and requires no physical device or emulator.
|
||||
- **Command**:
|
||||
```bash
|
||||
pytest
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Fixture Management (The Toolkit)
|
||||
To prevent offline tests from validating against outdated Instagram layouts, XML fixtures must be periodically synchronized. Use the **Testing Toolkit** located in `scripts/`.
|
||||
|
||||
### Interactive Guide (Full Sync)
|
||||
This guide walks you through 13 critical views (Home, Explore, DMs, etc.) and automatically captures the required dumps.
|
||||
- **Command**:
|
||||
```bash
|
||||
python3 scripts/sync_fixtures.py --config test_config.yml --interactive
|
||||
```
|
||||
|
||||
### Single Fixture Update
|
||||
If only a specific screen has changed:
|
||||
- **Command**:
|
||||
```bash
|
||||
python3 scripts/sync_fixtures.py --config test_config.yml --fixture explore_feed_dump.xml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Live Hardware Testing (Real Device Mode)
|
||||
Validates that the bot correctly interacts with a real device (emulator or physical phone) by performing actual clicks and swipes.
|
||||
|
||||
- **Concept**: Disables mocks. `pytest` connects via ADB to the active device.
|
||||
- **Primary Benefit**: Detects UI synchronization issues, animation delays, and ADB connection drops.
|
||||
- **Command**:
|
||||
```bash
|
||||
pytest --live
|
||||
```
|
||||
*(Note: Uses the device ID specified in your config or `conftest.py` defaults).*
|
||||
|
||||
---
|
||||
|
||||
## 4. AI & LLM Validation
|
||||
Verifies that the bot's "brain" (Telepathic Engine) still understands XML structures and correctly maps elements to actions (e.g., finding the "Like Button").
|
||||
|
||||
- **Concept**: Sends real prompts to your local LLM server (Ollama/Qwen).
|
||||
- **Primary Benefit**: Protects against "prompt drift" or performance regressions after model updates.
|
||||
- **Command**:
|
||||
```bash
|
||||
RUN_LIVE_AI_TESTS=1 pytest tests/integration/test_live_telepathy.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. E2E Functional Sequences
|
||||
These tests simulate full automation loops to ensure that different components (Goap, Navigation, SAE) play together correctly.
|
||||
|
||||
- **Concept**: Targeted scenario tests that verify a complete user flow from start to finish.
|
||||
- **Example Flows**:
|
||||
- `tests/e2e/test_e2e_explore_feed.py`: Validates the full "Explore -> Analyze -> Interact" loop.
|
||||
- `tests/e2e/test_e2e_dm_sequence.py`: Validates handling of message threads.
|
||||
- **Command**:
|
||||
```bash
|
||||
pytest tests/e2e/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Cognitive Benchmarking
|
||||
Measures the "IQ" and latency of your LLM models to ensure they are suitable for autonomous navigation.
|
||||
|
||||
- **Concept**: Runs a series of 13+ UI-parsing scenarios against your configured models and scores them on accuracy and speed.
|
||||
- **Benefit**: Identifies models that are too slow or "hallucinate" UI coordinates before you let them loose on your real account.
|
||||
- **Command (Ollama)**:
|
||||
```bash
|
||||
python3 benchmarks/run_competitive_benchmark.py --all-ollama
|
||||
```
|
||||
- **Command (Existing Config)**:
|
||||
```bash
|
||||
python3 benchmarks/run_competitive_benchmark.py --config test_config.yml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Latency and Adaptive Snap Validation
|
||||
Ensuring the agent handles slow network responses or missing feed markers (e.g., getting trapped in a Story) is critical for Full Self-Driving autonomy.
|
||||
|
||||
- **Concept**: Simulates UI rendering delays to trigger the `post_load_timeout` and verify the `Adaptive Snap` recovery logic.
|
||||
- **Implementation**: When testing `bot_flow.py`, mock `_wait_for_post_loaded` or the underlying `device.dump_hierarchy()` to return an incomplete or missing feed XML (like `reel_viewer_root`) to verify the bot presses `back` or wobbles successfully.
|
||||
- **Key Assertions**:
|
||||
- Verify that `nav_graph.do('align')` or `device.press("back")` is called when `_wait_for_post_loaded` fails to find `FEED_MARKERS`.
|
||||
- Validate that the timeout gracefully escapes loop-locks rather than blindly proceeding with bad UI state.
|
||||
|
||||
---
|
||||
|
||||
## 🛠 Troubleshooting
|
||||
- **Device offline**: Ensure that `adb devices` lists your device and it is authorized.
|
||||
- **LLM Timeout**: Verify that Ollama is running (`ollama list`) and the required model (e.g., `qwen3.5:latest`) is loaded.
|
||||
- **Missing Fixture**: If a test fails with `MISSING REAL DUMP`, use the Toolkit (Step 2) to capture the missing screen.
|
||||
- **Benchmark Failures**: If a model fails benchmarks, it is automatically marked as `is_unsuitable` and should not be used for critical navigation tasks.
|
||||
|
||||
---
|
||||
|
||||
## 💎 Golden Rules of Implementation
|
||||
|
||||
To maintain 100% reliability and "Tesla-level" autonomy, every developer (and AI agent) MUST follow these rules:
|
||||
|
||||
1. **Strict Green Light Policy**: All tests (both existing and new) MUST be green before a task is considered finished. No exceptions.
|
||||
2. **No Fix Without a Red Test**: Never implement a fix or a feature without first having a failing test that demonstrates the problem or the missing capability.
|
||||
3. **Explicit Test Summary**: Every completion summary must explicitly list exactly which tests were added or modified to verify the change.
|
||||
4. **Exhaustive Edge-Case Coverage**: Consider and test for failure modes: "What if the DB is down?", "What if the screen is empty?", "What if the user is in a state we've never seen?".
|
||||
5. **Efficient, Fail-Fast Testing**:
|
||||
* Do not run the entire suite if you know where the failure is.
|
||||
* Run targeted tests immediately after a change.
|
||||
* Fail fast: fix the first failing test before moving to the next.
|
||||
* Maintain a mental (or written) list of remaining failing tests to ensure none are forgotten.
|
||||
|
||||
---
|
||||
|
||||
## 💎 Best Practices & No-Gos
|
||||
- **Use Golden Fixtures**: Always use real, freshly pulled XML dumps. If the Instagram UI changes, update the fixtures immediately using the Testing Toolkit.
|
||||
- **Singleton Isolation**: Ensure all core singletons (`TelepathicEngine`, `GoalExecutor`) are reset between tests in `conftest.py`.
|
||||
- **Hermetic Tests**: Each test must be independent. Ensure on-disk caches (JSON files) are wiped before each run.
|
||||
- **Layered Validation**: Start with fast mock tests for logic, then verify with `--live` hardware tests for physical interaction.
|
||||
- **Relative Pathing**: Use `os.path.join` relative to `__file__` for all fixture loading.
|
||||
|
||||
### ❌ No-Gos
|
||||
- **"Lying" Mocks**: Never create hand-written or "guessed" XML structures. If you don't have a dump, pull a real one.
|
||||
- **Hardcoded Absolute Paths**: Never use paths like `/Users/name/...`. These break CI and other developers' environments.
|
||||
- **State Leakage**: Never rely on the side effects of a previous test. If a test fails, it should not cause subsequent tests to fail.
|
||||
- **Implicit Timing in Mocks**: Do not use `time.sleep()` for UI waiting in offline tests. Rely on the `VirtualClock` or state-based assertions.
|
||||
- **Mocking Navigation Logic**: In E2E tests, do not mock the internal decision-making of the `TelepathicEngine` or `GrowthBrain`. Force them to process real (fixture) data.
|
||||
@@ -1,25 +1,32 @@
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import json
|
||||
import time
|
||||
|
||||
# Root path alignment
|
||||
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.append(ROOT_DIR)
|
||||
|
||||
|
||||
def colored(text, color, attrs=None):
|
||||
colors = {
|
||||
"red": "\033[91m", "green": "\033[92m", "yellow": "\033[93m",
|
||||
"blue": "\033[94m", "magenta": "\033[95m", "cyan": "\033[96m",
|
||||
"white": "\033[97m"
|
||||
"red": "\033[91m",
|
||||
"green": "\033[92m",
|
||||
"yellow": "\033[93m",
|
||||
"blue": "\033[94m",
|
||||
"magenta": "\033[95m",
|
||||
"cyan": "\033[96m",
|
||||
"white": "\033[97m",
|
||||
}
|
||||
reset = "\033[0m"
|
||||
bold = "\033[1m" if attrs and "bold" in attrs else ""
|
||||
return f"{bold}{colors.get(color, '')}{text}{reset}"
|
||||
|
||||
|
||||
from GramAddict.core.config import Config
|
||||
from GramAddict.core.qdrant_memory import ParasocialCRMDB, CommentMemoryDB
|
||||
from GramAddict.core.qdrant_memory import CommentMemoryDB, ParasocialCRMDB
|
||||
from GramAddict.core.resonance_engine import ResonanceEngine
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
|
||||
class MockArgs:
|
||||
def __init__(self):
|
||||
@@ -31,10 +38,12 @@ class MockArgs:
|
||||
self.ai_vision_navigation = False
|
||||
self.ai_vision_context = False
|
||||
|
||||
|
||||
class MockConfig:
|
||||
def __init__(self):
|
||||
self.args = MockArgs()
|
||||
|
||||
|
||||
class AIMemoryDiagnosticRunner:
|
||||
def __init__(self):
|
||||
self.configs = MockConfig()
|
||||
@@ -42,7 +51,7 @@ class AIMemoryDiagnosticRunner:
|
||||
self.crm_db = ParasocialCRMDB()
|
||||
self.comment_db = CommentMemoryDB()
|
||||
self.resonance_oracle = ResonanceEngine("benchmark_agent", crm=self.crm_db)
|
||||
|
||||
|
||||
def setup(self):
|
||||
print(colored("🧹 Initializing benchmark data...", "cyan"))
|
||||
# We handle unique targets so we don't wipe the DB
|
||||
@@ -56,19 +65,24 @@ class AIMemoryDiagnosticRunner:
|
||||
fixture_path = os.path.join(ROOT_DIR, "tests", "fixtures", "comments_mock.xml")
|
||||
with open(fixture_path, "r", encoding="utf-8") as f:
|
||||
xml_data = f.read()
|
||||
|
||||
print(colored(f" -> Extracing comments using RAG Condenser ({self.configs.args.ai_condenser_model})...", "yellow"))
|
||||
|
||||
print(
|
||||
colored(
|
||||
f" -> Extracing comments using RAG Condenser ({self.configs.args.ai_condenser_model})...", "yellow"
|
||||
)
|
||||
)
|
||||
start = time.time()
|
||||
|
||||
|
||||
# Intercept the database write to bypass Qdrant indexing limits and solely test RAG filter logic
|
||||
intercepted_comments = []
|
||||
|
||||
|
||||
def mock_log(self, text: str, vibe: str, author: str = "unknown"):
|
||||
intercepted_comments.append(text)
|
||||
|
||||
|
||||
try:
|
||||
from unittest.mock import patch
|
||||
with patch.object(CommentMemoryDB, 'store_comment', new=mock_log):
|
||||
|
||||
with patch.object(CommentMemoryDB, "store_comment", new=mock_log):
|
||||
# Override the author logic
|
||||
test_author = f"benchmark_source_{int(time.time())}"
|
||||
self.resonance_oracle.extract_and_learn_comments(xml_data, self.configs, author=test_author)
|
||||
@@ -76,26 +90,41 @@ class AIMemoryDiagnosticRunner:
|
||||
except Exception as e:
|
||||
print(f"❌ EXCEPTION: {e}")
|
||||
return {"passed": False, "reason": str(e)}
|
||||
|
||||
|
||||
try:
|
||||
learned_texts = [c.lower() for c in intercepted_comments]
|
||||
dur = time.time() - start
|
||||
print(colored(f" -> Intercepted: {learned_texts}", "yellow"))
|
||||
|
||||
|
||||
toxic_count = sum(1 for t in learned_texts if "onlyfans" in t or "bitcoin" in t or "dm" in t or "$" in t)
|
||||
good_count = sum(1 for t in learned_texts if "majestic" in t or "lighting" in t)
|
||||
|
||||
|
||||
if toxic_count > 0:
|
||||
print(colored(" ❌ [Sub-Test] LLM Condenser hallucinated or failed to block toxic queries (OnlyFans/Crypto).", "red"))
|
||||
print(
|
||||
colored(
|
||||
" ❌ [Sub-Test] LLM Condenser hallucinated or failed to block toxic queries (OnlyFans/Crypto).",
|
||||
"red",
|
||||
)
|
||||
)
|
||||
return {"passed": False, "reason": "Toxic comments leaked"}
|
||||
|
||||
|
||||
if good_count == 0:
|
||||
print(colored(" ❌ [Sub-Test] LLM Condenser stripped everything or crashed. No good comments persisted.", "red"))
|
||||
print(
|
||||
colored(
|
||||
" ❌ [Sub-Test] LLM Condenser stripped everything or crashed. No good comments persisted.",
|
||||
"red",
|
||||
)
|
||||
)
|
||||
return {"passed": False, "reason": "Good comments dropped"}
|
||||
|
||||
print(colored(f" ✅ [Sub-Test] RAG Filter passed! 0 toxic comments, {good_count} valid comments mapped. Latency {dur:.2f}s", "green"))
|
||||
|
||||
print(
|
||||
colored(
|
||||
f" ✅ [Sub-Test] RAG Filter passed! 0 toxic comments, {good_count} valid comments mapped. Latency {dur:.2f}s",
|
||||
"green",
|
||||
)
|
||||
)
|
||||
return {"passed": True, "reason": "Toxic filtered, good preserved."}
|
||||
|
||||
|
||||
except Exception as e:
|
||||
return {"passed": False, "reason": f"DB Error: {e}"}
|
||||
|
||||
@@ -105,18 +134,18 @@ class AIMemoryDiagnosticRunner:
|
||||
"""
|
||||
target = "benchmark_target"
|
||||
context_string = "234 Posts | 1.2M Followers | 🏔️ Alpine Photographer | Link in bio"
|
||||
|
||||
|
||||
try:
|
||||
self.crm_db.log_profile_context(target, context_string)
|
||||
time.sleep(0.5) # indexing buffer
|
||||
|
||||
time.sleep(0.5) # indexing buffer
|
||||
|
||||
history = self.crm_db.get_conversation_context(target)
|
||||
if context_string in history or "1.2M Followers" in history:
|
||||
print(colored(" ✅ [Sub-Test] Profile context cleanly injected into RAG CRM payload.", "green"))
|
||||
return {"passed": True, "reason": "Context string found."}
|
||||
else:
|
||||
return {"passed": False, "reason": "Profile context missing from CRM retrieval."}
|
||||
|
||||
|
||||
except Exception as e:
|
||||
return {"passed": False, "reason": str(e)}
|
||||
|
||||
@@ -136,61 +165,60 @@ class AIMemoryDiagnosticRunner:
|
||||
self.crm_db.log_generated_comment(target, "Wow great photo!")
|
||||
self.crm_db.log_interaction(target, "tap_comment_button", new_stage=3)
|
||||
time.sleep(0.5)
|
||||
|
||||
|
||||
stage_info = self.crm_db.get_relationship_stage(target)
|
||||
stage = stage_info.get("stage", 0)
|
||||
|
||||
|
||||
if stage >= 3:
|
||||
print(colored(f" ✅ [Sub-Test] CRM safely advanced state memory to Stage {stage}.", "green"))
|
||||
return {"passed": True, "reason": "Evolution logic passed."}
|
||||
else:
|
||||
print(colored(f" ❌ [Sub-Test] CRM stalled at Stage {stage}!", "red"))
|
||||
return {"passed": False, "reason": "Failed to evolve stage"}
|
||||
|
||||
|
||||
except Exception as e:
|
||||
return {"passed": False, "reason": str(e)}
|
||||
|
||||
def execute_all(self):
|
||||
self.setup()
|
||||
results = {
|
||||
"timestamp": time.time(),
|
||||
"model": self.configs.args.ai_condenser_model,
|
||||
"scenarios": {}
|
||||
}
|
||||
|
||||
results = {"timestamp": time.time(), "model": self.configs.args.ai_condenser_model, "scenarios": {}}
|
||||
|
||||
def run_and_log(name, func):
|
||||
print(colored(f"\n--- SCENARIO: {name} ---", "magenta"))
|
||||
start_time = time.time()
|
||||
data = {"passed": False, "reason": "Unknown error", "latency_ms": 0}
|
||||
try:
|
||||
res = func()
|
||||
if isinstance(res, dict): data.update(res)
|
||||
elif res is True: data["passed"] = True
|
||||
if isinstance(res, dict):
|
||||
data.update(res)
|
||||
elif res is True:
|
||||
data["passed"] = True
|
||||
except Exception as e:
|
||||
print(colored(f"❌ EXCEPTION: {e}", "red"))
|
||||
data["reason"] = str(e)
|
||||
|
||||
|
||||
dur = time.time() - start_time
|
||||
data["latency_ms"] = int(dur * 1000)
|
||||
results["scenarios"][name] = data
|
||||
|
||||
|
||||
if data["passed"]:
|
||||
print(colored(f"🏁 {name} completed successfully in {dur:.2f}s", "green"))
|
||||
else:
|
||||
print(colored(f"🚨 {name} FAILED! (Elapsed: {dur:.2f}s)", "red", attrs=["bold"]))
|
||||
print(colored(f" Reason: {data['reason']}", "yellow"))
|
||||
|
||||
|
||||
run_and_log("RAG Comment Blacklist Extraction", self.test_rag_comment_extraction)
|
||||
run_and_log("CRM Profile Context Injection", self.test_crm_profile_context)
|
||||
run_and_log("CRM Sequential Evolution", self.test_crm_interaction_evolution)
|
||||
|
||||
self.setup() # Teardown
|
||||
|
||||
|
||||
self.setup() # Teardown
|
||||
|
||||
out_path = os.path.join(ROOT_DIR, "benchmarks", "data", "ai_memory_results.json")
|
||||
with open(out_path, "w") as f:
|
||||
json.dump(results, f, indent=4)
|
||||
print(colored(f"\n📄 Saved AI Memory Benchmark results to: {out_path}", "cyan", attrs=["bold"]))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
runner = AIMemoryDiagnosticRunner()
|
||||
runner.execute_all()
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import logging
|
||||
import json
|
||||
|
||||
from colorama import Fore, Style, init
|
||||
|
||||
# Init Colorama for cross-platform color support
|
||||
@@ -12,8 +13,8 @@ init(autoreset=True)
|
||||
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT_DIR)
|
||||
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
from GramAddict.core.qdrant_memory import UIMemoryDB
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
# Mute noisy loggers
|
||||
logging.getLogger("requests").setLevel(logging.WARNING)
|
||||
@@ -21,6 +22,7 @@ logging.getLogger("urllib3").setLevel(logging.WARNING)
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def colored(text, color, attrs=None):
|
||||
c = getattr(Fore, color.upper(), "")
|
||||
attr_str = ""
|
||||
@@ -28,6 +30,7 @@ def colored(text, color, attrs=None):
|
||||
attr_str = Style.BRIGHT
|
||||
return f"{attr_str}{c}{text}"
|
||||
|
||||
|
||||
class MockArgs:
|
||||
def __init__(self):
|
||||
self.ai_telepathic_model = "qwen3.5:latest"
|
||||
@@ -37,46 +40,55 @@ class MockArgs:
|
||||
self.ai_vision_navigation = True
|
||||
self.ai_vision_context = True
|
||||
|
||||
|
||||
import base64
|
||||
|
||||
|
||||
class MockDevice:
|
||||
def __init__(self):
|
||||
self.args = MockArgs()
|
||||
self.app_id = "com.instagram.android"
|
||||
|
||||
|
||||
def screenshot(self):
|
||||
# Return a simple 1x1 black pixel PNG to test the True Vision payload mapping
|
||||
# without crashing on invalid image data.
|
||||
return base64.b64decode("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAAXSURBVBhXY3jP4PgfAAWEAziO3O8MAAAAASUVORK5CYII=")
|
||||
return base64.b64decode(
|
||||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAAXSURBVBhXY3jP4PgfAAWEAziO3O8MAAAAASUVORK5CYII="
|
||||
)
|
||||
|
||||
|
||||
from GramAddict.core.config import Config
|
||||
|
||||
Config().args = MockArgs()
|
||||
|
||||
|
||||
class BrainDiagnosticRunner:
|
||||
"""
|
||||
Professional diagnostic suite for Live integration testing of the
|
||||
Singularity LLM Cognitive Stack and Vector DB (Qdrant) persistence.
|
||||
Tested against heavy real-world XML dumps from Instagram.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.device = MockDevice()
|
||||
self.engine = TelepathicEngine.get_instance()
|
||||
self.mem_db = UIMemoryDB()
|
||||
|
||||
|
||||
# Test Namespaces
|
||||
self.intents = {
|
||||
"modal": "diagnostics_dismiss_obstacle",
|
||||
"ad": "diagnostics_find_sponsored",
|
||||
"hallucination": "diagnostics_tap_like_button",
|
||||
"unfollow": "diagnostics_tap_following_button"
|
||||
"unfollow": "diagnostics_tap_following_button",
|
||||
}
|
||||
|
||||
|
||||
# Load heavy real-world XML files
|
||||
self.fixtures_dir = os.path.join(ROOT_DIR, "tests", "fixtures")
|
||||
self.xmls = {
|
||||
"modal": self._load_fixture("blocked_ui.xml"),
|
||||
"ad": self._load_fixture("peugeot_ad.xml"),
|
||||
"hallucination": self._load_fixture("vlm_hallucination.xml"),
|
||||
"unfollow": self._load_fixture("unfollow_list_dump.xml")
|
||||
"unfollow": self._load_fixture("unfollow_list_dump.xml"),
|
||||
}
|
||||
|
||||
def _load_fixture(self, filename) -> str:
|
||||
@@ -91,7 +103,7 @@ class BrainDiagnosticRunner:
|
||||
if not self.mem_db.is_connected:
|
||||
logger.error("❌ Qdrant is offline! Diagnostics cannot proceed.")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
print(colored("🧹 Initializing diagnostic namespace (clearing old cache)...", "yellow"))
|
||||
for intent in self.intents.values():
|
||||
pt_id = self.mem_db._deterministic_id(intent)
|
||||
@@ -111,20 +123,20 @@ class BrainDiagnosticRunner:
|
||||
xml = self.xmls["modal"]
|
||||
intent = self.intents["modal"]
|
||||
node = self.engine.find_best_node(xml, intent, min_confidence=0.8, device=self.device)
|
||||
|
||||
|
||||
if not node:
|
||||
print(colored(" ❌ LLM failed to find the dismiss button entirely.", "red"))
|
||||
return {"passed": False, "reason": "No node found"}
|
||||
|
||||
|
||||
semantic = str(node.get("semantic", "")).lower()
|
||||
if "try again later" in semantic or "action block" in semantic:
|
||||
print(colored(" ❌ LLM selected the title text instead of the dismiss button.", "red"))
|
||||
return {"passed": False, "reason": "Selected title instead of button"}
|
||||
|
||||
|
||||
if "dismiss" in semantic or "ok" in semantic:
|
||||
print(colored(f" ✅ VLM correctly reasoned the popup OK/Dismiss button: {semantic}", "green"))
|
||||
return {"passed": True, "reason": f"Found correct button: {semantic}"}
|
||||
|
||||
|
||||
return {"passed": False, "reason": f"Selected unrelated element: {semantic}"}
|
||||
|
||||
def test_ad_deception(self) -> dict:
|
||||
@@ -134,20 +146,21 @@ class BrainDiagnosticRunner:
|
||||
xml = self.xmls["ad"]
|
||||
intent = self.intents["ad"]
|
||||
node = self.engine.find_best_node(xml, intent, min_confidence=0.8, device=self.device)
|
||||
|
||||
|
||||
if not node:
|
||||
print(colored(" ❌ LLM failed to identify the sponsored indicator.", "red"))
|
||||
return {"passed": False, "reason": "Missed sponsored text"}
|
||||
|
||||
|
||||
semantic = str(node.get("semantic", "")).lower()
|
||||
if "sponsored" in semantic:
|
||||
print(colored(" ✅ VLM correctly identified the tiny 'Sponsored' label amidst a huge post.", "green"))
|
||||
|
||||
|
||||
# --- Test Fast Path Recall Sub-Scenario ---
|
||||
# Save it
|
||||
self.engine.confirm_click(intent)
|
||||
self.mem_db.store_memory(intent, xml, node)
|
||||
import time
|
||||
|
||||
time.sleep(0.5)
|
||||
# Try to grab it again
|
||||
start = time.time()
|
||||
@@ -159,7 +172,7 @@ class BrainDiagnosticRunner:
|
||||
else:
|
||||
print(colored(" ❌ [Sub-Test] Memory recall failed.", "red"))
|
||||
return {"passed": False, "reason": "Found ad, but memory persistence failed."}
|
||||
|
||||
|
||||
return {"passed": False, "reason": f"Picked wrong node: {semantic}"}
|
||||
|
||||
def test_vlm_hallucination(self) -> dict:
|
||||
@@ -169,63 +182,66 @@ class BrainDiagnosticRunner:
|
||||
xml = self.xmls["hallucination"]
|
||||
intent = self.intents["hallucination"]
|
||||
node = self.engine.find_best_node(xml, intent, min_confidence=0.8, device=self.device)
|
||||
|
||||
|
||||
if not node:
|
||||
print(colored(" ❌ LLM failed to find any like button.", "red"))
|
||||
return {"passed": False, "reason": "No node found"}
|
||||
|
||||
|
||||
semantic = str(node.get("semantic", "")).lower()
|
||||
|
||||
|
||||
is_caption = ("double tap" in semantic or "like" in semantic) and "row feed button" not in semantic
|
||||
if is_caption:
|
||||
print(colored(" ❌ LLM fell for the semantic hallucination gap and selected the text caption!", "red"))
|
||||
return {"passed": False, "reason": "Fell for caption text trap"}
|
||||
|
||||
|
||||
if "row feed button like" in semantic or "heart" in semantic:
|
||||
print(colored(" ✅ VLM successfully ignored the deceptive caption and found the structural like button.", "green"))
|
||||
print(
|
||||
colored(
|
||||
" ✅ VLM successfully ignored the deceptive caption and found the structural like button.", "green"
|
||||
)
|
||||
)
|
||||
return {"passed": True, "reason": "Ignored text trap, clicked structural button"}
|
||||
|
||||
|
||||
return {"passed": False, "reason": f"Picked unrelated node: {semantic}"}
|
||||
|
||||
def execute_all(self):
|
||||
self.setup()
|
||||
results = {
|
||||
"timestamp": time.time(),
|
||||
"model": self.device.args.ai_telepathic_model,
|
||||
"scenarios": {}
|
||||
}
|
||||
|
||||
results = {"timestamp": time.time(), "model": self.device.args.ai_telepathic_model, "scenarios": {}}
|
||||
|
||||
def run_and_log(name, func):
|
||||
print(colored(f"\n--- SCENARIO: {name} ---", "magenta"))
|
||||
start_time = time.time()
|
||||
data = {"passed": False, "reason": "Unknown error", "latency_ms": 0}
|
||||
try:
|
||||
res = func()
|
||||
if isinstance(res, dict): data.update(res)
|
||||
elif res is True: data["passed"] = True
|
||||
if isinstance(res, dict):
|
||||
data.update(res)
|
||||
elif res is True:
|
||||
data["passed"] = True
|
||||
except Exception as e:
|
||||
print(colored(f"❌ EXCEPTION: {e}", "red"))
|
||||
data["reason"] = str(e)
|
||||
|
||||
|
||||
dur = time.time() - start_time
|
||||
data["latency_ms"] = int(dur * 1000)
|
||||
results["scenarios"][name] = data
|
||||
|
||||
|
||||
if data["passed"]:
|
||||
print(colored(f"🏁 {name} completed successfully in {dur:.2f}s", "green"))
|
||||
else:
|
||||
print(colored(f"🚨 {name} FAILED! (Elapsed: {dur:.2f}s)", "red", attrs=["bold"]))
|
||||
|
||||
|
||||
run_and_log("The Modal Trap (Blocked UI)", self.test_modal_trap)
|
||||
run_and_log("The Ad Deception (Sponsored)", self.test_ad_deception)
|
||||
run_and_log("The VLM Hallucination Gap (Text Trap)", self.test_vlm_hallucination)
|
||||
self.teardown()
|
||||
|
||||
|
||||
out_path = os.path.join(ROOT_DIR, "benchmarks", "data", "live_learning_results.json")
|
||||
with open(out_path, "w") as f:
|
||||
json.dump(results, f, indent=4)
|
||||
print(colored(f"\n📄 Saved intensive learning results to: {out_path}", "cyan", attrs=["bold"]))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
runner = BrainDiagnosticRunner()
|
||||
runner.execute_all()
|
||||
|
||||
@@ -1,19 +1,23 @@
|
||||
import os
|
||||
import sys
|
||||
import json
|
||||
import time
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime
|
||||
|
||||
# Add root project path so we can import internal modules safely
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from GramAddict.core.llm_provider import query_telepathic_llm
|
||||
from GramAddict.core.llm_provider import query_llm, query_telepathic_llm
|
||||
|
||||
BENCHMARKS_FILE = os.path.join(os.path.dirname(__file__), "data/llm_benchmarks.json")
|
||||
SCENARIOS_FILE = os.path.join(os.path.dirname(__file__), "data/benchmark_scenarios.json")
|
||||
|
||||
# Minimum iterations for statistical significance
|
||||
MIN_ITERATIONS = 5
|
||||
|
||||
|
||||
def load_json(path):
|
||||
if os.path.exists(path):
|
||||
try:
|
||||
@@ -23,44 +27,49 @@ def load_json(path):
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def save_json(path, data):
|
||||
with open(path, "w") as f:
|
||||
json.dump(data, f, indent=4)
|
||||
|
||||
|
||||
def normalize_scores(db):
|
||||
"""Normalize relative performance by AVERAGE score per scenario, not raw totals."""
|
||||
if not db.get("models"):
|
||||
return db
|
||||
|
||||
# 1. Find the highest raw score across all models
|
||||
max_raw = 0
|
||||
|
||||
max_avg = 0
|
||||
leader_model = None
|
||||
|
||||
|
||||
for name, data in db["models"].items():
|
||||
if data.get("is_unsuitable"):
|
||||
continue
|
||||
|
||||
raw = data.get("raw_score", 0)
|
||||
if raw > max_raw:
|
||||
max_raw = raw
|
||||
|
||||
scenario_count = data.get("scenario_count", 1)
|
||||
avg = data.get("raw_score", 0) / max(scenario_count, 1)
|
||||
data["avg_score_per_scenario"] = round(avg, 1)
|
||||
|
||||
if avg > max_avg:
|
||||
max_avg = avg
|
||||
leader_model = name
|
||||
elif raw == max_raw and max_raw > 0:
|
||||
# Tie-breaker: Latency
|
||||
elif avg == max_avg and max_avg > 0:
|
||||
current_lat = data.get("latency_ms", 99999)
|
||||
leader_lat = db["models"][leader_model].get("latency_ms", 99999)
|
||||
if current_lat < leader_lat:
|
||||
leader_model = name
|
||||
|
||||
if max_raw == 0:
|
||||
if max_avg == 0:
|
||||
return db
|
||||
|
||||
# 2. Update relative performance
|
||||
for name, data in db["models"].items():
|
||||
raw = data.get("raw_score", 0)
|
||||
data["relative_performance_pct"] = round((raw / max_raw) * 100, 1)
|
||||
data["is_leader"] = (name == leader_model)
|
||||
|
||||
scenario_count = data.get("scenario_count", 1)
|
||||
avg = data.get("raw_score", 0) / max(scenario_count, 1)
|
||||
data["relative_performance_pct"] = round((avg / max_avg) * 100, 1)
|
||||
data["is_leader"] = name == leader_model
|
||||
|
||||
return db
|
||||
|
||||
|
||||
def get_installed_ollama_models():
|
||||
"""
|
||||
Finds truly local Ollama models by parsing 'ollama list'.
|
||||
@@ -71,31 +80,150 @@ def get_installed_ollama_models():
|
||||
models = []
|
||||
for line in output.split("\n")[1:]:
|
||||
if line.strip():
|
||||
# Format: NAME, ID, SIZE, MODIFIED
|
||||
parts = line.split()
|
||||
if len(parts) >= 3:
|
||||
name = parts[0]
|
||||
size = parts[2]
|
||||
|
||||
# 1. Skip if size is '-' (remote/cloud model)
|
||||
|
||||
if size == "-":
|
||||
continue
|
||||
|
||||
# 2. Skip ':cloud' tagged models explicitly
|
||||
if ":cloud" in name:
|
||||
continue
|
||||
|
||||
# 3. Filter out purely embedding models
|
||||
if any(k in name.lower() for k in ["embed", "minilm", "rerank"]):
|
||||
continue
|
||||
|
||||
|
||||
models.append(name)
|
||||
return models
|
||||
except Exception as e:
|
||||
print(f"⚠️ Could not list Ollama models: {e}")
|
||||
return []
|
||||
|
||||
def benchmark_model(model_name: str, url: str, force: bool = False):
|
||||
|
||||
def _run_telepathic_scenario(scenario, model_name, url, iterations):
|
||||
"""Run a telepathic (JSON element selection) scenario."""
|
||||
system_prompt = (
|
||||
"You identify which UI element to tap based ONLY on a JSON array of parsed Android elements. "
|
||||
'Output ONLY valid JSON: {"index": number, "reason": "brief reason"}'
|
||||
)
|
||||
|
||||
user_prompt = (
|
||||
f"Which element should I tap to: {scenario['task']}\n\n"
|
||||
f"Elements:\n{json.dumps(scenario['nodes'], indent=1)}\n\n"
|
||||
"Rules:\n"
|
||||
"- Pick the SMALLEST, most specific button or icon\n"
|
||||
"- NEVER pick large containers\n"
|
||||
'Return: {"index": number, "reason": "..."}'
|
||||
)
|
||||
|
||||
latencies = []
|
||||
scores = []
|
||||
successes = 0
|
||||
|
||||
for _ in range(iterations):
|
||||
start_time = time.time()
|
||||
try:
|
||||
resp_str = query_telepathic_llm(model_name, url, system_prompt, user_prompt)
|
||||
latency = int((time.time() - start_time) * 1000)
|
||||
latencies.append(latency)
|
||||
except Exception as e:
|
||||
print(f" ❌ API Request failed: {e}")
|
||||
scores.append(0)
|
||||
continue
|
||||
|
||||
raw_points = 0
|
||||
try:
|
||||
clean = resp_str.strip()
|
||||
if clean.startswith("```json"):
|
||||
clean = clean[7:]
|
||||
if clean.endswith("```"):
|
||||
clean = clean[:-3]
|
||||
data = json.loads(clean)
|
||||
|
||||
if "index" in data and "reason" in data:
|
||||
raw_points += 40
|
||||
if data["index"] == scenario["target_index"]:
|
||||
raw_points += 60
|
||||
successes += 1
|
||||
else:
|
||||
print(f" ❌ Wrong index ({data.get('index')}). Target was {scenario['target_index']}.")
|
||||
else:
|
||||
print(" ❌ JSON missing fields.")
|
||||
except Exception:
|
||||
print(" ❌ JSON Parsing failed.")
|
||||
|
||||
scores.append(raw_points)
|
||||
|
||||
return scores, latencies, successes
|
||||
|
||||
|
||||
def _run_brain_scenario(scenario, model_name, url, iterations):
|
||||
"""Run a brain action extraction scenario (format_json=False)."""
|
||||
system_prompt = (
|
||||
f"You are an autonomous Instagram agent. Your goal is: '{scenario['task']}'.\n"
|
||||
f"You are currently on screen: {scenario['screen_type']}.\n"
|
||||
f"Available actions: {scenario['available_actions']}\n"
|
||||
"INSTRUCTIONS: Reply with ONLY the action string. Nothing else."
|
||||
)
|
||||
|
||||
user_prompt = "Choose the next best action."
|
||||
|
||||
latencies = []
|
||||
scores = []
|
||||
successes = 0
|
||||
|
||||
for _ in range(iterations):
|
||||
start_time = time.time()
|
||||
try:
|
||||
# CRITICAL: Use format_json=False — this is the Brain code path
|
||||
ans = query_llm(
|
||||
url=url,
|
||||
model=model_name,
|
||||
prompt=user_prompt,
|
||||
system=system_prompt,
|
||||
format_json=False,
|
||||
timeout=30,
|
||||
temperature=0.0,
|
||||
max_tokens=50,
|
||||
)
|
||||
latency = int((time.time() - start_time) * 1000)
|
||||
latencies.append(latency)
|
||||
except Exception as e:
|
||||
print(f" ❌ API Request failed: {e}")
|
||||
scores.append(0)
|
||||
continue
|
||||
|
||||
raw_points = 0
|
||||
if ans and "response" in ans:
|
||||
response = ans["response"].strip().lower()
|
||||
|
||||
# Points for structural adherence (returned a clean string)
|
||||
if response and response in [a.lower() for a in scenario["available_actions"]]:
|
||||
raw_points += 40
|
||||
|
||||
# Points for correctness
|
||||
if scenario.get("accept_any_valid"):
|
||||
# Any valid action from the list is acceptable
|
||||
raw_points += 60
|
||||
successes += 1
|
||||
elif response == scenario["target_action"].lower():
|
||||
raw_points += 60
|
||||
successes += 1
|
||||
else:
|
||||
print(f" ⚠️ Valid but suboptimal: '{response}' (target: '{scenario['target_action']}')")
|
||||
raw_points += 20 # Partial credit for valid but wrong action
|
||||
else:
|
||||
print(f" ❌ Invalid response: '{response}' not in available actions")
|
||||
else:
|
||||
print(" ❌ Empty or null response from LLM")
|
||||
|
||||
scores.append(raw_points)
|
||||
|
||||
return scores, latencies, successes
|
||||
|
||||
|
||||
def benchmark_model(model_name: str, url: str, force: bool = False, iterations: int = MIN_ITERATIONS):
|
||||
iterations = max(iterations, MIN_ITERATIONS) # Enforce minimum
|
||||
|
||||
db = load_json(BENCHMARKS_FILE) or {"models": {}}
|
||||
scenarios_data = load_json(SCENARIOS_FILE)
|
||||
if not scenarios_data:
|
||||
@@ -105,107 +233,88 @@ def benchmark_model(model_name: str, url: str, force: bool = False):
|
||||
if not force and model_name in db.get("models", {}):
|
||||
pct = db["models"][model_name].get("relative_performance_pct", "N/A")
|
||||
if not db["models"][model_name].get("is_unsuitable"):
|
||||
print(f"Typical execution skip for {model_name} (Rel: {pct}%). Use --force.")
|
||||
return
|
||||
print(f"Typical execution skip for {model_name} (Rel: {pct}%). Use --force.")
|
||||
return
|
||||
|
||||
print(f"\n🚀 [Competitive Benchmarking] Model: {model_name} ({iterations} iterations)")
|
||||
|
||||
print(f"\n🚀 [Competitive Benchmarking] Model: {model_name}")
|
||||
|
||||
total_raw = 0
|
||||
total_latency = 0
|
||||
results_detail = {}
|
||||
passed_all = True
|
||||
|
||||
blank_b64 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII="
|
||||
system_prompt = (
|
||||
"You identify which UI element to tap based ONLY on a JSON array of parsed Android elements. "
|
||||
"Output ONLY valid JSON: {\"index\": number, \"reason\": \"brief reason\"}"
|
||||
)
|
||||
|
||||
scenarios = scenarios_data["scenarios"]
|
||||
for scenario in scenarios:
|
||||
print(f"--- Running: {scenario['name']} ---")
|
||||
|
||||
user_prompt = (
|
||||
f"Which element should I tap to: {scenario['task']}\n\n"
|
||||
f"Elements:\n{json.dumps(scenario['nodes'], indent=1)}\n\n"
|
||||
"Rules:\n"
|
||||
"- Pick the SMALLEST, most specific button or icon\n"
|
||||
"- NEVER pick large containers\n"
|
||||
"Return: {\"index\": number, \"reason\": \"...\"}"
|
||||
)
|
||||
scenario_type = scenario.get("type", "telepathic")
|
||||
print(f"--- [{scenario_type.upper()}] {scenario['name']} ---")
|
||||
|
||||
start_time = time.time()
|
||||
try:
|
||||
resp_str = query_telepathic_llm(model_name, url, system_prompt, user_prompt)
|
||||
latency = int((time.time() - start_time) * 1000)
|
||||
total_latency += latency
|
||||
except Exception as e:
|
||||
print(f" ❌ API Request failed for scenario {scenario['id']}: {e}")
|
||||
passed_all = False
|
||||
if scenario_type == "telepathic":
|
||||
scores, latencies, successes = _run_telepathic_scenario(scenario, model_name, url, iterations)
|
||||
elif scenario_type == "brain_action":
|
||||
scores, latencies, successes = _run_brain_scenario(scenario, model_name, url, iterations)
|
||||
else:
|
||||
print(f" ⚠️ Unknown scenario type: {scenario_type}")
|
||||
continue
|
||||
|
||||
raw_points = 0
|
||||
try:
|
||||
clean = resp_str.strip()
|
||||
if clean.startswith("```json"): clean = clean[7:]
|
||||
if clean.endswith("```"): clean = clean[:-3]
|
||||
data = json.loads(clean)
|
||||
|
||||
# Points for structural adherence
|
||||
if "index" in data and "reason" in data:
|
||||
raw_points += 40
|
||||
|
||||
# Points for correctness
|
||||
if data["index"] == scenario["target_index"]:
|
||||
raw_points += 60
|
||||
print(f" ✅ Correct index ({data['index']}).")
|
||||
else:
|
||||
passed_all = False
|
||||
print(f" ❌ Wrong index ({data['index']}). Target was {scenario['target_index']}.")
|
||||
else:
|
||||
passed_all = False
|
||||
print(" ❌ JSON missing fields.")
|
||||
except Exception:
|
||||
passed_all = False
|
||||
print(" ❌ JSON Parsing failed.")
|
||||
avg_score = int(sum(scores) / len(scores)) if scores else 0
|
||||
avg_latency = int(sum(latencies) / len(latencies)) if latencies else 0
|
||||
pass_rate = (successes / iterations) * 100
|
||||
|
||||
results_detail[scenario["id"]] = raw_points
|
||||
total_raw += raw_points
|
||||
if pass_rate < 100.0:
|
||||
passed_all = False
|
||||
|
||||
print(f" Result: {pass_rate:.0f}% Pass | Avg Score: {avg_score}/100 | Avg Latency: {avg_latency}ms")
|
||||
|
||||
# Consistent format: always an object
|
||||
results_detail[scenario["id"]] = {
|
||||
"avg_score": avg_score,
|
||||
"pass_rate": pass_rate,
|
||||
"latency": avg_latency,
|
||||
}
|
||||
total_raw += avg_score
|
||||
total_latency += avg_latency
|
||||
|
||||
avg_latency = total_latency // len(scenarios) if scenarios else 0
|
||||
print(f"\n📊 {model_name} Result: {'PASS' if passed_all else 'FAIL'} | Score: {total_raw} | Latency: {avg_latency}ms")
|
||||
|
||||
print(f"\n📊 {model_name}: {'PASS' if passed_all else 'FAIL'} | Total: {total_raw} | Latency: {avg_latency}ms")
|
||||
|
||||
if model_name not in db["models"]:
|
||||
db["models"][model_name] = {}
|
||||
|
||||
db["models"][model_name].update({
|
||||
"raw_score": total_raw,
|
||||
"telepathic_score": int((total_raw / (len(scenarios) * 100)) * 100) if scenarios else 0,
|
||||
"latency_ms": avg_latency,
|
||||
"last_tested": datetime.utcnow().isoformat() + "Z",
|
||||
"details": results_detail,
|
||||
"passed_all": passed_all,
|
||||
"is_unsuitable": not passed_all
|
||||
})
|
||||
|
||||
# Recalculate relative scores across all models
|
||||
|
||||
db["models"][model_name].update(
|
||||
{
|
||||
"raw_score": total_raw,
|
||||
"scenario_count": len(scenarios),
|
||||
"telepathic_score": int((total_raw / (len(scenarios) * 100)) * 100) if scenarios else 0,
|
||||
"latency_ms": avg_latency,
|
||||
"last_tested": datetime.utcnow().isoformat() + "Z",
|
||||
"details": results_detail,
|
||||
"passed_all": passed_all,
|
||||
"is_unsuitable": not passed_all,
|
||||
"iterations": iterations,
|
||||
}
|
||||
)
|
||||
|
||||
db = normalize_scores(db)
|
||||
save_json(BENCHMARKS_FILE, db)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
from GramAddict.core.config import Config
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser(description="Competitive Benchmark for Singularity", add_help=False)
|
||||
parser.add_argument("--config", type=str, help="Bot config file")
|
||||
parser.add_argument("--model", type=str, help="Explicit model name")
|
||||
parser.add_argument("--url", type=str, help="Explicit endpoint URL")
|
||||
parser.add_argument("--force", action="store_true", help="Force re-testing")
|
||||
parser.add_argument("--all-ollama", action="store_true", help="Automatically find and test all local Ollama models")
|
||||
|
||||
parser.add_argument(
|
||||
"--iterations", type=int, default=MIN_ITERATIONS, help=f"Iterations per scenario (min: {MIN_ITERATIONS})"
|
||||
)
|
||||
|
||||
args, unknown = parser.parse_known_args()
|
||||
|
||||
|
||||
models_to_test = []
|
||||
|
||||
|
||||
if args.all_ollama:
|
||||
ollama_models = get_installed_ollama_models()
|
||||
for m in ollama_models:
|
||||
@@ -215,8 +324,12 @@ if __name__ == "__main__":
|
||||
elif args.config:
|
||||
configs = Config(first_run=True, config=args.config)
|
||||
configs.parse_args()
|
||||
|
||||
for attr, pref in [("ai_telepathic_model", "ai_telepathic_url"), ("ai_model", "ai_model_url"), ("ai_condenser_model", "ai_condenser_url")]:
|
||||
|
||||
for attr, pref in [
|
||||
("ai_telepathic_model", "ai_telepathic_url"),
|
||||
("ai_model", "ai_model_url"),
|
||||
("ai_condenser_model", "ai_condenser_url"),
|
||||
]:
|
||||
m = getattr(configs.args, attr, None)
|
||||
u = getattr(configs.args, pref, "http://localhost:11434/api/generate")
|
||||
if m:
|
||||
@@ -224,7 +337,7 @@ if __name__ == "__main__":
|
||||
else:
|
||||
print("❌ Syntax: --all-ollama OR --config test_config.yml OR --model x --url y")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
for m, u in set(models_to_test):
|
||||
benchmark_model(m, u, args.force)
|
||||
benchmark_model(m, u, args.force, args.iterations)
|
||||
time.sleep(1)
|
||||
|
||||
@@ -89,6 +89,11 @@ telegram-reports: false # for using telegram-reports you have also to configure
|
||||
interactions-count: 30-40
|
||||
likes-count: 1-2
|
||||
likes-percentage: 100
|
||||
|
||||
plugins:
|
||||
dm_reply:
|
||||
enabled: false # Generates AI replies to unread DMs
|
||||
|
||||
stories-count: 1-2
|
||||
stories-percentage: 30-40
|
||||
carousel-count: 2-3
|
||||
|
||||
@@ -1,13 +0,0 @@
|
||||
from unittest.mock import MagicMock
|
||||
from GramAddict.core.q_nav_graph import QNavGraph
|
||||
|
||||
mock_device = MagicMock()
|
||||
mock_device._get_current_app.return_value = "com.android.vending"
|
||||
|
||||
mock_engine = MagicMock()
|
||||
mock_engine.find_best_node.return_value = {"x": 50, "y": 50, "semantic_string": "fake profile link", "source": "vlm"}
|
||||
|
||||
nav_graph = QNavGraph(mock_device)
|
||||
print(nav_graph._execute_transition("tap_post_username", zero_engine=mock_engine))
|
||||
print(mock_engine.find_best_node.called)
|
||||
|
||||
@@ -3,8 +3,8 @@ requires = ["flit_core >=3.2,<4"]
|
||||
build-backend = "flit_core.buildapi"
|
||||
|
||||
[project]
|
||||
name = "GramAddict"
|
||||
authors = [{ name = "GramAddict Team", email = "maintainers@gramaddict.org" }]
|
||||
name = "GramPilot"
|
||||
authors = [{ name = "Marc Mintel", email = "marc@mintel.me" }]
|
||||
readme = "README.md"
|
||||
classifiers = [
|
||||
"License :: Free for non-commercial use",
|
||||
@@ -12,29 +12,72 @@ classifiers = [
|
||||
"Programming Language :: Python :: 3"
|
||||
]
|
||||
license = { file = "LICENSE" }
|
||||
requires-python = ">=3.6"
|
||||
requires-python = ">=3.10"
|
||||
dynamic = ["version", "description"]
|
||||
dependencies = [
|
||||
"colorama==0.4.4",
|
||||
"ConfigArgParse==1.5.3",
|
||||
"ConfigArgParse==1.7",
|
||||
"PyYAML==6.0.1",
|
||||
"uiautomator2==2.16.14",
|
||||
"urllib3==1.26.18",
|
||||
"emoji==1.6.1",
|
||||
"uiautomator2>=3.0.0",
|
||||
"urllib3>=2.0.0",
|
||||
"emoji==2.12.1",
|
||||
"langdetect==1.0.9",
|
||||
"atomicwrites==1.4.0",
|
||||
"atomicwrites==1.4.1",
|
||||
"spintax==1.0.4",
|
||||
"requests~=2.31.0",
|
||||
"packaging~=20.9"
|
||||
"requests>=2.31.0",
|
||||
"packaging>=23.0",
|
||||
"python-dotenv==1.0.1",
|
||||
"qdrant-client>=1.7.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
analytics = ["matplotlib==3.4.2"]
|
||||
dev = ["flit", "pre-commit", "black", "flake8", "isort", "ruff", "pytest", "pytest-mock", "pytest-asyncio"]
|
||||
analytics = ["matplotlib>=3.8.0"]
|
||||
dev = [
|
||||
"flit",
|
||||
"pre-commit",
|
||||
"ruff",
|
||||
"pytest",
|
||||
"pytest-mock",
|
||||
"pytest-asyncio",
|
||||
"pytest-cov",
|
||||
"hypothesis",
|
||||
"diff-cover",
|
||||
]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
python_files = "test_*.py"
|
||||
addopts = "--strict-markers"
|
||||
markers = [
|
||||
"live: tests requiring a live ADB device",
|
||||
"chaos: chaos engineering / corruption tests",
|
||||
"property: hypothesis property-based tests",
|
||||
"live_llm: tests requiring a live local LLM via Ollama",
|
||||
]
|
||||
|
||||
[tool.coverage.run]
|
||||
source = ["GramAddict"]
|
||||
omit = ["GramAddict/plugins/*", "*/test_*"]
|
||||
|
||||
[tool.coverage.report]
|
||||
fail_under = 25
|
||||
show_missing = true
|
||||
exclude_lines = [
|
||||
"pragma: no cover",
|
||||
"if __name__ == .__main__.",
|
||||
"raise NotImplementedError",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py310"
|
||||
line-length = 120
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "W", "I"]
|
||||
ignore = ["E501"]
|
||||
|
||||
[project.urls]
|
||||
Documentation = "https://docs.gramaddict.org/#/"
|
||||
Source = "https://github.com/GramAddict/bot"
|
||||
Source = "https://github.com/marcmintel/grampilot"
|
||||
|
||||
[project.scripts]
|
||||
gramaddict = "GramAddict.__main__:main"
|
||||
grampilot = "GramAddict.__main__:main"
|
||||
|
||||
@@ -1,807 +0,0 @@
|
||||
============================= test session starts ==============================
|
||||
platform darwin -- Python 3.9.6, pytest-8.3.5, pluggy-1.5.0
|
||||
rootdir: /Volumes/Alpha SSD/Coding/bot
|
||||
configfile: pyproject.toml
|
||||
plugins: asyncio-0.23.5, cov-7.1.0, anyio-3.7.1, mock-3.14.0, xdist-3.6.1
|
||||
asyncio: mode=strict
|
||||
collected 152 items
|
||||
|
||||
tests/anomalies/test_bot_flow_edge_cases.py ... [ 1%]
|
||||
tests/anomalies/test_cognitive_edge_cases.py ... [ 3%]
|
||||
tests/anomalies/test_fsd_recovery.py F [ 4%]
|
||||
tests/anomalies/test_hardware_anomalies.py EEEE. [ 7%]
|
||||
tests/anomalies/test_hardware_edge_cases.py .. [ 9%]
|
||||
tests/anomalies/test_human_hesitation.py .. [ 10%]
|
||||
tests/anomalies/test_llm_hallucination_recovery.py .. [ 11%]
|
||||
tests/anomalies/test_nav_failure_tdd.py . [ 12%]
|
||||
tests/anomalies/test_nav_graph_edge_cases.py ... [ 14%]
|
||||
tests/anomalies/test_xml_dumps_fuzz.py s [ 15%]
|
||||
tests/integration/test_ad_detection.py FFF [ 17%]
|
||||
tests/integration/test_bot_flow_interaction.py ..........F.. [ 25%]
|
||||
tests/integration/test_bot_flow_start.py F [ 26%]
|
||||
tests/integration/test_cognitive_integration.py FF.F [ 28%]
|
||||
tests/integration/test_cognitive_stack_audit.py ....... [ 33%]
|
||||
tests/integration/test_darwin_engine.py .... [ 36%]
|
||||
tests/integration/test_deep_engagement.py s.. [ 38%]
|
||||
tests/integration/test_device_facade_full.py ........... [ 45%]
|
||||
tests/integration/test_dm_loop.py .. [ 46%]
|
||||
tests/integration/test_false_positive.py F [ 47%]
|
||||
tests/integration/test_llm_provider_full.py ....... [ 51%]
|
||||
tests/integration/test_q_nav_graph.py ... [ 53%]
|
||||
tests/integration/test_qdrant_memory_full.py ............ [ 61%]
|
||||
tests/integration/test_resonance_engine.py ....... [ 66%]
|
||||
tests/integration/test_scenarios_fsd.py EE [ 67%]
|
||||
tests/integration/test_swarm_protocol.py F... [ 70%]
|
||||
tests/integration/test_telepathic_edge_cases.py ...... [ 74%]
|
||||
tests/integration/test_telepathic_engine_extraction.py FFFFFEEFFFFFF [ 82%]
|
||||
tests/integration/test_telepathic_engine_vlm.py ...................... [ 97%]
|
||||
tests/integration/test_telepathic_keyword.py . [ 98%]
|
||||
tests/integration/test_unfollow_loop.py ... [100%]
|
||||
|
||||
==================================== ERRORS ====================================
|
||||
______________ ERROR at setup of test_slow_loading_post_recovery _______________
|
||||
|
||||
@pytest.fixture
|
||||
def test_dumps():
|
||||
dumps = {}
|
||||
> with open(DUMPS["organic"], "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/organic_post.xml'
|
||||
|
||||
tests/anomalies/test_hardware_anomalies.py:39: FileNotFoundError
|
||||
____________ ERROR at setup of test_wait_timeout_aborts_gracefully _____________
|
||||
|
||||
@pytest.fixture
|
||||
def test_dumps():
|
||||
dumps = {}
|
||||
> with open(DUMPS["organic"], "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/organic_post.xml'
|
||||
|
||||
tests/anomalies/test_hardware_anomalies.py:39: FileNotFoundError
|
||||
____________ ERROR at setup of test_empty_content_extraction_guard _____________
|
||||
|
||||
@pytest.fixture
|
||||
def test_dumps():
|
||||
dumps = {}
|
||||
> with open(DUMPS["organic"], "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/organic_post.xml'
|
||||
|
||||
tests/anomalies/test_hardware_anomalies.py:39: FileNotFoundError
|
||||
______________ ERROR at setup of test_missing_feed_markers_guard _______________
|
||||
|
||||
@pytest.fixture
|
||||
def test_dumps():
|
||||
dumps = {}
|
||||
> with open(DUMPS["organic"], "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/organic_post.xml'
|
||||
|
||||
tests/anomalies/test_hardware_anomalies.py:39: FileNotFoundError
|
||||
____________ ERROR at setup of test_full_mission_autopilot_sequence ____________
|
||||
|
||||
@pytest.fixture
|
||||
def fsd_fixtures():
|
||||
def _load(name):
|
||||
with open(os.path.join(FIX_DIR, name), "r") as f:
|
||||
return f.read()
|
||||
return {
|
||||
> "organic": _load("organic_post.xml"),
|
||||
"ad": _load("sponsored_reel.xml"),
|
||||
"modal": _load("survey_modal.xml")
|
||||
}
|
||||
|
||||
tests/integration/test_scenarios_fsd.py:64:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'organic_post.xml'
|
||||
|
||||
def _load(name):
|
||||
> with open(os.path.join(FIX_DIR, name), "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/organic_post.xml'
|
||||
|
||||
tests/integration/test_scenarios_fsd.py:61: FileNotFoundError
|
||||
_________________ ERROR at setup of test_feed_loop_chaos_mode __________________
|
||||
|
||||
@pytest.fixture
|
||||
def fsd_fixtures():
|
||||
def _load(name):
|
||||
with open(os.path.join(FIX_DIR, name), "r") as f:
|
||||
return f.read()
|
||||
return {
|
||||
> "organic": _load("organic_post.xml"),
|
||||
"ad": _load("sponsored_reel.xml"),
|
||||
"modal": _load("survey_modal.xml")
|
||||
}
|
||||
|
||||
tests/integration/test_scenarios_fsd.py:64:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'organic_post.xml'
|
||||
|
||||
def _load(name):
|
||||
> with open(os.path.join(FIX_DIR, name), "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/organic_post.xml'
|
||||
|
||||
tests/integration/test_scenarios_fsd.py:61: FileNotFoundError
|
||||
_ ERROR at setup of TestSafetyGuard.test_real_explore_fullscreen_container_rejected _
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestSafetyGuard object at 0x108e59eb0>
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def setup_real_nodes(self):
|
||||
"""Pre-parse real XML nodes BEFORE any mocking happens."""
|
||||
engine = TelepathicEngine()
|
||||
> explore_xml = load_fixture("explore_feed.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:140:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'explore_feed.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/explore_feed.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
------------------------------ Captured log setup ------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'telepathic_engine_cache': collection has <MagicMock name='mock.QdrantClient().get_collection().config.params.vectors.size' id='4446345248'>, expected 768. Recreating collection...
|
||||
___ ERROR at setup of TestSafetyGuard.test_real_explore_like_button_accepted ___
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestSafetyGuard object at 0x108e6c100>
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def setup_real_nodes(self):
|
||||
"""Pre-parse real XML nodes BEFORE any mocking happens."""
|
||||
engine = TelepathicEngine()
|
||||
> explore_xml = load_fixture("explore_feed.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:140:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'explore_feed.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/explore_feed.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
------------------------------ Captured log setup ------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'telepathic_engine_cache': collection has <MagicMock name='mock.QdrantClient().get_collection().config.params.vectors.size' id='4446345248'>, expected 768. Recreating collection...
|
||||
=================================== FAILURES ===================================
|
||||
___________________ test_fsd_handles_persistent_survey_modal ___________________
|
||||
|
||||
def test_fsd_handles_persistent_survey_modal():
|
||||
"""
|
||||
Simulates a case where the bot gets stuck on a survey modal.
|
||||
The FSD (Full Self Driving) anomaly handler should trigger,
|
||||
detect that 'Back' didn't work, and engage TelepathicEngine
|
||||
to find and tap the 'Not Now' or 'Dismiss' button.
|
||||
"""
|
||||
from GramAddict.core.bot_flow import _run_zero_latency_feed_loop
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
device = MagicMock()
|
||||
device.app_id = "com.instagram.android"
|
||||
device._get_current_app.return_value = "com.instagram.android"
|
||||
configs = ConfigMock()
|
||||
|
||||
# Mock the TelepathicEngine singleton behavior entirely
|
||||
mock_telepathic = MagicMock()
|
||||
mock_telepathic.find_best_node.return_value = {"x": 500, "y": 1400, "semantic": "Not Now"}
|
||||
mock_telepathic._extract_semantic_nodes.return_value = [{"x": 10}]
|
||||
|
||||
dopamine = MagicMock()
|
||||
dopamine.is_app_session_over.side_effect = [False, False, True] # Run twice, then exit
|
||||
dopamine.wants_to_change_feed.return_value = False
|
||||
dopamine.wants_to_doomscroll.return_value = False
|
||||
|
||||
ai = MagicMock()
|
||||
ai.get_sleep_modifier.return_value = 1.0
|
||||
cognitive_stack = {"dopamine": dopamine, "growth_brain": None, "active_inference": ai, "telepathic": mock_telepathic}
|
||||
|
||||
# Load the mock survey modal UI
|
||||
xml_path = os.path.join(FIXTURE_DIR, "survey_modal.xml")
|
||||
> with open(xml_path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/survey_modal.xml'
|
||||
|
||||
tests/anomalies/test_fsd_recovery.py:46: FileNotFoundError
|
||||
________________ test_real_sponsored_reel_flexcode_is_detected _________________
|
||||
|
||||
def test_real_sponsored_reel_flexcode_is_detected():
|
||||
"""
|
||||
Test: The manual_interrupt dump is a sponsored Reel (flexcode_systems).
|
||||
_detect_ad_structural MUST return True.
|
||||
"""
|
||||
xml_path = os.path.join(FIX_DIR, "sponsored_reel.xml")
|
||||
> with open(xml_path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/sponsored_reel.xml'
|
||||
|
||||
tests/integration/test_ad_detection.py:13: FileNotFoundError
|
||||
___________________________ test_normal_post_not_ad ____________________________
|
||||
|
||||
def test_normal_post_not_ad():
|
||||
"""
|
||||
Test: The manual_interrupt dump is a normal post.
|
||||
_detect_ad_structural MUST return False to avoid false positives.
|
||||
"""
|
||||
xml_path = os.path.join(FIX_DIR, "organic_post.xml")
|
||||
> with open(xml_path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/organic_post.xml'
|
||||
|
||||
tests/integration/test_ad_detection.py:24: FileNotFoundError
|
||||
_____________________ test_peugeot_carousel_ad_is_detected _____________________
|
||||
|
||||
def test_peugeot_carousel_ad_is_detected():
|
||||
"""
|
||||
Test: The 'peugeot.deutschland' carousel ad from manual_interrupt dump.
|
||||
_detect_ad_structural MUST return True.
|
||||
"""
|
||||
xml_path = os.path.join(FIX_DIR, "peugeot_ad.xml")
|
||||
> with open(xml_path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/peugeot_ad.xml'
|
||||
|
||||
tests/integration/test_ad_detection.py:36: FileNotFoundError
|
||||
___________________________ test_start_bot_interrupt ___________________________
|
||||
|
||||
def test_start_bot_interrupt():
|
||||
from GramAddict.core.bot_flow import start_bot
|
||||
|
||||
# Mock all the heavy initialization
|
||||
with patch('GramAddict.core.bot_flow.Config') as MockConfig, \
|
||||
patch('GramAddict.core.bot_flow.configure_logger'), \
|
||||
patch('GramAddict.core.bot_flow.check_if_updated'), \
|
||||
patch('GramAddict.core.benchmark_guard.check_model_benchmarks'), \
|
||||
patch('GramAddict.core.llm_provider.log_openrouter_burn'), \
|
||||
patch('GramAddict.core.bot_flow.create_device') as mock_create_device, \
|
||||
patch('GramAddict.core.bot_flow.set_time_delta') as mock_time_delta, \
|
||||
patch('GramAddict.core.bot_flow.SessionState') as MockSession, \
|
||||
patch('GramAddict.core.bot_flow.open_instagram', side_effect=KeyboardInterrupt()), \
|
||||
patch('GramAddict.core.bot_flow.dump_ui_state') as mock_dump:
|
||||
|
||||
MockConfig.return_value.args.feed = True
|
||||
MockConfig.return_value.args.explore = False
|
||||
MockConfig.return_value.args.reels = False
|
||||
MockConfig.return_value.args.stories = False
|
||||
MockConfig.return_value.args.working_hours = [10, 20]
|
||||
MockConfig.return_value.args.time_delta_session = 30
|
||||
|
||||
MockSession.inside_working_hours.return_value = (True, 0)
|
||||
|
||||
with pytest.raises(KeyboardInterrupt):
|
||||
> start_bot(username="test", device_id="123")
|
||||
E Failed: DID NOT RAISE <class 'KeyboardInterrupt'>
|
||||
|
||||
tests/integration/test_bot_flow_interaction.py:190: Failed
|
||||
----------------------------- Captured stdout call -----------------------------
|
||||
|
||||
==================================================
|
||||
🤖 MANUAL E2E DUMP CAPTURE SEQUENCE
|
||||
==================================================
|
||||
Please follow the instructions below to capture the required fixtures.
|
||||
If an IG update changed the layout, you can navigate there naturally.
|
||||
==================================================
|
||||
|
||||
|
||||
👉 1. COMMENT SHEET:
|
||||
Open Instagram, scroll to any post on the HomeFeed, and open the comment section.
|
||||
When the comment sheet is fully visible, press ENTER to capture...
|
||||
------------------------------ Captured log call -------------------------------
|
||||
ERROR GramAddict.core.dump_capturer:dump_capturer.py:105 💥 Capture Sequence crashed: pytest: reading from stdin while output is captured! Consider using `-s`.
|
||||
Traceback (most recent call last):
|
||||
File "/Volumes/Alpha SSD/Coding/bot/GramAddict/core/dump_capturer.py", line 43, in capture_all
|
||||
input("\n👉 1. COMMENT SHEET:\nOpen Instagram, scroll to any post on the HomeFeed, and open the comment section.\nWhen the comment sheet is fully visible, press ENTER to capture...")
|
||||
File "/Users/marcmintel/Library/Python/3.9/lib/python/site-packages/_pytest/capture.py", line 227, in read
|
||||
raise OSError(
|
||||
OSError: pytest: reading from stdin while output is captured! Consider using `-s`.
|
||||
__________________________ test_start_bot_normal_flow __________________________
|
||||
|
||||
MockConfig = <MagicMock name='Config' id='4457320064'>
|
||||
mock_logger = <MagicMock name='configure_logger' id='4458406768'>
|
||||
mock_update = <MagicMock name='check_if_updated' id='4458427008'>
|
||||
mock_benchmark = <MagicMock name='check_model_benchmarks' id='4458439056'>
|
||||
mock_burn = <MagicMock name='log_openrouter_burn' id='4458455248'>
|
||||
mock_create_device = <MagicMock name='create_device' id='4458463184'>
|
||||
mock_time_delta = <MagicMock name='set_time_delta' id='4458479376'>
|
||||
MockSession = <MagicMock name='SessionState' id='4458491472'>
|
||||
mock_open_ig = <MagicMock name='open_instagram' id='4458511760'>
|
||||
mock_ig_version = <MagicMock name='get_instagram_version' id='4458523808'>
|
||||
mock_close_ig = <MagicMock name='close_instagram' id='4458535808'>
|
||||
mock_sleep = <MagicMock name='random_sleep' id='4458552144'>
|
||||
mock_dump = <MagicMock name='dump_ui_state' id='4458568336'>
|
||||
mock_telepathic = <MagicMock name='TelepathicEngine' id='4458588624'>
|
||||
mock_nav = <MagicMock name='QNavGraph' id='4458604816'>
|
||||
mock_zero = <MagicMock name='ZeroLatencyEngine' id='4458621008'>
|
||||
mock_dopamine_class = <MagicMock name='DopamineEngine' id='4458637200'>
|
||||
mock_resonance = <MagicMock name='ResonanceEngine' id='4458653392'>
|
||||
mock_growth = <MagicMock name='GrowthBrain' id='4458669584'>
|
||||
mock_crm = <MagicMock name='ParasocialCRMDB' id='4458681680'>
|
||||
mock_radome = <MagicMock name='HoneypotRadome' id='4458693776'>
|
||||
mock_dojo = <MagicMock name='DojoEngine' id='4458709968'>
|
||||
mock_run_feed = <MagicMock name='_run_zero_latency_feed_loop' id='4458726160'>
|
||||
|
||||
@patch('GramAddict.core.bot_flow._run_zero_latency_feed_loop', return_value="SESSION_OVER")
|
||||
@patch('GramAddict.core.bot_flow.DojoEngine')
|
||||
@patch('GramAddict.core.bot_flow.HoneypotRadome')
|
||||
@patch('GramAddict.core.bot_flow.ParasocialCRMDB')
|
||||
@patch('GramAddict.core.bot_flow.GrowthBrain')
|
||||
@patch('GramAddict.core.bot_flow.ResonanceEngine')
|
||||
@patch('GramAddict.core.bot_flow.DopamineEngine')
|
||||
@patch('GramAddict.core.bot_flow.ZeroLatencyEngine')
|
||||
@patch('GramAddict.core.bot_flow.QNavGraph')
|
||||
@patch('GramAddict.core.bot_flow.TelepathicEngine')
|
||||
@patch('GramAddict.core.bot_flow.dump_ui_state')
|
||||
@patch('GramAddict.core.bot_flow.random_sleep')
|
||||
@patch('GramAddict.core.bot_flow.close_instagram')
|
||||
@patch('GramAddict.core.bot_flow.get_instagram_version', return_value="1.0")
|
||||
@patch('GramAddict.core.bot_flow.open_instagram', return_value=True)
|
||||
@patch('GramAddict.core.bot_flow.SessionState')
|
||||
@patch('GramAddict.core.bot_flow.set_time_delta')
|
||||
@patch('GramAddict.core.bot_flow.create_device')
|
||||
@patch('GramAddict.core.llm_provider.log_openrouter_burn')
|
||||
@patch('GramAddict.core.benchmark_guard.check_model_benchmarks')
|
||||
@patch('GramAddict.core.bot_flow.check_if_updated')
|
||||
@patch('GramAddict.core.bot_flow.configure_logger')
|
||||
@patch('GramAddict.core.bot_flow.Config')
|
||||
def test_start_bot_normal_flow(MockConfig, mock_logger, mock_update, mock_benchmark, mock_burn,
|
||||
mock_create_device, mock_time_delta, MockSession, mock_open_ig, mock_ig_version,
|
||||
mock_close_ig, mock_sleep, mock_dump, mock_telepathic, mock_nav, mock_zero,
|
||||
mock_dopamine_class, mock_resonance, mock_growth, mock_crm, mock_radome, mock_dojo, mock_run_feed):
|
||||
|
||||
MockConfig.return_value.args.feed = True
|
||||
MockConfig.return_value.args.explore = False
|
||||
MockConfig.return_value.args.reels = True
|
||||
MockConfig.return_value.args.stories = False
|
||||
MockConfig.return_value.args.working_hours = [10, 20]
|
||||
MockConfig.return_value.args.time_delta_session = 30
|
||||
|
||||
MockSession.inside_working_hours.return_value = (True, 0)
|
||||
|
||||
# Simulate dopamine session over after one loop
|
||||
mock_dopamine = mock_dopamine_class.return_value
|
||||
mock_dopamine.is_app_session_over.side_effect = [False, True]
|
||||
mock_dopamine.boredom = 10.0
|
||||
|
||||
# We need to intentionally throw an exception to break the "while True" loop
|
||||
MockSession.side_effect = [MagicMock(), Exception("Break infinite loop")]
|
||||
|
||||
try:
|
||||
start_bot(username="test", device_id="123")
|
||||
except Exception as e:
|
||||
if str(e) != "Break infinite loop":
|
||||
raise e
|
||||
|
||||
> assert mock_run_feed.called
|
||||
E AssertionError: assert False
|
||||
E + where False = <MagicMock name='_run_zero_latency_feed_loop' id='4458726160'>.called
|
||||
|
||||
tests/integration/test_bot_flow_start.py:56: AssertionError
|
||||
----------------------------- Captured stdout call -----------------------------
|
||||
|
||||
==================================================
|
||||
🤖 MANUAL E2E DUMP CAPTURE SEQUENCE
|
||||
==================================================
|
||||
Please follow the instructions below to capture the required fixtures.
|
||||
If an IG update changed the layout, you can navigate there naturally.
|
||||
==================================================
|
||||
|
||||
|
||||
👉 1. COMMENT SHEET:
|
||||
Open Instagram, scroll to any post on the HomeFeed, and open the comment section.
|
||||
When the comment sheet is fully visible, press ENTER to capture...
|
||||
------------------------------ Captured log call -------------------------------
|
||||
ERROR GramAddict.core.dump_capturer:dump_capturer.py:105 💥 Capture Sequence crashed: pytest: reading from stdin while output is captured! Consider using `-s`.
|
||||
Traceback (most recent call last):
|
||||
File "/Volumes/Alpha SSD/Coding/bot/GramAddict/core/dump_capturer.py", line 43, in capture_all
|
||||
input("\n👉 1. COMMENT SHEET:\nOpen Instagram, scroll to any post on the HomeFeed, and open the comment section.\nWhen the comment sheet is fully visible, press ENTER to capture...")
|
||||
File "/Users/marcmintel/Library/Python/3.9/lib/python/site-packages/_pytest/capture.py", line 227, in read
|
||||
raise OSError(
|
||||
OSError: pytest: reading from stdin while output is captured! Consider using `-s`.
|
||||
_____________________ test_full_content_to_resonance_flow ______________________
|
||||
|
||||
mock_engines = (<GramAddict.core.resonance_engine.ResonanceEngine object at 0x1093e2be0>, <GramAddict.core.growth_brain.GrowthBrain object at 0x108f3f040>)
|
||||
|
||||
def test_full_content_to_resonance_flow(mock_engines):
|
||||
"""
|
||||
REALITY CHECK: Tests the flow from RAW XML -> EXTRACED CONTENT -> RESONANCE SCORE.
|
||||
Using 'dump.xml' which contains an organic post and an ad.
|
||||
"""
|
||||
resonance, _ = mock_engines
|
||||
|
||||
> with open(DUMPS["organic"], "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/organic_post.xml'
|
||||
|
||||
tests/integration/test_cognitive_integration.py:51: FileNotFoundError
|
||||
________________________ test_ad_detection_integration _________________________
|
||||
|
||||
def test_ad_detection_integration():
|
||||
"""Verify that _detect_ad_structural works on the actual ad_dump.xml."""
|
||||
from GramAddict.core.bot_flow import _detect_ad_structural
|
||||
|
||||
> with open(DUMPS["ad"], "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/peugeot_ad.xml'
|
||||
|
||||
tests/integration/test_cognitive_integration.py:73: FileNotFoundError
|
||||
__________________________ test_extract_explore_reel ___________________________
|
||||
|
||||
def test_extract_explore_reel():
|
||||
"""Verify extraction logic works on the Explore Grid/Reels dump."""
|
||||
> with open(DUMPS["explore"], "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/mock_data/explore_feed.xml'
|
||||
|
||||
tests/integration/test_cognitive_integration.py:98: FileNotFoundError
|
||||
_______________________ test_real_normal_post_is_not_ad ________________________
|
||||
|
||||
def test_real_normal_post_is_not_ad():
|
||||
"""
|
||||
Test: Ensures the ad detector correctly ignores a standard organic post.
|
||||
"""
|
||||
xml_path = os.path.join(FIX_DIR, "organic_post.xml")
|
||||
> with open(xml_path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/tests/fixtures/organic_post.xml'
|
||||
|
||||
tests/integration/test_false_positive.py:12: FileNotFoundError
|
||||
_____________________________ test_emit_pheromone ______________________________
|
||||
|
||||
swarm = <GramAddict.core.swarm_protocol.SwarmProtocol object at 0x1093cde50>
|
||||
|
||||
def test_emit_pheromone(swarm):
|
||||
"""Verify that emitting a pheromone calls Qdrant upsert with correct payload."""
|
||||
with patch("GramAddict.core.qdrant_memory.QdrantBase.is_connected", new_callable=PropertyMock, return_value=True):
|
||||
path_hash = "some_ui_path_hash"
|
||||
outcome = "success"
|
||||
|
||||
swarm.emit_pheromone(path_hash, outcome)
|
||||
|
||||
# Check if upsert was called with the expected payload
|
||||
swarm.client.upsert.assert_called_once()
|
||||
args, kwargs = swarm.client.upsert.call_args
|
||||
points = kwargs.get('points')
|
||||
> assert points[0].payload['path_hash'] == path_hash
|
||||
E AssertionError: assert <MagicMock name='mock.PointStruct().payload.__getitem__()' id='4444684496'> == 'some_ui_path_hash'
|
||||
|
||||
tests/integration/test_swarm_protocol.py:22: AssertionError
|
||||
------------------------------ Captured log setup ------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'gramaddict_swarm_pheromones': collection has <MagicMock name='QdrantClient().get_collection().config.params.vectors.size' id='4444982096'>, expected 4. Recreating collection...
|
||||
____________ TestNodeExtraction.test_home_feed_extracts_like_button ____________
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestNodeExtraction object at 0x108d91fd0>
|
||||
|
||||
def test_home_feed_extracts_like_button(self):
|
||||
"""
|
||||
In a real Home Feed dump, the parser MUST find the Like button node
|
||||
with resource-id 'row_feed_button_like'.
|
||||
"""
|
||||
engine = TelepathicEngine()
|
||||
> xml = load_fixture("home_feed_with_ad.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:43:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'home_feed_with_ad.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/home_feed_with_ad.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
------------------------------ Captured log call -------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'telepathic_engine_cache': collection has <MagicMock name='mock.QdrantClient().get_collection().config.params.vectors.size' id='4446345248'>, expected 768. Recreating collection...
|
||||
______________ TestNodeExtraction.test_home_feed_extracts_tab_bar ______________
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestNodeExtraction object at 0x108e59430>
|
||||
|
||||
def test_home_feed_extracts_tab_bar(self):
|
||||
"""
|
||||
The parser must find the bottom tab bar items (Home, Reels, Search, Profile).
|
||||
These are critical for navigation.
|
||||
"""
|
||||
engine = TelepathicEngine()
|
||||
> xml = load_fixture("home_feed_with_ad.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:66:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'home_feed_with_ad.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/home_feed_with_ad.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
------------------------------ Captured log call -------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'telepathic_engine_cache': collection has <MagicMock name='mock.QdrantClient().get_collection().config.params.vectors.size' id='4446345248'>, expected 768. Recreating collection...
|
||||
__________ TestNodeExtraction.test_home_feed_node_count_is_realistic ___________
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestNodeExtraction object at 0x108e59610>
|
||||
|
||||
def test_home_feed_node_count_is_realistic(self):
|
||||
"""
|
||||
A real Instagram home feed XML produces 20-40 interactive nodes.
|
||||
If we get <10 or >100, the parser is broken.
|
||||
"""
|
||||
engine = TelepathicEngine()
|
||||
> xml = load_fixture("home_feed_with_ad.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:80:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'home_feed_with_ad.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/home_feed_with_ad.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
------------------------------ Captured log call -------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'telepathic_engine_cache': collection has <MagicMock name='mock.QdrantClient().get_collection().config.params.vectors.size' id='4446345248'>, expected 768. Recreating collection...
|
||||
__________ TestNodeExtraction.test_explore_feed_extracts_like_button ___________
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestNodeExtraction object at 0x108e59820>
|
||||
|
||||
def test_explore_feed_extracts_like_button(self):
|
||||
"""
|
||||
In the real Explore/Reels feed, the Like button has id 'like_button'
|
||||
and description 'Like'. The parser must find it.
|
||||
"""
|
||||
engine = TelepathicEngine()
|
||||
> xml = load_fixture("explore_feed.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:94:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'explore_feed.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/explore_feed.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
------------------------------ Captured log call -------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'telepathic_engine_cache': collection has <MagicMock name='mock.QdrantClient().get_collection().config.params.vectors.size' id='4446345248'>, expected 768. Recreating collection...
|
||||
________ TestNodeExtraction.test_explore_feed_has_fullscreen_containers ________
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestNodeExtraction object at 0x108e59a30>
|
||||
|
||||
def test_explore_feed_has_fullscreen_containers(self):
|
||||
"""
|
||||
Verify that the parser extracts the fullscreen containers
|
||||
(swipeable_nav_view_pager_inner_recycler_view, clips_viewer_view_pager)
|
||||
so that the Safety Guard has something to reject.
|
||||
"""
|
||||
engine = TelepathicEngine()
|
||||
> xml = load_fixture("explore_feed.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:112:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'explore_feed.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/explore_feed.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
------------------------------ Captured log call -------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'telepathic_engine_cache': collection has <MagicMock name='mock.QdrantClient().get_collection().config.params.vectors.size' id='4446345248'>, expected 768. Recreating collection...
|
||||
_______________ TestAdDetection.test_real_explore_feed_is_not_ad _______________
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestAdDetection object at 0x108e6c520>
|
||||
|
||||
def test_real_explore_feed_is_not_ad(self):
|
||||
"""
|
||||
The explore_feed.xml is a real Reel without any ad markers.
|
||||
It should NOT be flagged.
|
||||
"""
|
||||
from GramAddict.core.bot_flow import _detect_ad_structural
|
||||
|
||||
> xml = load_fixture("explore_feed.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:241:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'explore_feed.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/explore_feed.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
_______________ TestFeedMarkers.test_real_home_feed_has_markers ________________
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestFeedMarkers object at 0x108e6c8e0>
|
||||
|
||||
def test_real_home_feed_has_markers(self):
|
||||
"""The real home feed XML must match our feed markers."""
|
||||
from GramAddict.core.bot_flow import FEED_MARKERS
|
||||
|
||||
> xml = load_fixture("home_feed_with_ad.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:256:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'home_feed_with_ad.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/home_feed_with_ad.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
______________ TestFeedMarkers.test_real_explore_feed_has_markers ______________
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestFeedMarkers object at 0x108e6caf0>
|
||||
|
||||
def test_real_explore_feed_has_markers(self):
|
||||
"""The real explore feed XML must match our feed markers."""
|
||||
from GramAddict.core.bot_flow import FEED_MARKERS
|
||||
|
||||
> xml = load_fixture("explore_feed.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:267:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'explore_feed.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/explore_feed.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
______ TestTelepathicResolutionCascade.test_keyword_fast_path_bypasses_ai ______
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestTelepathicResolutionCascade object at 0x108e6ceb0>
|
||||
mock_get_embedding = <MagicMock name='_get_embedding' id='4449038736'>
|
||||
mock_vlm = <MagicMock name='query_telepathic_llm' id='4447908240'>
|
||||
|
||||
@patch('GramAddict.core.telepathic_engine.query_telepathic_llm')
|
||||
@patch('GramAddict.core.qdrant_memory.QdrantBase._get_embedding')
|
||||
def test_keyword_fast_path_bypasses_ai(self, mock_get_embedding, mock_vlm):
|
||||
"""
|
||||
A direct keyword match (like 'tap like button') MUST be resolved by Stage 1.5.
|
||||
It must never reach the Embedding (Stage 2) or VLM (Stage 3).
|
||||
"""
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
engine = TelepathicEngine()
|
||||
engine._embedding_cache.clear()
|
||||
engine._intent_cache.clear()
|
||||
|
||||
# home_feed_with_ad.xml contains standard UI elements
|
||||
> xml_content = load_fixture("home_feed_with_ad.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:294:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'home_feed_with_ad.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/home_feed_with_ad.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
------------------------------ Captured log call -------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'telepathic_engine_cache': collection has <MagicMock name='mock.QdrantClient().get_collection().config.params.vectors.size' id='4446345248'>, expected 768. Recreating collection...
|
||||
_ TestTelepathicResolutionCascade.test_embedding_fallback_bypasses_vlm_if_confident _
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestTelepathicResolutionCascade object at 0x108e6cdf0>
|
||||
mock_get_embedding = <MagicMock name='_get_embedding' id='4457325136'>
|
||||
mock_vlm = <MagicMock name='query_telepathic_llm' id='4449935808'>
|
||||
|
||||
@patch('GramAddict.core.telepathic_engine.query_telepathic_llm')
|
||||
@patch('GramAddict.core.qdrant_memory.QdrantBase._get_embedding')
|
||||
def test_embedding_fallback_bypasses_vlm_if_confident(self, mock_get_embedding, mock_vlm):
|
||||
"""
|
||||
If we ask something without an exact keyword match, it should fail Stage 1.5,
|
||||
hit Stage 2 (Embeddings), and if confident enough, avoid Stage 3 (VLM).
|
||||
"""
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
engine = TelepathicEngine()
|
||||
engine._embedding_cache.clear()
|
||||
engine._intent_cache.clear()
|
||||
|
||||
> xml_content = load_fixture("home_feed_with_ad.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:318:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'home_feed_with_ad.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/home_feed_with_ad.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
------------------------------ Captured log call -------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'telepathic_engine_cache': collection has <MagicMock name='mock.QdrantClient().get_collection().config.params.vectors.size' id='4446345248'>, expected 768. Recreating collection...
|
||||
_ TestTelepathicResolutionCascade.test_vlm_fallback_triggered_on_low_confidence _
|
||||
|
||||
self = <test_telepathic_engine_extraction.TestTelepathicResolutionCascade object at 0x108e6c430>
|
||||
mock_get_embedding = <MagicMock name='_get_embedding' id='4457118352'>
|
||||
mock_vlm = <MagicMock name='query_telepathic_llm' id='4444565312'>
|
||||
|
||||
@patch('GramAddict.core.telepathic_engine.query_telepathic_llm')
|
||||
@patch('GramAddict.core.qdrant_memory.QdrantBase._get_embedding')
|
||||
def test_vlm_fallback_triggered_on_low_confidence(self, mock_get_embedding, mock_vlm):
|
||||
"""
|
||||
If Embeddings fail to find a confident match (< 0.82), it must trigger
|
||||
the Stage 3 VLM fallback.
|
||||
"""
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
engine = TelepathicEngine()
|
||||
engine._embedding_cache.clear()
|
||||
engine._intent_cache.clear()
|
||||
|
||||
> xml_content = load_fixture("home_feed_with_ad.xml")
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:353:
|
||||
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
|
||||
|
||||
name = 'home_feed_with_ad.xml'
|
||||
|
||||
def load_fixture(name: str) -> str:
|
||||
"""Load a real XML capture from tests/mock_data/"""
|
||||
path = os.path.join(FIXTURE_DIR, name)
|
||||
> with open(path, "r") as f:
|
||||
E FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/Alpha SSD/Coding/bot/tests/integration/mock_data/home_feed_with_ad.xml'
|
||||
|
||||
tests/integration/test_telepathic_engine_extraction.py:27: FileNotFoundError
|
||||
------------------------------ Captured log call -------------------------------
|
||||
WARNING GramAddict.core.qdrant_memory:qdrant_memory.py:35 Qdrant dimension mismatch for 'telepathic_engine_cache': collection has <MagicMock name='mock.QdrantClient().get_collection().config.params.vectors.size' id='4446345248'>, expected 768. Recreating collection...
|
||||
=============================== warnings summary ===============================
|
||||
../../../../Users/marcmintel/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py:35
|
||||
/Users/marcmintel/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
|
||||
warnings.warn(
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
=========================== short test summary info ============================
|
||||
FAILED tests/anomalies/test_fsd_recovery.py::test_fsd_handles_persistent_survey_modal
|
||||
FAILED tests/integration/test_ad_detection.py::test_real_sponsored_reel_flexcode_is_detected
|
||||
FAILED tests/integration/test_ad_detection.py::test_normal_post_not_ad - File...
|
||||
FAILED tests/integration/test_ad_detection.py::test_peugeot_carousel_ad_is_detected
|
||||
FAILED tests/integration/test_bot_flow_interaction.py::test_start_bot_interrupt
|
||||
FAILED tests/integration/test_bot_flow_start.py::test_start_bot_normal_flow
|
||||
FAILED tests/integration/test_cognitive_integration.py::test_full_content_to_resonance_flow
|
||||
FAILED tests/integration/test_cognitive_integration.py::test_ad_detection_integration
|
||||
FAILED tests/integration/test_cognitive_integration.py::test_extract_explore_reel
|
||||
FAILED tests/integration/test_false_positive.py::test_real_normal_post_is_not_ad
|
||||
FAILED tests/integration/test_swarm_protocol.py::test_emit_pheromone - Assert...
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestNodeExtraction::test_home_feed_extracts_like_button
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestNodeExtraction::test_home_feed_extracts_tab_bar
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestNodeExtraction::test_home_feed_node_count_is_realistic
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestNodeExtraction::test_explore_feed_extracts_like_button
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestNodeExtraction::test_explore_feed_has_fullscreen_containers
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestAdDetection::test_real_explore_feed_is_not_ad
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestFeedMarkers::test_real_home_feed_has_markers
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestFeedMarkers::test_real_explore_feed_has_markers
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestTelepathicResolutionCascade::test_keyword_fast_path_bypasses_ai
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestTelepathicResolutionCascade::test_embedding_fallback_bypasses_vlm_if_confident
|
||||
FAILED tests/integration/test_telepathic_engine_extraction.py::TestTelepathicResolutionCascade::test_vlm_fallback_triggered_on_low_confidence
|
||||
ERROR tests/anomalies/test_hardware_anomalies.py::test_slow_loading_post_recovery
|
||||
ERROR tests/anomalies/test_hardware_anomalies.py::test_wait_timeout_aborts_gracefully
|
||||
ERROR tests/anomalies/test_hardware_anomalies.py::test_empty_content_extraction_guard
|
||||
ERROR tests/anomalies/test_hardware_anomalies.py::test_missing_feed_markers_guard
|
||||
ERROR tests/integration/test_scenarios_fsd.py::test_full_mission_autopilot_sequence
|
||||
ERROR tests/integration/test_scenarios_fsd.py::test_feed_loop_chaos_mode - Fi...
|
||||
ERROR tests/integration/test_telepathic_engine_extraction.py::TestSafetyGuard::test_real_explore_fullscreen_container_rejected
|
||||
ERROR tests/integration/test_telepathic_engine_extraction.py::TestSafetyGuard::test_real_explore_like_button_accepted
|
||||
== 22 failed, 120 passed, 2 skipped, 1 warning, 8 errors in 76.34s (0:01:16) ===
|
||||
@@ -7,7 +7,8 @@ emoji==2.12.1
|
||||
langdetect==1.0.9
|
||||
atomicwrites==1.4.1
|
||||
spintax==1.0.4
|
||||
requests>=2.31.0
|
||||
requests>=2.32.0
|
||||
packaging>=23.0
|
||||
python-dotenv==1.0.1
|
||||
qdrant-client>=1.7.0
|
||||
psutil==5.9.5
|
||||
|
||||
2
run.py
2
run.py
@@ -1,10 +1,12 @@
|
||||
import sys
|
||||
import warnings
|
||||
|
||||
import GramAddict
|
||||
|
||||
warnings.filterwarnings("ignore", category=UserWarning, module="urllib3")
|
||||
try:
|
||||
from urllib3.exceptions import NotOpenSSLWarning
|
||||
|
||||
warnings.filterwarnings("ignore", category=NotOpenSSLWarning)
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
import pytest
|
||||
from tests.unit.test_profile_interaction_sync import test_profile_grid_sync_delay_after_follow
|
||||
import sys
|
||||
pytest.main(["-v", "-s", "tests/unit/test_profile_interaction_sync.py"])
|
||||
27
run_test2.py
27
run_test2.py
@@ -1,27 +0,0 @@
|
||||
from unittest.mock import patch, MagicMock
|
||||
from GramAddict.core.bot_flow import _interact_with_profile
|
||||
from tests.unit.test_profile_interaction_sync import FakeConfig
|
||||
|
||||
mock_device = MagicMock()
|
||||
mock_configs = FakeConfig()
|
||||
mock_session_state = MagicMock()
|
||||
mock_session_state.check_limit.return_value = False
|
||||
manager = MagicMock()
|
||||
|
||||
with patch("GramAddict.core.bot_flow.QNavGraph") as MockQNavGraph, \
|
||||
patch("GramAddict.core.bot_flow.sleep") as mock_sleep, \
|
||||
patch("GramAddict.core.bot_flow.random.random", return_value=0.0):
|
||||
|
||||
mock_nav_instance = MagicMock()
|
||||
mock_nav_instance._execute_transition.return_value = True
|
||||
MockQNavGraph.return_value = mock_nav_instance
|
||||
|
||||
manager.attach_mock(mock_nav_instance._execute_transition, 'execute_transition')
|
||||
manager.attach_mock(mock_sleep, 'sleep')
|
||||
|
||||
_interact_with_profile(mock_device, mock_configs, "test_user", mock_session_state, 1.0, MagicMock())
|
||||
|
||||
print("MOCK CALLS:")
|
||||
for method, args, kwargs in manager.mock_calls:
|
||||
print(f"{method}: args={args}, kwargs={kwargs}")
|
||||
|
||||
@@ -1,41 +0,0 @@
|
||||
import os
|
||||
import glob
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
dumps = glob.glob('debug/xml_dumps/*.xml')
|
||||
|
||||
edge_cases = {
|
||||
'dialogs': set(),
|
||||
'bottom_sheets': set(),
|
||||
'errors': set(),
|
||||
'weird_states': set()
|
||||
}
|
||||
|
||||
for dump in dumps:
|
||||
try:
|
||||
tree = ET.parse(dump)
|
||||
root = tree.getroot()
|
||||
for node in root.iter('node'):
|
||||
rid = node.get('resource-id', '')
|
||||
class_name = node.get('class', '')
|
||||
text = node.get('text', '')
|
||||
|
||||
if 'dialog' in rid.lower() or 'alert' in rid.lower() or 'popup' in rid.lower():
|
||||
edge_cases['dialogs'].add(rid)
|
||||
elif 'bottom_sheet' in rid.lower() or 'action_sheet' in rid.lower():
|
||||
edge_cases['bottom_sheets'].add(rid)
|
||||
elif 'error' in rid.lower() or 'fail' in rid.lower():
|
||||
edge_cases['errors'].add(rid)
|
||||
|
||||
# Unusual views that might break logic
|
||||
if 'survey' in rid.lower() or 'rate' in rid.lower() or 'nux' in rid.lower():
|
||||
edge_cases['weird_states'].add(rid)
|
||||
except:
|
||||
pass
|
||||
|
||||
print("=== Discovered Edge Cases in Dumps ===")
|
||||
for k, v in edge_cases.items():
|
||||
print(f"\n[{k.upper()}]")
|
||||
for item in list(v)[:10]:
|
||||
print(f" - {item}")
|
||||
|
||||
42
scripts/debug_intent.py
Normal file
42
scripts/debug_intent.py
Normal file
@@ -0,0 +1,42 @@
|
||||
import re
|
||||
|
||||
from GramAddict.core.perception.intent_resolver import _humanize_desc
|
||||
from GramAddict.core.perception.spatial_parser import SpatialParser
|
||||
|
||||
|
||||
def main():
|
||||
with open("tests/fixtures/user_profile_dump.xml", "r", encoding="utf-8") as f:
|
||||
xml = f.read()
|
||||
|
||||
parser = SpatialParser()
|
||||
root = parser.parse(xml)
|
||||
candidates = parser.get_clickable_nodes(root)
|
||||
|
||||
intent_description = "tap 'following' list"
|
||||
quotes = re.findall(r"['\"](.*?)['\"]", intent_description)
|
||||
target_text = quotes[0].lower()
|
||||
localized_targets = [target_text]
|
||||
|
||||
semantic_candidates = []
|
||||
for node in candidates:
|
||||
n_text = _humanize_desc((node.text or "").lower())
|
||||
n_desc = _humanize_desc((node.content_desc or "").lower())
|
||||
|
||||
for loc_target in localized_targets:
|
||||
pattern = r"\b" + re.escape(loc_target) + r"\b"
|
||||
if (
|
||||
re.search(pattern, n_text)
|
||||
or re.search(pattern, n_desc)
|
||||
or loc_target in (node.resource_id or "").lower()
|
||||
):
|
||||
semantic_candidates.append(node)
|
||||
break
|
||||
|
||||
print(f"Quotes: {quotes}")
|
||||
print(f"Num semantic candidates: {len(semantic_candidates)}")
|
||||
for i, n in enumerate(semantic_candidates):
|
||||
print(f"[{i}] id={n.resource_id} desc={n.content_desc} text={n.text}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
20
scripts/debug_sort.py
Normal file
20
scripts/debug_sort.py
Normal file
@@ -0,0 +1,20 @@
|
||||
from GramAddict.core.telepathic_engine import TelepathicEngine
|
||||
|
||||
DUMP_PATH = "/Volumes/Alpha SSD/Coding/bot/debug/xml_dumps/post_load_timeout__2026-04-17_15-02-36.xml"
|
||||
with open(DUMP_PATH, "r") as f:
|
||||
xml_content = f.read()
|
||||
|
||||
engine = TelepathicEngine.get_instance()
|
||||
nodes = engine._extract_semantic_nodes(xml_content)
|
||||
grid_nodes = []
|
||||
for node in nodes:
|
||||
if node.get("resource_id") in [
|
||||
"com.instagram.android:id/grid_card_layout_container",
|
||||
"com.instagram.android:id/image_button",
|
||||
]:
|
||||
grid_nodes.append(node)
|
||||
|
||||
grid_nodes.sort(key=lambda n: (round(n["y"] / 5) * 5, n["x"], n["naf"], -n["area"]))
|
||||
|
||||
for n in grid_nodes[:5]:
|
||||
print(f"Y={n['y']} (rnd={round(n['y']/5)*5}), NAF={n['naf']}, Area={n['area']}, ID={n['resource_id']}")
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user