GramPilot Architecture

Welcome to the internal workings of GramPilot (formerly GramAddict). This document outlines the radical shift from fixed-state deterministic scripting to our current Vision-Language-Action (VLA) architecture that powers our "Full Self-Driving" behavior.

Core Design Philosophy

We treat the Instagram Android App like a dynamic, partially-observable environment. Instead of maintaining thousands of fragile XPaths, the bot relies on a Cognitive Stack to infer intent, learn layouts dynamically, and mathematically avoid detection.

1. The Telepathic Engine (3-Stage Resolution Cascade)

At the center of UI interactions is the TelepathicEngine which resolves semantic intent ("tap the like button") into precise screen coordinates via a strictly enforced performance cascade:

Stage 1.5: Deterministic Keyword Fast Path. Over 90% of interactions are handled by a high-performance string matcher that costs 0 API tokens and executes in <2ms.
Stage 2: Vector Similarity Engine. If keywords fail, an Ollama Semantic Embedding of the intent is generated and compared (Cosine Similarity) against cached UI vectors via Qdrant. Highly reliable for semantic synonyms.
Stage 3: Agentic Fallback. The ultimate safety net. If visual confidence drops <0.82, it falls back to an OpenRouter LLM (e.g., gemini-3.1-flash-lite-preview) which parses the raw XML to structurally guarantee a hit without hallucination.

2. Telepathic Memory & Autonomy

When Stage 3 successfully resolves an unknown interaction, the bot records the semantic signature into its positive memory (telepathic_memory.json). The next time the bot requires this action, it is instantly resolved via the local cache, guaranteeing that expensive LLM operations are only ever performed once per UI permutation.

3. The Cognitive Stack

⚖️ Active Inference (Shadow Mode)

Found in active_inference.py. Based on the free-energy principle, the bot calculates "Surprise" (prediction errors).

Shadow Mode: Before transitioning screens, the bot predicts the target UI. If it lands somewhere unexpected (a popup), it registers a prediction error, hits "Back", and averts a crash.

🛡️ Honeypot Radome & Anti-Trap Sensors

Found in sensors/honeypot_radome.py.

Topological Traps: Instagram deploys 1x1 pixel or 0x0 traps to detect bots. The Radome strictly strips these nodes prior to processing.
The Interceptor Sentinel: Detects and purges full-screen invisible clickable="true" overlays that act as touch traps (e.g., bounds >= 90% with no content description).
Ghost Engagement Guard: Strips DOM nodes explicitly tagged with visible-to-user="false" to prevent triggering Accessibility Hooks.
VLM Sanity Guard: Woven into telepathic_engine.py, it sends semantic matches for destructive actions (Like/Follow) through a Vision Language Model step to prevent executing semantic "Bait and Switch" tricks.

🦾 Biometric Facade (Gaussian Clicks)

Found in device_facade.py.

Human touches do not follow a flat mathematical uniform grid. The GramPilot simulates genuine biometric dispersion using random.gauss(mu, sigma), strictly centering clicks inside a thumb-bias radius (bottom-left skew for right-handers). In tests, this hits a 68% standard deviation precision.

💉 Dopamine Engine & Resonance Oracle

Instead of hardcoding limits like max_likes = 50, the bot stops interacting based on simulated boredom.

The ResonanceEngine calculates the aesthetic score of content.
The DopamineEngine uses this score to modulate pace. High resonance = engagement. Low resonance over multiple posts = early session termination (simulating human fatigue).

4. The 100% Autonomy Directive (Zero Hardcoding)

GramPilot is designed as a true agent, not a state-machine script. It operates on absolute zero hardcoded UI states or edge cases.

No Manual Guards: Features like if "row_feed_button_like" not in xml: or if state == "ReelsFeed": are strictly prohibited. The bot must understand the screen via its Vision-Language-Action (VLA) pipeline.
No Hand-Holding: If the LLM makes a mistake (e.g., clicking the wrong button in a DM), the solution is to improve the VLM prompt, the system architecture, or the Visual Critic. We never insert if is_dm_thread: hacks.
Smart like a human: The bot navigates by visually confirming targets, detecting obstacles when the UI organically stops responding, and inferring context precisely like a real user scrolling.

4.5 KiB Raw Blame History