3.3 KiB
3.3 KiB
GEMINI.md - Codebase Architecture & Structure
Internal technical overview of the Scraping & CDN Service (apps/scraper) for automated data extraction and image persistence.
🌍 Context
apps/scraper: Specialized backend engine (Axum).apps/solidjs: Frontend consumer.packages/services: Shared logic.
🤖 AI Assistant Guidelines
AI assistants (like Claude, Gemini, GPT) interacting with this codebase MUST adhere to the following protocols defined in AGENT.md:
- Professional Tone: Maintain a cold, technical, and objective tone.
- No Hyperbole: PROHIBITED from using marketing-speak or exaggerated praise (e.g., "amazing", "unparalleled", "powerful", "revolutionary").
- Technical Accuracy: Focus purely on implementation facts, data structures, and performance metrics.
- Minimalist Adherence: Always prioritize the Zero-Bloat Policy. If a request introduces unnecessary dependencies or logic, challenge the user and suggest a leaner alternative.
- Documentation Consistency: Ensure any generated documentation follows the established professional and objective style of the
docs/folder.
🦀 apps/scraper - Backend Service
An asynchronous service for scraping and image proxying. All secondary web framework features (Authentication, Social, GraphQL) have been removed to reduce complexity.
📊 Tech Stack
- Framework: Axum (0.8.8) - Asynchronous Rust HTTP.
- ORM: SeaORM (MySQL) - Database abstraction.
- Caching:
deadpool-redis&redis- In-memory cache mapping. - Observability: Request ID tracing and structured logging.
- Scraping:
scraper(CSS Selectors) & remote Chrome via HTTP.
📂 Directory Structure (apps/scraper/src)
Organized as a hybrid of Vertical Slice and Clean Architecture.
| Directory | Description |
|---|---|
bin/ |
Binary entry points and CLI tools. |
config/ |
Strongly-typed environment configuration. |
entities/ |
SeaORM Entities. Database schema mapping. |
routes/ |
API Handlers. Automatic routing system. |
services/ |
Business logic (e.g., ImageCache service). |
scraping/ |
Data extraction engines and parsers. |
helpers/ |
Shared utilities and cache helpers. |
middleware/ |
Axum layers (CORS, Compression). |
events/ |
Internal event bus for repair state updates. |
jobs/ |
Background task processing. |
scheduler/ |
Periodic tasks (Daily CDN audit). |
observability/ |
OpenAPI documentation, request ID tracing, and structured logging. |
🔑 Logic Flows
- Scraping:
Request->Router->Handler->Scraper Engine->Redis->Response. - Image Proxy/CDN:
Request->ImageCache->Cache Lookup->Picser Upload (on miss)->CDN URL.
📜 Commands
- Standard Run:
cargo run - Optimized Build:
cargo build --release - External Audit:
POST /api/proxy/image-cache/audit
🏗 Maintenance Constraints
- Minimalist Approach: New dependencies require impact evaluation.
- Lint Compliance: Suppression flags (
#[allow]) are prohibited. - Performance-First: Use
spawn_blockingfor CPU-heavy work (HTML parsing).