GEMINI.md - Codebase Architecture & Structure

Internal technical overview of the Scraping & CDN Service (apps/scraper) for automated data extraction and image persistence.

🌍 Context

apps/scraper: Specialized backend engine (Axum).
apps/solidjs: Frontend consumer.
packages/services: Shared logic.

🤖 AI Assistant Guidelines

AI assistants (like Claude, Gemini, GPT) interacting with this codebase MUST adhere to the following protocols defined in AGENT.md:

Professional Tone: Maintain a cold, technical, and objective tone.
No Hyperbole: PROHIBITED from using marketing-speak or exaggerated praise (e.g., "amazing", "unparalleled", "powerful", "revolutionary").
Technical Accuracy: Focus purely on implementation facts, data structures, and performance metrics.
Minimalist Adherence: Always prioritize the Zero-Bloat Policy. If a request introduces unnecessary dependencies or logic, challenge the user and suggest a leaner alternative.
Documentation Consistency: Ensure any generated documentation follows the established professional and objective style of the docs/ folder.

🦀 `apps/scraper` - Backend Service

An asynchronous service for scraping and image proxying. All secondary web framework features (Authentication, Social, GraphQL) have been removed to reduce complexity.

📊 Tech Stack

Framework: Axum (0.8.8) - Asynchronous Rust HTTP.
ORM: SeaORM (MySQL) - Database abstraction.
Caching: deadpool-redis & redis - In-memory cache mapping.
Observability: Request ID tracing and structured logging.
Scraping: scraper (CSS Selectors) & remote Chrome via HTTP.

📂 Directory Structure (`apps/scraper/src`)

Organized as a hybrid of Vertical Slice and Clean Architecture.

Directory	Description
`bin/`	Binary entry points and CLI tools.
`config/`	Strongly-typed environment configuration.
`entities/`	SeaORM Entities. Database schema mapping.
`routes/`	API Handlers. Automatic routing system.
`services/`	Business logic (e.g., `ImageCache` service).
`scraping/`	Data extraction engines and parsers.
`helpers/`	Shared utilities and cache helpers.
`middleware/`	Axum layers (CORS, Compression).
`events/`	Internal event bus for repair state updates.
`jobs/`	Background task processing.
`scheduler/`	Periodic tasks (Daily CDN audit).
`observability/`	OpenAPI documentation, request ID tracing, and structured logging.

🔑 Logic Flows

Scraping: Request -> Router -> Handler -> Scraper Engine -> Redis -> Response.
Image Proxy/CDN: Request -> ImageCache -> Cache Lookup -> Picser Upload (on miss) -> CDN URL.

📜 Commands

Standard Run: cargo run
Optimized Build: cargo build --release
External Audit: POST /api/proxy/image-cache/audit

🏗 Maintenance Constraints

Minimalist Approach: New dependencies require impact evaluation.
Lint Compliance: Suppression flags (#[allow]) are prohibited.
Performance-First: Use spawn_blocking for CPU-heavy work (HTML parsing).

3.3 KiB Raw Blame History