asepharyana
966d86ff67
Deploy Scraper / build-and-deploy (push) Canceled after 0s
- bootstrap: TracingLayer from mytheclipse-tracing (composed with scraper env filter), RuntimeConfig::auto() thread logging, init job queue + cron. - proxy_fetch: leader task via ::mytheclipse::spawn_io, gzip decompression offloaded to ::mytheclipse::compute (sized rayon pool, panic-isolated), bounded by new async fetch limiter (tokio Semaphore bridge). - queue: new infrastructure/queue module over mytheclipse-queue (InMemoryQueue + WorkerPool + BackpressureEnforcer) for repair jobs. - scheduler: new infrastructure/scheduler.rs using mytheclipse::cron for the daily 02:00 UTC image-cache cleanup. - deps: add mytheclipse-queue + mytheclipse-tracing path deps; mytheclipse -> full feature; rayon 1.12; keep path deps for unpublished crates. - tests: infra_round2 runtime smoke tests (compute panic isolation, spawn_io, backpressure admission, cron parse, queue roundtrip). - Fix: local cache::mytheclipse bridge module shadowed the mytheclipse crate name; use leading :: at spawn_io/compute call sites.
Scraper API
Backend service berbasis Axum untuk scraping, image proxy/cache, dan endpoint API inti.
Stack
- Rust + Axum
- SeaORM (MySQL)
- Redis (deadpool-redis)
- Utoipa + Swagger UI
Quick Start
cargo run
Server bind ke 0.0.0.0:${PORT} dengan default port 4091.
Required Environment Variables
DATABASE_URL=mysql://asephs:hunterz@localhost:3306/sosmed
JWT_SECRET=change-me
REDIS_URL=redis://localhost:6379
Optional yang sering dipakai:
RUST_LOG=info
MINIO_ENDPOINT=
MINIO_BUCKET_NAME=
MINIO_ACCESS_KEY=
MINIO_SECRET_KEY=
Useful Endpoints
GET /docs- Swagger UIGET /api-docs/openapi.json- OpenAPI JSONGET /api/anime2/*GET /api/komik/*POST /api/proxy/image-cachePOST /api/proxy/image-cache/audit
Notes
- Database akan dicek/dibuat saat startup jika
DATABASE_URLbertipe MySQL. - Service ini juga menginisialisasi browser pool dan scheduler saat boot.
License
MIT