Add comprehensive documentation for FlowSight project

- Introduced AGENT-SPECS.md detailing the specifications for seven agents including their inputs, processing steps, and outputs. - Created API-REFERENCE.md outlining the Sectors API v2 endpoints, parameters, costs, and usage. - Developed API.md to specify backend routes, request/response structures, and error handling. - Established ARCHITECTURE.md to describe the project layout, conventions, scheduler, and citation pipeline. - Added DATA-MODEL.md to define the database schema, tables, and seed strategy. - Compiled PLAN.md to outline the project concept, problem statement, unique features, and implementation timeline. - Created README.md as an index for documentation with links to all relevant files. - Documented ROUTINES.md detailing the seven automated routines, their schedules, inputs, detection logic, and delivery formats. - Introduced TECH-STACK.md to specify the technology choices and rationale for both backend and frontend components.
2026-09-14 20:35:02 +07:00
commit 3882c5aef0
11 changed files with 856 additions and 0 deletions
@@ -0,0 +1,71 @@
+# Agent specs
+
+Contract: `analyze(ticker_or_scope, snapshots) -> AgentResult(values[], score, citations[])`.
+Agents never fetch live; they read snapshots. Fixtures in `tests/fixtures/` prove each.
+
+## A1 — Smart Money Tracker
+
+- Reads: broker-summary/top, broker-activity/top, foreign-flow.
+- Steps: (1) top buyers/sellers per ticker; (2) per-broker accumulation ranks;
+  (3) foreign inflow 90d trend; (4) correlate broker net direction vs foreign
+  direction; (5) classify phase: accumulation / distribution / neutral / conflict.
+- Output: score −100..+100, phase, top-3 players with net Rp, direction-agreement flag.
+- Fixture: BBCA 5d — 3 domestic brokers net-buy Rp 1.2T + foreign inflow → score > +60.
+
+## A2 — Broker Intel
+
+- Reads: brokers/ registry cache, broker-activity per code, brokers/top daily.
+- Steps: (1) classify each active broker by origin/cohort; (2) behavior class per
+  broker (accumulating/distributing/neutral from top/ ranks); (3) sector exposure
+  shift week-over-week; (4) emit rotation signal on sign flip with evidence rows.
+- Output: behavior map, rotation signal (from→to + net Rp delta).
+- Fixture: Financials net −Rp 800M → Consumer +Rp 1.1T flip detected.
+
+## A3 — News Sentiment (Adaptive RAG)
+
+- Reads: news (incremental), filings, suspensions.
+- Steps: (1) fetch candidate articles; (2) LLM confidence check — confident →
+  answer from context, uncertain (rare ticker) → force retrieval + ground;
+  (3) per-article sentiment + confidence; (4) aggregate trend
+  improving/deteriorating/stable; (5) insider summary from filings.
+- Output: score −1..+1, trend, key events (≤5, cited), insider line.
+- Fixture: ticker with 2 bullish + 1 neutral + 1 director buy → positive trend cited.
+
+## A4 — Fundamental
+
+- Reads: company/report (sections=overview,valuation,financials,dividend),
+  financials/quarterly (n≤8), get-segments.
+- Steps: (1) P/E, P/B vs subsector median (subsector/report statistics);
+  (2) revenue/earnings 8Q trend; (3) ROE trajectory, debt/equity, payout ratio;
+  (4) grade A–F from weighted rubric (profitability 35, growth 25, leverage 20, payout 20).
+- Output: score 0–100, grade, vs-peers table, red flags.
+- Fixture: BBCA — premium P/E vs banks median, declining ROE flagged.
+
+## A5 — Technical
+
+- Reads: daily (≤90d), most-traded, top-changes, free-float.
+- Steps: (1) volume vs 20d avg multiples; (2) momentum positioning from movers;
+  (3) relative volume vs market; (4) liquidity grade from free-float %.
+- Output: momentum signal (strong/up/flat/down), anomaly flag with dates,
+  liquidity grade.
+- Fixture: 3.2× volume spike flagged with date + mover rank cited.
+
+## A6 — Event Catalyst
+
+- Reads: corporate-actions, quarterly-dates, listing-performance.
+- Steps: (1) upcoming dividends/splits/AGM with dates; (2) next earnings estimate;
+  (3) IPO-window context for recent listings; (4) score opportunity 0–100
+  (yield × certainty − earnings-risk).
+- Output: catalyst calendar rows (event, date, H−N, score).
+- Fixture: ex-div in 23d with yield + payout flag rendered.
+
+## A7 — Master Synthesizer
+
+- Reads: A1–A6 outputs + risk profile + accuracy-ledger weights.
+- Steps: (1) weight signals (conservative→fundamental-heavy, aggressive→
+  technical+broker-heavy); (2) agreement bonus when ≥3 agents align, conflict
+  flag when fundamentals oppose flows; (3) conviction 1–5 from weighted score
+  spread; (4) position size via capped Kelly (max 10% single name);
+  (5) thesis ≤5 sentences, each claim cited.
+- Output: BUY/HOLD/AVOID, conviction, size %, thesis, conflict flags.
+- Fixture: good fundamental + broker selling → HOLD-or-lower with conflict cited.
@@ -0,0 +1,112 @@
+# FlowSight — Sectors API v2 Reference (learned from schema.json + docs)
+
+Source: `https://docs.sectors.app/schema.json` (OpenAPI, 70 paths) + llms.txt.
+Base: `https://api.sectors.app/v2/`. Auth header: `Authorization: <raw-key>` (REST).
+v1 discontinued 2026-05-11 — all `/v1/*` return 410. Use v2 only.
+
+## Global constraints
+- IDX symbol: 4 letters, optional `.jk`, case-insensitive (`BBCA`, `bbca.jk`).
+- Broker codes: 2-letter exchange-member IDs (`MG`, `AK`, `CC`) — valid list from `GET /v2/brokers/`.
+- Date windows: broker endpoints max 14 days; daily/foreign-flow/idx-total/most-traded max 90 days (clamped). Future `end` → 400.
+- Pagination: `limit`/`offset` where listed. `GET /v2/close/` paginated per trading day.
+- Credit traps (defaults are expensive — always narrow params):
+  - `top-changes` default (2 class × 5 periods) = 10 credits → always set `classifications` + `periods`.
+  - `company/report` default (all 8 sections) = 8 credits → always set `sections`.
+  - `subsector/report` default (all 6 sections) = 6 credits → always set `sections`.
+  - `financials/quarterly` = 1 credit per quarter → bound `n_quarters`.
+  - Universe quarterly-dates full sweep ≈ 32 pages → poll incrementally with `since`.
+  - `free-float` = 1 credit per 100 companies; filters mutually exclusive (one per request).
+  - `news`: `extension=idx` vs `extension=mining` params mutually exclusive (400 if mixed).
+
+## A. Screener & taxonomy (7)
+| Method + path | Params | Cost | FlowSight use |
+|---|---|---|---|
+| GET `/v2/companies/` | `where`, `q`, `order_by`, `desc`, `limit`≤200, `offset`, `include_query_values` (`q` overrides all) | 1 (structured) | NL + SQL screener core |
+| GET `/v2/free-float/` | one of `sector`/`sub_sector`/`industry`/`sub_industry` | 1/100 cos | Liquidity grade, sector sweep |
+| GET `/v2/subsectors/` | — | 1 | Slug source (cache daily) |
+| GET `/v2/industries/` | — | 1 | Slug source (cache daily) |
+| GET `/v2/subindustries/` | — | 1 | Slug source (cache daily) |
+| GET `/v2/tags/` | — | 1 | News/filing tag filter values (cache daily) |
+| GET `/v2/companies/list_companies_with_segments/` | — | 1 | Check segment availability (cache weekly) |
+
+## B. Company core (7)
+| Method + path | Params | Cost | FlowSight use |
+|---|---|---|---|
+| GET `/v2/company/report/{symbol}/` (or `?symbol=`) | `sections` ∈ overview, valuation, future, peers, financials, dividend, management, ownership | 1/section | Fundamental agent; report sections |
+| GET `/v2/company/get-segments/{symbol}/` | `financial_year` | 1 | Revenue breakdown (Sankey-ready) |
+| GET `/v2/company/get_quarterly_financial_dates/{symbol}/` | — | 1 | Valid `report_date` values per ticker |
+| GET `/v2/financials/quarterly/{symbol}/` | `report_date`, `approx`, `n_quarters` | 1/quarter | Earnings trend (banks add net_interest_income, gross_loan, total_deposit) |
+| GET `/v2/company/corporate-actions/{symbol}/` | — | 1 | Splits/rights/warrants/AGM/dividends → Dividend Calendar, Event agent |
+| GET `/v2/company/shareholders-composition/{symbol}/` | `year` (≥2021) | 1 | Local vs foreign holder mix (9 categories × _l/_f) |
+| GET `/v2/listing-performance/{symbol}/` | — (post-May-2005 only) | 1 | IPO context (7/30/90/365d windows) |
+
+## C. Universe polling (2) — cheap sweeps, no per-ticker loop
+| Method + path | Params | Cost | FlowSight use |
+|---|---|---|---|
+| GET `/v2/close/` | `date` (default latest), `limit`, `offset` | 1/page | Full-universe close, one sweep per cycle |
+| GET `/v2/companies/quarterly-financial-dates/` | `year`, `since`, `limit`≤30, `offset` | 1/page | Freshness polling: `since=` returns only newly-reported companies |
+
+## D. Market & rankings (5)
+| Method + path | Params | Cost | FlowSight use |
+|---|---|---|---|
+| GET `/v2/daily/{symbol}/` | `start`, `end` (≤90d) | 1 | Price+volume+MCap series per watchlist ticker |
+| GET `/v2/idx-total/` | `start`, `end` (≤90d, ≥2021-01-01) | 1 | IHSG total MCap trend (macro context) |
+| GET `/v2/index-daily/{index_code}/` | lq45, idx30, kompas100, ihsg, jii70… (≥2019-01-02) | 1 | Index benchmark for beta/correlation |
+| GET `/v2/companies/top-changes/` | `classifications` top_gainers/top_losers, `periods` 1d/7d/14d/30d/365d, `sub_sector`, `n_stock`, `min_mcap_billion` | 1 per class×period | Momentum input (request minimal combos) |
+| GET `/v2/most-traded/` | `start`, `end`, `sub_sector`, `n_stock`, `adjusted` | 2 | Relative volume leaders |
+
+## E. Brokers — the moat (7)
+| Method + path | Params | Cost | FlowSight use |
+|---|---|---|---|
+| GET `/v2/brokers/` | `cohort` (retail/mixed/institutional/unknown), `origin` (foreign/domestic) | 1 | Registry cache → classify every code seen |
+| GET `/v2/brokers/top/` | `date`, `metric`, `n_brokers`, `origin`, `cohort` | 2 | Daily broker ranking → who is active today |
+| GET `/v2/broker-activity/{broker_code}/` | `symbol`, `start`, `end` (≤14d) | 1 | All (stock,day) rows per broker |
+| GET `/v2/broker-activity/{broker_code}/top/` | `start`, `end`, `n_brokers` | 2 | Top accumulations/distributions per broker |
+| GET `/v2/broker-summary/{symbol}/` | `broker_code`, `start`, `end` (≤14d) | 1 | Per-broker daily rows per ticker (lots, freq, avg price) |
+| GET `/v2/broker-summary/{symbol}/top/` | `start`, `end`, `cohort`, `origin`, `n_brokers` | 2 | Top buyers/sellers per ticker → accumulation rule |
+| GET `/v2/foreign-flow/{symbol}/` | `start`, `end` (≤90d) | 1 | Net foreign inflow series → reversal + sentiment |
+
+## F. News & events (3)
+| Method + path | Params | Cost | FlowSight use |
+|---|---|---|---|
+| GET `/v2/news/` | `extension`=idx, `sector`, `sub_sector`, `tags`, `symbols`, `keyword`, `start`, `end` | 1 | Sentiment agent input (incremental via `since`-style start) |
+| GET `/v2/filings/` | `symbol`, `sector`, `sub_sector`, `tags`, `transaction_type`, `holder_type`, `start`, `end` | 1 | Insider Tape routine |
+| GET `/v2/suspensions/` | `symbol`, `start`, `end` | 1 | Suspension Watch (reason + IDX PDF link) |
+
+## G. Subsector (2)
+| Method + path | Params | Cost | FlowSight use |
+|---|---|---|---|
+| GET `/v2/subsector/report/{sub_sector}/` (or `?sub_sector=`) | kebab-case slug; `sections` ∈ statistics, market_cap, stability, valuation, growth, companies | 1/section | Sector rotation map, peer medians |
+
+## H. SGX (9) — phase 2, regional extension
+`sgx/companies/` (where/q), `sgx/companies/top/`, `sgx/company/report[/{symbol}]`,
+`sgx/daily/{symbol}/`, `sgx/filings/`, `sgx/news/`, `sgx/buybacks/`, `sgx/short-sell/`,
+`sgx/sectors/`, `sgx/subsectors/`, `sgx/tags/`. Symbols 3–4 chars, output carries `.SI`.
+Killer angle: apply the same routine engine to SGX (short-sell + buybacks have no IDX equivalent).
+
+## I. KLSE (4) — phase 2
+`klse/sectors/`, `klse/companies/?sector=`, `klse/companies/top/`, `klse/company/report[/{symbol}]`.
+Symbols are 4-digit codes (`1155`). Basic coverage only.
+
+## J. Mining (19) — optional commodity vertical
+Companies: list/detail/financials (USD millions)/ownership/performance by `slug`.
+Trade: commodities list, price history (≤3y range), exports (Gold/Copper/Coal),
+global-commodity, sales-destination by slug.
+Sites: index + detail (lat/long), resources-reserves index + per-province detail,
+total-production. Licenses: IUP/IUPK list, auctions + WIUP detail, contracts.
+Angle: commodity-price → mining-stock linkage routine (coal/nickel price moves → watchlist miners).
+
+## Credit budget (per 30-min cycle, W = 20 watchlist tickers)
+| Step | Calls | Credits |
+|---|---|---|
+| close/ sweep | ~10 pages | ~10 |
+| top-changes (1 class × 2 periods) | 1 | 2 |
+| most-traded | 1 | 2 |
+| idx-total | 1 | 1 |
+| brokers/top | 1 | 2 |
+| broker-summary/top + foreign-flow + daily per ticker | 3 × 20 | 80 |
+| news + filings + suspensions (incremental) | 3 | 3 |
+| quarterly-dates universe (`since=`) | ~2 pages | ~2 |
+| **Total per cycle** | | **≈100** |
+Morning briefing extra: report (2–3 sections × 5 tickers ≈ 10–15) + corporate-actions ×5 + quarterly (n=4 ×5 = 20) ≈ 35–40.
+Rules: never full universe quarterly sweep without `since`; cache registry/taxonomy/tags daily; `sections` always explicit.
@@ -0,0 +1,44 @@
+# Backend API
+
+Base `/api`. Demo auth: `X-User-Key` header (single demo key for hackathon).
+Errors: `{error: {code, message}}` with HTTP 400/404/422/502 (502 = upstream Sectors).
+
+## Flow
+
+- `GET /api/flow/summary?date=` → foreign net total, top-5 accumulation rows,
+  rotation signal, mover of day. Each value with citations.
+- `GET /api/flow/broker?ticker=&start=&end=` → buyers/sellers + 5d net series.
+- `GET /api/flow/foreign?ticker=&start=&end=` → inflow series + reversal flag.
+- `GET /api/stream` → SSE (channels: agents, alerts, activity; heartbeat 15 s).
+
+## Screen
+
+- `POST /api/screen` body `{where?, q?, institutional?: {broker_score_min, foreign_trend, insider_buying, volume_anomaly}, limit?}` → ranked rows
+  `{symbol, name, composite, breakdown: {broker, foreign, insider, fundamental}, citations}`.
+
+## Routines & briefing
+
+- `GET /api/routines` → list with enabled + last run status.
+- `POST /api/routines` body `{type, schedule_cron?, channels[]}` → created.
+- `PATCH /api/routines/:id` body `{enabled?, schedule_cron?, channels?}`.
+- `GET /api/routine-runs?routine_id=&limit=` → run history.
+- `GET /api/briefing/today` → latest briefing payload + citations.
+
+## Alerts
+
+- `GET /api/alerts`, `POST /api/alerts` body `{name, rule, channels[]}`,
+  `DELETE /api/alerts/:id`, `GET /api/alert-events?since=&ticker=`.
+
+## Report
+
+- `POST /api/report/:ticker` query `?format=json|html|pdf|md` → 7-section payload
+  with `citations[]` per section. PDF rendered server-side.
+
+## Watchlist / portfolio / accuracy / chat / health
+
+- `GET /api/watchlist`, `POST /api/watchlist` `{ticker}`, `DELETE /api/watchlist/:ticker`.
+- `GET /api/portfolio/risk` → concentration[], correlation[][], beta, warnings[].
+- `GET /api/accuracy` → per-agent `{calls, resolved, hits, hit_rate}`.
+- `POST /api/chat` body `{message, scope?: {report_id}}` → cited answer (report scope
+  restricts grounding to that report's citations).
+- `GET /api/health` → `{last_cycle_at, credits_today, scheduler_ok, stale_flags}`.
@@ -0,0 +1,84 @@
+# Architecture
+
+## Layout
+
+```
+flowsight/
+  TODO.md / ROADMAP.md        # work tracking (shiro-neko style)
+  docs/                       # specs (this folder) — change before code
+  backend/                    # Go 1.23 module (see TECH-STACK.md)
+    cmd/server/main.go        # entrypoint: HTTP server + scheduler in one process
+    internal/
+      config/                 # env (SECTORS_API_KEY), credit budget, schedules
+      sectors/                # SectorsClient + endpoint packages per category
+        client.go             # retry, credit counter, param narrowing defaults
+        screener.go           # companies/, free-float/, taxonomy
+        company.go            # report, segments, quarterly, actions, shareholders
+        market.go             # close/, daily, idx-total, index-daily, movers
+        brokers.go            # registry cache, activity, summary, foreign-flow
+        events.go             # news, filings, suspensions
+      store/
+        db.go                 # database/sql connect + numbered migrations
+        seed.go               # historical-replay fixture loader
+        cache.go              # Redis wrapper + in-memory TTL fallback
+      agents/                 # 7 specialists + synthesizer (see AGENT-SPECS.md)
+        agent.go              # Agent contract: Analyze() -> AgentResult + citations
+        smart_money.go broker_intel.go sentiment.go fundamental.go
+        technical.go catalyst.go synthesizer.go
+      routines/               # 7 routines (see ROUTINES.md)
+        engine.go             # cron dispatch, run recording, delivery
+        briefing.go radar.go reversal.go insider.go earnings.go dividend.go weekly.go
+      alerts/
+        rules.go              # 6 detection rules over snapshots
+        evaluate.go           # per-cycle evaluation
+        notify.go             # Telegram/Discord webhooks
+      reports/
+        builder.go            # 7-section assembly + citations[]
+        render.go             # PDF/HTML/MD/JSON exporters
+      api/                    # chi route handlers (see API.md)
+        flow.go screen.go routines.go briefing.go alerts.go
+        report.go watchlist.go portfolio.go accuracy.go chat.go health.go stream.go
+      scheduler/              # robfig/cron wiring (ingestion + routines)
+  web/                        # SolidJS 1.9 + Vite 6 + StyleX
+    src/
+      pages/                  # Dashboard, Routines, Alerts, Screener, Portfolio, Report
+      components/             # cards, tables, rotation map, correlation matrix
+      lib/api.ts              # typed backend client + SSE hooks
+      styles/                 # StyleX tokens + themes
+  tests/
+    fixtures/                 # historical snapshots (one trading week)
+    agents_*_test.go          # per-agent fixture tests
+    rules_test.go report_test.go budget_test.go
+```
+
+## Conventions
+
+- Spec-first: docs/ updated before code; a PR without a doc touch needs a reason.
+- Every outbound Sectors call goes through `SectorsClient` (credit counted, sections
+  explicit, classification combos minimal). No raw HTTP to the API elsewhere.
+- Every number in user-visible output carries `{endpoint, snapshot_at}` citation.
+  Builders that emit numbers without citations fail review.
+- Tests: one test file per agent/rule + budget test asserting per-cycle credits ≤ cap
+  on fixtures. `go vet` + `gofmt` and `tsc` + `vite build` before commit.
+
+## Scheduler
+
+- Ingestion cycle every 30 min, 09:00–16:00 WIB (market hours). Steps in order:
+  reference cache check → universe sweep → market context → per-watchlist depth →
+  incremental events → quarterly freshness → rule evaluation → routine dispatch.
+- Routine schedules are cron exprs stored per routine row; engine records each run
+  (started_at, status, payload) for the Routine Manager history view.
+
+## Citation pipeline
+
+1. Ingestion stores raw payload + `snapshot_at` in `snapshots`.
+2. Agents/detectors read snapshots, emit values tagged with snapshot IDs.
+3. Reports/alerts/briefings serialize `citations[]` alongside values.
+4. Frontend renders citation chips (endpoint + time); stale (>1 session) chips are
+   visually marked.
+
+## SSE design
+
+- `GET /api/stream` (EventSource): channels `agents` (status+scores during runs),
+  `alerts` (new events), `activity` (feed rows). Heartbeat 15 s; reconnect resumes
+  from last event ID. No WebSocket — one-directional push is all the UI needs.
@@ -0,0 +1,36 @@
+# Data model
+
+SQLite for the hackathon; schema kept Postgres-compatible (serial → integer PK,
+JSON → TEXT with JSON1, no SQLite-only DDL). Migrations numbered in
+`backend/app/store/migrations/`.
+
+## Tables
+
+- `snapshots(id, ticker, date, source, payload_json, fetched_at)` — raw API rows.
+  Index (ticker, date, source). Retention: 180d, then compact to weekly.
+- `broker_activity(broker_code, ticker, date, buy, sell, net, lots, freq, avg_price)`
+  Index (ticker, date), (broker_code, date).
+- `foreign_flow(ticker, date, net_inflow)` — PK (ticker, date).
+- `news_items(id, ticker, date, source, sentiment, confidence, url, title)` —
+  Index (ticker, date).
+- `filings(id, ticker, date, holder_type, txn_type, volume, price)` — Index (ticker, date).
+- `routines(id, user_key, type, schedule_cron, channels_json, enabled)` — 7 types (R1–R7).
+- `routine_runs(id, routine_id, started_at, status, payload_json, credits_used)`.
+- `alerts(id, user_key, name, rule_json, channels_json, last_fired)`.
+- `alert_events(id, alert_id, ticker, date, message, context_json, citations_json)`.
+- `watchlists(user_key, ticker, added_at)` — PK (user_key, ticker).
+- `reports(id, ticker, generated_at, payload_json, citations_json)`.
+- `agent_accuracy(id, agent, ticker, prediction, predict_date, resolved, hit, actual_return)`.
+- `briefings(date, payload_json, citations_json)` — PK date.
+- `credit_ledger(date, endpoint, calls, credits)` — daily spend audit.
+
+## Seed strategy
+
+`seed.py` loads one historical trading week into snapshots + derived tables so the
+full demo (briefing → radar → report → interrogation) runs offline. Fixtures live in
+`tests/fixtures/` as JSON exports of real API shapes (field names match schema.json).
+
+## Cursors
+
+- `meta(key, value)`: `quarterly_since` (universe poll cursor), `news_since`,
+  `filings_since` — persisted so restarts resume incrementally.
@@ -0,0 +1,187 @@
+# FlowSight — Track 02: Automation & Workflows
+
+**One-liner:** FlowSight is for Indonesian retail investors who can't monitor the market all day — it automates institutional money-flow tracking and pushes actionable alerts so they never miss what big players are doing.
+
+## 1. Problem
+
+6M+ retail SID di Indonesia mengambil keputusan dari harga dan rumor. Data yang dipakai
+institusi — arus broker, foreign flow, insider filings — tersedia lewat Sectors API tapi
+mentah dan tercecer di 70 endpoint. Tidak ada retail tool yang mengubahnya menjadi
+rutinitas otomatis: setiap hari investor harus buka app, tarik data manual, dan
+interpret sendiri. Produk existing (StockPilot, Invezgo, Stockbit) semuanya on-demand:
+user bertanya, AI menjawab, selesai. Tidak ada yang bekerja saat user tidur.
+
+## 2. Concept: Autopilot Routines
+
+FlowSight bukan tool yang ditanya — ia rutinitas yang berjalan sendiri. User berlangganan
+routine sekali, agent mengeksekusinya sesuai jadwal, hasilnya tiba di Telegram/Discord
+tanpa user membuka app.
+
+### Routine bawaan (v1)
+1. **Morning Briefing (07:30 WIB)** — top 5 akumulasi semalam, foreign flow kemarin,
+   agenda earnings & ex-div minggu ini. Satu digest, langsung kirim.
+2. **Accumulation Radar (tiap 30 mnt, 09:00–16:00)** — deteksi ≥3 broker net-buy +
+   volume anomali; temuan langsung jadi alert dengan konteks (siapa, berapa, sejak kapan).
+3. **Foreign Reversal Watch** — outflow 5 hari berbalik inflow: sinyal pembalikan yang
+   hampir tidak pernah terpantau manual.
+4. **Insider Tape** — setiap ada director/major-holder buy di watchlist, user tahu
+   hari yang sama beserta volumenya vs rata-rata 30 hari.
+5. **Earnings Countdown** — H-7, H-3, H-1 sebelum laporan kuartalan ticker watchlist,
+   lengkap dengan ekspektasi dari tren 8 kuartal terakhir.
+6. **Dividend Calendar** — ex-date mendekat + yield proyeksi + histori payout, otomatis
+   dari corporate-actions.
+7. **Weekend Review (Sabtu 09:00)** — ringkasan mingguan portofolio: apa yang bergerak,
+   kenapa, dan apa yang perlu perhatian minggu depan.
+
+### Kenapa ini unik
+- Kompetitor menunggu ditanya. FlowSight bekerja tanpa ditanya.
+- Setiap routine = pipeline nyata (ingest → detect → synthesize → deliver), bukan
+  satu LLM call. Inilah inti Track 02: data Sectors hidup di dalam rutinitas berulang.
+- User membangun kebiasaan lewat produk, bukan lewat usaha: buka Telegram pagi,
+  briefing sudah ada.
+
+## 3. Verifiable AI (differentiator kedua)
+
+Setiap angka di setiap output menempel ke sumbernya: endpoint Sectors + timestamp snapshot.
+Contoh: "Foreign inflow Rp 340M/hari (foreign-flow/BBCA, snapshot 12 Sep 16:00 WIB)".
+User bisa klik dan memverifikasi. Tidak ada klaim tanpa jejak. Ini menjawab masalah
+terbesar AI finansial: halusinasi angka yang terdengar meyakinkan.
+
+- Report menyimpan `citations[]`: setiap section menunjuk ke snapshot ID.
+- Alert menyertakan data mentah ringkas + link ke dashboard detail.
+- Jika data basi (>1 sesi), output menandainya eksplisit sebagai stale.
+
+## 4. Accuracy Ledger (differentiator ketiga)
+
+Setiap rekomendasi BUY/HOLD/AVOID dicatat dengan tanggal, lalu dievaluasi 30 hari
+kemudian terhadap actual return. Hasilnya tampil publik per agent di dashboard:
+"Smart Money Tracker: 68% tepat (47/69 calls)". Bobot agent di synthesizer mengikuti
+rekam jejak, bukan asumsi. Tidak ada kompetitor IDX yang membuka track record
+modelnya sendiri.
+
+## 5. Agent System
+
+7 specialist agents, dieksekusi paralel via asyncio, diorkestrasi scheduler + on-demand.
+
+| Agent | Input (Sectors API) | Output |
+|---|---|---|
+| Smart Money Tracker | broker-summary-top, broker-activity-top, foreign-flow | Skor -100..+100, fase akumulasi, pemain kunci |
+| Broker Intel | broker-registry, broker-activity-by-code, brokers/top | Klasifikasi perilaku broker, sinyal rotasi sektor |
+| News Sentiment (Adaptive RAG) | news, filings, suspensions | Skor sentimen + tren, ringkasan insider, event kunci |
+| Fundamental | company/report, quarterly-financials, segments | Skor fundamental, valuasi vs peers, grade A–F |
+| Technical | daily-transaction, most-traded, top-changes, free-float | Sinyal momentum, anomali volume, grade likuiditas |
+| Event Catalyst | corporate-actions, quarterly-dates, IPO performance | Kalender katalis, skor peluang event |
+| Master Synthesizer | 6 output + risk profile + bobot accuracy | BUY/HOLD/AVOID + conviction 1–5 + tesis + sizing |
+
+### Agent features
+1. **Watchtower mode** — agents jalan tiap 30 menit saat market hours; temuan penting
+   langsung jadi alert.
+2. **Cross-signal correlation** — confidence naik saat sinyal selaras (fundamental
+   bullish + akumulasi + sentimen naik); conflict flag saat bertentangan (fundamental
+   bagus tapi broker jualan).
+3. **Report interrogation** — tiap report bisa ditanya follow-up ("kenapa conviction
+   cuma 3?"), jawaban grounding ke data report itu.
+4. **Natural-language screener** — parameter `q=` Sectors + filter institusional
+   (broker score, foreign trend, insider buying) yang di-compute sendiri.
+5. **Live agent panel** — SSE stream status 7 agents + skor real-time di dashboard.
+
+## 6. Platform Features
+1. **Smart Money Dashboard** — foreign net flow, tabel akumulasi broker, peta rotasi
+   sektor, activity feed real-time.
+2. **Routine Manager** — subscribe/unsubscribe routine, atur jadwal + kanal notifikasi
+   per routine, riwayat eksekusi.
+3. **Institutional Screener** — query builder SQL-like + NL toggle, saved screeners,
+   hasil berperingkat + breakdown sinyal.
+4. **Alert Engine** — user rules + auto alert → webhook Telegram/Discord.
+5. **One-Click Report** — research report 7 section, export PDF/HTML/MD/JSON,
+   lengkap dengan citations.
+6. **Portfolio Risk** — konsentrasi sektor, matriks korelasi, beta vs IHSG.
+7. **AI Chat sidebar** — context-aware dari watchlist.
+
+## 7. Architecture (detail: docs/ARCHITECTURE.md; stack: docs/TECH-STACK.md)
+- Frontend: Next.js 15 + React 19 + TypeScript + Tailwind v4 + Recharts (SSE streaming, responsive)
+- Backend: Python 3.12 + FastAPI + Uvicorn + httpx (async) + Pydantic v2 (eksekusi agent paralel)
+- LLM: OpenAI SDK v1 provider-agnostic (`LLM_BASE_URL`), gpt-4o-mini triage + gpt-4o synthesis
+- Data: Sectors API v2 `https://api.sectors.app/v2/`, auth `Authorization: <key>`
+  dari env `SECTORS_API_KEY`
+- Store: SQLite (stdlib, skema Postgres-compatible) + Redis 7 cache (degradasi in-memory jika kosong)
+- Scheduler: APScheduler AsyncIO, ingestion tiap 30 min saat market hours + routine harian/mingguan
+- Notify: outbound webhook → Telegram / Discord
+- PDF: ReportLab (tanpa system deps); test: pytest + respx + fakeredis; gate: ruff + mypy + tsc + next build
+
+## 8. Data model
+- `snapshots(ticker, date, source, payload)` — raw ingestion
+- `broker_activity(broker_code, ticker, date, buy, sell, net, lots, freq)`
+- `foreign_flow(ticker, date, net_inflow)`
+- `news_items(ticker, date, source, sentiment, confidence, url)`
+- `filings(ticker, date, holder_type, txn_type, volume)`
+- `routines(id, user_key, type, schedule, channels, enabled)`
+- `routine_runs(routine_id, started_at, status, payload_json)`
+- `alerts(id, user_key, name, rule_json, channels, last_fired)`
+- `alert_events(alert_id, ticker, date, message, context_json, citations_json)`
+- `watchlists(user_key, ticker, added_at)`
+- `reports(id, ticker, generated_at, payload_json, citations_json)`
+- `agent_accuracy(agent, ticker, prediction, date, resolved, hit)`
+- `briefings(date, payload_json, citations_json)`
+
+## 9. Backend routes
+- `GET /api/flow/summary`, `GET /api/flow/broker`, `GET /api/flow/foreign`
+- `POST /api/screen`
+- `GET /api/routines`, `POST /api/routines`, `PATCH /api/routines/:id`, `GET /api/routine-runs`
+- `GET /api/briefing/today`
+- `GET /api/alerts`, `POST /api/alerts`, `DELETE /api/alerts/:id`, `GET /api/alert-events`
+- `POST /api/report/:ticker`
+- `POST /api/watchlist`, `GET /api/watchlist`
+- `GET /api/portfolio/risk`, `GET /api/accuracy`
+- `POST /api/chat`, `GET /api/health`
+
+## 10. Frontend pages
+- `/` Smart Money Dashboard + live agent panel + activity feed
+- `/routines` Routine Manager (subscribe, jadwal, kanal, riwayat)
+- `/screener` institutional screener
+- `/alerts` rule builder + event history
+- `/portfolio` risk heatmap + correlation + accuracy ledger
+- `/report/:ticker` report + citations + interrogation scoped ke report
+- Global: watchlist drawer + AI chat sidebar
+
+## 11. Detection rules v1
+1. Accumulation: ≥3 broker net-buy 5d + volume > 1.5× avg 20d
+2. Foreign reversal: net outflow 5d lalu inflow 1d
+3. Insider spike: director buy > 2× avg 30d
+4. Unusual volume: >3× avg 20d, bukan earnings date
+5. Sector rotation: net broker flow subsector balik arah week-over-week
+6. Suspension watch: suspensi baru di watchlist
+
+## 12. Sectors endpoints (70 paths — detail: docs/API-REFERENCE.md)
+Company core: company/report (8 sections), get-segments, quarterly-financial-dates,
+financials/quarterly, corporate-actions, shareholders-composition, listing-performance.
+Universe sweeps: close/ (full-universe, paginated), companies/quarterly-financial-dates
+(`since=` incremental). Market: daily, idx-total, index-daily, top-changes, most-traded.
+Brokers: brokers/ (registry), brokers/top, broker-activity, broker-activity/top,
+broker-summary, broker-summary/top, foreign-flow. Events: news, filings, suspensions.
+Subsector: subsector/report (6 sections). Phase 2: SGX (9), KLSE (4), mining (19).
+
+## 13. 48h timeline
+| 0–3 | Setup: repo, API client, DB schema, health |
+| 3–8 | Ingestion scheduler + snapshots |
+| 8–14 | 7 agents + cross-signal correlation |
+| 14–20 | Routine engine (briefing + radar) + webhooks |
+| 20–26 | Screener + citations pipeline |
+| 26–32 | Report generator + interrogation |
+| 32–38 | Portfolio risk + accuracy ledger |
+| 38–44 | Frontend wiring + SSE live panel |
+| 44–48 | Polish, demo script, deck |
+
+## 14. Verification
+- `/api/health` last cycle < 35 min saat market hours
+- Routine briefing generate dari snapshot tanpa empty section + citations lengkap
+- Fixture akumulasi → alert event + webhook terkirim ke kanal uji
+- Screener balikin ranked list + breakdown per row
+- Report BBCA < 15s, 7 section terisi dari live API + citations
+- Key hanya dari env, v2 paths only
+- Market tutup → demo pakai historical replay seed
+
+## 15. Risks
+- Butuh Insider API key sebelum jam 0
+- Rate limit → cache Redis + siklus 30 min, tanpa loop per-ticker agresif
+- v1 mati (410) — pakai v2 saja
@@ -0,0 +1,14 @@
+# docs index
+
+Spec-driven source of truth. Code follows these docs; docs change before code.
+
+| Doc | Contents |
+|---|---|
+| [PLAN.md](PLAN.md) | Concept: Autopilot Routines, Verifiable AI, Accuracy Ledger; agents, features, architecture, data model, routes, pages, rules, timeline, verification, risks |
+| [API-REFERENCE.md](API-REFERENCE.md) | All 70 Sectors v2 paths with params, costs, FlowSight usage, per-cycle credit budget |
+| [TECH-STACK.md](TECH-STACK.md) | Pinned versions, deps, why-chosen, declined alternatives, CI gates |
+| [ARCHITECTURE.md](ARCHITECTURE.md) | Backend/frontend layout, scheduler, agent contracts, citation pipeline, SSE design |
+| [ROUTINES.md](ROUTINES.md) | 7 routine specs: schedule, inputs, detection logic, delivery format |
+| [AGENT-SPECS.md](AGENT-SPECS.md) | 7 agent contracts: inputs, processing steps, outputs, verification fixtures |
+| [DATA-MODEL.md](DATA-MODEL.md) | Table schemas, indexes, retention, seed strategy |
+| [API.md](API.md) | Backend route specs: request/response shapes, errors, auth |
@@ -0,0 +1,59 @@
+# Routines
+
+Each routine: schedule, inputs (Sectors endpoints), detection logic, delivery format.
+All routines read snapshots (never live-fetch inside delivery), attach citations,
+and record a `routine_runs` row.
+
+## R1 — Morning Briefing (07:30 WIB daily)
+
+- Inputs: broker-summary/top + foreign-flow (yesterday), corporate-actions (week
+  ahead), quarterly-dates universe (`since=` 7d), top-changes (1d).
+- Logic: top-5 accumulation by net-buy sum; foreign net per watchlist ticker;
+  earnings + ex-div agenda next 7d; biggest 1d mover with one-line cause (news match).
+- Delivery: one Telegram/Discord message, ≤25 lines: header date, 5 accumulation
+  rows (ticker, net Rp, #brokers), foreign table, agenda list, mover of the day.
+
+## R2 — Accumulation Radar (every 30 min, 09:00–16:00 WIB)
+
+- Inputs: broker-summary/top + broker-activity/top per active broker + daily volume.
+- Logic: rule 1 (≥3 brokers net-buy 5d + volume > 1.5× 20d avg). First-fire only
+  per (ticker, 5d window); re-fire requires net-buy sum growth > 25%.
+- Delivery: alert card — ticker, score, top-3 brokers with net values, volume
+  multiple, link to `/report/:ticker`.
+
+## R3 — Foreign Reversal Watch (every 30 min)
+
+- Inputs: foreign-flow per watchlist ticker (rolling 6d).
+- Logic: rule 2 (5d cumulative outflow then 1d inflow, or reverse). Threshold:
+  1d flow magnitude > 2× trailing 5d daily average.
+- Delivery: alert card — direction flip, amounts, 6d mini-series, context line
+  (e.g. "first inflow after 5 selling days").
+
+## R4 — Insider Tape (every 30 min)
+
+- Inputs: filings/ incremental (transaction_type=buy, holder director/major).
+- Logic: rule 3 (buy volume > 2× 30d avg for that ticker, or ≥3 distinct insiders
+  in 7d). Watchlist tickers only for push; others land in dashboard feed.
+- Delivery: alert card — who (holder type), volume, price if present, vs-average
+  multiple, filing date.
+
+## R5 — Earnings Countdown (daily 08:00; fires at H−7, H−3, H−1)
+
+- Inputs: company quarterly-dates per watchlist ticker + financials/quarterly (n≤8).
+- Logic: next expected report ≈ last report + ~90d (refined when universe
+  quarterly-dates shows a new date). Attach 8-quarter revenue/earnings mini-trend.
+- Delivery: countdown card with trend summary + link to full quarterly table.
+
+## R6 — Dividend Calendar (daily 08:00; fires at H−14, H−3)
+
+- Inputs: corporate-actions per watchlist ticker (upcoming + historical dividends).
+- Logic: ex-date within window; projected yield from last close; payout-ratio check
+  from report dividend section (flag > 80% as aggressive).
+- Delivery: calendar card — ex-date, DPS, est. yield, payout flag, history sparkline.
+
+## R7 — Weekend Review (Saturday 09:00)
+
+- Inputs: week snapshots (daily closes, flows, news, filings, routine run history).
+- Logic: week movers per watchlist position, what drove them (top cited event each),
+  open risks (conflict flags, concentration), next-week agenda (earnings/ex-div).
+- Delivery: longer digest (report-lite) + archived to `briefings`.
@@ -0,0 +1,60 @@
+# Tech stack
+
+Pinned versions. Change here before code. CI enforces the gates at the bottom.
+
+## Backend — `backend/` (Go 1.23)
+
+| Piece | Choice | Why |
+|---|---|---|
+| Runtime | Go 1.23 | Single binary, fast cold start on demo machines, `net/http` routing mature since 1.22 |
+| Router | chi v5 | Thin router over stdlib mux (middleware, route groups); no framework lock-in |
+| HTTP client | stdlib `net/http` + tuned `Transport` | One shared client for all Sectors calls (pooling, per-endpoint timeouts) |
+| Validation | go-playground/validator v10 | Request struct tags = kontrak docs/API.md; gagal validasi → 422 |
+| DB | `database/sql` + modernc.org/sqlite (pure Go) | Nol CGO — `mattn/go-sqlite3` butuh gcc dan gagal di mesin juri tanpa toolchain; schema Postgres-compatible, migrasi SQL polos bernomor, tanpa ORM |
+| Cache | go-redis v9; in-memory TTL fallback bila `REDIS_URL` kosong | Cache registry/taxonomy (TTL 24 jam); demo tetap jalan tanpa Redis |
+| Scheduler | robfig/cron v3 | Cron per routine + interval ingestion dalam satu proses |
+| LLM | Plain HTTPS ke endpoint OpenAI-compatible (`LLM_BASE_URL`) | Function calling untuk synthesis/report/briefing; `LLM_MODEL_TRIAGE` murah + `LLM_MODEL_SYNTH` kuat, override lewat env |
+| PDF export | gofpdf (jung-kurt fork) | Pure Go, tanpa system deps |
+| Config | env via `os.Getenv` + `godotenv` untuk dev | Semua secret dari env; contoh di `.env.example` |
+| Test | `go test` + `httptest` (mock upstream Sectors) | Agent/rule/budget tests atas fixture JSON bentuk API asli |
+| Lint/type | `gofmt -l` + `go vet ./...` (+ golangci-lint bila tersedia) | Compiler sudah strict; vet menangkap yang penting |
+
+## Frontend — `web/` (SolidJS + StyleX)
+
+| Piece | Choice | Why |
+|---|---|---|
+| Framework | SolidJS 1.9 + Vite 6 + TypeScript 5.6 | Fine-grained reactivity — ideal untuk live feed/SSE tanpa re-render tree; bundle kecil untuk demo cepat |
+| Styling | StyleX (@stylexjs/stylex + @stylexjs/vite-plugin) | Atomic CSS deterministic, typed via TS, tanpa runtime; ganti Tailwind sepenuhnya |
+| Charts | Chart.js 4 via solid-chartjs | Wrapper Solid resmi untuk flow/series; Recharts React-only jadi tidak dipakai |
+| Data fetch | `fetch` + typed client (`lib/api.ts`) + `EventSource` untuk SSE | Backend satu-satunya sumber kebenaran; web tidak pernah manggil Sectors langsung |
+| Test/gate | `tsc --noEmit` + `vite build` | Cukup untuk hackathon; tanpa e2e framework |
+
+## Package / runtime management
+
+| Piece | Choice | Why |
+|---|---|---|
+| Go deps | Go modules (`go.mod`, vendoring opsional via `go mod vendor`) | Build reproducible; `go build ./...` satu perintah |
+| JS deps | pnpm 9 + `pnpm-lock.yaml` | Install deterministik; fallback `npm` jika pnpm tidak ada |
+| Env | `.env` (tidak di-commit) — `SECTORS_API_KEY`, `LLM_API_KEY`, `LLM_BASE_URL`, `LLM_MODEL_SYNTH`, `LLM_MODEL_TRIAGE`, `TELEGRAM_BOT_TOKEN`, `DISCORD_WEBHOOK_URL`, `REDIS_URL` | Semua secret dari env; contoh di `.env.example` |
+| Procfile dev | dua proses: `go run ./cmd/server` (atau `air` untuk reload) + `vite dev` (+ redis opsional) | Demo tetap jalan tanpa Redis |
+
+## Alternatives declined
+
+- **Python/FastAPI** — startup + packaging demo lebih rapuh (venv, pip) dibanding satu binary Go; konkurensi agent paralel setara via goroutine.
+- **Next.js/React** — overhead framework + re-render model untuk dashboard live; Solid memberi update granular dengan bundle lebih kecil.
+- **Tailwind** — diganti StyleX: atomic, typed, nol runtime, tanpa scanning step.
+- **Recharts** — React-only; Chart.js via solid-chartjs menutup kebutuhan chart di Solid.
+- **mattn/go-sqlite3** — butuh CGO/gcc; modernc pure-Go selalu bisa build.
+- **GORM / sqlc / Alembic-style migrator** — overhead untuk 13 tabel; SQL polos + skrip bernomor cukup dan mudah diaudit juri.
+- **Celery / job queue eksternal** — butuh broker; cron in-process cukup untuk siklus 30 menit.
+- **WebSocket** — push satu arah saja (agents/alerts/activity); SSE lebih simpel + auto-reconnect.
+- **tRPC / GraphQL** — REST + JSON typed tanpa layer tambahan.
+- **MongoDB** — data relasional time-series (ticker × date); SQLite + indeks tepat lebih cepat dibangun.
+
+## CI gates (per commit)
+
+1. `gofmt -l backend/` kosong + `go vet ./...` bersih
+2. `go test ./...` (termasuk budget test: kredit per siklus ≤ cap pada fixture)
+3. `tsc --noEmit` + `vite build` di `web/`
+4. Larangan: tidak ada HTTP call ke `api.sectors.app` di luar `backend/sectors/`;
+   tidak ada angka user-visible tanpa citation (review checklist, bukan linter).