commit 3882c5aef0d77a6f42ee34d823ef1bb2786eaefd Author: asepharyana Date: Mon Sep 14 20:35:02 2026 +0700 Add comprehensive documentation for FlowSight project - Introduced AGENT-SPECS.md detailing the specifications for seven agents including their inputs, processing steps, and outputs. - Created API-REFERENCE.md outlining the Sectors API v2 endpoints, parameters, costs, and usage. - Developed API.md to specify backend routes, request/response structures, and error handling. - Established ARCHITECTURE.md to describe the project layout, conventions, scheduler, and citation pipeline. - Added DATA-MODEL.md to define the database schema, tables, and seed strategy. - Compiled PLAN.md to outline the project concept, problem statement, unique features, and implementation timeline. - Created README.md as an index for documentation with links to all relevant files. - Documented ROUTINES.md detailing the seven automated routines, their schedules, inputs, detection logic, and delivery formats. - Introduced TECH-STACK.md to specify the technology choices and rationale for both backend and frontend components. diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..56816fa --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,82 @@ +# FlowSight Roadmap + +What is built, what is next, and what has been deliberately declined. Reordered when +evidence says the order is wrong. + +Nothing here is a date. Items move to [TODO.md](TODO.md) when they are next up. + +--- + +## Shipped + +- **Plan + API reference from live schema.** docs/PLAN.md (concept, routines, + verifiable AI, accuracy ledger) + docs/API-REFERENCE.md (all 70 v2 paths with + params, costs, per-cycle credit budget). Source: schema.json + docs site. + +--- + +## Next + +### Hackathon core (maps to TODO Now, in order) + +**Foundation — data in, health out.** Scaffold (FastAPI + Next.js), Sectors client +with credit counting + param narrowing, SQLite schema (§8 tables), historical seed. +Health endpoint proves the pipeline breathes. + +**Ingestion — the 30-minute heartbeat.** Scheduler pulls universe sweep (close/), +market context (top-changes minimal combos, most-traded, idx-total, brokers/top), +per-watchlist depth (broker-summary/top, foreign-flow, daily), incremental events +(news/filings/suspensions), quarterly freshness (`since=`). Redis caches slow-moving +reference data. Credit spend stays inside the §12 budget. + +**Agents — seven specialists, one synthesis.** Smart Money Tracker and Broker Intel +are built first (the moat: no competitor fuses broker × foreign flow). Sentiment +(Adaptive RAG), Fundamental, Technical, and Event Catalyst follow the same +input→detect→score contract. Master Synthesizer correlates cross-signal agreement +into conviction, flags conflicts, and sizes positions by risk profile. + +**Routines — data becomes habit.** Routine engine with cron schedules turns agent +output into deliveries: Morning Briefing (07:30), Accumulation Radar (30-min), +Foreign Reversal Watch, Insider Tape, Earnings Countdown, Dividend Calendar, +Weekend Review. Each run is recorded; each delivery carries citations. + +**Trust — verify, then track.** Every number cites endpoint + snapshot timestamp; +stale data is labeled, never hidden. Accuracy Ledger records each recommendation +and resolves it at +30d against actual return; agent weights follow the ledger. + +**Surface — see it, run it, export it.** Dashboard with live agent panel (SSE), +Routine Manager, Institutional Screener, Alert Engine UI, One-Click Report with +interrogation scoped to citations, Portfolio Risk + accuracy table. Demo seed + +script + deck close the loop for judges. + +### Post-hackathon + +- SGX extension (buybacks + short-sell have no IDX equivalent — new angles). +- KLSE basic coverage through the same screener. +- Mining vertical: commodity price → miner watchlist linkage. +- WhatsApp delivery; per-user keys and watchlists; backtest harness for rules. + +--- + +## Later + +- **Push-first mobile.** Native-feel PWA with push for radar alerts; web stays primary. +- **Community routines.** Shareable routine templates (e.g. "dividend hunter", + "foreign follower") with fork counts. +- **Multi-market synthesis.** One briefing spanning IDX + SGX + KLSE positions. +- **LLM cost control.** Spend ceiling per routine run; cheaper model for sentiment + triage, strong model only for synthesis. + +--- + +## Declined + +- **Order execution / trading.** Read-only intelligence; executing trades adds + regulatory surface no hackathon needs. +- **Price prediction models.** Directional forecasting competes on accuracy claims no + 48h build can defend; the ledger tracks recommendations, not price targets. +- **Real-time tick streaming.** Sectors data is end-of-day granularity; pretending + otherwise would fake the product. 30-min cycles match the source. +- **A general chatbot.** Conversational UI exists as a scoped sidebar only; the + product is routines that run without being asked. +- **v1 API support.** v1 returns 410 Gone; no compat layer will be built. diff --git a/TODO.md b/TODO.md new file mode 100644 index 0000000..62a7e15 --- /dev/null +++ b/TODO.md @@ -0,0 +1,107 @@ +# FlowSight TODO + +One item, one outcome, verifiable when done. Longer-term direction lives in +[ROADMAP.md](ROADMAP.md). Feature specs live in [docs/](docs/). + +Conventions: spec-first (docs updated before code), key only from `SECTORS_API_KEY` +env, v2 API paths only, every number in output carries a citation. + +--- + +## Now + +- [ ] **Scaffold + Sectors client + health.** Monorepo `backend/` (Go module, + chi router) + `web/` (SolidJS + Vite + StyleX). `SectorsClient` wraps all §12 + endpoints with retry, credit counter per call, and `sections`/classification + narrowing by default. `GET /api/health` returns last cycle time + credits spent + today. Verify: health 200, one live call to `subsectors/` succeeds. +- [ ] **DB schema + snapshots.** SQLite via modernc.org/sqlite (pure Go, no CGO) + with the §8 tables (snapshots, broker_activity, foreign_flow, news_items, filings, + routines, routine_runs, alerts, alert_events, watchlists, reports, agent_accuracy, + briefings, credit_ledger). Numbered migrations + seed with one historical trading + day. Verify: seed loads, row counts match fixture. +- [ ] **Scheduler + ingestion cycle.** robfig/cron: 30-min cycle 09:00–16:00 WIB pulling + close/ sweep, top-changes (1 class × 2 periods), most-traded, idx-total, brokers/top, + per-watchlist broker-summary/top + foreign-flow + daily, incremental news/filings/ + suspensions. Redis cache (registry/taxonomy daily). Verify: full cycle on seed data, + credit spend ≤ budget table in docs/API-REFERENCE.md. +- [ ] **Smart Money Tracker agent.** Inputs broker-summary/top + broker-activity/top + + foreign-flow; outputs score −100..+100, accumulation phase, key players. Rule: + ≥3 brokers net-buy 5d + volume > 1.5× 20d avg. Verify: fixture BBCA accumulation + scores > +60 with 3 named brokers cited. +- [ ] **Broker Intel agent.** Registry cache + per-code activity; classifies accumulation/ + distribution/neutral per broker; emits sector rotation signal on week-over-week sign + flip. Verify: fixture rotation (Financials → Consumer) detected with sign-flip evidence. +- [ ] **News Sentiment agent (Adaptive RAG).** Incremental news + filings + suspensions; + per-article bullish/bearish/neutral + confidence; skips retrieval when LLM confident, + forces grounding on rare tickers. Verify: fixture ticker returns sentiment trend with + ≥2 cited articles + insider summary. +- [ ] **Fundamental agent.** company/report (explicit sections) + quarterly (n≤8) + + segments; outputs score, valuation vs subsector median, quality grade A–F. Verify: + BBCA fixture shows P/E vs banks median with cited sections. +- [ ] **Technical agent.** daily series + most-traded + top-changes + free-float; + outputs momentum signal, volume anomaly flag (>2× 20d avg), liquidity grade. Verify: + fixture spike 3.2× avg flagged with dates. +- [ ] **Event Catalyst agent.** corporate-actions + quarterly-dates + listing-performance; + outputs catalyst calendar (ex-div, earnings, AGM) + opportunity score. Verify: fixture + ex-div date + yield appear with H−N countdown. +- [ ] **Master Synthesizer.** Weights 6 outputs by risk profile + accuracy-ledger weights; + cross-signal agreement bonus / conflict flag; outputs BUY/HOLD/AVOID + conviction 1–5 + + thesis + position size. Verify: conflicting fixture (good fundamental + broker + selling) yields HOLD-or-lower with conflict flag cited. +- [ ] **Alert engine + webhooks.** Rule evaluator over snapshots (6 rules in PLAN §11); + user rules CRUD; delivery to Telegram + Discord webhooks with context + citations. + Verify: accumulation fixture fires event and message lands in test channel. +- [ ] **Routine engine + Morning Briefing.** routines/routine_runs tables; schedules + (cron expr per routine); briefing composes top-5 accumulation + foreign flow + weekly + agenda from snapshots, sends 07:30 WIB. Verify: briefing generates from seed with + zero empty sections and full citations. +- [ ] **Institutional Screener.** `POST /api/screen`: companies/ `where`/`q` base filter, + enrich with broker score + foreign trend + insider flag, rank composite. UI with + SQL-like + NL toggle + saved screeners. Verify: banks query returns ranked list with + per-row signal breakdown. +- [ ] **One-Click Report.** 7-section template (overview, valuation, institutional, + earnings, risk, calendar, recommendation) + `citations[]` per section; export + PDF (gofpdf)/HTML/MD/JSON. Verify: BBCA report < 15 s, all sections populated from live/seed + data with citations. +- [ ] **Portfolio Risk + Accuracy Ledger.** Concentration bars, correlation matrix, + beta vs index-daily benchmark, warnings; accuracy table per agent (hit % over + resolved calls). Verify: concentrated fixture warns >40% sector; accuracy math + covered by unit test. +- [ ] **Dashboard + live agent panel.** `/` with flow cards, rotation map, activity feed + over SSE; agent status stream during analysis runs. Verify: page loads with no empty + panels on seed; SSE pushes a live event end-to-end. +- [ ] **Routine Manager + Alerts + Report UI.** `/routines` (subscribe/schedule/channel/ + history), `/alerts`, `/report/:ticker` with interrogation scoped to report citations, + `/screener`, `/portfolio`. Verify: subscribe → run → history row appears. +- [ ] **Demo seed + deck.** Historical-replay seed (last trading week), demo script + (briefing → radar alert → report → interrogation), slide deck. Verify: full demo + runs offline from seed with no empty screen. + +--- + +## Next + +- SGX extension: same routine engine on SGX endpoints (buybacks + short-sell angles). +- KLSE basic coverage: sectors/companies/report wired to screener. +- Mining vertical: commodity-price → miner watchlist linkage routine. +- WhatsApp delivery channel alongside Telegram/Discord. +- Simple user keys + per-key watchlists (beyond single demo key). +- Backtest harness for detection rules against historical snapshots. + +--- + +## Maintenance + +- Credit-budget guard: per-cycle credit cap with abort + alert when exceeded. +- Stale-data marking: any output older than one session labeled stale with timestamp. +- Registry/taxonomy/tag cache refresh (daily) with failure fallback to last good. +- `since=` cursor persistence for quarterly-dates and news incremental polls. +- Accuracy resolution job: resolve predictions at +30d, update agent weights. + +--- + +## Done + +- [x] Plan + full API reference from live schema.json (70 paths, costs, budget). +- [x] Tech stack pinned: Go 1.23 backend (chi, modernc sqlite, robfig/cron, gofpdf) + SolidJS + StyleX frontend (Vite, Chart.js). diff --git a/docs/AGENT-SPECS.md b/docs/AGENT-SPECS.md new file mode 100644 index 0000000..f6f3954 --- /dev/null +++ b/docs/AGENT-SPECS.md @@ -0,0 +1,71 @@ +# Agent specs + +Contract: `analyze(ticker_or_scope, snapshots) -> AgentResult(values[], score, citations[])`. +Agents never fetch live; they read snapshots. Fixtures in `tests/fixtures/` prove each. + +## A1 — Smart Money Tracker + +- Reads: broker-summary/top, broker-activity/top, foreign-flow. +- Steps: (1) top buyers/sellers per ticker; (2) per-broker accumulation ranks; + (3) foreign inflow 90d trend; (4) correlate broker net direction vs foreign + direction; (5) classify phase: accumulation / distribution / neutral / conflict. +- Output: score −100..+100, phase, top-3 players with net Rp, direction-agreement flag. +- Fixture: BBCA 5d — 3 domestic brokers net-buy Rp 1.2T + foreign inflow → score > +60. + +## A2 — Broker Intel + +- Reads: brokers/ registry cache, broker-activity per code, brokers/top daily. +- Steps: (1) classify each active broker by origin/cohort; (2) behavior class per + broker (accumulating/distributing/neutral from top/ ranks); (3) sector exposure + shift week-over-week; (4) emit rotation signal on sign flip with evidence rows. +- Output: behavior map, rotation signal (from→to + net Rp delta). +- Fixture: Financials net −Rp 800M → Consumer +Rp 1.1T flip detected. + +## A3 — News Sentiment (Adaptive RAG) + +- Reads: news (incremental), filings, suspensions. +- Steps: (1) fetch candidate articles; (2) LLM confidence check — confident → + answer from context, uncertain (rare ticker) → force retrieval + ground; + (3) per-article sentiment + confidence; (4) aggregate trend + improving/deteriorating/stable; (5) insider summary from filings. +- Output: score −1..+1, trend, key events (≤5, cited), insider line. +- Fixture: ticker with 2 bullish + 1 neutral + 1 director buy → positive trend cited. + +## A4 — Fundamental + +- Reads: company/report (sections=overview,valuation,financials,dividend), + financials/quarterly (n≤8), get-segments. +- Steps: (1) P/E, P/B vs subsector median (subsector/report statistics); + (2) revenue/earnings 8Q trend; (3) ROE trajectory, debt/equity, payout ratio; + (4) grade A–F from weighted rubric (profitability 35, growth 25, leverage 20, payout 20). +- Output: score 0–100, grade, vs-peers table, red flags. +- Fixture: BBCA — premium P/E vs banks median, declining ROE flagged. + +## A5 — Technical + +- Reads: daily (≤90d), most-traded, top-changes, free-float. +- Steps: (1) volume vs 20d avg multiples; (2) momentum positioning from movers; + (3) relative volume vs market; (4) liquidity grade from free-float %. +- Output: momentum signal (strong/up/flat/down), anomaly flag with dates, + liquidity grade. +- Fixture: 3.2× volume spike flagged with date + mover rank cited. + +## A6 — Event Catalyst + +- Reads: corporate-actions, quarterly-dates, listing-performance. +- Steps: (1) upcoming dividends/splits/AGM with dates; (2) next earnings estimate; + (3) IPO-window context for recent listings; (4) score opportunity 0–100 + (yield × certainty − earnings-risk). +- Output: catalyst calendar rows (event, date, H−N, score). +- Fixture: ex-div in 23d with yield + payout flag rendered. + +## A7 — Master Synthesizer + +- Reads: A1–A6 outputs + risk profile + accuracy-ledger weights. +- Steps: (1) weight signals (conservative→fundamental-heavy, aggressive→ + technical+broker-heavy); (2) agreement bonus when ≥3 agents align, conflict + flag when fundamentals oppose flows; (3) conviction 1–5 from weighted score + spread; (4) position size via capped Kelly (max 10% single name); + (5) thesis ≤5 sentences, each claim cited. +- Output: BUY/HOLD/AVOID, conviction, size %, thesis, conflict flags. +- Fixture: good fundamental + broker selling → HOLD-or-lower with conflict cited. diff --git a/docs/API-REFERENCE.md b/docs/API-REFERENCE.md new file mode 100644 index 0000000..0be403b --- /dev/null +++ b/docs/API-REFERENCE.md @@ -0,0 +1,112 @@ +# FlowSight — Sectors API v2 Reference (learned from schema.json + docs) + +Source: `https://docs.sectors.app/schema.json` (OpenAPI, 70 paths) + llms.txt. +Base: `https://api.sectors.app/v2/`. Auth header: `Authorization: ` (REST). +v1 discontinued 2026-05-11 — all `/v1/*` return 410. Use v2 only. + +## Global constraints +- IDX symbol: 4 letters, optional `.jk`, case-insensitive (`BBCA`, `bbca.jk`). +- Broker codes: 2-letter exchange-member IDs (`MG`, `AK`, `CC`) — valid list from `GET /v2/brokers/`. +- Date windows: broker endpoints max 14 days; daily/foreign-flow/idx-total/most-traded max 90 days (clamped). Future `end` → 400. +- Pagination: `limit`/`offset` where listed. `GET /v2/close/` paginated per trading day. +- Credit traps (defaults are expensive — always narrow params): + - `top-changes` default (2 class × 5 periods) = 10 credits → always set `classifications` + `periods`. + - `company/report` default (all 8 sections) = 8 credits → always set `sections`. + - `subsector/report` default (all 6 sections) = 6 credits → always set `sections`. + - `financials/quarterly` = 1 credit per quarter → bound `n_quarters`. + - Universe quarterly-dates full sweep ≈ 32 pages → poll incrementally with `since`. + - `free-float` = 1 credit per 100 companies; filters mutually exclusive (one per request). + - `news`: `extension=idx` vs `extension=mining` params mutually exclusive (400 if mixed). + +## A. Screener & taxonomy (7) +| Method + path | Params | Cost | FlowSight use | +|---|---|---|---| +| GET `/v2/companies/` | `where`, `q`, `order_by`, `desc`, `limit`≤200, `offset`, `include_query_values` (`q` overrides all) | 1 (structured) | NL + SQL screener core | +| GET `/v2/free-float/` | one of `sector`/`sub_sector`/`industry`/`sub_industry` | 1/100 cos | Liquidity grade, sector sweep | +| GET `/v2/subsectors/` | — | 1 | Slug source (cache daily) | +| GET `/v2/industries/` | — | 1 | Slug source (cache daily) | +| GET `/v2/subindustries/` | — | 1 | Slug source (cache daily) | +| GET `/v2/tags/` | — | 1 | News/filing tag filter values (cache daily) | +| GET `/v2/companies/list_companies_with_segments/` | — | 1 | Check segment availability (cache weekly) | + +## B. Company core (7) +| Method + path | Params | Cost | FlowSight use | +|---|---|---|---| +| GET `/v2/company/report/{symbol}/` (or `?symbol=`) | `sections` ∈ overview, valuation, future, peers, financials, dividend, management, ownership | 1/section | Fundamental agent; report sections | +| GET `/v2/company/get-segments/{symbol}/` | `financial_year` | 1 | Revenue breakdown (Sankey-ready) | +| GET `/v2/company/get_quarterly_financial_dates/{symbol}/` | — | 1 | Valid `report_date` values per ticker | +| GET `/v2/financials/quarterly/{symbol}/` | `report_date`, `approx`, `n_quarters` | 1/quarter | Earnings trend (banks add net_interest_income, gross_loan, total_deposit) | +| GET `/v2/company/corporate-actions/{symbol}/` | — | 1 | Splits/rights/warrants/AGM/dividends → Dividend Calendar, Event agent | +| GET `/v2/company/shareholders-composition/{symbol}/` | `year` (≥2021) | 1 | Local vs foreign holder mix (9 categories × _l/_f) | +| GET `/v2/listing-performance/{symbol}/` | — (post-May-2005 only) | 1 | IPO context (7/30/90/365d windows) | + +## C. Universe polling (2) — cheap sweeps, no per-ticker loop +| Method + path | Params | Cost | FlowSight use | +|---|---|---|---| +| GET `/v2/close/` | `date` (default latest), `limit`, `offset` | 1/page | Full-universe close, one sweep per cycle | +| GET `/v2/companies/quarterly-financial-dates/` | `year`, `since`, `limit`≤30, `offset` | 1/page | Freshness polling: `since=` returns only newly-reported companies | + +## D. Market & rankings (5) +| Method + path | Params | Cost | FlowSight use | +|---|---|---|---| +| GET `/v2/daily/{symbol}/` | `start`, `end` (≤90d) | 1 | Price+volume+MCap series per watchlist ticker | +| GET `/v2/idx-total/` | `start`, `end` (≤90d, ≥2021-01-01) | 1 | IHSG total MCap trend (macro context) | +| GET `/v2/index-daily/{index_code}/` | lq45, idx30, kompas100, ihsg, jii70… (≥2019-01-02) | 1 | Index benchmark for beta/correlation | +| GET `/v2/companies/top-changes/` | `classifications` top_gainers/top_losers, `periods` 1d/7d/14d/30d/365d, `sub_sector`, `n_stock`, `min_mcap_billion` | 1 per class×period | Momentum input (request minimal combos) | +| GET `/v2/most-traded/` | `start`, `end`, `sub_sector`, `n_stock`, `adjusted` | 2 | Relative volume leaders | + +## E. Brokers — the moat (7) +| Method + path | Params | Cost | FlowSight use | +|---|---|---|---| +| GET `/v2/brokers/` | `cohort` (retail/mixed/institutional/unknown), `origin` (foreign/domestic) | 1 | Registry cache → classify every code seen | +| GET `/v2/brokers/top/` | `date`, `metric`, `n_brokers`, `origin`, `cohort` | 2 | Daily broker ranking → who is active today | +| GET `/v2/broker-activity/{broker_code}/` | `symbol`, `start`, `end` (≤14d) | 1 | All (stock,day) rows per broker | +| GET `/v2/broker-activity/{broker_code}/top/` | `start`, `end`, `n_brokers` | 2 | Top accumulations/distributions per broker | +| GET `/v2/broker-summary/{symbol}/` | `broker_code`, `start`, `end` (≤14d) | 1 | Per-broker daily rows per ticker (lots, freq, avg price) | +| GET `/v2/broker-summary/{symbol}/top/` | `start`, `end`, `cohort`, `origin`, `n_brokers` | 2 | Top buyers/sellers per ticker → accumulation rule | +| GET `/v2/foreign-flow/{symbol}/` | `start`, `end` (≤90d) | 1 | Net foreign inflow series → reversal + sentiment | + +## F. News & events (3) +| Method + path | Params | Cost | FlowSight use | +|---|---|---|---| +| GET `/v2/news/` | `extension`=idx, `sector`, `sub_sector`, `tags`, `symbols`, `keyword`, `start`, `end` | 1 | Sentiment agent input (incremental via `since`-style start) | +| GET `/v2/filings/` | `symbol`, `sector`, `sub_sector`, `tags`, `transaction_type`, `holder_type`, `start`, `end` | 1 | Insider Tape routine | +| GET `/v2/suspensions/` | `symbol`, `start`, `end` | 1 | Suspension Watch (reason + IDX PDF link) | + +## G. Subsector (2) +| Method + path | Params | Cost | FlowSight use | +|---|---|---|---| +| GET `/v2/subsector/report/{sub_sector}/` (or `?sub_sector=`) | kebab-case slug; `sections` ∈ statistics, market_cap, stability, valuation, growth, companies | 1/section | Sector rotation map, peer medians | + +## H. SGX (9) — phase 2, regional extension +`sgx/companies/` (where/q), `sgx/companies/top/`, `sgx/company/report[/{symbol}]`, +`sgx/daily/{symbol}/`, `sgx/filings/`, `sgx/news/`, `sgx/buybacks/`, `sgx/short-sell/`, +`sgx/sectors/`, `sgx/subsectors/`, `sgx/tags/`. Symbols 3–4 chars, output carries `.SI`. +Killer angle: apply the same routine engine to SGX (short-sell + buybacks have no IDX equivalent). + +## I. KLSE (4) — phase 2 +`klse/sectors/`, `klse/companies/?sector=`, `klse/companies/top/`, `klse/company/report[/{symbol}]`. +Symbols are 4-digit codes (`1155`). Basic coverage only. + +## J. Mining (19) — optional commodity vertical +Companies: list/detail/financials (USD millions)/ownership/performance by `slug`. +Trade: commodities list, price history (≤3y range), exports (Gold/Copper/Coal), +global-commodity, sales-destination by slug. +Sites: index + detail (lat/long), resources-reserves index + per-province detail, +total-production. Licenses: IUP/IUPK list, auctions + WIUP detail, contracts. +Angle: commodity-price → mining-stock linkage routine (coal/nickel price moves → watchlist miners). + +## Credit budget (per 30-min cycle, W = 20 watchlist tickers) +| Step | Calls | Credits | +|---|---|---| +| close/ sweep | ~10 pages | ~10 | +| top-changes (1 class × 2 periods) | 1 | 2 | +| most-traded | 1 | 2 | +| idx-total | 1 | 1 | +| brokers/top | 1 | 2 | +| broker-summary/top + foreign-flow + daily per ticker | 3 × 20 | 80 | +| news + filings + suspensions (incremental) | 3 | 3 | +| quarterly-dates universe (`since=`) | ~2 pages | ~2 | +| **Total per cycle** | | **≈100** | +Morning briefing extra: report (2–3 sections × 5 tickers ≈ 10–15) + corporate-actions ×5 + quarterly (n=4 ×5 = 20) ≈ 35–40. +Rules: never full universe quarterly sweep without `since`; cache registry/taxonomy/tags daily; `sections` always explicit. diff --git a/docs/API.md b/docs/API.md new file mode 100644 index 0000000..da8f886 --- /dev/null +++ b/docs/API.md @@ -0,0 +1,44 @@ +# Backend API + +Base `/api`. Demo auth: `X-User-Key` header (single demo key for hackathon). +Errors: `{error: {code, message}}` with HTTP 400/404/422/502 (502 = upstream Sectors). + +## Flow + +- `GET /api/flow/summary?date=` → foreign net total, top-5 accumulation rows, + rotation signal, mover of day. Each value with citations. +- `GET /api/flow/broker?ticker=&start=&end=` → buyers/sellers + 5d net series. +- `GET /api/flow/foreign?ticker=&start=&end=` → inflow series + reversal flag. +- `GET /api/stream` → SSE (channels: agents, alerts, activity; heartbeat 15 s). + +## Screen + +- `POST /api/screen` body `{where?, q?, institutional?: {broker_score_min, foreign_trend, insider_buying, volume_anomaly}, limit?}` → ranked rows + `{symbol, name, composite, breakdown: {broker, foreign, insider, fundamental}, citations}`. + +## Routines & briefing + +- `GET /api/routines` → list with enabled + last run status. +- `POST /api/routines` body `{type, schedule_cron?, channels[]}` → created. +- `PATCH /api/routines/:id` body `{enabled?, schedule_cron?, channels?}`. +- `GET /api/routine-runs?routine_id=&limit=` → run history. +- `GET /api/briefing/today` → latest briefing payload + citations. + +## Alerts + +- `GET /api/alerts`, `POST /api/alerts` body `{name, rule, channels[]}`, + `DELETE /api/alerts/:id`, `GET /api/alert-events?since=&ticker=`. + +## Report + +- `POST /api/report/:ticker` query `?format=json|html|pdf|md` → 7-section payload + with `citations[]` per section. PDF rendered server-side. + +## Watchlist / portfolio / accuracy / chat / health + +- `GET /api/watchlist`, `POST /api/watchlist` `{ticker}`, `DELETE /api/watchlist/:ticker`. +- `GET /api/portfolio/risk` → concentration[], correlation[][], beta, warnings[]. +- `GET /api/accuracy` → per-agent `{calls, resolved, hits, hit_rate}`. +- `POST /api/chat` body `{message, scope?: {report_id}}` → cited answer (report scope + restricts grounding to that report's citations). +- `GET /api/health` → `{last_cycle_at, credits_today, scheduler_ok, stale_flags}`. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..8e12bdb --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,84 @@ +# Architecture + +## Layout + +``` +flowsight/ + TODO.md / ROADMAP.md # work tracking (shiro-neko style) + docs/ # specs (this folder) — change before code + backend/ # Go 1.23 module (see TECH-STACK.md) + cmd/server/main.go # entrypoint: HTTP server + scheduler in one process + internal/ + config/ # env (SECTORS_API_KEY), credit budget, schedules + sectors/ # SectorsClient + endpoint packages per category + client.go # retry, credit counter, param narrowing defaults + screener.go # companies/, free-float/, taxonomy + company.go # report, segments, quarterly, actions, shareholders + market.go # close/, daily, idx-total, index-daily, movers + brokers.go # registry cache, activity, summary, foreign-flow + events.go # news, filings, suspensions + store/ + db.go # database/sql connect + numbered migrations + seed.go # historical-replay fixture loader + cache.go # Redis wrapper + in-memory TTL fallback + agents/ # 7 specialists + synthesizer (see AGENT-SPECS.md) + agent.go # Agent contract: Analyze() -> AgentResult + citations + smart_money.go broker_intel.go sentiment.go fundamental.go + technical.go catalyst.go synthesizer.go + routines/ # 7 routines (see ROUTINES.md) + engine.go # cron dispatch, run recording, delivery + briefing.go radar.go reversal.go insider.go earnings.go dividend.go weekly.go + alerts/ + rules.go # 6 detection rules over snapshots + evaluate.go # per-cycle evaluation + notify.go # Telegram/Discord webhooks + reports/ + builder.go # 7-section assembly + citations[] + render.go # PDF/HTML/MD/JSON exporters + api/ # chi route handlers (see API.md) + flow.go screen.go routines.go briefing.go alerts.go + report.go watchlist.go portfolio.go accuracy.go chat.go health.go stream.go + scheduler/ # robfig/cron wiring (ingestion + routines) + web/ # SolidJS 1.9 + Vite 6 + StyleX + src/ + pages/ # Dashboard, Routines, Alerts, Screener, Portfolio, Report + components/ # cards, tables, rotation map, correlation matrix + lib/api.ts # typed backend client + SSE hooks + styles/ # StyleX tokens + themes + tests/ + fixtures/ # historical snapshots (one trading week) + agents_*_test.go # per-agent fixture tests + rules_test.go report_test.go budget_test.go +``` + +## Conventions + +- Spec-first: docs/ updated before code; a PR without a doc touch needs a reason. +- Every outbound Sectors call goes through `SectorsClient` (credit counted, sections + explicit, classification combos minimal). No raw HTTP to the API elsewhere. +- Every number in user-visible output carries `{endpoint, snapshot_at}` citation. + Builders that emit numbers without citations fail review. +- Tests: one test file per agent/rule + budget test asserting per-cycle credits ≤ cap + on fixtures. `go vet` + `gofmt` and `tsc` + `vite build` before commit. + +## Scheduler + +- Ingestion cycle every 30 min, 09:00–16:00 WIB (market hours). Steps in order: + reference cache check → universe sweep → market context → per-watchlist depth → + incremental events → quarterly freshness → rule evaluation → routine dispatch. +- Routine schedules are cron exprs stored per routine row; engine records each run + (started_at, status, payload) for the Routine Manager history view. + +## Citation pipeline + +1. Ingestion stores raw payload + `snapshot_at` in `snapshots`. +2. Agents/detectors read snapshots, emit values tagged with snapshot IDs. +3. Reports/alerts/briefings serialize `citations[]` alongside values. +4. Frontend renders citation chips (endpoint + time); stale (>1 session) chips are + visually marked. + +## SSE design + +- `GET /api/stream` (EventSource): channels `agents` (status+scores during runs), + `alerts` (new events), `activity` (feed rows). Heartbeat 15 s; reconnect resumes + from last event ID. No WebSocket — one-directional push is all the UI needs. diff --git a/docs/DATA-MODEL.md b/docs/DATA-MODEL.md new file mode 100644 index 0000000..1a5cad6 --- /dev/null +++ b/docs/DATA-MODEL.md @@ -0,0 +1,36 @@ +# Data model + +SQLite for the hackathon; schema kept Postgres-compatible (serial → integer PK, +JSON → TEXT with JSON1, no SQLite-only DDL). Migrations numbered in +`backend/app/store/migrations/`. + +## Tables + +- `snapshots(id, ticker, date, source, payload_json, fetched_at)` — raw API rows. + Index (ticker, date, source). Retention: 180d, then compact to weekly. +- `broker_activity(broker_code, ticker, date, buy, sell, net, lots, freq, avg_price)` + Index (ticker, date), (broker_code, date). +- `foreign_flow(ticker, date, net_inflow)` — PK (ticker, date). +- `news_items(id, ticker, date, source, sentiment, confidence, url, title)` — + Index (ticker, date). +- `filings(id, ticker, date, holder_type, txn_type, volume, price)` — Index (ticker, date). +- `routines(id, user_key, type, schedule_cron, channels_json, enabled)` — 7 types (R1–R7). +- `routine_runs(id, routine_id, started_at, status, payload_json, credits_used)`. +- `alerts(id, user_key, name, rule_json, channels_json, last_fired)`. +- `alert_events(id, alert_id, ticker, date, message, context_json, citations_json)`. +- `watchlists(user_key, ticker, added_at)` — PK (user_key, ticker). +- `reports(id, ticker, generated_at, payload_json, citations_json)`. +- `agent_accuracy(id, agent, ticker, prediction, predict_date, resolved, hit, actual_return)`. +- `briefings(date, payload_json, citations_json)` — PK date. +- `credit_ledger(date, endpoint, calls, credits)` — daily spend audit. + +## Seed strategy + +`seed.py` loads one historical trading week into snapshots + derived tables so the +full demo (briefing → radar → report → interrogation) runs offline. Fixtures live in +`tests/fixtures/` as JSON exports of real API shapes (field names match schema.json). + +## Cursors + +- `meta(key, value)`: `quarterly_since` (universe poll cursor), `news_since`, + `filings_since` — persisted so restarts resume incrementally. diff --git a/docs/PLAN.md b/docs/PLAN.md new file mode 100644 index 0000000..edf3326 --- /dev/null +++ b/docs/PLAN.md @@ -0,0 +1,187 @@ +# FlowSight — Track 02: Automation & Workflows + +**One-liner:** FlowSight is for Indonesian retail investors who can't monitor the market all day — it automates institutional money-flow tracking and pushes actionable alerts so they never miss what big players are doing. + +## 1. Problem + +6M+ retail SID di Indonesia mengambil keputusan dari harga dan rumor. Data yang dipakai +institusi — arus broker, foreign flow, insider filings — tersedia lewat Sectors API tapi +mentah dan tercecer di 70 endpoint. Tidak ada retail tool yang mengubahnya menjadi +rutinitas otomatis: setiap hari investor harus buka app, tarik data manual, dan +interpret sendiri. Produk existing (StockPilot, Invezgo, Stockbit) semuanya on-demand: +user bertanya, AI menjawab, selesai. Tidak ada yang bekerja saat user tidur. + +## 2. Concept: Autopilot Routines + +FlowSight bukan tool yang ditanya — ia rutinitas yang berjalan sendiri. User berlangganan +routine sekali, agent mengeksekusinya sesuai jadwal, hasilnya tiba di Telegram/Discord +tanpa user membuka app. + +### Routine bawaan (v1) +1. **Morning Briefing (07:30 WIB)** — top 5 akumulasi semalam, foreign flow kemarin, + agenda earnings & ex-div minggu ini. Satu digest, langsung kirim. +2. **Accumulation Radar (tiap 30 mnt, 09:00–16:00)** — deteksi ≥3 broker net-buy + + volume anomali; temuan langsung jadi alert dengan konteks (siapa, berapa, sejak kapan). +3. **Foreign Reversal Watch** — outflow 5 hari berbalik inflow: sinyal pembalikan yang + hampir tidak pernah terpantau manual. +4. **Insider Tape** — setiap ada director/major-holder buy di watchlist, user tahu + hari yang sama beserta volumenya vs rata-rata 30 hari. +5. **Earnings Countdown** — H-7, H-3, H-1 sebelum laporan kuartalan ticker watchlist, + lengkap dengan ekspektasi dari tren 8 kuartal terakhir. +6. **Dividend Calendar** — ex-date mendekat + yield proyeksi + histori payout, otomatis + dari corporate-actions. +7. **Weekend Review (Sabtu 09:00)** — ringkasan mingguan portofolio: apa yang bergerak, + kenapa, dan apa yang perlu perhatian minggu depan. + +### Kenapa ini unik +- Kompetitor menunggu ditanya. FlowSight bekerja tanpa ditanya. +- Setiap routine = pipeline nyata (ingest → detect → synthesize → deliver), bukan + satu LLM call. Inilah inti Track 02: data Sectors hidup di dalam rutinitas berulang. +- User membangun kebiasaan lewat produk, bukan lewat usaha: buka Telegram pagi, + briefing sudah ada. + +## 3. Verifiable AI (differentiator kedua) + +Setiap angka di setiap output menempel ke sumbernya: endpoint Sectors + timestamp snapshot. +Contoh: "Foreign inflow Rp 340M/hari (foreign-flow/BBCA, snapshot 12 Sep 16:00 WIB)". +User bisa klik dan memverifikasi. Tidak ada klaim tanpa jejak. Ini menjawab masalah +terbesar AI finansial: halusinasi angka yang terdengar meyakinkan. + +- Report menyimpan `citations[]`: setiap section menunjuk ke snapshot ID. +- Alert menyertakan data mentah ringkas + link ke dashboard detail. +- Jika data basi (>1 sesi), output menandainya eksplisit sebagai stale. + +## 4. Accuracy Ledger (differentiator ketiga) + +Setiap rekomendasi BUY/HOLD/AVOID dicatat dengan tanggal, lalu dievaluasi 30 hari +kemudian terhadap actual return. Hasilnya tampil publik per agent di dashboard: +"Smart Money Tracker: 68% tepat (47/69 calls)". Bobot agent di synthesizer mengikuti +rekam jejak, bukan asumsi. Tidak ada kompetitor IDX yang membuka track record +modelnya sendiri. + +## 5. Agent System + +7 specialist agents, dieksekusi paralel via asyncio, diorkestrasi scheduler + on-demand. + +| Agent | Input (Sectors API) | Output | +|---|---|---| +| Smart Money Tracker | broker-summary-top, broker-activity-top, foreign-flow | Skor -100..+100, fase akumulasi, pemain kunci | +| Broker Intel | broker-registry, broker-activity-by-code, brokers/top | Klasifikasi perilaku broker, sinyal rotasi sektor | +| News Sentiment (Adaptive RAG) | news, filings, suspensions | Skor sentimen + tren, ringkasan insider, event kunci | +| Fundamental | company/report, quarterly-financials, segments | Skor fundamental, valuasi vs peers, grade A–F | +| Technical | daily-transaction, most-traded, top-changes, free-float | Sinyal momentum, anomali volume, grade likuiditas | +| Event Catalyst | corporate-actions, quarterly-dates, IPO performance | Kalender katalis, skor peluang event | +| Master Synthesizer | 6 output + risk profile + bobot accuracy | BUY/HOLD/AVOID + conviction 1–5 + tesis + sizing | + +### Agent features +1. **Watchtower mode** — agents jalan tiap 30 menit saat market hours; temuan penting + langsung jadi alert. +2. **Cross-signal correlation** — confidence naik saat sinyal selaras (fundamental + bullish + akumulasi + sentimen naik); conflict flag saat bertentangan (fundamental + bagus tapi broker jualan). +3. **Report interrogation** — tiap report bisa ditanya follow-up ("kenapa conviction + cuma 3?"), jawaban grounding ke data report itu. +4. **Natural-language screener** — parameter `q=` Sectors + filter institusional + (broker score, foreign trend, insider buying) yang di-compute sendiri. +5. **Live agent panel** — SSE stream status 7 agents + skor real-time di dashboard. + +## 6. Platform Features +1. **Smart Money Dashboard** — foreign net flow, tabel akumulasi broker, peta rotasi + sektor, activity feed real-time. +2. **Routine Manager** — subscribe/unsubscribe routine, atur jadwal + kanal notifikasi + per routine, riwayat eksekusi. +3. **Institutional Screener** — query builder SQL-like + NL toggle, saved screeners, + hasil berperingkat + breakdown sinyal. +4. **Alert Engine** — user rules + auto alert → webhook Telegram/Discord. +5. **One-Click Report** — research report 7 section, export PDF/HTML/MD/JSON, + lengkap dengan citations. +6. **Portfolio Risk** — konsentrasi sektor, matriks korelasi, beta vs IHSG. +7. **AI Chat sidebar** — context-aware dari watchlist. + +## 7. Architecture (detail: docs/ARCHITECTURE.md; stack: docs/TECH-STACK.md) +- Frontend: Next.js 15 + React 19 + TypeScript + Tailwind v4 + Recharts (SSE streaming, responsive) +- Backend: Python 3.12 + FastAPI + Uvicorn + httpx (async) + Pydantic v2 (eksekusi agent paralel) +- LLM: OpenAI SDK v1 provider-agnostic (`LLM_BASE_URL`), gpt-4o-mini triage + gpt-4o synthesis +- Data: Sectors API v2 `https://api.sectors.app/v2/`, auth `Authorization: ` + dari env `SECTORS_API_KEY` +- Store: SQLite (stdlib, skema Postgres-compatible) + Redis 7 cache (degradasi in-memory jika kosong) +- Scheduler: APScheduler AsyncIO, ingestion tiap 30 min saat market hours + routine harian/mingguan +- Notify: outbound webhook → Telegram / Discord +- PDF: ReportLab (tanpa system deps); test: pytest + respx + fakeredis; gate: ruff + mypy + tsc + next build + +## 8. Data model +- `snapshots(ticker, date, source, payload)` — raw ingestion +- `broker_activity(broker_code, ticker, date, buy, sell, net, lots, freq)` +- `foreign_flow(ticker, date, net_inflow)` +- `news_items(ticker, date, source, sentiment, confidence, url)` +- `filings(ticker, date, holder_type, txn_type, volume)` +- `routines(id, user_key, type, schedule, channels, enabled)` +- `routine_runs(routine_id, started_at, status, payload_json)` +- `alerts(id, user_key, name, rule_json, channels, last_fired)` +- `alert_events(alert_id, ticker, date, message, context_json, citations_json)` +- `watchlists(user_key, ticker, added_at)` +- `reports(id, ticker, generated_at, payload_json, citations_json)` +- `agent_accuracy(agent, ticker, prediction, date, resolved, hit)` +- `briefings(date, payload_json, citations_json)` + +## 9. Backend routes +- `GET /api/flow/summary`, `GET /api/flow/broker`, `GET /api/flow/foreign` +- `POST /api/screen` +- `GET /api/routines`, `POST /api/routines`, `PATCH /api/routines/:id`, `GET /api/routine-runs` +- `GET /api/briefing/today` +- `GET /api/alerts`, `POST /api/alerts`, `DELETE /api/alerts/:id`, `GET /api/alert-events` +- `POST /api/report/:ticker` +- `POST /api/watchlist`, `GET /api/watchlist` +- `GET /api/portfolio/risk`, `GET /api/accuracy` +- `POST /api/chat`, `GET /api/health` + +## 10. Frontend pages +- `/` Smart Money Dashboard + live agent panel + activity feed +- `/routines` Routine Manager (subscribe, jadwal, kanal, riwayat) +- `/screener` institutional screener +- `/alerts` rule builder + event history +- `/portfolio` risk heatmap + correlation + accuracy ledger +- `/report/:ticker` report + citations + interrogation scoped ke report +- Global: watchlist drawer + AI chat sidebar + +## 11. Detection rules v1 +1. Accumulation: ≥3 broker net-buy 5d + volume > 1.5× avg 20d +2. Foreign reversal: net outflow 5d lalu inflow 1d +3. Insider spike: director buy > 2× avg 30d +4. Unusual volume: >3× avg 20d, bukan earnings date +5. Sector rotation: net broker flow subsector balik arah week-over-week +6. Suspension watch: suspensi baru di watchlist + +## 12. Sectors endpoints (70 paths — detail: docs/API-REFERENCE.md) +Company core: company/report (8 sections), get-segments, quarterly-financial-dates, +financials/quarterly, corporate-actions, shareholders-composition, listing-performance. +Universe sweeps: close/ (full-universe, paginated), companies/quarterly-financial-dates +(`since=` incremental). Market: daily, idx-total, index-daily, top-changes, most-traded. +Brokers: brokers/ (registry), brokers/top, broker-activity, broker-activity/top, +broker-summary, broker-summary/top, foreign-flow. Events: news, filings, suspensions. +Subsector: subsector/report (6 sections). Phase 2: SGX (9), KLSE (4), mining (19). + +## 13. 48h timeline +| 0–3 | Setup: repo, API client, DB schema, health | +| 3–8 | Ingestion scheduler + snapshots | +| 8–14 | 7 agents + cross-signal correlation | +| 14–20 | Routine engine (briefing + radar) + webhooks | +| 20–26 | Screener + citations pipeline | +| 26–32 | Report generator + interrogation | +| 32–38 | Portfolio risk + accuracy ledger | +| 38–44 | Frontend wiring + SSE live panel | +| 44–48 | Polish, demo script, deck | + +## 14. Verification +- `/api/health` last cycle < 35 min saat market hours +- Routine briefing generate dari snapshot tanpa empty section + citations lengkap +- Fixture akumulasi → alert event + webhook terkirim ke kanal uji +- Screener balikin ranked list + breakdown per row +- Report BBCA < 15s, 7 section terisi dari live API + citations +- Key hanya dari env, v2 paths only +- Market tutup → demo pakai historical replay seed + +## 15. Risks +- Butuh Insider API key sebelum jam 0 +- Rate limit → cache Redis + siklus 30 min, tanpa loop per-ticker agresif +- v1 mati (410) — pakai v2 saja diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..30b0efd --- /dev/null +++ b/docs/README.md @@ -0,0 +1,14 @@ +# docs index + +Spec-driven source of truth. Code follows these docs; docs change before code. + +| Doc | Contents | +|---|---| +| [PLAN.md](PLAN.md) | Concept: Autopilot Routines, Verifiable AI, Accuracy Ledger; agents, features, architecture, data model, routes, pages, rules, timeline, verification, risks | +| [API-REFERENCE.md](API-REFERENCE.md) | All 70 Sectors v2 paths with params, costs, FlowSight usage, per-cycle credit budget | +| [TECH-STACK.md](TECH-STACK.md) | Pinned versions, deps, why-chosen, declined alternatives, CI gates | +| [ARCHITECTURE.md](ARCHITECTURE.md) | Backend/frontend layout, scheduler, agent contracts, citation pipeline, SSE design | +| [ROUTINES.md](ROUTINES.md) | 7 routine specs: schedule, inputs, detection logic, delivery format | +| [AGENT-SPECS.md](AGENT-SPECS.md) | 7 agent contracts: inputs, processing steps, outputs, verification fixtures | +| [DATA-MODEL.md](DATA-MODEL.md) | Table schemas, indexes, retention, seed strategy | +| [API.md](API.md) | Backend route specs: request/response shapes, errors, auth | diff --git a/docs/ROUTINES.md b/docs/ROUTINES.md new file mode 100644 index 0000000..473bc66 --- /dev/null +++ b/docs/ROUTINES.md @@ -0,0 +1,59 @@ +# Routines + +Each routine: schedule, inputs (Sectors endpoints), detection logic, delivery format. +All routines read snapshots (never live-fetch inside delivery), attach citations, +and record a `routine_runs` row. + +## R1 — Morning Briefing (07:30 WIB daily) + +- Inputs: broker-summary/top + foreign-flow (yesterday), corporate-actions (week + ahead), quarterly-dates universe (`since=` 7d), top-changes (1d). +- Logic: top-5 accumulation by net-buy sum; foreign net per watchlist ticker; + earnings + ex-div agenda next 7d; biggest 1d mover with one-line cause (news match). +- Delivery: one Telegram/Discord message, ≤25 lines: header date, 5 accumulation + rows (ticker, net Rp, #brokers), foreign table, agenda list, mover of the day. + +## R2 — Accumulation Radar (every 30 min, 09:00–16:00 WIB) + +- Inputs: broker-summary/top + broker-activity/top per active broker + daily volume. +- Logic: rule 1 (≥3 brokers net-buy 5d + volume > 1.5× 20d avg). First-fire only + per (ticker, 5d window); re-fire requires net-buy sum growth > 25%. +- Delivery: alert card — ticker, score, top-3 brokers with net values, volume + multiple, link to `/report/:ticker`. + +## R3 — Foreign Reversal Watch (every 30 min) + +- Inputs: foreign-flow per watchlist ticker (rolling 6d). +- Logic: rule 2 (5d cumulative outflow then 1d inflow, or reverse). Threshold: + 1d flow magnitude > 2× trailing 5d daily average. +- Delivery: alert card — direction flip, amounts, 6d mini-series, context line + (e.g. "first inflow after 5 selling days"). + +## R4 — Insider Tape (every 30 min) + +- Inputs: filings/ incremental (transaction_type=buy, holder director/major). +- Logic: rule 3 (buy volume > 2× 30d avg for that ticker, or ≥3 distinct insiders + in 7d). Watchlist tickers only for push; others land in dashboard feed. +- Delivery: alert card — who (holder type), volume, price if present, vs-average + multiple, filing date. + +## R5 — Earnings Countdown (daily 08:00; fires at H−7, H−3, H−1) + +- Inputs: company quarterly-dates per watchlist ticker + financials/quarterly (n≤8). +- Logic: next expected report ≈ last report + ~90d (refined when universe + quarterly-dates shows a new date). Attach 8-quarter revenue/earnings mini-trend. +- Delivery: countdown card with trend summary + link to full quarterly table. + +## R6 — Dividend Calendar (daily 08:00; fires at H−14, H−3) + +- Inputs: corporate-actions per watchlist ticker (upcoming + historical dividends). +- Logic: ex-date within window; projected yield from last close; payout-ratio check + from report dividend section (flag > 80% as aggressive). +- Delivery: calendar card — ex-date, DPS, est. yield, payout flag, history sparkline. + +## R7 — Weekend Review (Saturday 09:00) + +- Inputs: week snapshots (daily closes, flows, news, filings, routine run history). +- Logic: week movers per watchlist position, what drove them (top cited event each), + open risks (conflict flags, concentration), next-week agenda (earnings/ex-div). +- Delivery: longer digest (report-lite) + archived to `briefings`. diff --git a/docs/TECH-STACK.md b/docs/TECH-STACK.md new file mode 100644 index 0000000..2880b91 --- /dev/null +++ b/docs/TECH-STACK.md @@ -0,0 +1,60 @@ +# Tech stack + +Pinned versions. Change here before code. CI enforces the gates at the bottom. + +## Backend — `backend/` (Go 1.23) + +| Piece | Choice | Why | +|---|---|---| +| Runtime | Go 1.23 | Single binary, fast cold start on demo machines, `net/http` routing mature since 1.22 | +| Router | chi v5 | Thin router over stdlib mux (middleware, route groups); no framework lock-in | +| HTTP client | stdlib `net/http` + tuned `Transport` | One shared client for all Sectors calls (pooling, per-endpoint timeouts) | +| Validation | go-playground/validator v10 | Request struct tags = kontrak docs/API.md; gagal validasi → 422 | +| DB | `database/sql` + modernc.org/sqlite (pure Go) | Nol CGO — `mattn/go-sqlite3` butuh gcc dan gagal di mesin juri tanpa toolchain; schema Postgres-compatible, migrasi SQL polos bernomor, tanpa ORM | +| Cache | go-redis v9; in-memory TTL fallback bila `REDIS_URL` kosong | Cache registry/taxonomy (TTL 24 jam); demo tetap jalan tanpa Redis | +| Scheduler | robfig/cron v3 | Cron per routine + interval ingestion dalam satu proses | +| LLM | Plain HTTPS ke endpoint OpenAI-compatible (`LLM_BASE_URL`) | Function calling untuk synthesis/report/briefing; `LLM_MODEL_TRIAGE` murah + `LLM_MODEL_SYNTH` kuat, override lewat env | +| PDF export | gofpdf (jung-kurt fork) | Pure Go, tanpa system deps | +| Config | env via `os.Getenv` + `godotenv` untuk dev | Semua secret dari env; contoh di `.env.example` | +| Test | `go test` + `httptest` (mock upstream Sectors) | Agent/rule/budget tests atas fixture JSON bentuk API asli | +| Lint/type | `gofmt -l` + `go vet ./...` (+ golangci-lint bila tersedia) | Compiler sudah strict; vet menangkap yang penting | + +## Frontend — `web/` (SolidJS + StyleX) + +| Piece | Choice | Why | +|---|---|---| +| Framework | SolidJS 1.9 + Vite 6 + TypeScript 5.6 | Fine-grained reactivity — ideal untuk live feed/SSE tanpa re-render tree; bundle kecil untuk demo cepat | +| Styling | StyleX (@stylexjs/stylex + @stylexjs/vite-plugin) | Atomic CSS deterministic, typed via TS, tanpa runtime; ganti Tailwind sepenuhnya | +| Charts | Chart.js 4 via solid-chartjs | Wrapper Solid resmi untuk flow/series; Recharts React-only jadi tidak dipakai | +| Data fetch | `fetch` + typed client (`lib/api.ts`) + `EventSource` untuk SSE | Backend satu-satunya sumber kebenaran; web tidak pernah manggil Sectors langsung | +| Test/gate | `tsc --noEmit` + `vite build` | Cukup untuk hackathon; tanpa e2e framework | + +## Package / runtime management + +| Piece | Choice | Why | +|---|---|---| +| Go deps | Go modules (`go.mod`, vendoring opsional via `go mod vendor`) | Build reproducible; `go build ./...` satu perintah | +| JS deps | pnpm 9 + `pnpm-lock.yaml` | Install deterministik; fallback `npm` jika pnpm tidak ada | +| Env | `.env` (tidak di-commit) — `SECTORS_API_KEY`, `LLM_API_KEY`, `LLM_BASE_URL`, `LLM_MODEL_SYNTH`, `LLM_MODEL_TRIAGE`, `TELEGRAM_BOT_TOKEN`, `DISCORD_WEBHOOK_URL`, `REDIS_URL` | Semua secret dari env; contoh di `.env.example` | +| Procfile dev | dua proses: `go run ./cmd/server` (atau `air` untuk reload) + `vite dev` (+ redis opsional) | Demo tetap jalan tanpa Redis | + +## Alternatives declined + +- **Python/FastAPI** — startup + packaging demo lebih rapuh (venv, pip) dibanding satu binary Go; konkurensi agent paralel setara via goroutine. +- **Next.js/React** — overhead framework + re-render model untuk dashboard live; Solid memberi update granular dengan bundle lebih kecil. +- **Tailwind** — diganti StyleX: atomic, typed, nol runtime, tanpa scanning step. +- **Recharts** — React-only; Chart.js via solid-chartjs menutup kebutuhan chart di Solid. +- **mattn/go-sqlite3** — butuh CGO/gcc; modernc pure-Go selalu bisa build. +- **GORM / sqlc / Alembic-style migrator** — overhead untuk 13 tabel; SQL polos + skrip bernomor cukup dan mudah diaudit juri. +- **Celery / job queue eksternal** — butuh broker; cron in-process cukup untuk siklus 30 menit. +- **WebSocket** — push satu arah saja (agents/alerts/activity); SSE lebih simpel + auto-reconnect. +- **tRPC / GraphQL** — REST + JSON typed tanpa layer tambahan. +- **MongoDB** — data relasional time-series (ticker × date); SQLite + indeks tepat lebih cepat dibangun. + +## CI gates (per commit) + +1. `gofmt -l backend/` kosong + `go vet ./...` bersih +2. `go test ./...` (termasuk budget test: kredit per siklus ≤ cap pada fixture) +3. `tsc --noEmit` + `vite build` di `web/` +4. Larangan: tidak ada HTTP call ke `api.sectors.app` di luar `backend/sectors/`; + tidak ada angka user-visible tanpa citation (review checklist, bukan linter).