2 Commits
Author SHA1 Message Date
asepharyana 2b2b8d8759 docs: Phase 10 audit — verified all features live, recorded frontmatter fix
CI / typecheck + build (turbo) (push) Canceled after 0s
2026-08-20 11:50:08 +07:00
asepharyana ca2bd19c48 fix(core): strip YAML frontmatter in getDocument before web/MCP render
getDocument preferred the on-disk file via readFileSync (raw), returning
the full frontmatter block as visible text on doc pages and MCP resources.
ReactMarkdown rendered the '---' delimiters as <hr/>, exposing YAML keys
(id/title/type/etc.) as plain paragraphs.

Root cause: getDocument bypassed parseFile's gray-matter stripping.
The indexer correctly stripped frontmatter (DB body was clean), but
getDocument read raw from disk for the 'source of truth' path.

Fix: use parseFile(abs, relPath).body (same as the indexer) so the
on-disk path returns frontmatter-free markdown. DB fallback unchanged.

Affects web doc pages + MCP mcpedia://docs/{+slug} and /chunks resources.
Verified: frontmatter leakage 0/9 doc pages, headings render as <h2>,
browser snapshot clean.
2026-08-20 11:45:37 +07:00
2 changed files with 62 additions and 2 deletions

No files matched your search

+51
View File
@@ -257,6 +257,57 @@ mod: apps/api/src/index.ts (thin re-export), apps/api/src/router.ts (unchange
renamed: apps/mcp/src/smoke.test.ts -> smoke.ts (fixed stale assertions)
```
## Phase 10 — Audit + Bug fixes (live verification)
> Full feature audit against live services. Found + fixed one real bug.
### Bug: Frontmatter leaking into rendered doc pages + MCP doc body
- **Symptom:** Doc pages showed raw YAML frontmatter (`id: websocket-contract`,
`title: WebSocket Contract`, etc.) as visible plain text between `<hr/>`
markers. MCP `mcpedia://docs/{+slug}` resource had the same leak.
- **Root cause:** `getDocument()` in `@mcpedia/core` preferred the on-disk file
via `readFileSync(abs, "utf8")` — returning **raw** file content including the
`---` frontmatter block. The indexer correctly stripped frontmatter via
`parseFile` (gray-matter), but `getDocument` bypassed it. `ReactMarkdown`
rendered `---` as `<hr/>` and the YAML as paragraphs.
- **Fix:** `package/core/src/document.service.ts` — replaced `readFileSync` with
`parseFile(abs, row.path).body` (same frontmatter stripping as the indexer).
DB fallback (`row.body`) unchanged (already clean).
- **Verified:** 9/9 doc pages render clean (no frontmatter `id:` text, proper
`<h2>` + `<code>` elements in SSR HTML); MCP `mcpedia://` doc body resource
returns clean markdown (starts with `# WebSocket Contract`).
### Audit findings (all phases verified live)
| Phase | Feature | Live check | Status |
|-------|---------|------------|--------|
| P1 | Web UI `/docs/<section>/<slug>` | 200, renders markdown | ✅ |
| P1 | Search page (`?q=` + `?mode=hybrid`) | 200, returns results | ✅ |
| P1 | MCP stdio + HTTP (`/4021`) | 10 tools, 4 resources | ✅ |
| P2 | Semantic/hybrid search | returns ranked chunks | ✅ |
| P2 | tRPC API on domain (`/trpc/*`) | listDocuments → 4 docs | ✅ |
| P3 | BullMQ worker drains jobs | queue completed 17→19 after enqueue | ✅ |
| P3 | Revision system | listRevisions → rev #1 "phase4-final-clean" | ✅ |
| P3 | Git webhook auth gate | 401 w/o secret, 200 w/ secret | ✅ |
| P4 | Dashboard | `/dashboard` → 200 HTML | ✅ |
| P6 | All 4 systemd services | web/api/mcp/worker all `active` | ✅ |
| P6 | restoreRevision mutation locked | 401 w/o secret, executes w/ secret | ✅ |
| P7 | 10 MCP tools (6 read + 4 write) | tools/list → 10 | ✅ |
| P7 | Write-tool auth gate | reindex_all w/o secret → isError | ✅ |
| P7 | Prometheus metrics | `/metrics` → 7 gauges, 200 | ✅ |
| P8 | Dashboard live search | `fetch("/metrics")` + `hybrid_search` via `/mcp` | ✅ |
| P9 | Test suite | 6/6 packages, 32 tests, 0 fail | ✅ |
### Notes / non-bugs
- Doc URLs follow `/<section>/<slug>` (e.g. `/docs/caddy/reverse-proxy`,
`/writeups/infra/cloudflare-525`, `/notes/postgres/full-text-search`).
The route is `[section]/[...slug]` — `/docs/websocket/contract` works because
the section IS `docs` for that doc; `/notes/postgres/fts` does not (the correct
slug is `notes/postgres/full-text-search`).
- `restoreRevision` via tRPC needs the `x-webhook-secret` as an **HTTP header**
(not inside the JSON body) — the fetch adapter reads `c.req.raw.headers`.
## Decisions locked (from initial planning)
+11 -2
View File
@@ -2,7 +2,8 @@ import { and, eq, sql } from "drizzle-orm";
import { db } from "@mcpedia/db";
import { documentChunks, documents } from "@mcpedia/db/schema";
import { CONTENT_ROOT } from "@mcpedia/config";
import { existsSync, readFileSync } from "node:fs";
import { parseFile } from "@mcpedia/parser";
import { existsSync } from "node:fs";
import { join } from "node:path";
import type {
Document,
@@ -33,8 +34,16 @@ export async function getDocument(slug: string): Promise<Document | null> {
const [row] = await db.select().from(documents).where(eq(documents.slug, slug));
if (!row) return null;
// Prefer the on-disk file (source of truth); fall back to stored body.
// Use parseFile (gray-matter) so the frontmatter is stripped — matches what
// the indexer stores and what ReactMarkdown expects.
const abs = join(CONTENT_ROOT, row.path);
const body = existsSync(abs) ? readFileSync(abs, "utf8") : row.body;
let body: string;
if (existsSync(abs)) {
const { body: parsedBody } = parseFile(abs, row.path);
body = parsedBody;
} else {
body = row.body;
}
return { ...toMeta(row), body };
}