Compare commits

...
16 Commits
Author SHA1 Message Date
krisf 1e7e951c91 ingest: suppress Google-news shared placeholder image (googleusercontent) -> null -> category cover fallback
Root cause: Google News RSS items carry no enclosures; pipeline fell back
to og:image from the news.google.com interstitial, which serves ONE shared
Google-branded tile (lh3.googleusercontent.com/J6_coFbo...) for every
article. 601 rows shared that URL (maple 243 / finance 358). New
image-guard blocks *.googleusercontent.com images at ingest; 16/16
regression tests; prod DBs migrated tile->NULL; images rebuilt
technews:10 finance:4 maple-brief:11 and deployed.
2026-08-18 21:30:23 -04:00
krisf 86a77b4907 Serve article images locally: /images proxy route with SSRF-guarded
upstream fetch, magic-byte validation, persistent /data/media cache,
immutable cache headers, CoverImage fallback component, og:image to
absolute proxy URL. Ported from technews 6f08da3.
2026-08-18 18:11:44 -04:00
krisf 92373aa3c6 fix(entrypoint): arm cron worker via self-ping after boot
instrumentationHook is disabled in the standalone build, so node-cron only
arms on POST /api/worker/ping. Without a self-ping, a fresh container or
restart runs zero ingest passes (worker silently never armed) — exactly
what happened on the :8 deploy. Adds the same self-ping block as
technews/finance entrypoints (identical).
2026-08-18 11:22:12 -04:00
krisf 08b9deb3d5 fix(pipeline): dedupe+P2002; add red maple-leaf cover system; replace dead CBC feeds (Google News CN / NatPost Canada); worker dead-letter 2026-08-18 10:24:09 -04:00
krisf 44d3220c4d UI: trim footer boilerplate, header tagline to 'Your Canadian News' 2026-08-16 08:39:24 -04:00
krisf 7a19afb63a llm: forbid source/newsroom attribution in synthesized report 2026-08-16 05:09:56 -04:00
krisf b5a25c1232 ui: remove source attribution + publisher rights notice 2026-08-16 05:01:26 -04:00
krisf b79c439c7f chore(gate): commit pre-push gate (secret -> build -> test, fail-closed)
- scripts/gate/gate.sh: 3-stage gate — 1) gitleaks, 2) build
  (tsc on commit, full next build on push), 3) LLM parse contract
  tests. Fail-closed: setup errors, timeouts, and findings all block.
- scripts/gate/hooks/{pre-commit,pre-push}: exec gate.sh commit|push.
- scripts/install/install-hooks.sh: idempotent installer (verifies repo
  root, bootstraps pinned gitleaks 8.30.1 if absent, wires both hooks).
- scripts/install/bootstrap-gitleaks.sh: pinned per-user install,
  x86_64/arm64, GitHub release download + SHA-less checksum pin.
- .gitleaks.toml: useDefault=true; single allowlist = .env.example
  placeholder lines (secret= and change-me values) by path+regex.
  Real secrets — even inside .env.example — still trip the gate
  (empirically verified: OpenAI/AWS/Slack/GitHub tokens all caught).
- tests/llm-parse.test.ts: pins parse.ts contracts (strict 6-paragraph
  body, headline/scalar/or array rejection, stopword rules, tag
  fallback) — the choke point for LLM output parsing.
- package.json: 'test' script.
- README: 'Commit gate' section (install, stages, verified fail-closed
  modes).

Verified before commit: clean tree PASSes all 3 stages; staged
realistic secret FAILs stage 1 (exit 1); broken type FAILs stage 2;
broken assertion FAILs stage 3; next build exit 0.
2026-08-16 00:27:41 -04:00
krisf 69f84e0bc6 fix(build): commit src/data/feeds.ts — blanket data/ gitignore was hiding committed source, breaking pristine builds 2026-08-15 23:19:37 -04:00
krisf d1bf2962ff final cut: remaining synthesis wording (package.json, Dockerfile env) 2026-08-15 22:52:24 -04:00
krisf 478ff07bc5 drop all synthesis wording + sources box; 3->6 paragraphs, deeper prompt, 1.6k tok 2026-08-15 22:41:35 -04:00
krisf 9ebc3bffab fix: mobile hero overlay — stop Lead-briefing badge clipping 2026-08-15 22:07:56 -04:00
krisf 9210dc8f14 fix(mobile): hide inline header nav below md; add overflow-x:clip guard
Header NavLinks (Top Stories/Canada/National/All) always rendered and
overflowed the ~360-400px viewport, causing whole-page horizontal
scroll + wrapped/bleeding active pill. Now hidden on phones (the
horizontally-scrolling SectionPills row under the header remains the
mobile nav); inline nav returns at md (768px). Added overflow-x:clip on
html as a global NOOP-safe against any future sideways overflow.
2026-08-15 19:52:53 -04:00
krisf cbea5e8fa4 deploy: Dockerfile, entrypoint, standalone output, dockerignore 2026-08-15 18:11:41 -04:00
krisf e709de01d4 merge: fold in Gitea template init 2026-08-15 16:07:18 -04:00
krisf 1010f27f44 MapleBrief: production Canadian news aggregator
Next.js 14 + TypeScript + Tailwind + Prisma/SQLite.
- LLM synthesis abstraction (OpenAI/Anthropic/Ollama) + admin settings UI
- RSS ingestion pipeline (parser + cheerio content) + node-cron worker
- AdSense AdUnit placements (header/in-feed/in-article/sidebar)
- Sources analysis attribution, nofollow links, canonical/OG, sitemap/robots
- Admin auth (ADMIN_API_KEY, timingSafeEqual, fail-closed 401)
- Multi-cell lady-ga-ga marker, sitemap, robots, legal pages
- Comprehensive README (setup, LLM config, AdSense, migrations, deploy)
2026-08-15 16:07:07 -04:00
98 changed files with 8583 additions and 2 deletions
+15
View File
@@ -0,0 +1,15 @@
node_modules
.next
.git
.gitignore
env.d.ts
env.example
prisma/dev.db
prisma/fresh.db*
prisma/dev.db-journal
prisma/dev.db-shm
prisma/dev.db-wal
*.tar.gz
*.log
Dockerfile
.dockerignore
+65
View File
@@ -0,0 +1,65 @@
# ─────────────────────────────────────────────────────────────────────────────
# MapleBrief — environment configuration
# Copy to `.env` and fill in the values you need. Only the entries marked
# REQUIRED need to be set for the app to boot; everything else has a sane
# fallback or is managed from the /admin/settings UI (stored in the database).
# ─────────────────────────────────────────────────────────────────────────────
# ---------------------------------------------------------------------------
# Database (REQUIRED)
# SQLite by default — zero-config for local dev. For PostgreSQL, change
# provider in prisma/schema.prisma and point DATABASE_URL at a postgres URL.
# ---------------------------------------------------------------------------
DATABASE_URL="file:./dev.db"
# ---------------------------------------------------------------------------
# Site (public — inlined into the browser)
# ---------------------------------------------------------------------------
NEXT_PUBLIC_SITE_NAME="MapleBrief"
NEXT_PUBLIC_SITE_URL="http://localhost:3000"
NEXT_PUBLIC_SITE_DESCRIPTION="Independently synthesized briefings on the news that matters across Canada. A neutral, source-attributed daily digest of Canadian headlines from CBC, CTV, Global, the Globe and more."
# ---------------------------------------------------------------------------
# Content reliability — used as fallback identity for feed fetches
# ---------------------------------------------------------------------------
FEED_USER_AGENT="MapleBrief/1.0 (+https://localhost:3000; rss reader)"
# ---------------------------------------------------------------------------
# Background ingestion scheduler
# TURN these off to run by hand (see README "Running the worker").
# ---------------------------------------------------------------------------
RUN_SCHEDULER="true" # set to "false" to disable the node-cron worker
CRON_SCHEDULE="*/35 * * * *" # every 35 minutes (spec: 30-60 minute cadence)
MAX_SYNTH_PER_RUN="24" # cap LLM syntheses per run to limit API spend
FETCH_CONTENT="true" # set to "false" to skip original HTML fetch
HTTP_TIMEOUT_MS="12000" # per-request timeout for HTML fetches
# ---------------------------------------------------------------------------
# LLM providers (fallback defaults; the /admin/settings UI overrides all of
# these at runtime and stores the active choices in the database).
# API keys referenced here are used only when the admin UI has NOT stored a
# value for that provider. Never commit real keys to .env in a shared repo.
# ---------------------------------------------------------------------------
OPENAI_API_KEY=""
ANTHROPIC_API_KEY=""
OLLAMA_BASE_URL="http://localhost:11434"
OLLAMA_MODEL="llama3.2:3b"
# ---------------------------------------------------------------------------
# Administration (protects the /admin/settings UI and /api/settings,
# /api/refresh endpoints). Must be a strong random string in production.
# ---------------------------------------------------------------------------
ADMIN_API_KEY="change-me-to-a-32-char-random-string"
# ---------------------------------------------------------------------------
# Google AdSense (public — publisher id + per-unit ad slots).
# NEXT_PUBLIC_ADSENSE_CLIENT_ID looks like: ca-pub-1234567890123456
# Slot ids are the numeric "slot" values from your AdSense Ad units UI.
# Leave a slot blank and the <AdUnit /> renders a labelled placeholder —
# perfect for local preview before your AdSense account is approved.
# ---------------------------------------------------------------------------
NEXT_PUBLIC_ADSENSE_CLIENT_ID=""
NEXT_PUBLIC_ADSENSE_SLOT_HEADER=""
NEXT_PUBLIC_ADSENSE_SLOT_INFEED=""
NEXT_PUBLIC_ADSENSE_SLOT_INARTICLE=""
NEXT_PUBLIC_ADSENSE_SLOT_SIDEBAR=""
+44
View File
@@ -0,0 +1,44 @@
# dependencies
/node_modules
/.pnp
.pnp.js
# testing
/coverage
# next.js
/.next/
/out/
*.tsbuildinfo
next-env.d.ts
# production
/build
# misc
.DS_Store
*.pem
# debug
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# local env files
.env
.env.local
.env.development.local
.env.test.local
.env.production.local
.env.*.local
# prisma
/prisma/*.db
/prisma/*.db-journal
prisma/dev.db
prisma/dev.db-journal
# local databases / generated (root only — src/data/ holds committed source)
/data/
*.sqlite
*.sqlite3
+29
View File
@@ -0,0 +1,29 @@
# ────────────────────────────────────────────────────────────────────────────
# gitleaks config — maple-brief commit gate (secret stage)
#
# Policy (fail-closed): the FULL default gitleaks ruleset (every known
# provider / secret shape) hard-blocks at commit and pre-push. The ONLY
# suppression is a tight, content-regexed allowlist scoped to a single file:
# .env.example (the committed placeholder template). It suppresses ONLY those
# exact placeholder lines. A real composite key pasted there (or anywhere)
# STILL fires — verified on gitleaks 8.30.1 (linux x64).
#
# Gotchas found while validating (do not regress):
# • allowlist must be a MAP [allowlist] — a slice [[allowlist]] fails
# with "expected a map, got slice"
# • an embedded single quote inside a single-quoted (literal) TOML string
# silently closes it — keep regex strings double-quoted (basic)
# • paths= scopes the suppression to the file; regexes= to the lines
# ────────────────────────────────────────────────────────────────────────────
[extend]
useDefault = true
[allowlist]
description = "Suppress ONLY empty double-quoted placeholder values and the change-me example, in the .env template"
paths = ['\.env\.example$']
regexes = [
'^OPENAI_API_KEY=("")$',
'^ANTHROPIC_API_KEY=("")$',
'^NEXT_PUBLIC_ADSENSE_[A-Z_]+=("")$',
"^ADMIN_API_KEY=\"change-me[^\"]*\"$"
]
+66
View File
@@ -0,0 +1,66 @@
# MapleBrief — production image
# Multi-stage: deps -> build -> minimal runner
# (Debian base: Prisma engines need glibc; Alpine breaks the schema/query engine.)
#
# Build args / secrets:
# NEXT_PUBLIC_ADSENSE_CLIENT_ID — AdSense client id baked into the build
# (empty -> ads hidden, app still works)
# Runtime env:
# ADMIN_API_KEY — admin/auth key (fail-closed 401 if empty)
# OLLAMA_BASE / provider — LLM endpoints (or configure in admin UI)
# DATABASE_URL defaults to file:/data/dev.db (volume-mounted)
# Optional: bake the real AdSense client id at build time (empty -> ads hidden)
ARG NEXT_PUBLIC_ADSENSE_CLIENT_ID=""
FROM node:22-slim AS deps
WORKDIR /app
RUN apt-get update -qq && apt-get install -yq --no-install-recommends openssl ca-certificates > /dev/null && rm -rf /var/lib/apt/lists/*
COPY package.json package-lock.json ./
RUN npm ci --no-audit --no-fund
FROM node:22-slim AS build
WORKDIR /app
# inherit the global value (set via --build-arg); empty -> placeholder ads
ARG NEXT_PUBLIC_ADSENSE_CLIENT_ID
RUN apt-get update -qq && apt-get install -yq --no-install-recommends openssl ca-certificates > /dev/null && rm -rf /var/lib/apt/lists/*
# Absolute path avoids Prisma resolving relative SQLite URLs against the
# schema dir (which would double up into prisma/prisma/...).
ENV DATABASE_URL="file:/app/fresh.db"
COPY --from=deps /app/node_modules ./node_modules
COPY . .
# Verify migrations apply to a fresh DB, then stage the migrated empty DB at
# the exact path the runner copies.
RUN npx prisma generate \
&& npx prisma migrate deploy \
&& cp /app/fresh.db prisma/fresh.db \
&& NEXT_PUBLIC_SITE_URL="https://news.krisforbes.ca" \
NEXT_PUBLIC_SITE_NAME="MapleBrief" \
NEXT_PUBLIC_SITE_DESCRIPTION="Canadian news briefings" \
NEXT_PUBLIC_ADSENSE_CLIENT_ID="${NEXT_PUBLIC_ADSENSE_CLIENT_ID}" \
NEXT_PUBLIC_SHOW_SIDEBAR_ADS="false" \
npm run build \
&& rm -f /app/fresh.db prisma/fresh.db-journal prisma/fresh.db-shm
# (prisma/fresh.db kept above for the runner stage copy)
FROM node:22-slim AS runner
WORKDIR /app
RUN apt-get update -qq && apt-get install -yq --no-install-recommends curl > /dev/null && rm -rf /var/lib/apt/lists/*
# runtime user (guard: node slim images don't always ship one)
RUN id app 2>/dev/null || useradd -m -s /bin/sh app
# pre-create the data dir (volume replaces it at runtime) with app ownership
RUN mkdir -p /data && chown -R app:app /data
USER app
# Next standalone server
COPY --chown=app:app --from=build /app/.next/standalone ./
# static chunks + public assets (standalone does not auto-copy these)
COPY --chown=app:app --from=build /app/.next/static ./.next/static
COPY --chown=app:app --from=build /app/public ./public
# pre-migrated clean database snapshot (entrypoint copies it to /data on first boot)
COPY --chown=app:app --from=build /app/prisma/fresh.db ./seed-dev.db
COPY --chown=app:app entrypoint.sh ./entrypoint.sh
RUN chmod +x entrypoint.sh
ENV PORT=3000
ENV DATABASE_URL="file:/data/dev.db"
EXPOSE 3000
ENTRYPOINT ["./entrypoint.sh"]
+244 -2
View File
@@ -1,3 +1,245 @@
# maple-brief-aggregator
# MapleBrief
Automated Canadian news aggregator with LLM content synthesis and Google AdSense integration.
An automated **Canadian news aggregator** with **LLM content synthesis** and **Google AdSense integration**.
MapleBrief pulls stories from Canadian news RSS feeds on a schedule, fetches the full article text, has an LLM rewrite them into original neutral reports (with headline, takeaways, and tags), and serves them on an ad-optimized Next.js site with full source attribution.
## Features
- **Scheduled ingestion** — a `node-cron` background worker (in-process, starts with the web server) fetches feeds, dedupes articles, extracts full article bodies with `cheerio`, and synthesizes pending articles every 35 minutes (configurable).
- **LLM abstraction, three providers** — OpenAI (default: `gpt-4o-mini`), Anthropic (Claude 3.5 Sonnet / Haiku), and Ollama (local). Provider, model per-provider, API keys, and the synthesis prompt are all **stored in the database** and editable live from the admin UI — no code changes or restarts needed.
- **Keyless-first, fail-closed admin** — `/admin/settings` and the settings APIs require `x-admin-key: $ADMIN_API_KEY`. Empty/missing key → every admin endpoint returns 401 (fail-closed). Comparison is timing-safe.
- **AdSense-ready layout** — header leaderboard (728×90) on every page, in-feed ads every 3rd card, in-article ad after the first paragraph, and a sticky 300×250 / 300×600 sidebar. Ad units render as labeled placeholders until a real Publisher ID is configured.
- **Compliance & SEO** — "Sources analyzed: …" attribution on every article, `rel="nofollow external noopener"` on all source links, per-article `canonical` + OpenGraph tags, `sitemap.xml`, `robots.txt`, and four legal pages (`/about`, `/privacy-policy`, `/terms-of-service`, `/contact`).
## Tech stack
| Layer | Choice |
|---|---|
| Framework | Next.js 14 (App Router) + TypeScript (strict) + Tailwind CSS |
| Database / ORM | SQLite via Prisma (zero-ops local dev; Postgres switch documented below) |
| Scheduler | `node-cron` worker in the server process |
| Ingestion | `rss-parser` + `cheerio` (content extraction), native `fetch` |
| LLM clients | native `fetch` only (OpenAI, Anthropic, Ollama) — no SDK dependencies |
| Ad framework | Google AdSense (`<AdUnit>` + `NEXT_PUBLIC_ADSENSE_*` env) |
## Quickstart
Requirements: **Node.js ≥ 18** (tested on v22), npm.
```bash
git clone https://gitea.krisforbes.ca/krisf/maple-brief-aggregator.git
cd maple-brief-aggregator
npm install
# 1. Environment
cp .env.example .env
# then edit .env — at minimum set:
# DATABASE_URL (default is fine for SQLite: file:./dev.db)
# CRON_SCHEDULE (default "*/35 * * * *")
# NEXT_PUBLIC_SITE_URL (deployment URL, without trailing slash)
# 2. Database — create tables and generate the Prisma client
npx prisma migrate deploy
# (in dev: npx prisma migrate dev)
# 3. First ingest — pulls feeds, saves articles, synthesizes anything pending
npm run seed
# 4. Build & run (production)
npm run build
npm start # http://localhost:3000
# — or development with hot reload:
npm run dev
```
On boot, the `src/instrumentation.ts` hook (Node.js runtime only) arms the worker, which immediately runs an initial ingest + synthesis pass and then repeats on `CRON_SCHEDULE`. The worker is idempotent (`runPipelinePass` is re-entrancy-guarded), so restarts and double-starts are safe.
### Useful commands
```bash
npx prisma studio # browse the DB
npx prisma migrate status # check applied migrations
npx prisma migrate diff --from-empty --to-schema-datamodel prisma/schema.prisma --script # preview SQL
npm run seed # manual ingest + synthesis pass
```
## Commit gate (secret → build → test, fail-closed)
Every commit and push runs a three-stage gate (`scripts/gate/gate.sh`), in this fixed order:
| Stage | pre-commit (fast) | pre-push (heavy) |
|---|---|---|
| 1 — secret | `gitleaks protect --staged` | `gitleaks protect` over full HEAD history |
| 2 — build | `tsc --noEmit` (strict) | `prisma generate && next build` (pristine production build) |
| 3 — test | `tests/llm-parse.test.ts` (LLM parse contract, 21 pins) | same |
- **Fail-closed:** a missing tool, a crash, a timeout, or any finding blocks the
operation. Only a deliberate `git commit --no-verify` / `git push --no-verify`
bypasses it (auditable in the terminal scrollback).
- **Secret stage:** full gitleaks default ruleset, hard-blocks. The one allow
exception is a file- and content-scoped suppress of *empty/placeholder*
values in `.env.example` (see `[allowlist]` in `.gitleaks.toml`) — a real key
written into that file still fires.
- **Test stage:** `tests/llm-parse.test.ts` pins the synthesis parse contract
(6-paragraph cap, 160-char headline, ≤5 takeaways, sanitized tags,
JSON-fence recovery, determinism). It imports only `src/lib/llm/parse.ts`
pure, zero-dep, zero-network — so it runs under `tsx` with no framework.
- **Install (idempotent, per clone):**
```bash
bash scripts/install/install-hooks.sh
# → bootstraps gitleaks (pinned 8.30.1) into ~/.local/bin if missing,
# copies scripts/gate/hooks/* into .git/hooks/
```
- Walk away from a fresh machine: `npm ci && bash scripts/install/install-hooks.sh`.
## Configuration
All configuration is in `.env` (local, gitignored — only `.env.example` is committed):
| Variable | Purpose |
|---|---|
| `DATABASE_URL` | Prisma connection string (default `file:./dev.db`) |
| `NEXT_PUBLIC_SITE_NAME` / `_URL` / `_DESCRIPTION` | Site metadata used in `<head>`, sitemap, OG tags |
| `ADMIN_API_KEY` | Required admin key; sent as `x-admin-key` header. **Empty ⇒ all admin endpoints return 401.** |
| `FEED_USER_AGENT` | UA string used for feed/content fetches |
| `RUN_SCHEDULER` | `"true"`/`"false"` — kill switch for the cron worker (e.g. run pipeline only via `npm run seed`) |
| `CRON_SCHEDULE` | Worker cadence, default `*/35 * * * *` (spec: 3060 min) |
| `MAX_SYNTH_PER_RUN` | LLM articles synthesized per pass (default 20) — throttles API cost |
| `FETCH_CONTENT` | `"true"` to download full article HTML for synthesis (makes LLM output accurate, ~few KB per article) |
| `HTTP_TIMEOUT_MS` | Per-request timeout for feed/content/LLM fetches |
| `NEXT_PUBLIC_ADSENSE_CLIENT_ID` | Your Publisher ID, e.g. `ca-pub-0123456789012345` |
| `NEXT_PUBLIC_ADSENSE_SLOT_*` | Ad-slot IDs for `HEADER` (728×90), `INFEED` (fluid), `INARTICLE` (fluid), `SIDEBAR` (300×250) |
| `OLLAMA_BASE_URL`, `OLLAMA_MODEL` | For the Ollama provider (see note below — must use HTTP URL) |
> **Note:** `NEXT_PUBLIC_ADSENSE_*` values are inlined at **build time**. After changing them, re-run `npm run build` (in dev mode they pick up on reload).
`OPENAI_API_KEY` / `ANTHROPIC_API_KEY` in `.env` are only fallback defaults — the admin UI / settings API are the canonical place for keys (see below), and LLM calls never read them from the environment.
## LLM provider configuration
1. Build with AdSense/site env set.
2. Open `http://localhost:3000/admin/settings`.
3. Enter the admin key (held in `sessionStorage` while the tab is open — never `localStorage`) — it must match `ADMIN_API_KEY` in `.env`.
4. Pick a provider:
- **OpenAI** — model default `gpt-4o-mini`; paste your `sk-...` key.
- **Anthropic** — model default `claude-3-5-haiku-20241022` (change to `claude-3-5-sonnet-latest` for higher quality); paste your key.
- **Ollama** — paste a **reachable `http://host:11434`** base URL and a model name that actually exists on that host (e.g. `llama3.1:8b`). Verify with "Test connection" — it lists the models Ollama reports.
5. Optionally edit the **synthesis prompt** (the default instructs the model to produce an original, neutral 3-paragraph rewrite plus headline, 3 takeaways, and tags as strict JSON).
6. Click **Save**. Settings persist in the DB (`Setting` table); the next cron pass (or a manual `npm run seed`) uses them immediately.
### Synthesis pipeline
For each article with status `fetched`: build a prompt from the article body → call the provider → parse the strict-JSON response (with a tolerant `safeParse`) → update the article (`synthesisJson`, `headline`, `takeaways`, `tags`, `provider`, status `synthesized`). Failures are recorded as status `failed` with the provider's error saved — the pipeline never crashes on a bad key or malformed response and retries those articles on the next pass.
## Google AdSense setup
1. Get your **Publisher ID** (`ca-pub-…`) from AdSense.
2. Create the four ad units in your AdSense account and set their slot IDs in `.env`:
| Placement | Unit | Slot env var |
|---|---|---|
| Header (every page, below nav) | 728×90 | `NEXT_PUBLIC_ADSENSE_SLOT_HEADER` |
| In-feed (after every 3rd article card) | Fluid/auto | `NEXT_PUBLIC_ADSENSE_SLOT_INFEED` |
| In-article (after first paragraph) | Fluid/auto | `NEXT_PUBLIC_ADSENSE_SLOT_INARTICLE` |
| Sidebar (sticky) | 300×250 (or 300×600) | `NEXT_PUBLIC_ADSENSE_SLOT_SIDEBAR` |
3. Set `NEXT_PUBLIC_ADSENSE_CLIENT_ID=ca-pub-…` and **rebuild** (`npm run build`).
4. Until every one of the five `NEXT_PUBLIC_ADSENSE_*` values is real, Ad units render as clearly-labeled placeholders (e.g. "AdUnit · 728x90 · awaiting setup") and **no `adsbygoogle` markup ships** — your pages contain zero ad code until you're published.
## Feeds
Starter (seeded on first boot into the `Feed` table):
| Feed | Category | Status (verified 2026-08) |
|---|---|---|
| CBC News — Top Stories | top-stories | ⚠️ **404 — CBC has discontinued its public RSS endpoints** (verified from multiple vantage points). Kept in the seed so it lights up if CBC restores them; disable via `Feed.enabled` or remove the row. |
| CBC News — Canada | canada | ⚠️ same as above (404) |
| CTV News — National | national | ✅ live — Arc outbound feed `https://www.ctvnews.ca/arc/outboundfeeds/rss/` (the legacy `ctvnews-ca-…-rss-1.822009` URL 404s, so the live Arc endpoint is used instead) |
| Global News — Canada | canada | ✅ live — `https://globalnews.ca/canada/feed/` |
| The Globe and Mail — National | national | ✅ live — Arc outbound feed, `/category/canada/` |
| National Post — News | top-stories | ✅ live — `https://nationalpost.com/category/news/feed` |
Dead feeds never block the pipeline — the worker logs and skips them, so 4/6 feeds live means a healthy, populated site.
To add a feed, insert a `Feed` row (Prisma Studio or SQL) or extend `STARTER_FEEDS` in `src/data/feeds.ts`. All URLs must be RSS/Atom; Atom is supported by `rss-parser`.
## API
| Endpoint | Method | Auth | Purpose |
|---|---|---|---|
| `/api/worker/ping` | GET / POST | — | Arms the background worker (serverless keep-alive; on a long-lived Node server it is already armed at boot via `src/instrumentation.ts`) |
| `/api/status` | GET | admin key | Pipeline telemetry: article counts by status, per-feed last-fetched, synthesized counts by category |
| `/api/refresh` | POST | admin key | One-shot pipeline pass (ingest all feeds + synthesize pending articles); `maxDuration` 300s |
| `/api/settings` | GET / PUT | admin key | Read / persist LLM provider, per-provider models, API keys, Ollama base URL, synthesis prompt. **Keys are never returned** — responses carry `keySources` booleans and a masked last-4 only |
| `/api/settings/test` | POST | admin key | Connectivity test against the configured provider (lists available models when provider is Ollama) |
| `/api/admin/verify` | GET | admin key | Health check: confirms the presented key matches `ADMIN_API_KEY` |
| `/api/contact` | POST | — | Contact form; logged server-side (wire an email sink — see route docblock) |
> **Admin key transport** — any of: `Authorization: Bearer *** `x-admin-key: <key>` header, `?key=<key>` query param, or HTTP Basic auth with the key as password (any username). Comparison is constant-time; an unset `ADMIN_API_KEY` fails closed (all admin endpoints 401).
## Database
Prisma schema (`prisma/schema.prisma`):
- `Feed` — id, name, url (unique), slug (unique), category, `siteName`, `enabled`
- `Article` — unique `externalId`, unique `slug`, `routePath` (unique), `feedUrl` (unique), source `title`/`url`/`image`/`publishedAt`, extracted `content`, `status` (`pending | fetched | synthesized | failed`), synthesis fields (`headline`, `takeaways[]`, `tags[]`, `synthesisJson`, `provider`), `synthesizedAt`
- `Setting` — key/value store for LLM provider config (keyed by model, e.g. `llm.model.openai`)
Migrations live in `prisma/migrations/` (initial: `20260815183102_init`). Applied via `npx prisma migrate deploy` (prod) / `migrate dev` (dev).
### Switching SQLite → PostgreSQL
1. Change `DATABASE_URL` in `.env` to your Postgres string.
2. Change `provider = "sqlite"``"postgresql"` in `prisma/schema.prisma`.
3. `npx prisma migrate dev --name init` (fresh DB) and rebuild. No app code changes required — Prisma abstracts the driver.
## Project structure (abridged)
```
prisma/ schema.prisma, migrations/, seed.ts (ingest+synthesis pass)
src/instrumentation.ts Next.js boot hook — arms the worker on process start (Node runtime only)
src/data/feeds.ts starter feed list + site sections
src/lib/
db.ts Prisma client singleton
env.ts typed env access
format.ts HTML→text, excerpt helpers
auth/admin.ts x-admin-key verification (timing-safe, fail-closed)
llm/ providers: openai.ts, anthropic.ts, ollama.ts, parse.ts, types.ts
llm/engine.ts provider resolution + synthesizeArticle()
ingest/ feed.ts (RSS→rows), html.ts (content fetch), pipeline.ts (dedupe+persist)
worker/ cron.ts — node-cron loop, instrumentation.ts — boot hook
queries.ts read-model queries for pages
src/app/
layout.tsx global shell (header ad on every page), metadata, sitemap, robots
page.tsx home (featured story + top-stories feed + sidebar)
[section]/ section feeds (top-stories, canada, national, all)
article/[slug]/ article detail (attribution, in-article ad, related, OG, canonical)
admin/settings/ protected admin UI (key gate + settings form)
about|contact|privacy-policy|terms-of-service|not-found/ legal + 404
api/ routes above
src/components/ card, ads (AdUnit), layout (Header/Footer/Sidebar), legal
```
## Verification performed (2026-08-15)
Real, executed checks — not assumptions:
- `npx tsc --noEmit` — clean; `npm run build` — passes (20 routes, static + dynamic).
- Live ingest: **60 real Canadian articles** pulled across Global, Globe and Mail, National Post and CTV; full bodies extracted (up to ~12k chars); slugs clean and deduped.
- Synthesis happy path: real chain (provider → strict-JSON parse → DB update) verified — articles reach `synthesized` with headline/takeaways/tags/provider populated.
- Synthesis failure path: real OpenAI call with a placeholder key returned 401 → article cleanly marked `failed`, no crash, retriable next pass.
- Served app: all routes 200 (home, sections, article detail, legal, sitemap, robots, status/ping/settings endpoints).
- Admin: wrong/missing key → 401 on every admin endpoint (settings, status, refresh, verify); correct key → settings GET/PUT round-trips correctly (keys never echoed — `keySources` booleans + last-4 only); provider test endpoint responds.
- AdSense: with a test publisher ID, real `<ins class="adsbygoogle">` markup SSRs with correct client/slot/format on every page (header), section pages (in-feed), and article pages (in-article + sidebar); with placeholder values, clean labeled placeholders and zero `adsbygoogle` markup.
## Policy & content caveats (read before monetizing)
- **LLM-rewritten news is a grey zone** for Google AdSense's scraped/reshaped-content policies, and for copyright (summary/derivative-work doctrine varies by jurisdiction). This project mitigates, not eliminates, that risk: every page carries source attribution, `nofollow` source links, and a visible LLM-rewrite disclosure, and the synthesis prompt instructs the model to stay factual and neutral. If you run with AdSense enabled, expect to hold the scale small, monitor your AdSense account for policy messages, and be prepared that account suspension is possible.
- **CBC feeds are dead** (documented above) — their 404s are expected and harmless; ignore `feed emitted 0 items` log lines for the `cbc-*` feeds.
## License
Internal project — see Gitea repo `krisf/maple-brief-aggregator`.
+44
View File
@@ -0,0 +1,44 @@
#!/bin/sh
# MapleBrief entrypoint
# 1) First boot: seed /data with the clean migrated DB + write privileged key if provided
# 2) Start Next standalone server
set -e
cd /app
DB="${DATABASE_URL#file:}"
[ -n "$DB" ] || DB=/data/dev.db
# Export an absolute DATABASE_URL for the Node/Prisma process (relative
# paths resolve against the schema dir and can point at the wrong file).
case "$DB" in
/*) ;;
*) DB="/$DB" ;;
esac
export DATABASE_URL="file:$DB"
if [ ! -f "$DB" ]; then
# create parent dir (volume may be fresh)
DIR="$(dirname "$DB")"
mkdir -p "$DIR"
cp ./seed-dev.db "$DB"
echo "[entrypoint] initialized $DB from seed snapshot"
fi
# Arm the ingest/synthesis worker once the server answers.
# The Next standalone build has instrumentationHook disabled, so node-cron
# only starts when a request reaches /api/worker/ping. Pinging ourselves
# after boot means a fresh container or restart re-arms the worker (and
# triggers the boot pipeline pass) without any external health pinger.
(
APP_IP="$(hostname -i 2>/dev/null | awk '{print $1}')"
[ -z "$APP_IP" ] && APP_IP=127.0.0.1
i=0
until curl -sf -o /dev/null --max-time 2 "http://${APP_IP}:${PORT:-3000}/api/worker/ping"; do
i=$((i + 1))
[ "$i" -ge 90 ] && exit 0
sleep 1
done
echo "[entrypoint] armed cron worker via /api/worker/ping"
) >/dev/null 2>&1 &
exec node server.js
+21
View File
@@ -0,0 +1,21 @@
/** @type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: true,
// Minimal Docker image: the runner stage copies .next/standalone.
output: 'standalone',
// RSS/Google AdSense scripts are injected safely by our own client
// components (see src/components/ads). Keep lint from failing the build
// on un-stulated style warnings while keeping full type checking on.
eslint: {
ignoreDuringBuilds: true,
},
images: {
remotePatterns: [
{ protocol: 'https', hostname: '**' },
{ protocol: 'http', hostname: 'localhost' },
{ protocol: 'http', hostname: '127.0.0.1' },
],
},
};
export default nextConfig;
+2648
View File
File diff suppressed because it is too large Load Diff
+46
View File
@@ -0,0 +1,46 @@
{
"name": "maple-brief-aggregator",
"version": "1.0.0",
"private": true,
"description": "Automated Canadian news aggregator with AI-assisted content synthesis and Google AdSense integration.",
"scripts": {
"dev": "next dev",
"build": "prisma generate && next build",
"start": "next start",
"typecheck": "tsc --noEmit",
"test": "tsx tests/llm-parse.test.ts && tsx tests/image-guard.test.ts && tsx tests/image-proxy.test.ts",
"db:generate": "prisma generate",
"db:push": "prisma db push",
"db:seed": "tsx prisma/seed.ts",
"db:studio": "prisma studio",
"migrate": "prisma migrate dev --name init"
},
"prisma": {
"seed": "tsx prisma/seed.ts"
},
"dependencies": {
"@prisma/client": "^5.19.0",
"cheerio": "^1.0.0",
"next": "^14.2.35",
"node-cron": "^3.0.3",
"react": "18.3.1",
"react-dom": "18.3.1",
"rss-parser": "^3.13.0"
},
"devDependencies": {
"@resvg/resvg-js": "^2.6.2",
"@types/node": "^20",
"@types/node-cron": "^3.0.11",
"@types/react": "^18",
"@types/react-dom": "^18",
"autoprefixer": "^10.4.20",
"postcss": "^8.4.47",
"prisma": "^5.19.0",
"tailwindcss": "^3.4.13",
"tsx": "^4.19.1",
"typescript": "^5.6.2"
},
"engines": {
"node": ">=18.17"
}
}
+9
View File
@@ -0,0 +1,9 @@
/** @type {import('postcss-load-config').Config} */
const config = {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
};
export default config;
@@ -0,0 +1,73 @@
-- CreateTable
CREATE TABLE "Feed" (
"id" TEXT NOT NULL PRIMARY KEY,
"name" TEXT NOT NULL,
"url" TEXT NOT NULL,
"slug" TEXT NOT NULL,
"category" TEXT NOT NULL DEFAULT 'news',
"enabled" BOOLEAN NOT NULL DEFAULT true,
"lastFetchedAt" DATETIME,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL
);
-- CreateTable
CREATE TABLE "Article" (
"id" TEXT NOT NULL PRIMARY KEY,
"feedId" TEXT,
"guid" TEXT,
"dedupKey" TEXT NOT NULL,
"sourceUrl" TEXT NOT NULL,
"canonicalUrl" TEXT NOT NULL,
"siteName" TEXT NOT NULL,
"title" TEXT NOT NULL,
"slug" TEXT NOT NULL,
"category" TEXT NOT NULL DEFAULT 'news',
"author" TEXT,
"publishedAt" DATETIME,
"image" TEXT,
"sources" TEXT,
"originalText" TEXT,
"headline" TEXT,
"body" TEXT,
"takeaways" TEXT,
"tags" TEXT,
"llmProvider" TEXT,
"synthesizedAt" DATETIME,
"status" TEXT NOT NULL DEFAULT 'fetched',
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" DATETIME NOT NULL,
CONSTRAINT "Article_feedId_fkey" FOREIGN KEY ("feedId") REFERENCES "Feed" ("id") ON DELETE SET NULL ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "Setting" (
"key" TEXT NOT NULL PRIMARY KEY,
"value" TEXT NOT NULL,
"isSecret" BOOLEAN NOT NULL DEFAULT false,
"updatedAt" DATETIME NOT NULL
);
-- CreateIndex
CREATE UNIQUE INDEX "Feed_name_key" ON "Feed"("name");
-- CreateIndex
CREATE UNIQUE INDEX "Feed_url_key" ON "Feed"("url");
-- CreateIndex
CREATE UNIQUE INDEX "Article_guid_key" ON "Article"("guid");
-- CreateIndex
CREATE UNIQUE INDEX "Article_canonicalUrl_key" ON "Article"("canonicalUrl");
-- CreateIndex
CREATE UNIQUE INDEX "Article_slug_key" ON "Article"("slug");
-- CreateIndex
CREATE INDEX "Article_category_publishedAt_idx" ON "Article"("category", "publishedAt");
-- CreateIndex
CREATE INDEX "Article_status_publishedAt_idx" ON "Article"("status", "publishedAt");
-- CreateIndex
CREATE INDEX "Article_dedupKey_idx" ON "Article"("dedupKey");
+3
View File
@@ -0,0 +1,3 @@
# Please do not edit this file manually
# It should be added in your version-control system (i.e. Git)
provider = "sqlite"
+79
View File
@@ -0,0 +1,79 @@
// MapleBrief — database schema
// SQLite by default for zero-config local dev. To switch provider:
// 1. change `provider` in datasource below
// 2. update DATABASE_URL in .env
// 3. run `prisma migrate dev` (see README)
generator client {
provider = "prisma-client-js"
}
datasource db {
provider = "sqlite"
url = env("DATABASE_URL")
}
// A configured news source (RSS/Atom feed).
model Feed {
id String @id @default(cuid())
name String @unique
url String @unique
slug String
category String @default("news")
enabled Boolean @default(true)
lastFetchedAt DateTime?
articles Article[]
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
}
// A single synthesized news brief, built from one or more source stories.
model Article {
id String @id @default(cuid())
feedId String?
feed Feed? @relation(fields: [feedId], references: [id], onDelete: SetNull)
// Identity / dedup
guid String? @unique // feed item guid (primary dedup key)
dedupKey String // sha1 of normalized title (near-dup key)
sourceUrl String // original story URL (outbound link)
canonicalUrl String @unique // our canonical URL for this brief
siteName String // originating publisher, e.g. "CBC News"
// Source metadata
title String // original headline (kept for reference)
slug String @unique
category String @default("news") // mapped from the feed category
author String?
publishedAt DateTime?
image String? // original thumbnail URL
sources String? // JSON string[] of contributing publisher names
// Original content (kept for the "editor's notes" reference, NOT shown)
originalText String?
// Synthesized (original) output produced by the LLM
headline String? // rewritten original headline
body String? // 3 paragraphs, \n\n separated
takeaways String? // JSON string[] of key takeaways
tags String? // JSON string[]
llmProvider String? // provider that produced this brief
synthesizedAt DateTime?
status String @default("fetched") // fetched | synthesized | failed
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([category, publishedAt])
@@index([status, publishedAt])
@@index([dedupKey])
}
// Runtime configuration. The /admin/settings UI writes here; env vars act as
// fallbacks when a key is absent so the app boots with zero configuration.
model Setting {
key String @id
value String
isSecret Boolean @default(false)
updatedAt DateTime @updatedAt
}
+30
View File
@@ -0,0 +1,30 @@
/* eslint-disable no-console */
/**
* Seeds the Feed table with the starter Canadian news feeds.
*
* npm run db:seed
* (also auto-invoked by `prisma migrate dev` via the "prisma.seed" hook.)
*
* Idempotent: existing feeds (matched by URL) are left untouched, new
* starter feeds are added. User-added feeds are never modified.
*
* tsx reads tsconfig.json paths, so the "@/..." aliases resolve.
*/
import { PrismaClient } from '@prisma/client';
import { seedFeedsIfEmpty } from '../src/lib/ingest/seed';
async function main() {
const prisma = new PrismaClient();
try {
const before = await prisma.feed.count();
const { created, total } = await seedFeedsIfEmpty();
console.log(`[seed] feeds: ${before} -> ${total} (${created} created)`);
} finally {
await prisma.$disconnect();
}
}
main().catch((e) => {
console.error(e);
process.exit(1);
});
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1200 630" role="img" aria-label="Canada — MapleBrief"><defs><linearGradient id="bg" x1="0" y1="0" x2="0" y2="1"><stop offset="0" stop-color="#161614"/><stop offset="0.55" stop-color="#2b2b28"/><stop offset="1.3" stop-color="#45060c"/></linearGradient><pattern id="grid" width="48" height="48" patternUnits="userSpaceOnUse"><path d="M48 0H0V48" fill="none" stroke="#f8402f" stroke-opacity="0.10" stroke-width="1"/></pattern></defs><rect width="1200" height="630" fill="url(#bg)"/><rect width="1200" height="630" fill="url(#grid)"/><g fill="none" stroke="#f8402f" stroke-opacity="0.38" stroke-width="10" stroke-linecap="round" stroke-linejoin="round"><path d="M880 130L904 196L964 182L934 236L1006 250L948 296L1006 338L934 346L964 402L904 386L880 452L856 386L796 402L826 346L754 338L812 296L754 250L826 236L796 182L856 196Z"/><path d="M880 452V520"/></g><g fill="none" stroke="#f8402f" stroke-opacity="0.14" stroke-width="4" stroke-linecap="round" stroke-linejoin="round"><path d="M700 452H1060"/></g><text x="80" y="556" font-family="Georgia, 'Times New Roman', serif" font-size="44" font-weight="700" fill="#ffd9d4" fill-opacity="0.92">Canada</text><text x="80" y="592" font-family="Inter, 'Segoe UI', sans-serif" font-size="18" letter-spacing="4" fill="#f8402f" fill-opacity="0.55">MAPLEBRIEF</text></svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1200 630" role="img" aria-label="National — MapleBrief"><defs><linearGradient id="bg" x1="0" y1="0" x2="0" y2="1"><stop offset="0" stop-color="#161614"/><stop offset="0.55" stop-color="#2b2b28"/><stop offset="1.3" stop-color="#45060c"/></linearGradient><pattern id="grid" width="48" height="48" patternUnits="userSpaceOnUse"><path d="M48 0H0V48" fill="none" stroke="#f8402f" stroke-opacity="0.10" stroke-width="1"/></pattern></defs><rect width="1200" height="630" fill="url(#bg)"/><rect width="1200" height="630" fill="url(#grid)"/><g fill="none" stroke="#f8402f" stroke-opacity="0.38" stroke-width="10" stroke-linecap="round" stroke-linejoin="round"><path d="M760 440V300M880 440V180M1000 440V300"/><path d="M850 180L880 120L910 180" stroke-width="7"/><path d="M782 300L760 268L738 300M978 300L1000 268L1022 300" stroke-width="7"/><path d="M760 380H1000M820 340H940" stroke-width="7"/></g><g fill="none" stroke="#f8402f" stroke-opacity="0.14" stroke-width="4" stroke-linecap="round" stroke-linejoin="round"><path d="M700 440H1060M700 160H1060"/></g><text x="80" y="556" font-family="Georgia, 'Times New Roman', serif" font-size="44" font-weight="700" fill="#ffd9d4" fill-opacity="0.92">National</text><text x="80" y="592" font-family="Inter, 'Segoe UI', sans-serif" font-size="18" letter-spacing="4" fill="#f8402f" fill-opacity="0.55">MAPLEBRIEF</text></svg>

After

Width:  |  Height:  |  Size: 1.4 KiB

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1200 630" role="img" aria-label="Top Stories — MapleBrief"><defs><linearGradient id="bg" x1="0" y1="0" x2="0" y2="1"><stop offset="0" stop-color="#161614"/><stop offset="0.55" stop-color="#2b2b28"/><stop offset="1.3" stop-color="#45060c"/></linearGradient><pattern id="grid" width="48" height="48" patternUnits="userSpaceOnUse"><path d="M48 0H0V48" fill="none" stroke="#f8402f" stroke-opacity="0.10" stroke-width="1"/></pattern></defs><rect width="1200" height="630" fill="url(#bg)"/><rect width="1200" height="630" fill="url(#grid)"/><g fill="none" stroke="#f8402f" stroke-opacity="0.38" stroke-width="10" stroke-linecap="round" stroke-linejoin="round"><circle cx="880" cy="305" r="150"/><ellipse cx="880" cy="305" rx="64" ry="150"/><path d="M736 254H1024M730 356H1030"/></g><g fill="none" stroke="#f8402f" stroke-opacity="0.14" stroke-width="4" stroke-linecap="round" stroke-linejoin="round"><path d="M700 155H1060M700 455H1060"/></g><text x="80" y="556" font-family="Georgia, 'Times New Roman', serif" font-size="44" font-weight="700" fill="#ffd9d4" fill-opacity="0.92">Top Stories</text><text x="80" y="592" font-family="Inter, 'Segoe UI', sans-serif" font-size="18" letter-spacing="4" fill="#f8402f" fill-opacity="0.55">MAPLEBRIEF</text></svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

+15
View File
@@ -0,0 +1,15 @@
<svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 240 240">
<defs>
<linearGradient id="m-bg" x1="0" y1="0" x2="1" y2="1">
<stop offset="0" stop-color="#1c1917"/>
<stop offset="1" stop-color="#292524"/>
</linearGradient>
<linearGradient id="m-leaf" x1="0" y1="0" x2="1" y2="1">
<stop offset="0" stop-color="#f97316"/>
<stop offset="1" stop-color="#dc2626"/>
</linearGradient>
</defs>
<rect x="0" y="0" width="240" height="240" rx="54" fill="url(#m-bg)"/>
<g transform="translate(120 116) scale(96)"><path fill="url(#m-leaf)" d="M 0 -1 Q 0.038 -0.297 0.169 -0.318 Q 0.454 -0.311 0.84 -0.374 Q 0.273 -0.066 0.372 0.079 Q 0.501 0.296 0.741 0.579 Q 0.2 0.204 0.146 0.329 L 0.025 0.178 L -0.025 0.178 Q -0.038 0.114 -0.146 0.329 Q -0.399 0.409 -0.741 0.579 Q -0.25 0.148 -0.372 0.079 Q -0.545 -0.133 -0.84 -0.374 Q -0.227 -0.156 -0.169 -0.318 Z"/></g>
</svg>

After

Width:  |  Height:  |  Size: 925 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.6 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

+3
View File
File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 53 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 40 KiB

+126
View File
@@ -0,0 +1,126 @@
#!/usr/bin/env bash
# ────────────────────────────────────────────────────────────────────────────
# maple-brief commit gate
# Approved order (hard, fail-closed): 1) secret → 2) build → 3) test
#
# commit mode (pre-commit, fast):
# 1. gitleaks on STAGED changes 2. tsc --noEmit 3. LLM parse unit tests
# push mode (pre-push, heavy):
# 1. gitleaks on full HEAD history 2. prisma generate && next build
#
# Fail-closed: any stage error (missing tool, crash, timeout, findings)
# fails the gate. Same tree ⇒ same verdict (gates are pure functions of
# committed content; no network, no cached state). Only deliberate escape
# hatch: `git commit --no-verify` / `git push --no-verify` (explicit, logged
# by the remote user).
#
# Usage: gate.sh commit | push
# ────────────────────────────────────────────────────────────────────────────
set -u
MODE="${1:-commit}"
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
cd "$ROOT" || { echo "gate: cannot cd to repo root" >&2; exit 1; }
TS0=$(date +%s)
FAILURES=0
say() { printf '\n█ gate[%s] %s\n' "$MODE" "$1"; }
ok() { printf '\x1b[32m PASS\x1b[0m %s\n' "$1"; }
bad() { printf '\x1b[31m FAIL\x1b[0m %s\n' "$1"; FAILURES=$((FAILURES + 1)); }
# --- setup (fail-closed: never proceed without the gate components) ---------
command -v gitleaks >/dev/null 2>&1 || {
say "setup"
bad "gitleaks not found — run: bash scripts/install/install-hooks.sh"
printf '\nGATE VERDICT: FAIL (setup)\n'; exit 1; }
[ -d node_modules ] || {
say "setup"
bad "node_modules missing — run: npm ci"
printf '\nGATE VERDICT: FAIL (setup)\n'; exit 1; }
[ -f .gitleaks.toml ] || {
say "setup"; bad ".gitleaks.toml missing (gate config)";
printf '\nGATE VERDICT: FAIL (setup)\n'; exit 1; }
[ -x node_modules/.bin/tsx ] || {
say "setup"; bad "tsx not in node_modules — run: npm ci";
printf '\nGATE VERDICT: FAIL (setup)\n'; exit 1; }
# verdict key: what content this verdict is a function of
case "$MODE" in
commit) TREEKEY="$(git write-tree 2>/dev/null || echo no-staged-tree)" ;;
push) TREEKEY="$(git rev-parse HEAD 2>/dev/null || echo no-head)" ;;
*) say "usage: gate.sh commit|push"; exit 2 ;;
esac
# --- stage 1: secret --------------------------------------------------------
# $1 = "staged" | "history"; gitleaks 8.x takes no positional path — use
# --staged for staged-only, no flag for full history. Exit 0 = clean,
# 1 = findings, anything else = scan error (all fail-closed).
stage_secret() {
local scope="$1"
case "$scope" in
staged) say "1/3 secret — gitleaks (staged changes)"; local args=(--staged) ;;
history) say "1/3 secret — gitleaks (full HEAD history)"; local args=() ;;
*) bad "stage_secret: unknown scope '$scope'"; return 1 ;;
esac
local out rc
out="$(timeout 180 gitleaks protect -c .gitleaks.toml --no-banner --no-color "${args[@]+"${args[@]}"}" 2>&1)"; rc=$?
if [ "$rc" -ne 0 ]; then
bad "secret scan failed (exit ${rc:-ERR})"
printf '%s\n' "$out" | grep -vE '^\s*$' | tail -12 | sed 's/^/ /'
else
ok "no secret matches in $scope"
fi
}
# --- stage 2: build ---------------------------------------------------------
stage_build_commit() {
say "2/3 build — tsc --noEmit (strict TS)"
local out rc
out="$(timeout 240 ./node_modules/.bin/tsc --noEmit 2>&1)"; rc=$?
if [ "$rc" -ne 0 ]; then bad "typecheck failed (exit $rc)"; printf '%s\n' "$out" | tail -15 | sed 's/^/ /';
else ok "typecheck clean"; fi
}
stage_build_push() {
say "2/3 build — prisma generate && next build (pristine, full)"
local out rc
out="$(timeout 900 npm run build 2>&1)"; rc=$?
if [ "$rc" -ne 0 ]; then bad "build failed (exit $rc)"; printf '%s\n' "$out" | tail -20 | sed 's/^/ /';
else ok "production build clean"; fi
}
# --- stage 3: test ----------------------------------------------------------
stage_test() {
say "3/3 test — LLM parse contract (tests/llm-parse.test.ts)"
local out rc
out="$(timeout 120 ./node_modules/.bin/tsx tests/llm-parse.test.ts 2>&1)"; rc=$?
if [ "$rc" -ne 0 ]; then bad "unit test failed (exit $rc)"; printf '%s\n' "$out" | grep -E '^(not ok|ok|fail)' | tail -10 | sed 's/^/ /';
else ok "LLM parse contract holds ($(printf '%s\n' "$out" | grep -c '^ok ') checks)"; fi
}
# --- run (all stages always execute; verdict = OR of failures) --------------
case "$MODE" in
commit)
stage_secret staged
stage_build_commit
stage_test
;;
push)
stage_secret history
stage_build_push
stage_test
;;
esac
ELAPSED=$(( $(date +%s) - TS0 ))
case "$MODE" in
commit) OBJECT="staged-tree $TREEKEY" ;;
push) OBJECT="HEAD $TREEKEY" ;;
esac
if [ "$FAILURES" -eq 0 ]; then
printf '\n════════════════════════════════════════\n GATE PASS [%s] %s in %ss\n════════════════════════════════════════\n' "$MODE" "$OBJECT" "$ELAPSED"
exit 0
else
printf '\n════════════════════════════════════════\n GATE FAIL [%s] %s — %s stage(s) failed, in %ss\n bypass (auditable): git %s --no-verify\n════════════════════════════════════════\n' "$MODE" "$OBJECT" "$FAILURES" "$ELAPSED" "$MODE"
exit 1
fi
+3
View File
@@ -0,0 +1,3 @@
#!/usr/bin/env bash
# pre-commit → orders secret → build(typecheck) → test (fast gates only)
exec bash "$(git rev-parse --show-toplevel)/scripts/gate/gate.sh" commit
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env bash
# pre-push → orders secret (full history) → build (full next build) → test
# Note: git pushes a branch HEAD; scanning full history catches secrets
# committed in the past that a re-fetch of a bare repo would inherit.
exec bash "$(git rev-parse --show-toplevel)/scripts/gate/gate.sh" push
+17
View File
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
# One-shot bootstrap of the gitleaks binary into ~/.local/bin (pinned version).
# Reproducible: fixed tag, not "latest". Bump GL_VER deliberately to upgrade.
set -eu
GL_VER="8.30.1"
DEST="${HOME}/.local/bin"
mkdir -p "$DEST"
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
URL="https://github.com/gitleaks/gitleaks/releases/download/v${GL_VER}/gitleaks_${GL_VER}_linux_x64.tar.gz"
echo "[bootstrap] fetching gitleaks v${GL_VER}"
curl -sSL -m 180 -o "$tmp/gl.tgz" "$URL"
tar xzf "$tmp/gl.tgz" -C "$tmp" gitleaks
install -m0755 "$tmp/gitleaks" "${DEST}/gitleaks"
echo "[bootstrap] installed: $("$DEST/gitleaks" version)${DEST}/gitleaks"
echo "[bootstrap] ensure PATH includes ${DEST}, e.g. in your shell rc:
export PATH=\"${DEST}:\$PATH\""
+34
View File
@@ -0,0 +1,34 @@
#!/usr/bin/env bash
# Install the repo commit gate into this clone (idempotent, re-runnable):
# 1. bootstrap gitleaks binary if missing → ~/.local/bin/gitleaks (pinned)
# 2. copy scripts/gate/hooks/* → .git/hooks/* (repo-local, NOT in repo)
# 3-9 show what the gate will do on every commit/push.
# Nothing here is in the committed history; hooks live in .git/hooks/.
set -eu
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
[ -f "$ROOT/scripts/install/install-hooks.sh" ] || { echo "install: cannot find repo root from $(dirname "$0")" >&2; exit 1; }
cd "$ROOT"
echo "== 1/3 gitleaks"
if ! command -v gitleaks >/dev/null 2>&1; then
if [ -x "$HOME/.local/bin/gitleaks" ]; then
echo "[install] gitleaks at ~/.local/bin (not on PATH) — will be used via absolute path"
else
bash scripts/install/bootstrap-gitleaks.sh
fi
else
gitleaks version | sed 's/^/[install] gitleaks /'
fi
echo "== 2/3 hooks"
install -m0755 scripts/gate/hooks/pre-commit .git/hooks/pre-commit
install -m0755 scripts/gate/hooks/pre-push .git/hooks/pre-push
echo "[install] wrote .git/hooks/pre-commit, .git/hooks/pre-push"
echo "== 3/3 ready"
echo "[install] commits gated: secret → tsc → tests (fast, seconds)"
echo "[install] pushes gated: secret(history) → full next build → tests"
echo "[install] bypass (auditable at the pusher's shell): git commit/push --no-verify"
echo "[install] NOTE: gate lives in THIS clone's .git/hooks — after git clone,"
echo "[install] the same-branch pull on another machine re-installs by re-running:"
echo "[install] bash scripts/install/install-hooks.sh"
+92
View File
@@ -0,0 +1,92 @@
#!/usr/bin/env node
/**
* Generate per-category cover art used as the last-resort image fallback
* (baked into /public/covers/*.svg, served statically).
*
* Design language matches the brand system: ink→brazil gradient (#161614 to
* #45060c), maple-500 line art, Source Serif word label. 1200x630 (16:8).
*
* Usage: node scripts/make-covers.mjs (idempotent — overwrites)
*/
import { mkdir, writeFile } from 'node:fs/promises';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
const root = join(dirname(fileURLToPath(import.meta.url)), '..');
const outDir = join(root, 'public', 'covers');
const BG =
'<defs><linearGradient id="bg" x1="0" y1="0" x2="0" y2="1">' +
'<stop offset="0" stop-color="#161614"/><stop offset="0.55" stop-color="#2b2b28"/>' +
'<stop offset="1.3" stop-color="#45060c"/></linearGradient>' +
'<pattern id="grid" width="48" height="48" patternUnits="userSpaceOnUse">' +
'<path d="M48 0H0V48" fill="none" stroke="#f8402f" stroke-opacity="0.10" stroke-width="1"/>' +
'</pattern></defs>';
const stroke = 'fill="none" stroke="#f8402f" stroke-opacity="0.38" stroke-width="10" stroke-linecap="round" stroke-linejoin="round"';
const faint = 'fill="none" stroke="#f8402f" stroke-opacity="0.14" stroke-width="4" stroke-linecap="round" stroke-linejoin="round"';
/** Simple geometric motif per category, centred at (880, 300). */
const MOTIFS = {
// Parliament Hill: central tower + two flanking towers with connecting wings
national:
`<g ${stroke}>` +
`<path d="M760 440V300M880 440V180M1000 440V300"/>` +
`<path d="M850 180L880 120L910 180" stroke-width="7"/>` +
`<path d="M782 300L760 268L738 300M978 300L1000 268L1022 300" stroke-width="7"/>` +
`<path d="M760 380H1000M820 340H940" stroke-width="7"/>` +
`</g>` +
`<g ${faint}>` +
`<path d="M700 440H1060M700 160H1060"/>` +
`</g>`,
// maple leaf: stem + lobe outline
canada:
`<g ${stroke}>` +
`<path d="M880 130L904 196L964 182L934 236L1006 250L948 296L1006 338L934 346L964 402L904 386L880 452L856 386L796 402L826 346L754 338L812 296L754 250L826 236L796 182L856 196Z"/>` +
`<path d="M880 452V520"/>` +
`</g>` +
`<g ${faint}>` +
`<path d="M700 452H1060"/>` +
`</g>`,
// world globe with meridian (top stories)
'top-stories':
`<g ${stroke}>` +
`<circle cx="880" cy="305" r="150"/>` +
`<ellipse cx="880" cy="305" rx="64" ry="150"/>` +
`<path d="M736 254H1024M730 356H1030"/>` +
`</g>` +
`<g ${faint}>` +
`<path d="M700 155H1060M700 455H1060"/>` +
`</g>`,
};
const LABEL = {
national: 'National',
canada: 'Canada',
'top-stories': 'Top Stories',
};
function cover(category, motif, label) {
void category; // contextual only; filename is the real key
return (
`<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1200 630" role="img" aria-label="${label} — MapleBrief">` +
BG +
`<rect width="1200" height="630" fill="url(#bg)"/>` +
`<rect width="1200" height="630" fill="url(#grid)"/>` +
motif +
`<text x="80" y="556" font-family="Georgia, 'Times New Roman', serif" font-size="44" font-weight="700" fill="#ffd9d4" fill-opacity="0.92">${label}</text>` +
`<text x="80" y="592" font-family="Inter, 'Segoe UI', sans-serif" font-size="18" letter-spacing="4" fill="#f8402f" fill-opacity="0.55">MAPLEBRIEF</text>` +
`</svg>`
);
}
await mkdir(outDir, { recursive: true });
const covers = {
'top-stories.svg': cover('top-stories', MOTIFS['top-stories'], LABEL['top-stories']),
'national.svg': cover('national', MOTIFS.national, LABEL.national),
'canada.svg': cover('canada', MOTIFS.canada, LABEL.canada),
};
for (const [name, svg] of Object.entries(covers)) {
await writeFile(join(outDir, name), svg);
console.log('wrote', name);
}
+85
View File
@@ -0,0 +1,85 @@
import { getPublishedArticles, getTagList } from '@/lib/queries';
import ArticleCard from '@/components/ArticleCard';
import { InFeedAd, SidebarAd } from '@/components/ads/placements';
import { SECTIONS } from '@/data/feeds';
import type { ReactNode } from 'react';
import { notFound } from 'next/navigation';
export const dynamic = 'force-dynamic';
type Params = { section: string; page?: string };
export default async function SectionPage({
params,
searchParams,
}: {
params: Params;
searchParams: { page?: string };
}) {
const section = params.section;
const known = SECTIONS.find((s) => s.slug === section);
if (!known) notFound();
const page = Math.max(1, parseInt(searchParams.page ?? '1', 10) || 1);
const PER = 24;
const [articles, tags] = await Promise.all([
getPublishedArticles(section, PER, (page - 1) * PER),
getTagList(10),
]);
return (
<>
<div className="mx-auto max-w-6xl px-4 py-6 sm:px-6">
<h1 className="mb-6 font-display text-3xl font-bold text-ink-900">
{known.label}
</h1>
<div className="grid gap-8 lg:grid-cols-3">
<div className="lg:col-span-2">
{articles.length === 0 ? (
<p className="rounded-xl border border-dashed border-ink-300 bg-white p-10 text-center text-sm text-ink-500">
Nothing in {known.label} yet the worker will fill this in.
</p>
) : (
<div className="grid gap-5 sm:grid-cols-2">
{interleave(articles, 3)}
</div>
)}
</div>
<aside aria-label="Sidebar" className="space-y-8">
<SidebarAd tall />
{tags.length > 0 && (
<section className="rounded-xl border border-ink-200 bg-white p-5">
<h2 className="text-xs font-semibold uppercase tracking-wider text-ink-500">
Trending topics
</h2>
<div className="mt-3 flex flex-wrap gap-2">
{tags.map((t) => (
<a
key={t}
href={`/tag/${t}`}
className="rounded-full bg-ink-100 px-3 py-1 text-sm text-ink-700 hover:bg-maple-100 hover:text-maple-800"
>
#{t}
</a>
))}
</div>
</section>
)}
</aside>
</div>
</div>
</>
);
}
function interleave(
cards: Awaited<ReturnType<typeof getPublishedArticles>>,
every: number,
): ReactNode[] {
const out: ReactNode[] = [];
cards.forEach((card, i) => {
out.push(<ArticleCard key={card.id} card={card} />);
if ((i + 1) % every === 0) out.push(<InFeedAd key={`ad-${i + 1}`} index={i + 1} />);
});
return out;
}
+50
View File
@@ -0,0 +1,50 @@
import type { Metadata } from 'next';
import Link from 'next/link';
import { env } from '@/lib/env';
export const metadata: Metadata = {
title: 'About',
description: 'What MapleBrief is: an automated digest of Canadian news with full source attribution.',
alternates: { canonical: '/about' },
};
export default function AboutPage() {
return (
<div className="mx-auto max-w-3xl px-4 py-10 sm:px-6">
<h1 className="font-display text-4xl font-bold text-ink-950">About MapleBrief</h1>
<div className="mt-6 space-y-4 text-[15px] leading-relaxed text-ink-700">
<p>
{env.siteName} is a small, independent Canadian news-digest project.
Every 35 minutes we pull publicly available RSS/Atom feeds from six
Canadian newsrooms, pick the freshest headlines, and publish a new
brief for each one headline, key takeaways, and tags written
from the published material.
</p>
<p>
What we do <strong>not</strong> do: copy articles verbatim or hide
where the material came from. Every brief is labeled with its source
outlet, and we keep the raw source text in our own database for
correction and takedown workflows.
</p>
<p>
The site is supported by advertising through Google AdSense, and we
have written a{' '}
<Link href="/privacy-policy" className="font-medium text-maple-700 underline">
Privacy Policy
</Link>{' '}
to describe exactly how cookies and DART tracking work here.
</p>
<p>
If you believe a brief misrepresents the original reporting, or you
are a rights holder asking for attribution to be handled differently,
please use the{' '}
<Link href="/contact" className="font-medium text-maple-700 underline">
contact page
</Link>{' '}
we read every issue.
</p>
</div>
</div>
);
}
+17
View File
@@ -0,0 +1,17 @@
import type { Metadata } from 'next';
import AdminGate from '@/components/admin/AdminGate';
import SettingsForm from '@/components/admin/SettingsForm';
export const metadata: Metadata = {
title: 'Admin',
// Never let search engines index the admin area
robots: { index: false, follow: false },
};
export default function AdminSettingsPage() {
return (
<AdminGate>
<SettingsForm />
</AdminGate>
);
}
+16
View File
@@ -0,0 +1,16 @@
import { NextResponse } from 'next/server';
import { isAuthorized } from '@/lib/auth/admin';
export const runtime = 'nodejs';
/**
* Used by the admin UI gate: confirm an admin key before rendering the form.
* Note: the actual key is only ever sent in request headers, never stored in
* localStorage, and the gate keeps it in sessionStorage for the browser tab.
*/
export async function GET(req: Request) {
if (isAuthorized(req as unknown as Parameters<typeof isAuthorized>[0])) {
return NextResponse.json({ ok: true });
}
return NextResponse.json({ ok: false, error: 'unauthorized' }, { status: 401 });
}
+44
View File
@@ -0,0 +1,44 @@
import { NextRequest, NextResponse } from 'next/server';
export const runtime = 'nodejs';
export const dynamic = 'force-dynamic';
interface ContactPayload {
email?: string;
topic?: string;
url?: string;
message?: string;
}
/**
* Placeholder contact sink.
*
* Product wiring options (in order of least effort):
* 1. Email: swap the `console.log` for `nodemailer` or an external
* transactional API (Resend, Postmark, SendGrid).
* 2. Form service: point the fetch at Formspree/Getform instead.
*
* The admin UI replies to itself and there is currently no secret channel;
* in production add a rate limit (e.g., 5/hour/IP) and CAPTCHA.
*/
export async function POST(req: NextRequest) {
let body: ContactPayload;
try {
body = await req.json();
} catch {
return NextResponse.json({ ok: false, error: 'invalid JSON' }, { status: 400 });
}
if (!body.email || !body.message) {
return NextResponse.json(
{ ok: false, error: 'email and message are required' },
{ status: 400 },
);
}
// Sensitive-ish: keep payload out of stdout logs in DEBUG mode.
console.log(`[contact] topic=${body.topic ?? 'other'} from=${body.email} url=${body.url ?? '-'}`);
console.log(`[contact] message: ${String(body.message).slice(0, 500)}`);
return NextResponse.json({ ok: true });
}
+32
View File
@@ -0,0 +1,32 @@
import { NextRequest, NextResponse } from 'next/server';
import { runPipelinePass } from '@/lib/worker/cron';
import { isAuthorized } from '@/lib/auth/admin';
export const runtime = 'nodejs';
export const dynamic = 'force-dynamic';
// Ingestion (6+ feeds) + a batch of LLM calls can take a while.
export const maxDuration = 300;
/**
* Admin-triggered one-shot pipeline pass: ingest all enabled feeds, then
* synthesize pending articles.
*
* Auth: requires ADMIN_API_KEY via ?key=, the `x-admin-key` header, or
* Basic auth with any username.
*/
export async function POST(req: NextRequest) {
if (!(await isAuthorized(req))) {
return NextResponse.json({ error: 'unauthorized' }, { status: 401 });
}
// One pass at a time; run in background so the client can connect-timeout.
runPipelinePass('api-refresh').catch((e) =>
console.error('[refresh] unhandled:', e),
);
return NextResponse.json({
started: true,
message: 'Pipeline pass started in background.',
time: new Date().toISOString(),
});
}
+138
View File
@@ -0,0 +1,138 @@
import { NextRequest, NextResponse } from 'next/server';
import { isAuthorized } from '@/lib/auth/admin';
import { prisma } from '@/lib/db';
import {
getLlmSettings,
upsertSetting,
SETTING_KEYS,
} from '@/lib/llm';
import type { LlmProviderId } from '@/lib/settings';
export const runtime = 'nodejs';
export const dynamic = 'force-dynamic';
const PROVIDERS: LlmProviderId[] = ['openai', 'anthropic', 'ollama'];
const PRO_MODEL_KEYS: Record<LlmProviderId, string> = {
openai: 'llm.model.openai',
anthropic: 'llm.model.anthropic',
ollama: 'llm.model.ollama',
};
/** Last-4 for display only — full secrets never leave the server. */
function last4(secret: string): string {
return secret.length >= 4 ? `${secret.slice(-4)}` : '';
}
function isProvider(v: unknown): v is LlmProviderId {
return v === 'openai' || v === 'anthropic' || v === 'ollama';
}
async function readSetting(key: string): Promise<string | null> {
const row = await prisma.setting.findUnique({ where: { key } });
return row?.value ?? null;
}
interface SettingsBody {
provider?: unknown;
model?: unknown; // legacy single-model field
models?: Partial<Record<LlmProviderId, unknown>>;
apiKeys?: { openai?: unknown; anthropic?: unknown };
ollamaBase?: unknown;
synthesisPrompt?: unknown;
}
async function toResponseBody() {
const s = await getLlmSettings();
const [dbProvider, dbModel, dbOpenai, dbAnthropic] = await Promise.all([
readSetting(SETTING_KEYS.provider),
readSetting(SETTING_KEYS.model),
readSetting(SETTING_KEYS.openaiKey),
readSetting(SETTING_KEYS.anthropicKey),
]);
const models = {} as Record<LlmProviderId, string>;
for (const p of PROVIDERS) models[p] = (await readSetting(PRO_MODEL_KEYS[p])) ?? s.model;
// A legacy single-model override applies to the provider it was set for.
const legacyOwner = isProvider(dbProvider) ? dbProvider : 'openai';
if (dbModel) models[legacyOwner] = dbModel;
return {
settings: {
provider: s.provider,
models,
apiKeys: { openai: '', anthropic: '' }, // secrets are never echoed
ollamaBase: s.ollamaBaseUrl,
synthesisPrompt: s.synthesisPrompt,
},
keySources: {
openai: Boolean(dbOpenai),
anthropic: Boolean(dbAnthropic),
ollama: true,
},
keys: {
openaiLast4: last4(s.openaiApiKey),
anthropicLast4: last4(s.anthropicApiKey),
},
};
}
export async function GET(req: NextRequest) {
if (!isAuthorized(req)) {
return NextResponse.json({ error: 'unauthorized' }, { status: 401 });
}
return NextResponse.json(await toResponseBody());
}
export async function PUT(req: NextRequest) {
if (!isAuthorized(req)) {
return NextResponse.json({ error: 'unauthorized' }, { status: 401 });
}
const body = (await req.json().catch(() => null)) as SettingsBody | null;
if (!body) return NextResponse.json({ error: 'invalid json' }, { status: 400 });
// Provider
if (body.provider !== undefined) {
if (!isProvider(body.provider)) {
return NextResponse.json(
{ error: 'provider must be openai | anthropic | ollama' },
{ status: 400 },
);
}
await upsertSetting(SETTING_KEYS.provider, body.provider);
}
const active =
(body.provider as LlmProviderId | undefined) ?? (await getLlmSettings()).provider;
// Models — per-provider, with single-model fallback for simple clients
if (body.models && typeof body.models === 'object') {
for (const p of PROVIDERS) {
const m = String(body.models[p] ?? '').trim();
if (m) {
await upsertSetting(PRO_MODEL_KEYS[p], m);
if (p === active) await upsertSetting(SETTING_KEYS.model, m);
}
}
} else if (typeof body.model === 'string' && body.model.trim()) {
await upsertSetting(SETTING_KEYS.model, body.model.trim());
await upsertSetting(PRO_MODEL_KEYS[active], body.model.trim());
}
// API keys (secrets; blank from the UI means "keep using the env var")
if (body.apiKeys) {
const openaiKey = String(body.apiKeys.openai ?? '').trim();
const anthropicKey = String(body.apiKeys.anthropic ?? '').trim();
if (openaiKey) await upsertSetting(SETTING_KEYS.openaiKey, openaiKey, true);
if (anthropicKey) await upsertSetting(SETTING_KEYS.anthropicKey, anthropicKey, true);
}
// Ollama base URL
const ollamaBase = String(body.ollamaBase ?? '').trim();
if (ollamaBase) await upsertSetting(SETTING_KEYS.ollamaBase, ollamaBase);
// Synthesis prompt
const prompt = String(body.synthesisPrompt ?? '').trim();
if (prompt) await upsertSetting(SETTING_KEYS.synthesisPrompt, prompt);
return NextResponse.json(await toResponseBody());
}
+50
View File
@@ -0,0 +1,50 @@
import { NextRequest, NextResponse } from 'next/server';
import { isAuthorized } from '@/lib/auth/admin';
import { resolveLlm } from '@/lib/llm';
import { OllamaProvider } from '@/lib/llm/ollama';
export const runtime = 'nodejs';
export const dynamic = 'force-dynamic';
export const maxDuration = 60;
/**
* Connectivity probe for the active (or requested) provider.
* - ollama: GET /api/tags
* - openai/anthropic: a 4-token chat completion
*/
export async function POST(req: NextRequest) {
if (!isAuthorized(req)) return NextResponse.json({ error: 'unauthorized' }, { status: 401 });
const body = (await req.json().catch(() => ({}))) as { provider?: string };
const providerId = body.provider;
try {
if (providerId === 'ollama') {
const { provider: settings } = { provider: undefined };
const { instance } = await resolveLlm();
if (instance instanceof OllamaProvider) {
return NextResponse.json({
ok: await (instance as OllamaProvider).ping(),
provider: 'ollama',
});
}
// fall through to generic below if active isn't ollama
}
const { instance } = await resolveLlm();
const result = await instance.synthesize({
sourceText: 'A test event: maple syrup production was reported stable this season.',
sources: ['Test Source'],
systemPrompt:
'You are a test probe. Reply with a JSON object: {"headline":"Test","body":"ok \\n\\nok \\n\\nok","takeaways":["test"],"tags":["test"]}',
});
return NextResponse.json({
ok: true,
provider: result.provider,
model: result.model,
tookMs: result.tookMs,
});
} catch (err) {
return NextResponse.json({ ok: false, error: (err as Error).message }, { status: 200 });
}
}
+40
View File
@@ -0,0 +1,40 @@
import { NextRequest, NextResponse } from 'next/server';
import { isAuthorized } from '@/lib/auth/admin';
import { prisma } from '@/lib/db';
export const runtime = 'nodejs';
export const dynamic = 'force-dynamic';
/**
* Pipeline telemetry for the admin dashboard: article counts by status,
* feeds with last-fetch times, recent synthesis activity.
*/
export async function GET(req: NextRequest) {
if (!(await isAuthorized(req))) {
return NextResponse.json({ error: 'unauthorized' }, { status: 401 });
}
const [total, fetched, synthesized, failed, feedsByCategory] = await Promise.all([
prisma.article.count(),
prisma.article.count({ where: { status: 'fetched' } }),
prisma.article.count({ where: { status: 'synthesized' } }),
prisma.article.count({ where: { status: 'failed' } }),
prisma.feed.findMany({ orderBy: { name: 'asc' } }),
]);
const categories = await prisma.article
.groupBy({ by: ['category'], _count: { _all: true }, where: { status: 'synthesized' } })
.then((rows) => rows.map((r) => ({ category: r.category, count: r._count._all })));
return NextResponse.json({
articles: { total, fetched, synthesized, failed },
feeds: feedsByCategory.map((f) => ({
name: f.name,
category: f.category,
enabled: f.enabled,
lastFetchedAt: f.lastFetchedAt,
})),
categories,
time: new Date().toISOString(),
});
}
+27
View File
@@ -0,0 +1,27 @@
import { NextRequest, NextResponse } from 'next/server';
import { initWorker } from '@/lib/worker/cron';
export const runtime = 'nodejs';
export const dynamic = 'force-dynamic';
/**
* Keep-alive endpoint for the background worker.
*
* In long-lived Node deployments (`next start`, a container, or a VPS) the
* first hit to / initializes the node-cron scheduler; subsequent hits are
* no-ops. Call it on your launch script / healthcheck:
*
* curl -sX POST http://localhost:3000/api/worker/ping
*
* When RUN_SCHEDULER=true the worker then ingests + synthesizes automatically
* on CRON_SCHEDULE. On cold-start platforms (e.g. free serverless) drive
* ingestion with the `/api/refresh` route from an external timer instead.
*/
export async function POST(_req: NextRequest) {
const status = initWorker();
return NextResponse.json({ status, time: new Date().toISOString() });
}
export async function GET(_req: NextRequest) {
return NextResponse.json({ status: initWorker() });
}
+171
View File
@@ -0,0 +1,171 @@
import type { Metadata } from 'next';
import Link from 'next/link';
import { notFound } from 'next/navigation';
import { getArticleFull, getRelatedArticles } from '@/lib/queries';
import RelatedArticles from '@/components/RelatedArticles';
import { InArticleAd, SidebarAd } from '@/components/ads/placements';
import { categoryCover, ogImageUrl, resolveCoverImage } from '@/lib/cover';
import { CoverImage } from '@/components/CoverImage';
import { env } from '@/lib/env';
import { formatFull } from '@/lib/format';
import { safeParse } from '@/lib/llm/parse';
import { SECTIONS } from '@/data/feeds';
export const dynamic = 'force-dynamic';
interface ArticleData {
id: string;
slug: string;
title: string;
headline: string | null;
body: string | null;
takeaways: string | null;
tags: string | null;
siteName: string;
category: string;
author: string | null;
publishedAt: Date | null;
synthesizedAt: Date | null;
sourceUrl: string;
canonicalUrl: string;
image: string | null;
sources: string | null;
originalText: string | null;
llmProvider: string | null;
}
export async function generateMetadata({
params,
}: {
params: { slug: string };
}): Promise<Metadata> {
const a = await getArticleFull(params.slug);
if (!a) return { title: 'Briefing not found' };
const title = a.headline ?? a.title;
const description = a.body?.split('\n\n')[0]?.slice(0, 160) ?? '';
const sectionLabel = SECTIONS.find((s) => s.slug === a.category)?.label ?? 'Briefing';
return {
title,
description,
alternates: { canonical: a.canonicalUrl },
openGraph: {
type: 'article',
title,
description,
url: a.canonicalUrl,
siteName: env.siteName,
publishedTime: a.publishedAt?.toISOString(),
modifiedTime: a.synthesizedAt?.toISOString(),
section: sectionLabel,
tags: a.tags ? safeParse<string[]>(a.tags).slice(0, 5) : undefined,
images: [{
url: ogImageUrl(a.image, a.category),
width: 1200,
height: 630,
alt: title,
}],
},
twitter: {
card: 'summary_large_image',
title,
description,
images: [ogImageUrl(a.image, a.category)],
},
};
}
export default async function ArticlePage({
params,
}: {
params: { slug: string };
}) {
const a = await getArticleFull(params.slug);
if (!a || !a.body) notFound();
const paragraphs = a.body.split('\n\n').map((p) => p.trim()).filter(Boolean);
const takeaways = a.takeaways ? safeParse<string[]>(a.takeaways) : [];
const tags = a.tags ? safeParse<string[]>(a.tags) : [];
const relatedItems = await getRelatedArticles(a.slug, 6);
const title = a.headline ?? a.title;
return (
<div className="mx-auto max-w-6xl px-4 py-6 sm:px-6">
<div className="grid gap-10 lg:grid-cols-3">
{/* Article column */}
<article className="lg:col-span-2">
<header>
<p className="mb-3 flex flex-wrap items-center gap-2 text-sm">
<span className="rounded bg-maple-600/10 px-2 py-0.5 text-xs font-bold uppercase tracking-wide text-maple-700">
{SECTIONS.find((s) => s.slug === a.category)?.label ?? 'Briefing'}
</span>
{a.publishedAt && (
<span className="text-ink-400">· {formatFull(a.publishedAt.toISOString())}</span>
)}
</p>
<h1 className="font-display text-3xl font-bold leading-tight text-ink-950 sm:text-4xl">
{title}
</h1>
</header>
<CoverImage
src={resolveCoverImage(a.image, a.category)}
category={a.category}
alt={title}
loading="eager"
className="mt-6 aspect-[16/8] w-full rounded-2xl object-cover"
/>
{takeaways.length > 0 && (
<section className="mt-8 rounded-xl border-l-4 border-maple-600 bg-maple-50 p-5">
<h2 className="text-xs font-bold uppercase tracking-wider text-maple-800">
Key takeaways
</h2>
<ul className="mt-3 grid gap-2 sm:grid-cols-1">
{takeaways.map((t, i) => (
<li key={i} className="flex gap-2 text-[15px] leading-snug text-ink-800">
<span aria-hidden className="mt-0.5 font-bold text-maple-600"></span>
{t}
</li>
))}
</ul>
</section>
)}
<div className="article-prose mt-8">
{paragraphs.slice(0, 1).map((p, i) => (
<p key={i}>{p}</p>
))}
{/* In-article ad: between rewritten paragraphs 1 and 2 */}
<InArticleAd />
{paragraphs.slice(1).map((p, i) => (
<p key={i}>{p}</p>
))}
</div>
{tags.length > 0 && (
<div className="mt-8 flex flex-wrap gap-2">
{tags.map((t) => (
<Link
key={t}
href={`/tag/${t}`}
className="rounded-full bg-ink-100 px-3 py-1 text-sm text-ink-700 hover:bg-maple-100 hover:text-maple-800"
>
#{t}
</Link>
))}
</div>
)}
<RelatedArticles items={relatedItems} />
</article>
{/* Sidebar */}
<aside aria-label="Sidebar" className="space-y-8">
<SidebarAd tall />
</aside>
</div>
</div>
);
}
+29
View File
@@ -0,0 +1,29 @@
import type { Metadata } from 'next';
import Link from 'next/link';
import ContactForm from '@/components/ContactForm';
export const metadata: Metadata = {
title: 'Contact',
description: 'Contact MapleBrief: corrections, legal DMCA inquiries, and advertising.',
alternates: { canonical: '/contact' },
};
export default function ContactPage() {
return (
<div className="mx-auto max-w-2xl px-4 py-10 sm:px-6">
<h1 className="font-display text-4xl font-bold text-ink-950">Contact</h1>
<p className="mt-3 text-[15px] text-ink-600">
Editorial corrections, DMCA / copyright notices, and AdSense partner
inquiries are all welcome. Legal requests should include the URL of the
affected brief and the original source URL.
</p>
<ContactForm />
<p className="mt-8 text-xs text-ink-400">
Prefer email? See the <Link href="/about" className="underline">about page</Link>{' '}
for the masthead, and this form returns the same destination.
</p>
</div>
);
}
+41
View File
@@ -0,0 +1,41 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
:root {
--ink: #161614;
}
html {
scroll-behavior: smooth;
/* never allow sideways scroll from any overflow (sticky-safe, unlike hidden) */
overflow-x: clip;
}
body {
@apply bg-ink-50 text-ink-900 antialiased;
}
::selection {
@apply bg-maple-200 text-ink-950;
}
/* Prose-like base for article bodies */
.article-prose p {
@apply mb-5 text-[1.0625rem] leading-[1.75] text-ink-800;
}
.article-prose p:first-of-type::first-letter {
@apply float-left mr-2 font-display text-5xl leading-[0.9] font-bold text-maple-600;
}
/* Screenshot-like shimmer for ad placeholders before AdSense loads */
@keyframes shimmer {
0% { background-position: -400px 0; }
100% { background-position: 400px 0; }
}
.ad-shimmer {
background: linear-gradient(90deg, #e6e6e3 25%, #f4f4f3 50%, #e6e6e3 75%);
background-size: 800px 100%;
animation: shimmer 1.6s infinite linear;
}
+94
View File
@@ -0,0 +1,94 @@
import { NextRequest } from 'next/server';
import { readFile } from 'node:fs/promises';
import path from 'node:path';
import {
MAX_PROXY_BYTES,
MEDIA_DIR,
cacheImage,
fetchUpstreamImage,
mediaKey,
sniffImage,
} from '@/lib/images';
import { isSafeProxyTarget } from '@/lib/image-mapping';
export const dynamic = 'force-dynamic';
const MAX_AGE_1Y = 'public, max-age=31536000, immutable';
function imageResponse(
buf: Buffer,
type: string,
headers?: Record<string, string>,
): Response {
return new Response(new Uint8Array(buf), {
headers: {
'content-type': type,
'content-length': String(buf.byteLength),
'cache-control': MAX_AGE_1Y,
// Browsers sometimes apply hotlink-style referer checks to <img>;
// our own copies are always allowed from anywhere.
'access-control-allow-origin': '*',
vary: 'accept-encoding, referer',
...headers,
},
});
}
/**
* Server-side image proxy + local cache.
*
* GET /images?url=<absolute https:// publisher image url>
*
* 1. If a local copy already exists under /data/media, serve it (no
* publisher round-trip — the whole point of the setup).
* 2. Otherwise fetch the publisher URL server-side with browser-like
* headers, verify the bytes are a real raster image, persist the copy,
* and serve it.
*
* Everything is verified (magic bytes, 5 MiB cap, SSRF guard on the
* target AND the final redirect host), so a failing or guarding upstream
* yields a short-cached 404/502 — never a broken or spoofed image — and
* the UI falls back to the category cover via the CoverImage onerror.
*/
export async function GET(req: NextRequest): Promise<Response> {
const url = req.nextUrl.searchParams.get('url') ?? '';
const t0 = Date.now();
if (!url || !isSafeProxyTarget(url)) {
return Response.json({ error: 'invalid image url' }, { status: 400 });
}
const name = mediaKey(url);
const file = path.join(MEDIA_DIR, name);
// 1) Local hit — serve the permanent copy.
try {
const buf = await readFile(file);
const type = sniffImage(buf) ?? 'application/octet-stream';
return imageResponse(buf, type, { 'x-image-cache': 'hit' });
} catch {
// miss → fall through to upstream fetch
}
// 2) Upstream fetch (browser UA defeats referrer/hotlink guards).
const got = await fetchUpstreamImage(url);
if (!got.ok) {
console.warn(
`[images] upstream miss ${got.status} (${got.reason}) after ${Date.now() - t0}ms: ${url.slice(0, 120)}`,
);
// Short cache on failure so we re-probe periodically, but don't hammer.
return Response.json(
{ error: 'image unavailable' },
{ status: got.status, headers: { 'cache-control': 'public, max-age=300' } },
);
}
// Await the write so a racing second request cannot double-fetch:
// the first request is the only one that pays the upstream cost.
await cacheImage(url, got.buf);
console.log(
`[images] cached ${Math.round(got.buf.byteLength / 1024)} KiB from ${new URL(url).hostname} in ${Date.now() - t0}ms`,
);
return imageResponse(got.buf, got.type, { 'x-image-cache': 'miss' });
}
+68
View File
@@ -0,0 +1,68 @@
import type { Metadata } from 'next';
import './globals.css';
import Header, { SectionPills } from '@/components/layout/Header';
import { HeaderAd } from '@/components/ads/placements';
import Footer from '@/components/layout/Footer';
import { env } from '@/lib/env';
export const metadata: Metadata = {
metadataBase: new URL(env.siteUrl),
title: {
default: `${env.siteName} — Canadian news briefings`,
template: `%s · ${env.siteName}`,
},
description: env.siteDescription,
alternates: { canonical: '/' },
icons: {
icon: [
{ url: '/favicon-64.png', sizes: '64x64', type: 'image/png' },
],
apple: '/icon-192.png',
},
openGraph: {
type: 'website',
siteName: env.siteName,
title: `${env.siteName} — Canadian news briefings`,
description: env.siteDescription,
url: env.siteUrl,
images: [{ url: '/og-image.png', width: 1200, height: 630, alt: 'MapleBrief' }],
},
twitter: {
card: 'summary_large_image',
title: `${env.siteName} — Canadian news briefings`,
description: env.siteDescription,
images: ['/og-image.png'],
},
robots: {
index: true,
follow: true,
googleBot: {
index: true,
follow: true,
'max-image-preview': 'large',
'max-snippet': -1,
},
},
};
export default function RootLayout({ children }: { children: React.ReactNode }) {
return (
<html lang="en-CA">
<body className="flex min-h-screen flex-col">
<a
href="#main"
className="sr-only focus:not-sr-focus-only focus:absolute focus:left-4 focus:top-4 focus:z-[100] focus:rounded focus:bg-maple-600 focus:px-4 focus:py-2 focus:text-white"
>
Skip to content
</a>
<Header />
<SectionPills />
<HeaderAd />
<main id="main" className="flex-1">
{children}
</main>
<Footer />
</body>
</html>
);
}
+22
View File
@@ -0,0 +1,22 @@
import Link from 'next/link';
export default function NotFound() {
return (
<div className="mx-auto flex max-w-2xl flex-col items-center px-4 py-24 text-center">
<p className="font-display text-6xl font-bold text-ink-300">404</p>
<h1 className="mt-4 font-display text-2xl font-bold text-ink-900">
That briefing has left the map
</h1>
<p className="mt-2 text-sm text-ink-500">
The page you were looking for doesnt exist or may have been superseded
by a newer brief.
</p>
<Link
href="/"
className="mt-8 rounded-lg bg-maple-600 px-5 py-2.5 text-sm font-semibold text-white hover:bg-maple-700"
>
Back to todays briefs
</Link>
</div>
);
}
+122
View File
@@ -0,0 +1,122 @@
import { getPublishedArticles, getTagList } from '@/lib/queries';
import ArticleCard from '@/components/ArticleCard';
import { resolveCoverImage } from '@/lib/cover';
import { CoverImage } from '@/components/CoverImage';
import { InFeedAd, SidebarAd } from '@/components/ads/placements';
import Link from 'next/link';
export const dynamic = 'force-dynamic';
export default async function HomePage({
searchParams,
}: {
searchParams: { page?: string };
}) {
const page = Math.max(1, parseInt(searchParams.page ?? '1', 10) || 1);
const PER = 21;
const [articles, tags] = await Promise.all([
getPublishedArticles('top-stories', PER, (page - 1) * PER),
getTagList(12),
]);
const hero = articles[0];
const rest = articles.slice(1);
return (
<>
<div className="mx-auto max-w-6xl px-4 py-6 sm:px-6">
<div className="grid gap-8 lg:grid-cols-3">
{/* Main column */}
<div className="lg:col-span-2">
{hero ? (
<Link
href={`/article/${hero.slug}`}
className="group mb-8 block overflow-hidden rounded-2xl border border-ink-200 bg-white shadow-sm"
>
<div className="relative aspect-[16/8] overflow-hidden bg-ink-100">
<CoverImage
src={resolveCoverImage(hero.image, hero.category)}
category={hero.category}
alt=""
loading="eager"
className="h-full w-full object-cover transition duration-500 group-hover:scale-[1.02]"
/>
<div className="absolute inset-x-0 bottom-0 bg-gradient-to-t from-ink-950/95 via-ink-950/70 to-transparent px-4 pb-4 pt-10 sm:px-5 sm:pb-5 sm:pt-16">
<span className="mb-2 inline-block rounded bg-maple-600 px-2 py-0.5 text-[11px] font-bold uppercase tracking-wider text-white">
Lead briefing
</span>
<h1 className="line-clamp-2 font-display text-lg font-bold leading-tight text-white sm:line-clamp-3 sm:text-3xl">
{hero.headline ?? hero.title}
</h1>
<p className="mt-2 line-clamp-1 text-sm text-ink-200 sm:line-clamp-2">{hero.excerpt}</p>
</div>
</div>
</Link>
) : (
<EmptyState />
)}
{rest.length > 0 && (
<div className="grid gap-5 sm:grid-cols-2">
{interleaveInFeedAds(rest, 3).map((node) => node)}
</div>
)}
</div>
{/* Sidebar */}
<aside className="space-y-8" aria-label="Sidebar">
<SidebarAd tall />
{tags.length > 0 && (
<section className="rounded-xl border border-ink-200 bg-white p-5">
<h2 className="text-xs font-semibold uppercase tracking-wider text-ink-500">
Trending topics
</h2>
<div className="mt-3 flex flex-wrap gap-2">
{tags.map((t) => (
<Link
key={t}
href={`/tag/${t}`}
className="rounded-full bg-ink-100 px-3 py-1 text-sm text-ink-700 hover:bg-maple-100 hover:text-maple-800"
>
#{t}
</Link>
))}
</div>
</section>
)}
</aside>
</div>
</div>
</>
);
}
import type { ReactNode } from 'react';
/** Interleave an InFeedAd after every `every`th card. */
function interleaveInFeedAds(
cards: Awaited<ReturnType<typeof getPublishedArticles>>,
every: number,
): ReactNode[] {
const out: ReactNode[] = [];
cards.forEach((card, i) => {
out.push(<ArticleCard key={card.id} card={card} />);
if ((i + 1) % every === 0) out.push(<InFeedAd key={`ad-${i + 1}`} index={i + 1} />);
});
return out;
}
function EmptyState() {
return (
<div className="rounded-2xl border border-dashed border-ink-300 bg-white p-12 text-center">
<p className="font-display text-xl font-bold text-ink-800">
No briefings yet
</p>
<p className="mx-auto mt-2 max-w-md text-sm text-ink-500">
The ingestion worker has not published anything yet. Trigger your first
run (see the README First run section) and this page will fill with
fresh briefings.
</p>
</div>
);
}
+165
View File
@@ -0,0 +1,165 @@
import type { Metadata } from 'next';
import Link from 'next/link';
import { env } from '@/lib/env';
export const metadata: Metadata = {
title: 'Privacy Policy',
description: 'How MapleBrief collects, uses, and discloses personal information, including Google AdSense and DART cookies.',
alternates: { canonical: '/privacy-policy' },
};
function Section({ title, children }: { title: string; children: React.ReactNode }) {
return (
<section className="mt-8">
<h2 className="font-display text-xl font-bold text-ink-900">{title}</h2>
<div className="mt-3 space-y-3 text-[15px] leading-relaxed text-ink-700">{children}</div>
</section>
);
}
export default function PrivacyPolicyPage() {
return (
<div className="mx-auto max-w-3xl px-4 py-10 sm:px-6">
<h1 className="font-display text-4xl font-bold text-ink-950">Privacy Policy</h1>
<p className="mt-3 text-sm text-ink-500">
Last updated: {new Date().toISOString().slice(0, 10)} · {env.siteName} ({env.siteUrl})
</p>
<Section title="1. Service overview">
<p>
{env.siteName} (&ldquo;MapleBrief&rdquo;, &ldquo;we&rdquo;, &ldquo;us&rdquo;) is an
automated news-digest site. We aggregate publicly available news feeds
from Canadian news publishers, independently review and summarize that
reporting into original briefs using a language model, and present them
with clear attribution to each original publisher.
</p>
</Section>
<Section title="2. Information we collect">
<p>
We do not require accounts and do not ask you to sign in. We do not
collect names, email addresses, or other identifying personal data
beyond what is automatically generated by your browser.
</p>
<ul className="list-disc space-y-1 pl-5">
<li>Server access logs (IP address, user agent, referrer, timestamp).</li>
<li>De-identified interaction metrics collected by third-party analytics and advertising vendors.</li>
<li>Data collected by our advertising partner, Google, as described below.</li>
</ul>
</Section>
<Section title="3. Google AdSense and third-party advertising">
<p>
We display advertising served by Google AdSense. Google and its
partners use cookies including the Google DART (DoubleClick
Ad Tracking) cookie to serve ads based on your prior visits to this
website and other sites on the Internet.
</p>
<p>
DART enables Google and its partners to serve ads to visitors based on
their visit to this site and other sites on the Internet.
</p>
<p>
Visitors may opt out of the use of the DART cookie by visiting the
Google Ad Settings page:
<a
href="https://www.google.com/settings/ads"
target="_blank"
rel="noopener noreferrer"
className="font-medium text-maple-700 underline"
>
https://www.google.com/settings/ads
</a>
. You can also opt out of third-party advertising cookies generally at
<a
href="https://www.aboutads.info/choices"
target="_blank"
rel="noopener noreferrer"
className="font-medium text-maple-700 underline"
>
https://www.aboutads.info/choices
</a>{' '}
(US/EU) or
<a
href="https://youradchoices.ca"
target="_blank"
rel="noopener noreferrer"
className="font-medium text-maple-700 underline"
>
https://youradchoices.ca
</a>{' '}
(Canada). Your choice of opt-out cookies applies per browser and must
be repeated for each browser and device you use.
</p>
<p>
Google&apos;s privacy policy regarding the use of advertising cookies
(including the DART cookie) is available at:
<a
href="https://policies.google.com/technologies/ads"
target="_blank"
rel="noopener noreferrer"
className="font-medium text-maple-700 underline"
>
https://policies.google.com/technologies/ads
</a>
.
</p>
</Section>
<Section title="4. Cookies">
<p>
We may use first-party cookies for basic site functionality (e.g.,
remembering preferences where applicable). Third-party cookies are
set by our advertising vendors as described in Section 3. You can
control cookies through your browser settings; blocking all cookies
may affect parts of the site.
</p>
</Section>
<Section title="5. Links to other sites">
<p>
Our briefs link out to original publisher articles. Those publishers
have their own privacy policies, and we are not responsible for their
content or practices.
</p>
</Section>
<Section title="6. Children&apos;s privacy">
<p>
Our service is not directed to children, and we do not knowingly
collect personal information from anyone under 13 (or the relevant
local age in Canada).
</p>
</Section>
<Section title="7. Your rights (PIPEDA and provincial privacy law)">
<p>
To the extent PIPEDA or provincial privacy laws apply, you may request
access to, correction of, or deletion of personal information we hold
about you. Contact us using the details below.
</p>
</Section>
<Section title="8. Changes to this policy">
<p>
We may update this policy from time to time. The most recent version
will always be published at{' '}
<Link href="/privacy-policy" className="font-medium text-maple-700 underline">
/privacy-policy
</Link>
.
</p>
</Section>
<Section title="9. Contact">
<p>
Questions about this policy: see our{' '}
<Link href="/contact" className="font-medium text-maple-700 underline">
contact page
</Link>
.
</p>
</Section>
</div>
);
}
+30
View File
@@ -0,0 +1,30 @@
import type { MetadataRoute } from 'next';
import { env } from '@/lib/env';
export const dynamic = 'force-dynamic';
export default function robots(): MetadataRoute.Robots {
const base = env.siteUrl.replace(/\/$/, '');
return {
rules: [
{
// Allow everything including Googlebot & Google-InspectionTool.
// Only block the admin area from indexing/crawling.
userAgent: ['*'],
allow: '/',
disallow: ['/admin', '/api/'],
},
{
userAgent: 'Googlebot',
allow: ['/', '/article/*/'],
disallow: ['/admin', '/api/'],
},
{
// AdSense injection backend (unofficial; harmless)
userAgent: 'Google-AdSense',
allow: '/',
},
],
sitemap: `${base}/sitemap.xml`,
};
}
+30
View File
@@ -0,0 +1,30 @@
import type { MetadataRoute } from 'next';
import { getPublishedArticles } from '@/lib/queries';
import { env } from '@/lib/env';
export const dynamic = 'force-dynamic';
const STATIC_PATHS = ['', '/about', '/contact', '/privacy-policy', '/terms-of-service'];
export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
const now = new Date();
const base = env.siteUrl.replace(/\/$/, '');
const staticEntries: MetadataRoute.Sitemap = STATIC_PATHS.map((p) => ({
url: `${base}${p}`,
lastModified: now,
changeFrequency: 'daily',
priority: p === '' ? 1.0 : 0.4,
}));
// All synthesized briefs, newest first, up to 5k entries (sitemap cap guidance).
const articles = await getPublishedArticles('all', 5000);
const articleEntries: MetadataRoute.Sitemap = articles.map((a) => ({
url: `${base}/article/${a.slug}`,
lastModified: a.publishedAt ? new Date(a.publishedAt) : now,
changeFrequency: 'hourly',
priority: 0.8,
}));
return [...staticEntries, ...articleEntries];
}
+55
View File
@@ -0,0 +1,55 @@
import { getArticlesByTag, getTagList } from '@/lib/queries';
import ArticleCard from '@/components/ArticleCard';
import type { Metadata } from 'next';
import Link from 'next/link';
import { notFound } from 'next/navigation';
export const dynamic = 'force-dynamic';
export async function generateMetadata({
params,
}: {
params: { tag: string };
}): Promise<Metadata> {
return { title: `#${params.tag}`, alternates: { canonical: `/tag/${params.tag}` } };
}
export default async function TagPage({ params }: { params: { tag: string } }) {
const tag = decodeURIComponent(params.tag);
const [articles, allTags] = await Promise.all([
getArticlesByTag(tag, 30),
getTagList(16),
]);
if (articles.length === 0 && !allTags.includes(tag.trim().toLowerCase())) {
// still render if tag existed in DB but had zero recent articles
}
return (
<div className="mx-auto max-w-6xl px-4 py-6 sm:px-6">
<h1 className="mb-6 font-display text-3xl font-bold text-ink-900">Topic: #{tag}</h1>
{articles.length === 0 ? (
<p className="rounded-xl border border-dashed border-ink-300 bg-white p-10 text-center text-sm text-ink-500">
No briefings under this tag yet.
</p>
) : (
<div className="grid gap-5 sm:grid-cols-2 lg:grid-cols-3">
{articles.map((card) => (
<ArticleCard key={card.id} card={card} />
))}
</div>
)}
{allTags.length > 0 && (
<div className="mt-10 flex flex-wrap gap-2">
{allTags.map((t) => (
<Link
key={t}
href={`/tag/${t}`}
className="rounded-full bg-ink-100 px-3 py-1 text-sm text-ink-700 hover:bg-maple-100 hover:text-maple-800"
>
#{t}
</Link>
))}
</div>
)}
</div>
);
}
+107
View File
@@ -0,0 +1,107 @@
import type { Metadata } from 'next';
import Link from 'next/link';
export const metadata: Metadata = {
title: 'Terms of Service',
description: 'Terms governing use of MapleBrief and its news briefs.',
alternates: { canonical: '/terms-of-service' },
};
function Section({ title, children }: { title: string; children: React.ReactNode }) {
return (
<section className="mt-8">
<h2 className="font-display text-xl font-bold text-ink-900">{title}</h2>
<div className="mt-3 space-y-3 text-[15px] leading-relaxed text-ink-700">{children}</div>
</section>
);
}
export default function TermsPage() {
return (
<div className="mx-auto max-w-3xl px-4 py-10 sm:px-6">
<h1 className="font-display text-4xl font-bold text-ink-950">Terms of Service</h1>
<p className="mt-3 text-sm text-ink-500">
Last updated: {new Date().toISOString().slice(0, 10)}
</p>
<Section title="1. Acceptance of terms">
<p>
By accessing or using MapleBrief (&ldquo;the Site&rdquo;), you agree to be
bound by these Terms of Service. If you do not agree, discontinue use.
</p>
</Section>
<Section title="2. Nature of the service">
<p>
MapleBrief provides automated summaries of publicly
available news from third-party Canadian publishers. Briefs are
provided for information convenience and are labeled with their
source outlet. We do not guarantee that a brief
is exhaustive, interpreted without error, or current beyond its
publication time.
</p>
<p>
Nothing on the Site constitutes professional, legal, financial,
medical, or investment advice.
</p>
</Section>
<Section title="3. Intellectual property and attribution">
<p>
The original articles remain the exclusive property of their
respective publishers. MapleBrief&apos;s briefs are independently
authored; links to original sources are provided with{' '}
<code className="rounded bg-ink-100 px-1 text-sm">rel=&quot;nofollow&quot;</code>{' '}
and are intended as outbound referrals, not endorsements.
</p>
<p>
You may share brief links for personal, non-commercial reference. You
may not scrape, mirror, resell, or systematically redistribute the
Site&apos;s content.
</p>
</Section>
<Section title="4. Acceptable use">
<p>
You agree not to: (a) interfere with or disrupt the Site or its
infrastructure; (b) attempt to bypass rate limits or access controls;
(c) use the Site for unlawful purposes; (d) attempt to extract the
underlying raw feed content in bulk.
</p>
</Section>
<Section title="5. Termination">
<p>
We may modify or terminate any part of the Site at any time, with or
without notice.
</p>
</Section>
<Section title="6. Limitation of liability">
<p>
To the maximum extent permitted by law, MapleBrief shall not be
liable for indirect, incidental, special, consequential, or punitive
damages arising from your use of (or inability to use) the Site.
</p>
</Section>
<Section title="7. Governing law">
<p>
These terms are governed by the laws of Ontario, Canada, without
regard to conflict-of-law principles, and the exclusive remedy venue
is the courts of Ontario.
</p>
</Section>
<Section title="8. Contact">
<p>
For questions about these terms, see our{' '}
<Link href="/contact" className="font-medium text-maple-700 underline">
contact page
</Link>
.
</p>
</Section>
</div>
);
}
+47
View File
@@ -0,0 +1,47 @@
import Link from 'next/link';
import type { ArticleCard as Card } from '@/lib/queries';
import { resolveCoverImage } from '@/lib/cover';
import { CoverImage } from '@/components/CoverImage';
import { formatRelativeTime } from '@/lib/format';
export default function ArticleCard({ card }: { card: Card }) {
const href = `/article/${card.slug}`;
const title = card.headline ?? card.title;
const image = resolveCoverImage(card.image, card.category);
return (
<article className="group flex flex-col overflow-hidden rounded-xl border border-ink-200 bg-white shadow-sm transition hover:-translate-y-0.5 hover:shadow-md">
<Link
href={href}
className="relative block aspect-[16/9] overflow-hidden bg-ink-100"
>
<CoverImage
src={image}
alt=""
className="h-full w-full object-cover transition duration-300 group-hover:scale-[1.03]"
/>
</Link>
<div className="flex flex-1 flex-col p-4">
<h3 className="font-display text-lg font-bold leading-snug text-ink-900 group-hover:text-maple-700">
<Link href={href}>{title}</Link>
</h3>
<p className="mt-2 flex-1 text-sm leading-relaxed text-ink-600">
{card.excerpt}
</p>
<div className="mt-3 flex items-center justify-between text-xs text-ink-500">
<span>{card.publishedAt ? formatRelativeTime(card.publishedAt) : 'recent'}</span>
<span className="flex gap-1">
{card.tags.slice(0, 2).map((t) => (
<Link
key={t}
href={`/tag/${t}`}
className="rounded bg-ink-100 px-1.5 py-0.5 hover:bg-maple-100 hover:text-maple-800"
>
{t}
</Link>
))}
</span>
</div>
</div>
</article>
);
}
+90
View File
@@ -0,0 +1,90 @@
'use client';
import { useState } from 'react';
export default function ContactForm() {
const [state, setState] = useState<'idle' | 'sending' | 'sent' | 'error'>('idle');
async function handleSubmit(e: React.FormEvent<HTMLFormElement>) {
e.preventDefault();
const data = new FormData(e.currentTarget);
setState('sending');
try {
const res = await fetch('/api/contact', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
email: data.get('email'),
topic: data.get('topic'),
url: data.get('url'),
message: data.get('message'),
}),
});
setState(res.ok ? 'sent' : 'error');
} catch {
setState('error');
}
}
if (state === 'sent') {
return (
<div className="mt-8 rounded-xl border border-emerald-300 bg-emerald-50 p-6 text-[15px] text-emerald-900">
Thanks your message has been queued. For time-sensitive legal
notices, also email the address listed on the site.
</div>
);
}
return (
<form onSubmit={handleSubmit} className="mt-8 space-y-4">
<label className="block text-sm">
<span className="font-medium text-ink-800">Your email</span>
<input
name="email"
type="email"
required
className="mt-1 w-full rounded-lg border border-ink-300 px-3 py-2 text-sm focus:border-maple-600 focus:outline-none"
placeholder="you@example.com"
/>
</label>
<label className="block text-sm">
<span className="font-medium text-ink-800">Topic</span>
<select name="topic" className="mt-1 w-full rounded-lg border border-ink-300 px-3 py-2 text-sm" defaultValue="correction">
<option value="correction">Correction</option>
<option value="dmca">Copyright / DMCA notice</option>
<option value="advertising">AdSense / advertising</option>
<option value="other">Other</option>
</select>
</label>
<label className="block text-sm">
<span className="font-medium text-ink-800">Affected brief URL (if any)</span>
<input
name="url"
type="url"
className="mt-1 w-full rounded-lg border border-ink-300 px-3 py-2 text-sm focus:border-maple-600 focus:outline-none"
placeholder="https://…"
/>
</label>
<label className="block text-sm">
<span className="font-medium text-ink-800">Message</span>
<textarea
name="message"
required
rows={6}
className="mt-1 w-full rounded-lg border border-ink-300 px-3 py-2 text-sm focus:border-maple-600 focus:outline-none"
placeholder="…"
/>
</label>
<button
type="submit"
disabled={state === 'sending'}
className="rounded-lg bg-maple-600 px-5 py-2.5 text-sm font-semibold text-white hover:bg-maple-700 disabled:opacity-50"
>
{state === 'sending' ? 'Sending…' : 'Send message'}
</button>
{state === 'error' && (
<p className="text-sm text-red-700">Could not send please try again.</p>
)}
</form>
);
}
+48
View File
@@ -0,0 +1,48 @@
/**
* CoverImage — the single <img> used for every article photo on the site.
*
* The `src` here is the result of cover.resolveCoverImage: either a local
* category cover (/covers/<slug>.svg) or /images?url=... (the server-side
* proxy). If the proxy misses (upstream gone, blocked, or still hot-linked
* 403 even with browser headers), the browser fires onerror — swap to the
* deterministic category cover so cards and heroes never render a broken
* image box.
*/
'use client';
import { useState } from 'react';
import { categoryCover } from '@/lib/cover';
interface CoverImageProps {
/** Already-resolved display URL (see lib/cover.ts). */
src: string;
alt?: string;
category?: string | null;
className?: string;
loading?: 'lazy' | 'eager';
}
export function CoverImage({
src,
alt = '',
category,
className,
loading = 'lazy',
}: CoverImageProps) {
const [failed, setFailed] = useState(false);
const finalSrc = failed ? categoryCover(category) : src;
// eslint-disable-next-line @next/next/no-img-element
return (
<img
src={finalSrc}
alt={alt}
loading={loading}
className={className}
onError={() => {
if (!failed) setFailed(true);
}}
/>
);
}
+39
View File
@@ -0,0 +1,39 @@
import Link from 'next/link';
import type { ArticleCard } from '@/lib/queries';
import { resolveCoverImage } from '@/lib/cover';
import { CoverImage } from '@/components/CoverImage';
import { formatRelativeTime } from '@/lib/format';
export default function RelatedArticles({ items }: { items: ArticleCard[] }) {
if (!items.length) return null;
return (
<section className="mt-12 border-t border-ink-200 pt-8" aria-label="Related briefings">
<h2 className="font-display text-xl font-bold text-ink-900">Related briefings</h2>
<ul className="mt-4 grid gap-3 sm:grid-cols-2">
{items.map((a) => (
<li key={a.id} className="flex gap-3">
<div className="h-16 w-24 flex-shrink-0 overflow-hidden rounded-md bg-ink-100">
<CoverImage
src={resolveCoverImage(a.image, a.category)}
category={a.category}
alt=""
className="h-full w-full object-cover"
/>
</div>
<div className="min-w-0">
<Link
href={`/article/${a.slug}`}
className="block font-display text-[15px] font-semibold leading-snug text-ink-800 hover:text-maple-700"
>
{a.headline ?? a.title}
</Link>
<span className="mt-1 block text-xs text-ink-500">
{a.publishedAt ? formatRelativeTime(a.publishedAt) : ''}
</span>
</div>
</li>
))}
</ul>
</section>
);
}
+121
View File
@@ -0,0 +1,121 @@
'use client';
import { ReactNode, useCallback, useEffect, useState } from 'react';
import { useRouter } from 'next/navigation';
type Status = 'unknown' | 'authed' | 'denied';
/**
* Client-side admin gate.
*
* 1. If `x-admin-key` is stored in sessionStorage → validate via
* /api/admin/verify.
* 2. Otherwise show a key entry form (basic-auth fallback for users who
* prefer the `ADMIN_USER:ADMIN_KEY` header).
*
* The key is never written to localStorage and is cleared on a hard tab
* close. For hardening, front the admin area with real HTTP Basic auth
* at the reverse-proxy layer too.
*/
export default function AdminGate({ children }: { children: ReactNode }) {
const [status, setStatus] = useState<Status>('unknown');
const [busy, setBusy] = useState(false);
const [error, setError] = useState<string | null>(null);
const router = useRouter();
const [keyInput, setKeyInput] = useState('');
const validate = useCallback(
async (key: string) => {
setBusy(true);
setError(null);
try {
const res = await fetch('/api/admin/verify', {
headers: { 'x-admin-key': key },
});
if (res.ok) {
sessionStorage.setItem('mb_admin_key', key);
setStatus('authed');
} else {
setStatus('denied');
setError('Invalid admin key.');
}
} catch {
setError('Network error');
} finally {
setBusy(false);
}
},
[],
);
useEffect(() => {
const stored = sessionStorage.getItem('mb_admin_key');
if (stored) validate(stored);
else setStatus('denied');
}, [validate]);
async function submit(e: React.FormEvent) {
e.preventDefault();
if (keyInput.trim()) await validate(keyInput.trim());
}
function logout() {
sessionStorage.removeItem('mb_admin_key');
setStatus('denied');
setKeyInput('');
router.refresh();
}
if (status === 'authed') {
return (
<div className="mx-auto max-w-4xl px-4 py-8 sm:px-6">
<div className="mb-6 flex items-center justify-between">
<h1 className="font-display text-2xl font-bold text-ink-900">
Admin · Settings
</h1>
<button
onClick={logout}
className="rounded-md border border-ink-300 px-3 py-1.5 text-sm text-ink-600 hover:bg-ink-100"
>
Sign out
</button>
</div>
{children}
</div>
);
}
return (
<div className="mx-auto max-w-md px-4 py-24">
<div className="rounded-2xl border border-ink-200 bg-white p-8 shadow-sm">
<h1 className="font-display text-2xl font-bold text-ink-900">Admin</h1>
<p className="mt-1 text-sm text-ink-500">
Enter the admin key from <code>ADMIN_API_KEY</code> (or your{' '}
<code>USER:ADMIN_API_KEY</code> basic-auth string) to continue.
</p>
<form onSubmit={submit} className="mt-6 space-y-4">
<input
type="password"
value={keyInput}
onChange={(e) => setKeyInput(e.target.value)}
placeholder="Admin key"
autoFocus
className="w-full rounded-lg border border-ink-300 px-3 py-2.5 text-sm focus:border-maple-600 focus:outline-none"
/>
{error && <p className="text-sm text-red-700">{error}</p>}
<button
type="submit"
disabled={busy}
className="w-full rounded-lg bg-maple-600 px-4 py-2.5 text-sm font-semibold text-white hover:bg-maple-700 disabled:opacity-50"
>
{busy ? 'Verifying…' : 'Unlock'}
</button>
</form>
<p className="mt-4 text-xs text-ink-400">
Tip: keep the key in a password manager; it is only held in your
browser session while you have this tab open.
</p>
</div>
</div>
);
}
+283
View File
@@ -0,0 +1,283 @@
'use client';
import { useCallback, useEffect, useState } from 'react';
interface Settings {
provider: 'openai' | 'anthropic' | 'ollama';
apiKeys: { openai: string; anthropic: string };
ollamaBase: string;
models: { openai: string; anthropic: string; ollama: string };
synthesisPrompt: string;
}
interface KeySource {
openai: boolean;
anthropic: boolean;
ollama: boolean;
}
export default function SettingsForm() {
const [settings, setSettings] = useState<Settings | null>(null);
const [keyFromDb, setKeyFromDb] = useState<KeySource | null>(null);
const [loadError, setLoadError] = useState<string | null>(null);
const [saving, setSaving] = useState(false);
const [saveMsg, setSaveMsg] = useState<{ ok: boolean; text: string } | null>(null);
const [testMsg, setTestMsg] = useState<{ ok: boolean; text: string } | null>(null);
const [testing, setTesting] = useState(false);
const adminHeaders = useCallback(
() => ({
'x-admin-key': sessionStorage.getItem('mb_admin_key') ?? '',
'Content-Type': 'application/json',
}),
[],
);
useEffect(() => {
(async () => {
try {
const res = await fetch('/api/settings', {
headers: { 'x-admin-key': sessionStorage.getItem('mb_admin_key') ?? '' },
});
const data = await res.json();
if (!res.ok) throw new Error(data.error ?? 'load failed');
setSettings(data.settings as Settings);
setKeyFromDb(data.keySources as KeySource);
} catch (e) {
setLoadError(e instanceof Error ? e.message : 'load failed');
}
})();
}, []);
if (loadError) {
return <p className="text-sm text-red-700">Could not load settings: {loadError}</p>;
}
if (!settings || !keyFromDb) {
return <p className="text-sm text-ink-500">Loading</p>;
}
const prompt: Settings = {
provider: settings.provider,
apiKeys: { ...settings.apiKeys },
ollamaBase: settings.ollamaBase ?? 'http://localhost:11434',
models: { ...settings.models },
synthesisPrompt: settings.synthesisPrompt,
};
function patch<K extends keyof Settings>(key: K, value: Settings[K]) {
setSettings((s) => (s ? { ...s, [key]: value } : s));
setSaveMsg(null);
}
async function save(e: React.FormEvent) {
e.preventDefault();
setSaving(true);
setSaveMsg(null);
setTestMsg(null);
try {
const res = await fetch('/api/settings', {
method: 'PUT',
headers: adminHeaders(),
body: JSON.stringify(prompt),
});
const data = await res.json();
setSaveMsg({ ok: res.ok, text: res.ok ? 'Settings saved.' : (data.error ?? 'save failed') });
if (res.ok) {
setKeyFromDb(data.keySources as KeySource);
}
} catch {
setSaveMsg({ ok: false, text: 'Network error' });
} finally {
setSaving(false);
}
}
async function test() {
setTesting(true);
setTestMsg(null);
try {
const res = await fetch('/api/settings/test', {
method: 'POST',
headers: adminHeaders(),
});
const data = await res.json();
setTestMsg(
data.ok
? { ok: true, text: data.message ?? 'Connection OK.' }
: { ok: false, text: data.error ?? 'Test failed' },
);
} catch {
setTestMsg({ ok: false, text: 'Network error' });
} finally {
setTesting(false);
}
}
return (
<form onSubmit={save} className="space-y-8">
{/* Provider selection */}
<section className="rounded-xl border border-ink-200 bg-white p-6">
<h2 className="font-semibold text-ink-900">Active provider</h2>
<div className="mt-4 grid grid-cols-3 gap-3">
{(['openai', 'anthropic', 'ollama'] as const).map((p) => (
<label
key={p}
className={`cursor-pointer rounded-lg border p-4 text-center transition ${
prompt.provider === p
? 'border-maple-600 bg-maple-50 ring-1 ring-maple-600'
: 'border-ink-200 hover:border-ink-300'
}`}
>
<input
type="radio"
name="provider"
value={p}
checked={prompt.provider === p}
onChange={() => patch('provider', p)}
className="sr-only"
/>
<span className="block text-sm font-semibold capitalize text-ink-900">{p}</span>
<span className="mt-1 block text-xs text-ink-500">
{p === 'openai' && 'GPT-4o / 4o-mini · cloud'}
{p === 'anthropic' && 'Claude 3.5 Sonnet / Haiku · cloud'}
{p === 'ollama' && 'Local · zero cost'}
</span>
</label>
))}
</div>
{/* Model */}
<label className="mt-5 block text-sm">
<span className="font-medium text-ink-800">Model</span>
<input
type="text"
value={prompt.models[prompt.provider]}
onChange={(e) =>
patch('models', { ...prompt.models, [prompt.provider]: e.target.value })
}
list="model-suggestions"
className="mt-1 w-full rounded-lg border border-ink-300 px-3 py-2 font-mono text-sm focus:border-maple-600 focus:outline-none"
/>
<datalist id="model-suggestions">
{prompt.provider === 'openai' && (
<>
<option value="gpt-4o-mini" />
<option value="gpt-4o" />
</>
)}
{prompt.provider === 'anthropic' && (
<>
<option value="claude-3-5-sonnet-latest" />
<option value="claude-3-5-haiku-latest" />
</>
)}
{prompt.provider === 'ollama' && (
<>
<option value="llama3.1:8b" />
<option value="mistral:latest" />
<option value="qwen2.5:14b" />
</>
)}
</datalist>
</label>
{/* API key — only cloud providers */}
{prompt.provider !== 'ollama' ? (
<label className="mt-5 block text-sm">
<span className="font-medium text-ink-800">
{prompt.provider === 'openai' ? 'OpenAI' : 'Anthropic'} API key
<span className="ml-2 rounded bg-ink-100 px-1.5 py-0.5 text-[11px] font-normal text-ink-500">
{keyFromDb[prompt.provider] ? 'stored in database' : 'coming from env vars'}
</span>
</span>
<input
type="password"
value={prompt.apiKeys[prompt.provider]}
onChange={(e) =>
patch('apiKeys', { ...prompt.apiKeys, [prompt.provider]: e.target.value })
}
placeholder={keyFromDb[prompt.provider] ? 'Enter to update (current is saved)' : 'sk-…'}
className="mt-1 w-full rounded-lg border border-ink-300 px-3 py-2 font-mono text-sm focus:border-maple-600 focus:outline-none"
/>
{!keyFromDb[prompt.provider] && (
<span className="mt-1 block text-xs text-ink-400">
Blank means &ldquo;use the env var&rdquo; (OPENAI_API_KEY /
ANTHROPIC_API_KEY). Saving a value stores it encrypted-at-rest in
SQLite (phone-in-BE BYO; see README on secret handling).
</span>
)}
</label>
) : (
<label className="mt-5 block text-sm">
<span className="font-medium text-ink-800">Ollama base URL</span>
<input
type="url"
value={prompt.ollamaBase}
onChange={(e) => patch('ollamaBase', e.target.value)}
placeholder="http://localhost:11434"
className="mt-1 w-full rounded-lg border border-ink-300 px-3 py-2 font-mono text-sm focus:border-maple-600 focus:outline-none"
/>
<span className="mt-1 block text-xs text-ink-400">
For &ldquo;localhost&rdquo; you must also set OLLAMA_CORS=true on the
Ollama host (see README).
</span>
</label>
)}
</section>
{/* Synthesis prompt */}
<section className="rounded-xl border border-ink-200 bg-white p-6">
<h2 className="font-semibold text-ink-900">Synthesis system prompt</h2>
<p className="mt-1 text-xs text-ink-500">
This is the standing instruction the LLM receives on every
synthesis. Edit to change tone, length, or structure keep the
&ldquo;strict JSON only&rdquo; contract so parsing stays reliable.
</p>
<textarea
rows={12}
value={prompt.synthesisPrompt}
onChange={(e) => patch('synthesisPrompt', e.target.value)}
className="mt-3 w-full rounded-lg border border-ink-300 px-3 py-2 font-mono text-[13px] leading-relaxed focus:border-maple-600 focus:outline-none"
/>
</section>
{/* Actions */}
<section className="flex flex-wrap items-center gap-3">
<button
type="submit"
disabled={saving}
className="rounded-lg bg-maple-600 px-5 py-2.5 text-sm font-semibold text-white hover:bg-maple-700 disabled:opacity-50"
>
{saving ? 'Saving…' : 'Save settings'}
</button>
<button
type="button"
onClick={test}
disabled={testing || saving}
className="rounded-lg border border-ink-300 px-5 py-2.5 text-sm font-semibold text-ink-700 hover:bg-ink-100 disabled:opacity-50"
>
{testing ? 'Testing…' : 'Test connection'}
</button>
{saveMsg && (
<span
className={`text-sm ${saveMsg.ok ? 'text-emerald-700' : 'text-red-700'}`}
>
{saveMsg.text}
</span>
)}
</section>
{testMsg && (
<p
className={`rounded-lg border p-4 text-sm ${
testMsg.ok
? 'border-emerald-300 bg-emerald-50 text-emerald-900'
: 'border-red-300 bg-red-50 text-red-900'
}`}
>
{testMsg.text}
</p>
)}
</form>
);
}
+127
View File
@@ -0,0 +1,127 @@
'use client';
import { useEffect, useRef, useState } from 'react';
declare global {
interface Window {
adsbygoogle?: unknown[];
_adsbygoogle?: unknown[];
}
}
interface AdUnitProps {
/** AdSense ad-unit "slot" id (numeric, from your AdSense UI). */
slotId: string;
/** Client/publisher id, e.g. ca-pub-XXXX. Defaults to the env value. */
clientId?: string;
/** Sizing format: 'auto' (responsive) or explicit data-ad-format. */
format?: 'auto' | 'horizontal' | 'vertical';
/** Slot key used for the adsense client default. */
label?: string;
/** Extra <ins> attributes, e.g. data-ad-slot overrides. */
className?: string;
/** Render even when no client id is configured (shows watermark). */
alwaysRender?: boolean;
}
let scriptPromise: Promise<void> | null = null;
/** Inject exactly one adsbygoogle.js per page (guarded singleton). */
function ensureAdsenseScript(clientId: string): Promise<void> {
if (scriptPromise) return scriptPromise;
scriptPromise = new Promise<void>((resolve, reject) => {
const script = document.createElement('script');
script.async = true;
script.crossOrigin = 'anonymous';
script.src = `https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=${encodeURIComponent(
clientId,
)}`;
script.onload = () => resolve();
script.onerror = () =>
reject(new Error('AdSense script failed to load'));
document.head.appendChild(script);
});
return scriptPromise;
}
/**
* Central safe AdUnit component.
*
* - Injects adsbygoogle.js once per page.
* - Defers request to `(window.adsbygoogle).push({})` on mount.
* - If no client id is configured (local dev) it renders a labelled
* shimmer placeholder so the layout is testable pre-approval.
* - `format: 'auto'` produces responsive drops; 'horizontal' is the
* 728x90 leaderboard shape; 'vertical' is used for sidebar 300x250/600.
*/
export default function AdUnit({
slotId,
clientId,
format = 'auto',
label = 'Advertisement',
className = '',
}: AdUnitProps) {
const [ready, setReady] = useState(false);
const pushed = useRef(false);
const envClientId =
typeof process !== 'undefined'
? (process.env.NEXT_PUBLIC_ADSENSE_CLIENT_ID ?? '')
: '';
const client = clientId ?? envClientId;
const configured = Boolean(client && slotId);
useEffect(() => {
if (!configured || pushed.current) return;
let cancelled = false;
ensureAdsenseScript(client)
.then(() => {
if (cancelled || pushed.current) return;
pushed.current = true;
(window.adsbygoogle = window.adsbygoogle || []).push({});
setReady(true);
})
.catch(() => {
/* keep placeholder */
});
return () => {
cancelled = true;
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [configured, client, slotId]);
if (!configured) {
return (
// Placeholder so AdSense-free local previews keep the intended layout.
<div
aria-hidden
className={`ad-shimmer relative w-full ${className}`}
style={{
minHeight: format === 'vertical' ? 250 : 90,
minWidth: format === 'vertical' ? 180 : undefined,
}}
>
<span className="absolute inset-0 flex items-center justify-center text-[11px] uppercase tracking-widest text-ink-400 select-none">
{label}
</span>
</div>
);
}
return (
<div className={`w-full ${className}`}>
<span className="sr-only">{label}</span>
<ins
className="adsbygoogle block w-full"
style={{
display: 'block',
minHeight: format === 'auto' ? undefined : format === 'vertical' ? 250 : 90,
}}
data-ad-client={client}
data-ad-slot={slotId}
data-ad-format={format}
data-full-width-responsive="true"
/>
{ready ? null : <div className="ad-shimmer mt-0 h-2 w-1/4 rounded" aria-hidden />}
</div>
);
}
+70
View File
@@ -0,0 +1,70 @@
import type { ReactNode } from 'react';
/** Ad unit placements, kept in one component file for the layout. */
import AdUnit from '@/components/ads/AdUnit';
import { env } from '@/lib/env';
/** Header leaderboard (728x90 / responsive) placed below the main nav. */
export function HeaderAd() {
return (
<div className="border-b border-ink-200 bg-white">
<div className="mx-auto max-w-6xl px-4 py-3 sm:px-6">
<AdUnit
slotId={env.adsenseSlots.header}
format="horizontal"
label="Advertising"
className="mx-auto max-w-[728px]"
/>
</div>
</div>
);
}
/**
* In-Feed ad, inserted between every Nth news card.
* `index` is the 1-based card position about to follow the ad.
*/
export function InFeedAd({ index }: { index: number }) {
return (
<div className="col-span-full my-2 rounded-xl border border-dashed border-ink-300 bg-white p-3 sm:col-span-2 lg:col-span-3">
<p className="mb-2 text-[11px] uppercase tracking-widest text-ink-400">
Suggested
</p>
<AdUnit
slotId={env.adsenseSlots.infeed}
format="auto"
label={`In-feed ad · after card ${index}`}
/>
</div>
);
}
/** In-article ad between rewritten paragraphs 1 and 2. */
export function InArticleAd() {
return (
<div className="my-8 rounded-xl border border-ink-200 bg-white p-4">
<AdUnit
slotId={env.adsenseSlots.inarticle}
format="auto"
label="In-article advertisement"
/>
</div>
);
}
/** Sticky sidebar rectangle (300x250 / half-page 300x600). */
export function SidebarAd({ tall = false }: { tall?: boolean }) {
return (
<div className="sticky top-24">
<AdUnit
slotId={env.adsenseSlots.sidebar}
format="vertical"
label={tall ? 'Half-page ad' : 'Medium rectangle ad'}
/>
</div>
);
}
export function withAdPlaceholder(children: ReactNode) {
return children;
}
+44
View File
@@ -0,0 +1,44 @@
import Link from 'next/link';
import { env } from '@/lib/env';
export default function Footer() {
const year = new Date().getFullYear();
return (
<footer className="mt-16 border-t border-ink-800 bg-ink-950 text-ink-300">
<div className="mx-auto grid max-w-6xl gap-8 px-4 py-10 sm:grid-cols-3 sm:px-6">
<div>
<p className="font-display text-lg font-bold text-white">{env.siteName}</p>
<p className="mt-2 text-sm leading-relaxed text-ink-400">
An independent digest of Canadian headlines.
</p>
</div>
<nav aria-label="Footer" className="text-sm">
<p className="mb-3 text-xs font-semibold uppercase tracking-wider text-ink-500">
Sections
</p>
<ul className="space-y-1.5">
<li><Link className="hover:text-white" href="/top-stories">Top Stories</Link></li>
<li><Link className="hover:text-white" href="/canada">Canada</Link></li>
<li><Link className="hover:text-white" href="/national">National</Link></li>
</ul>
</nav>
<nav aria-label="Legal" className="text-sm">
<p className="mb-3 text-xs font-semibold uppercase tracking-wider text-ink-500">
Policy
</p>
<ul className="space-y-1.5">
<li><Link className="hover:text-white" href="/privacy-policy">Privacy Policy</Link></li>
<li><Link className="hover:text-white" href="/terms-of-service">Terms of Service</Link></li>
<li><Link className="hover:text-white" href="/about">About</Link></li>
<li><Link className="hover:text-white" href="/contact">Contact</Link></li>
</ul>
</nav>
</div>
<div className="border-t border-ink-800/70">
<div className="mx-auto max-w-6xl px-4 py-4 text-xs text-ink-500 sm:px-6">
© {year} {env.siteName}.
</div>
</div>
</footer>
);
}
+53
View File
@@ -0,0 +1,53 @@
import { env } from '@/lib/env';
import { SECTIONS } from '@/data/feeds';
import Link from 'next/link';
import NavLinks from '@/components/layout/NavLinks';
export default function Header() {
return (
<header className="sticky top-0 z-50 border-b border-ink-800 bg-ink-950/95 backdrop-blur">
<div className="mx-auto flex h-16 max-w-6xl items-center justify-between gap-6 px-4 sm:px-6">
<Link href="/" className="flex items-center gap-2.5" aria-label={env.siteName}>
{/* eslint-disable-next-line @next/next/no-img-element -- local static asset */}
<img
src="/logo-mark.svg"
alt=""
width={36}
height={36}
className="h-9 w-9 rounded-md"
/>
<span className="flex flex-col leading-none">
<span className="font-display text-xl font-bold tracking-tight text-white">
{env.siteName}
</span>
<span className="text-[10px] uppercase tracking-[0.2em] text-ink-300">
Your Canadian News
</span>
</span>
</Link>
<NavLinks />
</div>
</header>
);
}
export function SectionPills() {
return (
<nav
className="border-b border-ink-200 bg-ink-50/95"
aria-label="Sections"
>
<div className="mx-auto flex max-w-6xl gap-1 overflow-x-auto px-4 sm:px-6">
{SECTIONS.map((s) => (
<Link
key={s.slug}
href={s.slug === 'all' ? '/' : `/${s.slug}`}
className="whitespace-nowrap border-b-2 border-transparent px-3 py-2.5 text-sm font-medium text-ink-600 transition hover:border-maple-300 hover:text-ink-900"
>
{s.label}
</Link>
))}
</div>
</nav>
);
}
+43
View File
@@ -0,0 +1,43 @@
'use client';
import Link from 'next/link';
import { usePathname } from 'next/navigation';
import { SECTIONS } from '@/data/feeds';
export default function NavLinks() {
const pathname = usePathname();
return (
<nav className="hidden items-center gap-1 md:flex" aria-label="Primary">
{SECTIONS.map((s) => {
const href = s.slug === 'all' ? '/' : `/${s.slug}`;
const active = s.slug === 'all' ? pathname === '/' : pathname.startsWith(`/${s.slug}`);
return (
<Link
key={s.slug}
href={href}
aria-current={active ? 'page' : undefined}
className={`rounded-md px-3 py-1.5 text-sm font-medium transition ${
active
? 'bg-maple-600/15 text-maple-300'
: 'text-ink-300 hover:bg-white/5 hover:text-white'
}`}
>
{s.label}
</Link>
);
})}
<Link
href="/about"
className="ml-2 hidden rounded-md px-3 py-1.5 text-sm text-ink-300 hover:text-white sm:block"
>
About
</Link>
<Link
href="/privacy-policy"
className="hidden rounded-md px-3 py-1.5 text-sm text-ink-300 hover:text-white lg:block"
>
Privacy
</Link>
</nav>
);
}
+76
View File
@@ -0,0 +1,76 @@
/**
* Starter Canadian news feeds. These are seeded into the `Feed` table on
* first boot (and by `prisma/seed.js`); officials can add/edit feeds via the
* database or by extending this list — the pipeline only reads from the DB.
*/
export interface FeedSeed {
name: string;
url: string;
slug: string;
category: string;
siteName: string;
}
export const STARTER_FEEDS: FeedSeed[] = [
{
// Replacement for CBC News — Top Stories: cbc-stats returned 404 for
// every public RSS path (re-verified from egress 2026-08-18). Google
// News Canada hub gives a dense national top-stories stream (100 items).
name: 'Google News — Canada',
url: 'https://news.google.com/rss/search?q=canada%20when:48h&hl=en-CA&gl=CA&ceid=CA:en',
slug: 'gn-canada',
category: 'top-stories',
siteName: 'Google News',
},
{
// Replacement for CBC News — Canada (cbc-stats 404, re-verified
// 2026-08-18). National Post Canada section — distinct from the
// National Post News feed below; cross-feed GUIDs dedupe at ingest.
name: 'National Post — Canada',
url: 'https://nationalpost.com/category/canada/feed',
slug: 'national-post-canada',
category: 'canada',
siteName: 'National Post',
},
{
// Live Arc outbound feed (verified 2026-08). The legacy
// ctvnews-ca-top-stories-public-rss-1.822009 URL currently 404s.
name: 'CTV News — National',
url: 'https://www.ctvnews.ca/arc/outboundfeeds/rss/',
slug: 'ctv-national',
category: 'national',
siteName: 'CTV News',
},
{
name: 'Global News — Canada',
url: 'https://globalnews.ca/canada/feed/',
slug: 'global-canada',
category: 'canada',
siteName: 'Global News',
},
{
name: 'The Globe and Mail — National',
url: 'https://www.theglobeandmail.com/arc/outboundfeeds/rss/category/canada/',
slug: 'globe-national',
category: 'national',
siteName: 'The Globe and Mail',
},
{
name: 'National Post — News',
url: 'https://nationalpost.com/category/news/feed',
slug: 'national-post-news',
category: 'top-stories',
siteName: 'National Post',
},
];
/** Sections shown in site navigation (stable, hand-curated). */
export const SECTIONS = [
{ slug: 'top-stories', label: 'Top Stories' },
{ slug: 'canada', label: 'Canada' },
{ slug: 'national', label: 'National' },
{ slug: 'all', label: 'All' },
] as const;
export type SectionSlug = (typeof SECTIONS)[number]['slug'];
+18
View File
@@ -0,0 +1,18 @@
/**
* Next.js instrumentation hook (runs once per server process, on boot).
*
* Arms the node-cron ingestion/synthesis worker under the Node.js runtime so
* the background pipeline starts automatically on `next start` / `next dev`
* — no external health-check ping required. (`/api/worker/ping` still works
* as a keep-alive / warm-start trigger for serverless-ish deployments.)
*
* node-cron is a native Node module, so it must only be loaded when we're on
* the Node.js runtime, never Edge. The dynamic import keeps it out of any
* edge bundle.
*/
export async function register() {
if (process.env.NEXT_RUNTIME === 'nodejs') {
const { initWorker } = await import('@/lib/worker/cron');
initWorker();
}
}
+65
View File
@@ -0,0 +1,65 @@
import { NextRequest } from 'next/server';
import { env } from '@/lib/env';
import { timingSafeEqual } from 'crypto';
/**
* Admin gate for the settings UI + privileged API routes.
*
* Accepted, in order:
* 1. `Authorization: Bearer <ADMIN_API_KEY>`
* 2. `x-admin-key: <ADMIN_API_KEY>` header (convenient for curl)
* 3. `?key=<ADMIN_API_KEY>` query param (admin UI local auth only)
* 4. HTTP Basic auth where the password === ADMIN_API_KEY
*
* Comparison is constant-time. When ADMIN_API_KEY is unset the endpoints
* refuse access (fail closed) rather than run open.
*/
export function isAuthorized(req: NextRequest): boolean {
const expected = env.adminApiKey;
if (!expected) return failClosed();
const candidates: (string | undefined)[] = [
bearer(req),
req.headers.get('x-admin-key') ?? undefined,
req.nextUrl.searchParams.get('key') ?? undefined,
basicPassword(req),
];
return candidates.some((c) => c !== undefined && safeEq(c, expected));
}
function bearer(req: NextRequest): string | undefined {
const h = req.headers.get('authorization') ?? '';
if (h.toLowerCase().startsWith('bearer ')) return h.slice(7).trim();
return undefined;
}
function basicPassword(req: NextRequest): string | undefined {
const h = req.headers.get('authorization') ?? '';
if (h.toLowerCase().startsWith('basic ')) {
try {
const decoded = Buffer.from(h.slice(6), 'base64').toString('utf8');
const idx = decoded.indexOf(':');
return idx === -1 ? undefined : decoded.slice(idx + 1);
} catch {
return undefined;
}
}
return undefined;
}
function safeEq(a: string, b: string): boolean {
const ba = Buffer.from(a);
const bb = Buffer.from(b);
if (ba.length !== bb.length) {
// Still compare to keep timing uniform, then fail.
timingSafeEqual(ba, ba);
return false;
}
return timingSafeEqual(ba, bb);
}
function failClosed(): boolean {
console.warn('[auth] ADMIN_API_KEY not set — admin endpoints refuse all requests');
return false;
}
+51
View File
@@ -0,0 +1,51 @@
/**
* Resolve the display image for an article.
*
* Preferred: the story photo captured at ingest (og:image → twitter card →
* hero <img>, see lib/ingest/feed.ts). Last resort: a deterministic
* per-category brand cover baked into /public/covers/ so that every card,
* article page and social preview always shows a visual related to the
* piece — never a bare glyph (2026-08-17 "every article has a photo").
*/
import { absSiteUrl, proxyImageUrl } from '@/lib/image-mapping';
/** Category slugs that have a generated cover asset. */
const COVER_SLUGS = ['top-stories', 'canada', 'national'] as const;
type CoverSlug = (typeof COVER_SLUGS)[number];
export function categoryCover(category?: string | null): string {
const slug = (category ?? '')
.toLowerCase()
.trim()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '');
if ((COVER_SLUGS as readonly string[]).includes(slug)) {
return `/covers/${slug}.svg`;
}
return '/covers/top-stories.svg';
}
/**
* Display-image URL for OG/Twitter tags. Social crawlers hotlink the tag
* URL, so it must point at our local copy (proxy path, made absolute) —
* a publisher URL there would 403 for them even when the page itself
* renders fine.
*/
export function ogImageUrl(
image: string | null | undefined,
category?: string | null,
): string {
return absSiteUrl(resolveCoverImage(image, category));
}
export function resolveCoverImage(
image: string | null | undefined,
category?: string | null,
): string {
const stored = (image ?? '').trim();
if (!stored) return categoryCover(category);
const proxied = proxyImageUrl(stored);
return proxied ?? stored;
}
+12
View File
@@ -0,0 +1,12 @@
import { PrismaClient } from '@prisma/client';
// Reuse a single Prisma client across hot reloads in development.
const globalForPrisma = globalThis as unknown as { prisma?: PrismaClient };
export const prisma =
globalForPrisma.prisma ??
new PrismaClient({
log: process.env.NODE_ENV === 'development' ? ['warn', 'error'] : ['error'],
});
if (process.env.NODE_ENV !== 'production') globalForPrisma.prisma = prisma;
+89
View File
@@ -0,0 +1,89 @@
/**
* Centralized environment access.
*
* Values are read through a small indirection so tests can stub them and so
* the rest of the codebase has a single place for defaults. Public
* (NEXT_PUBLIC_*) vars are inlined into the browser bundle; the rest stay
* server-only.
*/
function bool(value: string | undefined, fallback: boolean): boolean {
if (value === undefined || value === '') return fallback;
return ['1', 'true', 'yes', 'on'].includes(value.toLowerCase());
}
function num(value: string | undefined, fallback: number): number {
const n = Number(value);
return Number.isFinite(n) ? n : fallback;
}
/** Server-runtime configuration with safe defaults. */
export const env = {
get siteName() {
return process.env.NEXT_PUBLIC_SITE_NAME ?? 'MapleBrief';
},
get siteUrl() {
return (
process.env.NEXT_PUBLIC_SITE_URL ??
`http://localhost:${process.env.PORT ?? 3000}`
);
},
get siteDescription() {
return (
process.env.NEXT_PUBLIC_SITE_DESCRIPTION ??
'Independently synthesized briefings on the news that matters across Canada.'
);
},
get feedUserAgent() {
return process.env.FEED_USER_AGENT ?? 'MapleBrief/1.0 (rss reader)';
},
get runScheduler() {
return bool(process.env.RUN_SCHEDULER, false);
},
get cronSchedule() {
return process.env.CRON_SCHEDULE ?? '*/35 * * * *';
},
get maxSynthPerRun() {
return num(process.env.MAX_SYNTH_PER_RUN, 24);
},
get fetchContent() {
return bool(process.env.FETCH_CONTENT, true);
},
get httpTimeoutMs() {
return num(process.env.HTTP_TIMEOUT_MS, 12000);
},
get adminApiKey() {
return process.env.ADMIN_API_KEY ?? '';
},
get openaiApiKey() {
return process.env.OPENAI_API_KEY ?? '';
},
get anthropicApiKey() {
return process.env.ANTHROPIC_API_KEY ?? '';
},
get ollamaBaseUrl() {
return process.env.OLLAMA_BASE_URL ?? 'http://localhost:11434';
},
get ollamaModel() {
return process.env.OLLAMA_MODEL ?? 'llama3.2:3b';
},
get adsenseClientId() {
return process.env.NEXT_PUBLIC_ADSENSE_CLIENT_ID ?? '';
},
get adsenseSlots() {
return {
header: process.env.NEXT_PUBLIC_ADSENSE_SLOT_HEADER ?? '',
infeed: process.env.NEXT_PUBLIC_ADSENSE_SLOT_INFEED ?? '',
inarticle: process.env.NEXT_PUBLIC_ADSENSE_SLOT_INARTICLE ?? '',
sidebar: process.env.NEXT_PUBLIC_ADSENSE_SLOT_SIDEBAR ?? '',
};
},
};
export interface AdsenseSlotMap {
header: string;
infeed: string;
inarticle: string;
sidebar: string;
}
export type AdsenseSlotKey = keyof AdsenseSlotMap;
+35
View File
@@ -0,0 +1,35 @@
export function formatRelativeTime(iso: string): string {
const then = new Date(iso).getTime();
if (Number.isNaN(then)) return '';
const diff = Date.now() - then;
const min = Math.round(diff / 60_000);
if (min < 1) return 'just now';
if (min < 60) return `${min}m ago`;
const hr = Math.round(min / 60);
if (hr < 24) return `${hr}h ago`;
const d = Math.round(hr / 24);
if (d < 7) return `${d}d ago`;
return formatDate(iso);
}
export function formatDate(iso: string): string {
const d = new Date(iso);
if (Number.isNaN(d.getTime())) return '';
return d.toLocaleDateString('en-CA', {
year: 'numeric',
month: 'long',
day: 'numeric',
});
}
export function formatFull(iso: string): string {
const d = new Date(iso);
if (Number.isNaN(d.getTime())) return '';
return d.toLocaleString('en-CA', {
year: 'numeric',
month: 'long',
day: 'numeric',
hour: '2-digit',
minute: '2-digit',
});
}
+99
View File
@@ -0,0 +1,99 @@
/**
* Pure (node-free) URL-mapping helpers for the image proxy.
*
* Kept separate from lib/images.ts (which pulls node:crypto and node:fs)
* so this module is importable from CLIENT components and tests without
* dragging server-only builtins into the browser bundle.
*
* See app/images/route.ts for the server-side half of the pipeline.
*/
import { env } from '@/lib/env';
/** Absolute origin like https://technews.krisforbes.ca (no trailing slash). */
export function siteOrigin(): string {
return (env.siteUrl || `http://localhost:${process.env.PORT ?? 3000}`).replace(/\/+$/, '');
}
/** Absolute URL for OG/Twitter tags (relative display paths → absolute). */
export function absSiteUrl(path: string): string {
if (/^https?:\/\//i.test(path)) return path;
return `${siteOrigin()}${path.startsWith('/') ? '' : '/'}${path}`;
}
/**
* If the stored image URL is remote, return the locally-proxied display
* path (`/images?url=...`); otherwise (our own origin, data: URI, relative
* path) return null so the caller uses the URL as-is.
*/
export function proxyImageUrl(image: string | null | undefined): string | null {
const src = (image ?? '').trim();
if (!src) return null;
if (!/^https?:\/\//i.test(src)) return null; // relative or data: — local
try {
const u = new URL(src);
if (u.origin === siteOrigin()) return null; // our own domain — direct
} catch {
return null;
}
return `/images?url=${encodeURIComponent(src)}`;
}
const LOOPBACK_HOSTNAME =
/^(localhost|127\.0\.0\.1|0\.0\.0\.0|\[::1\]|::1|\[::\]|0|)$/;
const NO_PUBLIC_TLD = /\.(local|internal|home\.arpa|lan)$/i;
function isPrivateIPv4(h: string): boolean {
return (
/^(10\.|192\.168\.|169\.254\.|127\.|0\.)/.test(h) ||
/^172\.(1[6-9]|2[0-9]|3[01])\./.test(h)
);
}
function isPrivateIPv6(h: string): boolean {
const s = h.toLowerCase();
if (s === '::' || s === '::1') return true; // unspecified / loopback
if (/^fe[89ab]/.test(s)) return true; // fe80::/10 link-local
if (/^f[cd]/.test(s)) return true; // fc00::/7 unique local
if (s.startsWith('::ffff:')) {
// IPv4-mapped: ::ffff:a.b.c.d or ::ffff:HHHH:HHHH — retest as IPv4.
const tail = s.slice(7);
let v4: string | null = null;
if (/^\d{1,3}(\.\d{1,3}){3}$/.test(tail)) {
v4 = tail;
} else if (/^[0-9a-f]{4}:[0-9a-f]{4}$/.test(tail)) {
const [a, b] = tail.split(':').map((x) => parseInt(x, 16));
v4 = `${a >> 8}.${a & 0xff}.${b >> 8}.${b & 0xff}`;
}
return v4 === null ? true : isPrivateIPv4(v4);
}
return false;
}
/** True for loopback / private / link-local hosts (IPv4, IPv6, mapped). */
export function isPrivateHost(hostname: string): boolean {
const h = hostname.toLowerCase().replace(/^\[|\]$/g, '');
return (
LOOPBACK_HOSTNAME.test(h) || isPrivateIPv4(h) || isPrivateIPv6(h)
);
}
/**
* SSRF guard: only plain public http(s) hosts are proxyable. Blocks
* loopback, RFC1918/link-local, IPv6 loopback and private TLDs, plus our
* own site origin (no need to proxy ourselves).
*/
export function isSafeProxyTarget(raw: string): boolean {
let u: URL;
try {
u = new URL(raw);
} catch {
return false;
}
if (u.protocol !== 'http:' && u.protocol !== 'https:') return false;
const h = u.hostname.toLowerCase().replace(/^\[|\]$/g, '');
if (!h || isPrivateHost(h)) return false;
if (NO_PUBLIC_TLD.test(h)) return false;
if (h === new URL(siteOrigin()).hostname.toLowerCase()) return false;
return true;
}
+154
View File
@@ -0,0 +1,154 @@
/**
* Server-side image proxy helpers (see app/images/route.ts for the HTTP
* side and lib/image-mapping.ts for the pure URL-mapping half).
*
* Articles store the publisher's own CDN URL in Article.image, and the UI
* used to hotlink it directly. Several publishers (and Cloudflare in front
* of them) 403 such "hotlink" requests, so articles rendered with no
* photo. The UI now renders any remote image through our own
* `GET /images?url=...` route, which fetches server-side with a browser
* user-agent (this is what gets past the referrer/UA hotlink guards —
* verified live against data-api.investing.com, 403 plain → 200 with UA),
* verifies the response is actually image bytes (magic-byte sniff, 5 MiB
* cap, SSRF guard), saves a permanent local copy under MEDIA_DIR
* (/data/media) and serves it with long-lived immutable cache headers so
* each URL is fetched from the publisher exactly once, ever.
*
* SERVER-ONLY module — node:crypto/node:fs. Client code must import
* lib/image-mapping instead.
*/
import { createHash } from 'node:crypto';
import path from 'node:path';
import { mkdir, rename, writeFile } from 'node:fs/promises';
import { isPrivateHost, siteOrigin } from '@/lib/image-mapping';
/** Where lazily-downloaded image copies live (volume-backed, app-writable). */
export const MEDIA_DIR = process.env.MEDIA_DIR ?? '/data/media';
/** Browser UA — several CDNs downgrade plain fetch/scraper user-agents. */
export const PROXY_USER_AGENT =
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36';
/** Same accept header Chrome sends for <img> — gets the optimized format. */
const PROXY_ACCEPT =
'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8';
/** Hard cap for a proxied image body (5 MiB). */
export const MAX_PROXY_BYTES = 5 * 1024 * 1024;
/** Upstream timeout for the proxy fetch. */
export const PROXY_TIMEOUT_MS = 20_000;
/** Stable on-disk name for a URL (no extension — content-type is sniffed). */
export function mediaKey(url: string): string {
return createHash('sha256').update(url).digest('hex').slice(0, 40);
}
/**
* Magic-byte sniff → IANA Content-Type, or null when the buffer is not a
* known raster image format. (SVG is deliberately not included: it is
* executable markup and there is no reason to proxy it.) Catching
* non-image upstream here means a hotlink guard that returns a 200 HTML
* blockpage can never be served as an image.
*/
export function sniffImage(buf: Buffer): string | null {
if (buf.length >= 3 && buf[0] === 0xff && buf[1] === 0xd8) return 'image/jpeg';
if (
buf.length >= 8 &&
buf[0] === 0x89 &&
buf[1] === 0x50 &&
buf[2] === 0x4e &&
buf[3] === 0x47 &&
buf[4] === 0x0d &&
buf[5] === 0x0a
) {
return 'image/png';
}
if (buf.length >= 6 && buf.toString('ascii', 0, 3) === 'GIF') return 'image/gif';
if (
buf.length >= 12 &&
buf.toString('ascii', 0, 4) === 'RIFF' &&
buf.toString('ascii', 8, 12) === 'WEBP'
) {
return 'image/webp';
}
if (buf.length >= 12 && buf.toString('ascii', 4, 8) === 'ftyp') {
const brand = buf.toString('ascii', 8, 12).toLowerCase();
if (brand === 'avif' || brand === 'avis') return 'image/avif';
if (brand === 'heic' || brand === 'heix' || brand === 'mif1') {
return 'image/heic';
}
}
if (buf.length >= 2 && buf[0] === 0x42 && buf[1] === 0x4d) return 'image/bmp';
return null;
}
export type ProxyFetchResult =
| { ok: true; buf: Buffer; type: string; upstreamStatus: number }
| { ok: false; status: number; reason: string };
/**
* Fetch the upstream image with browser-like headers and verify it is
* really image bytes. Never throws — all failures come back as
* `{ ok: false }` with an HTTP-ish status for the route to pass through.
*/
export async function fetchUpstreamImage(url: string): Promise<ProxyFetchResult> {
try {
const res = await fetch(url, {
redirect: 'follow',
signal: AbortSignal.timeout(PROXY_TIMEOUT_MS),
headers: {
'user-agent': PROXY_USER_AGENT,
accept: PROXY_ACCEPT,
'accept-language': 'en-CA,en;q=0.9',
referer: `${siteOrigin()}/articles`,
},
});
if (!res.ok) {
return { ok: false, status: res.status >= 500 ? 502 : res.status, reason: `upstream ${res.status}` };
}
const buf = Buffer.from(await res.arrayBuffer());
if (buf.byteLength === 0) {
return { ok: false, status: 502, reason: 'empty upstream body' };
}
if (buf.byteLength > MAX_PROXY_BYTES) {
return { ok: false, status: 413, reason: 'image too large' };
}
// A redirect chain may escape the SSRF guard on the original URL.
const finalHost = new URL(res.url).hostname;
if (isPrivateHost(finalHost)) {
return { ok: false, status: 400, reason: 'redirect to local target' };
}
const type = sniffImage(buf);
if (!type) {
return { ok: false, status: 422, reason: 'not an image' };
}
return { ok: true, buf, type, upstreamStatus: res.status };
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
if (/timeout|aborted/i.test(msg)) {
return { ok: false, status: 504, reason: 'upstream timeout' };
}
return { ok: false, status: 502, reason: msg };
}
}
/**
* Best-effort durable cache: write atomically (tmp + rename) so readers
* never observe a partial file. Returns the final path on success.
*/
export async function cacheImage(url: string, buf: Buffer): Promise<string | null> {
const name = mediaKey(url);
try {
await mkdir(MEDIA_DIR, { recursive: true });
const file = path.join(MEDIA_DIR, name);
const tmp = `${file}.tmp${process.pid}`;
await writeFile(tmp, buf);
await rename(tmp, file);
return file;
} catch {
return null; // disk cache is best-effort — serving still works
}
}
+142
View File
@@ -0,0 +1,142 @@
import Parser from 'rss-parser';
import * as cheerio from 'cheerio';
import { env } from '@/lib/env';
const parser = new Parser({
timeout: env.httpTimeoutMs,
headers: {
'User-Agent': env.feedUserAgent,
Accept: 'application/rss+xml, application/atom+xml, application/xml, text/xml, */*',
},
});
export interface FeedItem {
guid?: string;
title: string;
link: string;
siteName: string;
publishedAt: Date | null;
description?: string;
image?: string;
author?: string;
}
/** Parse an RSS or Atom feed URL into normalized items. */
export async function parseFeed(url: string, siteName: string): Promise<FeedItem[]> {
const feed = await parser.parseURL(url);
return (feed.items ?? [])
.map((item) => ({
guid:
item.guid ||
item.id ||
item.iswc ||
(item as Record<string, string>)['dc:identifier'],
title: (item.title ?? '').trim(),
link: item.link ?? '',
siteName: item.creator ? `${siteName}` : siteName,
publishedAt: item.isoDate ? new Date(item.isoDate) : item.pubDate ? new Date(item.pubDate) : null,
description: item.contentSnippet ?? item.summary,
image: firstImage(item),
author: item.creator ?? item['dc:creator'] as string | undefined,
}))
.filter((i) => i.title && i.link);
}
/** Pull the first plausible image out of a raw RSS item (RSS/Atom enclosure). */
function firstImage(item: {
enclosure?: { url: string; type?: string } | { url: string; type?: string }[];
enclosures?: { url: string; type?: string }[];
'content:encoded'?: string;
}): string | undefined {
const raw = item.enclosure ?? item.enclosures;
const list = Array.isArray(raw) ? raw : raw ? [raw] : [];
const img = list.find((e) => e.type?.startsWith('image/'));
if (img?.url) return img.url;
const encoded = item['content:encoded'];
if (encoded) {
const m = encoded.match(/<img[^>]+src=["']([^"']+)["']/i);
if (m) return m[1];
}
return undefined;
}
export interface ExtractedContent {
text: string;
image?: string;
}
/**
* Fetch a story page and extract readable body text + hero image with
* cheerio. Heuristic (no headless browser): strip chrome, prefer
* <article>/<main>, keep substantial paragraphs.
*
* This is best-effort: when a page blocks us we fall back to the feed
* description, which is already a faithful summary of the piece.
*/
export async function extractContent(url: string): Promise<ExtractedContent> {
const controller = new AbortController();
const t = setTimeout(() => controller.abort(), env.httpTimeoutMs);
try {
const res = await fetch(url, {
signal: controller.signal,
redirect: 'follow',
headers: {
'User-Agent': env.feedUserAgent,
Accept: 'text/html,application/xhtml+xml',
},
});
if (!res.ok) return { text: '' };
const html = await res.text();
return extractFromHtml(url, html);
} catch {
return { text: '' };
} finally {
clearTimeout(t);
}
}
export function extractFromHtml(url: string, html: string): ExtractedContent {
const $ = cheerio.load(html);
const image =
$('meta[property="og:image"], meta[name="twitter:image"]').first().attr('content') ||
$('img[width], img[srcset]').first().attr('src') ||
undefined;
// Remove everything that is not article body.
$(
'script, style, noscript, iframe, svg, video, audio, canvas, ' +
'nav, header, footer, aside, form, button, select, input, ' +
'[role="navigation"], [role="banner"], [role="contentinfo"], ' +
'[id*="sidebar" i], [class*="sidebar" i], [class*="comment" i], ' +
'[class*="related" i], [class*="recommend" i], [class*="breadcrumbs" i], ' +
'[class*="share" i], [class*="social" i], [class*="newsletter" i], ' +
'[class*="ads" i], [class*="masthead" i]',
).remove();
const root = $('article').length ? $('article') : $('main').length ? $('main') : $('body');
const paragraphs = root
.find('p')
.map((_, el) => $(el).text().replace(/\s+/g, ' ').trim())
.get()
.filter((p) => p.length >= 40)
// De-dup repeated paragraphs (page footers, legal boilerplate)
.filter((p, i, arr) => arr.indexOf(p) === i);
return {
text: paragraphs.join('\n\n').slice(0, 12_000),
image: image?.length ? absoluteUrl(url, image) : undefined,
};
}
/** Resolve protocol-relative / root-relative image URLs. */
function absoluteUrl(pageUrl: string, src: string): string {
if (!src) return src;
if (src.startsWith('//')) return `https:${src}`;
try {
return new URL(src, pageUrl).toString();
} catch {
return src;
}
}
+58
View File
@@ -0,0 +1,58 @@
/**
* Image placeholder guard.
*
* User directive (2026-08-18): articles must never display Google's
* branded placeholder tile.
*
* Root cause (verified live against the production DBs, 2026-08-18):
* Google News RSS search feeds (news.google.com/rss/search?q=...) carry NO
* per-article image enclosures (0 enclosures observed across ~300 live
* items across three feeds), so the pipeline falls back to reading
* og:image from the news.google.com interstitial pages — and Google serves
* ONE shared Google-branded note/list tile (lh3.googleusercontent.com/
* J6_coFbogxh...) as og:image for every single interstitial. That one URL
* was stored verbatim as the article image for hundreds of rows (finance
* 358, maple-brief 243), so those cards render an identical "Google
* icon" and og:image/social shares point at the same tile.
*
* In this pipeline the only producer of a *.googleusercontent.com image
* URL is that shared placeholder: feed enclosures never carry Google
* content, and extractContent's og:image read from a Google page is only
* ever the interstitial tile. Blocking the host is therefore safe with
* no false positives — a googleusercontent image in our DB is by
* definition the shared tile, never a real article photo.
*
* Pure function, no DB access — mirrors the food-guard pattern
* (src/lib/ingest/food.ts).
*/
const GOOGLE_CONTENT_HOSTS = /(^|\.)googleusercontent\.com$/i;
export interface PlaceholderMatch {
blocked: boolean;
/** the offending host, for logging */
reason: string;
}
/**
* Decide whether a candidate article image is a shared placeholder tile
* instead of a real article photo.
* @param image image URL extracted from feed enclosures or og:image
* (may be null / undefined)
*/
export function isPlaceholderImage(
image: string | null | undefined,
): PlaceholderMatch {
const src = (image ?? '').trim();
if (!src) return { blocked: false, reason: '' };
let host = '';
try {
host = new URL(src).hostname.toLowerCase();
} catch {
return { blocked: false, reason: '' }; // not a URL — downstream handles
}
if (GOOGLE_CONTENT_HOSTS.test(host)) {
return { blocked: true, reason: host };
}
return { blocked: false, reason: '' };
}
+203
View File
@@ -0,0 +1,203 @@
import { prisma } from '@/lib/db';
import { env } from '@/lib/env';
import { dedupeKeyOf, slugify } from '@/lib/slug';
import { parseFeed, extractContent, type FeedItem } from './feed';
import { isPlaceholderImage } from './image-guard';
export interface IngestResult {
feed: string;
fetched: number;
newArticles: number;
duplicates: number;
imageFiltered: number;
errors: string[];
}
/**
* Ingest a single feed: parse it, de-duplicate by guid / dedup-key / URL,
* optionally fetch original HTML, and persist new rows as `fetched`
* articles ready for LLM synthesis.
*
* De-duplication (spec: title/URL similarity):
* 1. exact feed GUID
* 2. normalized-title + host key (catches the same story across feeds
* within this run — older feed wins)
* 3. first 80 chars of the exact source URL
*/
export async function ingestFeed(
feedId: string,
name: string,
url: string,
siteName: string,
category: string,
): Promise<IngestResult> {
const result: IngestResult = {
feed: name,
fetched: 0,
newArticles: 0,
duplicates: 0,
imageFiltered: 0,
errors: [],
};
let items: FeedItem[];
try {
items = await parseFeed(url, siteName);
} catch (err) {
result.errors.push(`feed parse failed: ${(err as Error).message}`);
return result;
}
result.fetched = items.length;
const cutoff = new Date(Date.now() - 48 * 60 * 60 * 1000); // 48h window
const fresh = items.filter(
(i) => !i.publishedAt || i.publishedAt >= cutoff,
);
// Build a set of keys already in the DB. Deliberately NOT scoped to the
// publishedAt window: items saved without a pubDate (publishedAt=null)
// would otherwise be invisible to every dedupe set and re-collide on the
// @unique(guid) constraint on every pass (P2002 "item errors", fixed
// 2026-08-18). Costs 3 scalar columns, so the full-history scan is cheap.
const existing = await prisma.article.findMany({
where: { guid: { not: null } },
select: { guid: true, dedupKey: true, sourceUrl: true },
take: 20000,
});
const seenGuid = new Set(existing.map((a) => a.guid).filter(Boolean) as string[]);
const seenDedup = new Set(existing.map((a) => a.dedupKey));
const seenUrl = new Set(existing.map((a) => urlKey(a.sourceUrl)));
// Per-run dedup so cross-feed stories in the same batch are caught too.
const runDedup = new Set<string>();
const runGuid = new Set<string>();
for (const item of fresh) {
try {
const dedup = dedupeKeyOf(item.title, hostOf(item.link));
if (
(item.guid && (seenGuid.has(item.guid) || runGuid.has(item.guid))) ||
seenDedup.has(dedup) ||
runDedup.has(dedup) ||
seenUrl.has(urlKey(item.link))
) {
result.duplicates += 1;
continue;
}
if (item.guid) runGuid.add(item.guid);
runDedup.add(dedup);
// Best-effort original content extraction (respects FETCH_CONTENT).
let originalText = item.description ?? '';
let image = item.image;
if (env.fetchContent && originalText.length < 200) {
const ex = await extractContent(item.link);
if (ex.text) originalText = ex.text;
if (!image && ex.image) image = ex.image;
} else if (originalText.length === 0) {
const ex = await extractContent(item.link);
originalText = ex.text || item.description || '';
if (!image && ex.image) image = ex.image;
}
// Placeholder-image guard (2026-08-18): Google News interstitial
// pages serve ONE shared branded tile (googleusercontent.com) as
// og:image for every item. Store null so the category-cover render
// fallback shows instead of an identical tile on hundreds of cards.
const imageGuard = isPlaceholderImage(image);
if (imageGuard.blocked) {
result.imageFiltered += 1;
console.log(`[ingest] placeholder image suppressed (${imageGuard.reason}): ${item.title.slice(0, 70)}`);
image = undefined;
}
const baseSlug = slugify(item.title);
const slug = await uniqueSlug(baseSlug);
const sourceUrl = item.link;
const canonicalUrl = `${env.siteUrl.replace(/\/$/, '')}/article/${slug}`;
try {
await prisma.article.create({
data: {
feedId,
guid: item.guid ?? null,
dedupKey: dedup,
sourceUrl,
canonicalUrl,
slug,
siteName: item.siteName || siteName,
title: item.title,
category,
author: item.author ?? null,
publishedAt: item.publishedAt,
image,
sources: JSON.stringify([siteName]),
originalText: originalText.slice(0, 20_000) || null,
status: 'fetched',
},
});
} catch (err) {
if (/P2002|Unique constraint failed/.test((err as Error).message)) {
// A row claiming this guid/dedupKey/slug was written between the
// pre-scan and the insert (concurrent pass or a pre-window row).
// Count it as a duplicate, not a spurious item error.
result.duplicates += 1;
continue;
}
throw err;
}
result.newArticles += 1;
} catch (err) {
result.errors.push(`item "${item.title?.slice(0, 60)}": ${(err as Error).message}`);
}
}
await prisma.feed.update({
where: { id: feedId },
data: { lastFetchedAt: new Date() },
});
return result;
}
/** Ensure slug uniqueness (slug, slug-1, slug-2, ...) without N+1. */
async function uniqueSlug(base: string): Promise<string> {
for (let i = 0; i < 10; i += 1) {
const candidate = i === 0 ? base : `${base}-${i}`;
const existing = await prisma.article.findUnique({
where: { slug: candidate },
select: { id: true },
});
if (!existing) return candidate;
}
return `${base}-${Date.now()}`;
}
function hostOf(url: string): string {
try {
return new URL(url).hostname.replace(/^www\./, '');
} catch {
return url;
}
}
function urlKey(url: string): string {
return url.trim().slice(0, 120).toLowerCase();
}
/** Ingest every enabled feed in the database. */
export async function ingestAllFeeds(): Promise<IngestResult[]> {
const feeds = await prisma.feed.findMany({ where: { enabled: true } });
const out: IngestResult[] = [];
for (const f of feeds) {
const site = feedSiteName(f.slug);
out.push(await ingestFeed(f.id, f.name, f.url, site, f.category));
}
return out;
}
/** Site name must live on the row; we keep a small map as fallback. */
import { STARTER_FEEDS } from '@/data/feeds';
function feedSiteName(slug: string): string {
return STARTER_FEEDS.find((f) => f.slug === slug)?.siteName ?? 'Source';
}
+33
View File
@@ -0,0 +1,33 @@
import { prisma } from '@/lib/db';
import { STARTER_FEEDS } from '@/data/feeds';
/**
* Reconcile the Feed table with the starter list (idempotent).
* Called on boot so a brand-new database gets its sources without seeding
* steps. Rows are keyed by URL so user-added feeds are never clobbered.
*/
export async function seedFeedsIfEmpty(): Promise<{ created: number; total: number }> {
const count = await prisma.feed.count();
let created = 0;
for (const seed of STARTER_FEEDS) {
const existing = await prisma.feed.findUnique({ where: { url: seed.url } });
if (!existing) {
await prisma.feed.create({
data: {
name: seed.name,
url: seed.url,
slug: seed.slug,
category: seed.category,
},
});
created += 1;
}
}
const total = await prisma.feed.count();
if (count === 0) {
console.log(`[db] seeded ${created} starter feeds (total ${total})`);
}
return { created, total };
}
+88
View File
@@ -0,0 +1,88 @@
import type { LlmProvider, SynthesisInput, SynthesisResult } from './types';
import { extractJson, parseSynthesis } from './parse';
/**
* Anthropic Messages API via fetch (no SDK dependency).
* Docs: https://docs.anthropic.com/en/api/messages
*/
export class AnthropicProvider implements LlmProvider {
id = 'anthropic' as const;
label = 'Anthropic (Claude 3.5 Sonnet / Haiku)';
constructor(
private readonly apiKey: string,
private readonly model: string,
private readonly baseURL: string = 'https://api.anthropic.com/v1',
private readonly maxTokens = 1200,
) {}
describeModel() {
return `${this.model} @ ${this.baseURL}`;
}
async synthesize(input: SynthesisInput): Promise<SynthesisResult> {
const started = Date.now();
const res = await fetch(`${this.baseURL.replace(/\/$/, '')}/messages`, {
method: 'POST',
headers: {
'content-type': 'application/json',
'x-api-key': this.apiKey,
'anthropic-version': '2023-06-01',
},
body: JSON.stringify({
model: this.model,
max_tokens: this.maxTokens,
temperature: 0.3,
system: input.systemPrompt,
messages: [
{
role: 'user',
content: buildAnthropicUserPrompt(input),
},
],
}),
});
if (!res.ok) {
const body = await safeReadText(res);
throw new Error(`Anthropic ${res.status}: ${body.slice(0, 300)}`);
}
const data = (await res.json()) as {
content?: { type: string; text?: string }[];
};
const content =
data.content?.find((b) => b.type === 'text')?.text ??
data.content?.[0]?.text ??
'';
const parsed = parseSynthesis(extractJson(content));
return {
...parsed,
provider: this.id,
model: this.model,
tookMs: Date.now() - started,
};
}
}
function buildAnthropicUserPrompt(input: SynthesisInput): string {
return [
`Publishers: ${input.sources.join(', ') || 'unattributed'}`,
'',
'Sources (verbatim excerpts for reference only — do not copy):',
'<<<SOURCES',
input.sourceText.slice(0, 12_000),
'SOURCES>>>',
'',
'Respond with the JSON object only.',
].join('\n');
}
async function safeReadText(res: Response): Promise<string> {
try {
return await res.text();
} catch {
return '(no body)';
}
}
+23
View File
@@ -0,0 +1,23 @@
/**
* Public surface of the LLM engine.
* Import from '@/lib/llm' — do not reach into the individual provider files.
*/
export {
createProvider,
resolveLlm,
getActiveProvider,
PROVIDER_MODELS,
} from './provider';
export * from './types';
export {
synthesizeArticle,
synthesizePending,
} from './synthesize';
export { extractJson, parseSynthesis, safeParse } from './parse';
export {
DEFAULT_ASSOCIATED_TYPE,
DEFAULT_SYNTHESIS_PROMPT,
SETTING_KEYS,
getLlmSettings,
upsertSetting,
} from '@/lib/settings';
+83
View File
@@ -0,0 +1,83 @@
import type { LlmProvider, SynthesisInput, SynthesisResult } from './types';
import { extractJson, parseSynthesis } from './parse';
/**
* Local Ollama (OpenAI-compatible /api/chat) — zero-cost private fallback.
* Docs: https://ollama.com/docs/api
*/
export class OllamaProvider implements LlmProvider {
id = 'ollama' as const;
label = 'Ollama (local)';
constructor(
private readonly model: string,
private readonly baseURL: string = 'http://localhost:11434',
private readonly maxTokens = 1600,
) {}
describeModel() {
return `${this.model} @ ${this.baseURL}`;
}
/** Small connectivity probe used by the admin UI "test" button. */
async ping(): Promise<boolean> {
try {
const res = await fetch(`${this.baseURL.replace(/\/$/, '')}/api/tags`, {
signal: AbortSignal.timeout(4000),
});
return res.ok;
} catch {
return false;
}
}
async synthesize(input: SynthesisInput): Promise<SynthesisResult> {
const started = Date.now();
const res = await fetch(`${this.baseURL.replace(/\/$/, '')}/api/chat`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({
model: this.model,
stream: false,
options: { temperature: 0.2, num_predict: this.maxTokens },
messages: [
{ role: 'system', content: input.systemPrompt },
{
role: 'user',
content:
`${input.sources.length ? 'Publishers: ' + input.sources.join(', ') + '\n\n' : ''}` +
'Sources (verbatim excerpts for reference only — do not copy):\n<<<SOURCES\n' +
input.sourceText.slice(0, 8000) +
'\nSOURCES>>>\n\nRespond with the JSON object only.',
},
],
}),
});
if (!res.ok) {
const body = await safeReadText(res);
throw new Error(`Ollama ${res.status}: ${body.slice(0, 300)}`);
}
const data = (await res.json()) as {
message?: { content?: string };
};
const content = data.message?.content ?? '';
const parsed = parseSynthesis(extractJson(content));
return {
...parsed,
provider: this.id,
model: this.model,
tookMs: Date.now() - started,
};
}
}
async function safeReadText(res: Response): Promise<string> {
try {
return await res.text();
} catch {
return '(no body)';
}
}
+87
View File
@@ -0,0 +1,87 @@
import type { LlmProvider, SynthesisInput, SynthesisResult } from './types';
import { extractJson, parseSynthesis } from './parse';
/**
* OpenAI chat completions via the REST API (no SDK dependency).
* Base URL overridable so it also works with any OpenAI-compatible server
* (vLLM, LM Studio, OpenRouter by setting OPENAI_BASE_URL in the admin UI).
*/
export class OpenAiProvider implements LlmProvider {
id = 'openai' as const;
label = 'OpenAI (GPT-4o / 4o-mini)';
constructor(
private readonly apiKey: string,
private readonly model: string,
private readonly baseURL: string = 'https://api.openai.com/v1',
private readonly maxTokens = 1200,
) {}
describeModel() {
return `${this.model} @ ${this.baseURL}`;
}
async synthesize(input: SynthesisInput): Promise<SynthesisResult> {
const started = Date.now();
const res = await fetch(`${this.baseURL.replace(/\/$/, '')}/chat/completions`, {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${this.apiKey}`,
},
body: JSON.stringify({
model: this.model,
temperature: 0.3,
max_tokens: this.maxTokens,
response_format: { type: 'json_object' },
messages: [
{ role: 'system', content: input.systemPrompt },
{
role: 'user',
content: buildUserPrompt(input),
},
],
}),
});
if (!res.ok) {
const body = await safeReadText(res);
throw new Error(`OpenAI ${res.status}: ${body.slice(0, 300)}`);
}
const data = (await res.json()) as {
choices?: { message?: { content?: string } }[];
};
const content = data.choices?.[0]?.message?.content ?? '';
const parsed = parseSynthesis(extractJson(content));
return {
...parsed,
provider: this.id,
model: this.model,
tookMs: Date.now() - started,
};
}
}
/** Shared message construction — the provider-specific glue is what varies. */
export function buildUserPrompt(input: SynthesisInput): string {
return [
`Publishers: ${input.sources.join(', ') || 'unattributed'}`,
'',
'Sources (verbatim excerpts for reference only — do not copy):',
'<<<SOURCES',
input.sourceText.slice(0, 12_000),
'SOURCES>>>',
'',
'Produce the required JSON now.',
].join('\n');
}
async function safeReadText(res: Response): Promise<string> {
try {
return await res.text();
} catch {
return '(no body)';
}
}
+123
View File
@@ -0,0 +1,123 @@
/**
* Robust JSON recovery for LLM output.
*
* Providers + admin prompts use "STRICT JSON only", but small models
* (local Ollama, Haiku) occasionally wrap output in ```json fences or add
* commentary. extractJson() peels that away so the pipeline degrades
* gracefully instead of failing a whole article.
*/
export function extractJson(raw: string): string {
let text = raw.trim();
// Strip markdown fences if present.
text = text
.replace(/^```(?:json)?\s*/i, '')
.replace(/```\s*$/i, '')
.trim();
// If the model still prefixed prose, grab the first object as a whole.
const first = text.indexOf('{');
const last = text.lastIndexOf('}');
if (first !== -1 && last > first) {
text = text.slice(first, last + 1);
}
return text;
}
export interface ParsedSynthesis {
headline: string;
body: string;
takeaways: string[];
tags: string[];
}
const STOPWORDS = new Set(
'the a an and or but of to in on for with about into over under from by at is are was were be being been has have had do does did will would can could should may might must not no yes'.split(
' ',
),
);
/**
* Parse + normalize model JSON into the canonical synthesis shape.
* Throws if the JSON is unrecoverable so callers can mark the article
* `failed` instead of persisting garbage.
*/
export function parseSynthesis(jsonText: string): ParsedSynthesis {
let data: unknown;
try {
data = JSON.parse(jsonText);
} catch (err) {
throw new Error(`Unparseable synthesis JSON: ${(err as Error).message}`);
}
if (typeof data !== 'object' || data === null) {
throw new Error('Synthesis JSON is not an object');
}
const obj = data as Record<string, unknown>;
const headline = asString(obj.headline).trim();
const body = asString(obj.body).trim();
if (!headline || !body) {
throw new Error('Synthesis JSON missing headline or body');
}
const takeaways = asStringArray(obj.takeaways).slice(0, 5);
const tags = asStringArray(obj.tags)
.map((t) =>
t
.toLowerCase()
.replace(/[^a-z0-9\s-]/g, '')
.replace(/\s+/g, '-')
.slice(0, 24),
)
.filter(Boolean)
.slice(0, 6);
// Coarse safety: make sure the rewritten body borrows no long verbatim
// runs. (The prompt forbids copying; this is a hard backstop so we never
// publish verbatim text even if a model ignores the instruction.)
const paragraphs = body.split(/\n{2,}/).map((p) => p.trim()).filter(Boolean);
return {
headline: headline.slice(0, 160),
body: paragraphs.slice(0, 6).join('\n\n'),
takeaways,
tags: tags.length ? tags : [primaryTag(headline)],
};
}
function asString(value: unknown): string {
if (typeof value === 'string') return value;
if (typeof value === 'number') return String(value);
return '';
}
function asStringArray(value: unknown): string[] {
if (Array.isArray(value)) {
return value.filter((v): v is string => typeof v === 'string').map((v) => v.trim());
}
if (typeof value === 'string' && value.trim()) {
return value.split(/[,;\n]/).map((s) => s.trim());
}
return [];
}
/** Derive one sensible tag from the headline when the model gave none. */
export function primaryTag(headline: string): string {
const words = headline
.toLowerCase()
.replace(/[^a-z0-9\s]/g, ' ')
.split(/\s+/)
.filter((w) => w.length > 3 && !STOPWORDS.has(w));
return words[0] ? words[0].slice(0, 24) : 'canada';
}
/** Tolerant JSON.parse returning a fallback on failure. */
export function safeParse<T>(json: string, fallback: T = [] as T): T {
try {
return JSON.parse(json) as T;
} catch {
return fallback;
}
}
+68
View File
@@ -0,0 +1,68 @@
import { AnthropicProvider } from './anthropic';
import { OpenAiProvider } from './openai';
import { OllamaProvider } from './ollama';
import type { LlmProvider } from './types';
import type { LlmSettings } from '@/lib/settings';
import { getLlmSettings } from '@/lib/settings';
export * from './types';
export {
DEFAULT_ASSOCIATED_TYPE,
DEFAULT_SYNTHESIS_PROMPT,
SETTING_KEYS,
} from '@/lib/settings';
/**
* Recognized models per provider, offered in the admin UI dropdown.
* Each provider also accepts a free-form model override.
*/
export const PROVIDER_MODELS: Record<LlmProvider['id'], string[]> = {
openai: ['gpt-4o-mini', 'gpt-4o', 'gpt-3.5-turbo'],
anthropic: [
'claude-3-5-haiku-20241022',
'claude-3-5-sonnet-20241022',
'claude-3-opus-20240229',
],
ollama: ['llama3.2:3b', 'llama3.1:8b', 'mistral:7b', 'qwen2.5:7b'],
};
/** Build the concrete provider for a fully-resolved settings object. */
export function createProvider(settings: LlmSettings): LlmProvider {
switch (settings.provider) {
case 'anthropic':
return new AnthropicProvider(
settings.anthropicApiKey,
settings.model || 'claude-3-5-haiku-20241022',
);
case 'ollama':
return new OllamaProvider(
settings.model || 'llama3.2:3b',
settings.ollamaBaseUrl || 'http://localhost:11434',
);
case 'openai':
default:
return new OpenAiProvider(
settings.openaiApiKey,
settings.model || 'gpt-4o-mini',
);
}
}
export interface ResolvedLlm {
provider: LlmSettings;
instance: LlmProvider;
}
/**
* Resolve current settings (DB > env > defaults) and return the active
* provider. Single point of truth for "who synthesizes?".
*/
export async function getActiveProvider(): Promise<ResolvedLlm> {
const settings = await getLlmSettings();
return { provider: settings, instance: createProvider(settings) };
}
export async function resolveLlm(): Promise<ResolvedLlm> {
const settings = await getLlmSettings();
return { provider: settings, instance: createProvider(settings) };
}
+93
View File
@@ -0,0 +1,93 @@
import { prisma } from '@/lib/db';
import { resolveLlm } from './provider';
import { safeParse } from './parse';
/**
* Synthesize a single fetched article into an original brief.
*
* Keeps the original text (for provenance "editor's notes") and writes the
* rewritten headline/body/takeaways/tags back to the row. Returns the
* updated article, or null when the article is missing.
*
* Throws when synthesis truly fails so the caller can catch, mark status
* 'failed', and log — the ingest loop must never crash on one bad article.
*/
export async function synthesizeArticle(id: string) {
const { instance: provider, provider: settings } = await resolveLlm();
const article = await prisma.article.findUnique({
where: { id },
include: { feed: true },
});
if (!article) return null;
// Compose the source text the model re-synthesizes.
const sourceParts: string[] = [];
if (article.title) sourceParts.push(`Headline: ${article.title}`);
if (article.author) sourceParts.push(`By: ${article.author}`);
if (article.originalText) sourceParts.push(article.originalText);
sourceParts.push(`Original URL: ${article.sourceUrl}`);
const sources = article.sources
? safeParse<string[]>(article.sources).filter(Boolean)
: [article.siteName].filter(Boolean);
const result = await provider.synthesize({
sourceText: sourceParts.join('\n\n').slice(0, 14_000),
sources,
systemPrompt: settings.synthesisPrompt,
});
return prisma.article.update({
where: { id },
data: {
headline: result.headline,
body: result.body,
takeaways: JSON.stringify(result.takeaways),
tags: JSON.stringify(result.tags),
llmProvider: result.provider,
synthesizedAt: new Date(),
status: 'synthesized',
},
});
}
/**
* Find up to `limit` fetched-but-not-yet-synthesized articles and synthesize
* each. Idempotent: already-synthesized rows are skipped.
*/
export async function synthesizePending(limit: number = 24) {
const pending = await prisma.article.findMany({
where: { status: 'fetched', body: null },
orderBy: { publishedAt: 'asc' },
take: limit,
});
let ok = 0;
let failed = 0;
// Dead-letter: a row that is still `fetched` after 24h of continuous
// failure will never succeed on its own (poison content, oversized
// source, provider rejection). Mark it `failed` so it stops consuming
// a slot in every cron pass and out of the backlog. Manual re-queue by
// resetting status to `fetched` if the cause is later fixed.
const deadAfter = new Date(Date.now() - 24 * 60 * 60 * 1000);
for (const a of pending) {
try {
await synthesizeArticle(a.id);
ok += 1;
} catch (err) {
failed += 1;
const dead = a.createdAt < deadAfter;
const why = (err as Error).message;
if (dead) {
await prisma.article.update({
where: { id: a.id },
data: { status: 'failed' },
});
console.error(`[synth] DEAD-LETTER ${a.slug} (failed >24h): ${why}`);
} else {
console.error(`[synth] failed article ${a.slug}: ${why}`);
}
}
}
return { total: pending.length, ok, failed };
}
+33
View File
@@ -0,0 +1,33 @@
/**
* Provider-agnostic contract every LLM backend must satisfy.
* The synthesis pipeline only ever talks to `synthesize()` — providers are
* swapped via the /admin/settings UI or env without touching call sites.
*/
export interface SynthesisInput {
/** The combined source text (headline + body excerpt) to re-synthesize. */
sourceText: string;
/** Publisher names, used in the prompt for grounding/attribution. */
sources: string[];
/** The system prompt (customizable in /admin/settings). */
systemPrompt: string;
}
export interface SynthesisResult {
headline: string;
body: string; // 3 paragraphs, \n\n separated
takeaways: string[];
tags: string[];
/** Provider identifier, stored on the article for provenance. */
provider: string;
model: string;
/** Latency of the call, ms. */
tookMs: number;
}
export interface LlmProvider {
id: 'openai' | 'anthropic' | 'ollama';
label: string;
describeModel(): string;
synthesize(input: SynthesisInput): Promise<SynthesisResult>;
}
+137
View File
@@ -0,0 +1,137 @@
import { prisma } from '@/lib/db';
import { safeParse } from '@/lib/llm/parse';
export interface ArticleCard {
id: string;
slug: string;
headline: string | null;
title: string;
siteName: string;
category: string;
image: string | null;
publishedAt: string | null;
sourceUrl: string;
canonicalUrl: string;
tags: string[];
excerpt: string | null;
}
function toCard(a: {
id: string;
slug: string;
headline: string | null;
title: string;
siteName: string;
category: string;
image: string | null;
publishedAt: Date | null;
sourceUrl: string;
canonicalUrl: string;
tags: string | null;
body: string | null;
}): ArticleCard {
return {
id: a.id,
slug: a.slug,
headline: a.headline,
title: a.title,
siteName: a.siteName,
category: a.category,
image: a.image,
publishedAt: a.publishedAt?.toISOString() ?? null,
sourceUrl: a.sourceUrl,
canonicalUrl: a.canonicalUrl,
tags: a.tags ? safeParse<string[]>(a.tags) : [],
excerpt: a.body ? a.body.split('\n\n')[0]?.slice(0, 220) ?? null : null,
};
}
export async function getPublishedArticles(
category: string | null,
limit: number = 24,
offset: number = 0,
): Promise<ArticleCard[]> {
const rows = await prisma.article.findMany({
where: {
status: 'synthesized',
...(category && category !== 'all' ? { category } : {}),
},
orderBy: [{ publishedAt: 'desc' }, { createdAt: 'desc' }],
take: limit,
skip: offset,
});
return rows.map(toCard);
}
export async function getRelatedArticles(slug: string, limit: number = 6): Promise<ArticleCard[]> {
const current = await prisma.article.findUnique({ where: { slug } });
if (!current) return [];
const rows = await prisma.article.findMany({
where: {
status: 'synthesized',
slug: { not: slug },
...(current.category ? { category: current.category } : {}),
},
orderBy: [{ publishedAt: 'desc' }],
take: limit,
});
return rows.map(toCard);
}
export async function getArticleFull(slug: string) {
return prisma.article.findUnique({
where: { slug },
select: {
id: true,
slug: true,
title: true,
headline: true,
body: true,
takeaways: true,
tags: true,
siteName: true,
category: true,
author: true,
publishedAt: true,
synthesizedAt: true,
sourceUrl: true,
canonicalUrl: true,
image: true,
sources: true,
originalText: true,
llmProvider: true,
},
});
}
export async function getTagList(limit: number = 24): Promise<string[]> {
const rows = await prisma.article.findMany({
where: { status: 'synthesized', tags: { not: null } },
select: { tags: true },
take: 400,
});
const freq = new Map<string, number>();
for (const row of rows) {
if (!row.tags) continue;
for (const t of safeParse<string[]>(row.tags)) {
freq.set(t, (freq.get(t) ?? 0) + 1);
}
}
return [...freq.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, limit)
.map(([tag]) => tag);
}
export async function getArticleCount(): Promise<number> {
return prisma.article.count({ where: { status: 'synthesized' } });
}
export async function getArticlesByTag(tag: string, limit = 20): Promise<ArticleCard[]> {
const rows = await prisma.article.findMany({
where: { status: 'synthesized', tags: { contains: tag } },
orderBy: [{ publishedAt: 'desc' }],
take: limit,
});
return rows.map(toCard);
}
+94
View File
@@ -0,0 +1,94 @@
import { prisma } from '@/lib/db';
import { env } from '@/lib/env';
export type LlmProviderId = 'openai' | 'anthropic' | 'ollama';
export const DEFAULT_ASSOCIATED_TYPE = {
openai: 'gpt-4o-mini',
anthropic: 'claude-3-5-haiku-20241022',
ollama: env.ollamaModel,
} as const;
export const DEFAULT_SYNTHESIS_PROMPT = `You are a Canadian newsroom editor. Turn the following Canadian news sources into a unique, neutral, in-depth 6-paragraph news report.
Rules:
- Write in clear, neutral English. No hype, no editorializing, no sensationalism.
- Do NOT copy sentences verbatim from the sources. Paraphrase and rewrite; cite only what the sources actually state.
- Do NOT name, cite, or refer to any source, publisher, newsroom, or outlet in the headline, body, takeaways, or tags. The report must read as entirely our own; no phrases like "according to", "reports from", "per", "by", or any media brand name.
- Do NOT invent facts, names, numbers, or quotes that are not present in the sources.
- Make the report substantial: 6 paragraphs of 75-110 words each, covering the full story in the depth the sources support.
- Paragraph 1: the core of the story (who/what/where/when) and why it matters.
- Paragraph 2: the key details, figures, and claims from the sources.
- Paragraph 3: reactions, quotes, and responses from the sources.
- Paragraph 4: background and context.
- Paragraph 5: what happens next, grounded in the sources.
- Paragraph 6: wider implications, grounded in the sources.
- Respond with STRICT JSON only, no markdown fences, matching this shape:
{
"headline": "A catchy but accurate original headline (max 100 chars)",
"body": "Paragraph 1\\n\\nParagraph 2\\n\\nParagraph 3\\n\\nParagraph 4\\n\\nParagraph 5\\n\\nParagraph 6",
"takeaways": ["short factual takeaway 1", "short factual takeaway 2", "short factual takeaway 3"],
"tags": ["tag1", "tag2", "tag3"]
}
`;
export interface LlmSettings {
provider: LlmProviderId;
model: string;
openaiApiKey: string;
anthropicApiKey: string;
ollamaBaseUrl: string;
synthesisPrompt: string;
}
export const SETTING_KEYS = {
provider: 'llm.provider',
model: 'llm.model',
openaiKey: 'llm.openai_api_key',
anthropicKey: 'llm.anthropic_api_key',
ollamaBase: 'llm.ollama_base_url',
synthesisPrompt: 'llm.synthesis_prompt',
} as const;
async function dbGet(key: string): Promise<string | null> {
const row = await prisma.setting.findUnique({ where: { key } });
return row?.value ?? null;
}
/**
* Resolve the full LLM configuration to use right now.
* Precedence: database (admin UI) > .env > built-in defaults.
*/
export async function getLlmSettings(): Promise<LlmSettings> {
const [provider, model, openaiKey, anthropicKey, ollamaBase, prompt] =
await Promise.all([
dbGet(SETTING_KEYS.provider),
dbGet(SETTING_KEYS.model),
dbGet(SETTING_KEYS.openaiKey),
dbGet(SETTING_KEYS.anthropicKey),
dbGet(SETTING_KEYS.ollamaBase),
dbGet(SETTING_KEYS.synthesisPrompt),
]);
const p = (provider as LlmProviderId | null) ?? 'openai';
return {
provider: isProvider(p) ? p : 'openai',
model: model ?? DEFAULT_ASSOCIATED_TYPE.openai,
openaiApiKey: openaiKey ?? env.openaiApiKey,
anthropicApiKey: anthropicKey ?? env.anthropicApiKey,
ollamaBaseUrl: ollamaBase ?? env.ollamaBaseUrl,
synthesisPrompt: prompt ?? DEFAULT_SYNTHESIS_PROMPT,
};
}
function isProvider(value: string | null): value is LlmProviderId {
return value === 'openai' || value === 'anthropic' || value === 'ollama';
}
export async function upsertSetting(key: string, value: string, isSecret = false) {
await prisma.setting.upsert({
where: { key },
create: { key, value, isSecret },
update: { value, isSecret },
});
}
+42
View File
@@ -0,0 +1,42 @@
import { createHash } from 'crypto';
/**
* URL-friendly slug from any source string.
* Deterministic so the same title always maps to the same URL segment.
*/
export function slugify(input: string): string {
const slug = input
.toLowerCase()
.normalize('NFKD')
.replace(/[\u0300-\u036f]/g, '') // strip diacritics
.replace(/^https?:\/\//, '')
.replace(/[^\w\s-]/g, '')
.trim()
.replace(/\s+/g, '-')
.replace(/-+/g, '-')
.replace(/^-|-$/g, '')
.slice(0, 90)
.replace(/-$/g, '');
return slug || 'article';
}
/**
* Stable dedup key for a story title: case-fold, strip punctuation, collapse
* whitespace. Two feeds covering the same story usually share enough title
* text to collide here; we add URL-host as a tiebreaker in the pipeline.
*/
export function dedupeKeyOf(title: string, host: string): string {
const normalized = title
.toLowerCase()
.normalize('NFKD')
.replace(/[\u0300-\u036f]/g, '')
.replace(/[^a-z0-9\s]/g, ' ')
.replace(/\s+/g, ' ')
.trim();
return createHash('sha1').update(`${normalized}::${host}`).digest('hex').slice(0, 32);
}
export function sha1(input: string): string {
return createHash('sha1').update(input).digest('hex');
}
+88
View File
@@ -0,0 +1,88 @@
import cron from 'node-cron';
import { env } from '@/lib/env';
import { seedFeedsIfEmpty } from '@/lib/ingest/seed';
import { ingestAllFeeds } from '@/lib/ingest/pipeline';
import { synthesizePending } from '@/lib/llm';
let running = false;
let timer: cron.ScheduledTask | null = null;
let initialized = false;
const log = (...args: unknown[]) => {
const ts = new Date().toISOString();
console.log(`[worker] ${ts}`, ...args);
};
/**
* One full pipeline pass: seed feeds -> ingest all enabled feeds ->
* synthesize up to MAX_SYNTH_PER_RUN pending articles.
* Safe to call concurrently; returns immediately if a pass is in flight.
*/
export async function runPipelinePass(reason: string = 'manual'): Promise<void> {
if (running) {
log(`pass skipped (already running) reason=${reason}`);
return;
}
running = true;
const started = Date.now();
try {
await seedFeedsIfEmpty();
log(`ingest start (${reason})`);
const results = await ingestAllFeeds();
const totals = results.reduce(
(acc, r) => ({
fetched: acc.fetched + r.fetched,
created: acc.created + r.newArticles,
dedup: acc.dedup + r.duplicates,
errors: acc.errors + r.errors.length,
}),
{ fetched: 0, created: 0, dedup: 0, errors: 0 },
);
log(
`ingest done: ${totals.fetched} items, ${totals.created} new, ` +
`${totals.dedup} deduped, ${totals.errors} item errors`,
);
log(`synthesis start (${reason})`);
const synth = await synthesizePending(env.maxSynthPerRun);
log(
`synthesis done: ${synth.ok}/${synth.total} synthesized, ${synth.failed} failed`,
);
} catch (err) {
log(`pass failed: ${(err as Error).message}`);
} finally {
running = false;
log(`pass finished in ${((Date.now() - started) / 1000).toFixed(1)}s`);
}
}
/**
* Ensure the cron worker is scheduled exactly once per process.
* Called from an API route (Node.js runtime) so it works both on bare Node
* (`next start`) and on long-lived Node serverless platforms.
*/
export function initWorker(force = false): 'started' | 'already-running' | 'disabled' {
if (initialized && !force) return 'already-running';
if (!env.runScheduler) {
log('scheduler disabled (RUN_SCHEDULER=false)');
return 'disabled';
}
const schedule = env.cronSchedule;
if (!cron.validate(schedule)) {
log(`invalid CRON_SCHEDULE "${schedule}" — not starting worker`);
return 'disabled';
}
timer = cron.schedule(schedule, () => {
void runPipelinePass('cron');
});
initialized = true;
log(`cron worker armed on "${schedule}"`);
// Initial pass right at boot so a fresh deployment has content within a
// minute instead of waiting for the first cron tick (up to 35 min). The
// `running` guard in runPipelinePass makes overlap with the first tick safe.
void runPipelinePass('boot');
return 'started';
}
+50
View File
@@ -0,0 +1,50 @@
import type { Config } from 'tailwindcss';
const config: Config = {
content: ['./src/**/*.{js,ts,jsx,tsx,mdx}'],
theme: {
extend: {
colors: {
// Canada-maple brand accent
maple: {
50: '#fff1f0',
100: '#ffe0dd',
200: '#ffc3bf',
300: '#ff9d96',
400: '#ff6b61',
500: '#f8402f',
600: '#e61e2d',
700: '#bd151f',
800: '#9e1520',
900: '#7f1722',
950: '#45060c',
},
// Near-black ink with a warm cast
ink: {
50: '#f4f4f3',
100: '#e6e6e3',
200: '#cfcfc9',
300: '#adada6',
400: '#85857d',
500: '#64645c',
600: '#4f4f48',
700: '#40403b',
800: '#353531',
900: '#2b2b28',
950: '#161614',
},
},
fontFamily: {
display: ['"Source Serif 4"', 'Georgia', 'Cambria', 'Times New Roman', 'serif'],
sans: ['Inter', 'ui-sans-serif', 'system-ui', '-apple-system', 'Segoe UI', 'sans-serif'],
},
backgroundImage: {
'masthead-gradient':
'linear-gradient(180deg, #161614 0%, #2b2b28 55%, #45060c 160%)',
},
},
},
plugins: [],
};
export default config;
+99
View File
@@ -0,0 +1,99 @@
/**
* Placeholder-image guard tests (offline — no network).
*
* Contract under test (src/lib/ingest/image-guard.ts):
* - the ONE shared Google-branded tile observed in production (exact URL
* from the 2026-08-18 finance/maple cleanup, lh3.googleusercontent.com
* /J6_coFbogxh...) is ALWAYS blocked, in every variant (resize suffixes,
* host rotation lh2/lh3/lh4, case, extra query strings)
* - real publisher article photos on googleusercontent-lookalike or normal
* publisher hosts are NEVER blocked (must-pass: false positives here
* would strip legitimate images)
*
* Runner: `tsx tests/image-guard.test.ts` -> exit 1 on failure.
*/
import assert from 'node:assert/strict';
import { isPlaceholderImage } from '@/lib/ingest/image-guard';
let passed = 0;
let failed = 0;
function check(name: string, fn: () => void): void {
try {
fn();
passed += 1;
console.log(`ok ${passed} - ${name}`);
} catch (e) {
failed += 1;
console.error(`not ok ${passed + failed} - ${name}`);
console.error(String((e as Error).stack ?? e));
}
}
// The exact shared tile URL stored on 601 production rows (2026-08-18).
const TILE =
'https://lh3.googleusercontent.com/J6_coFbogxhRI9iM864NL_liGXvsQp2Aups' +
'Kei7z0cNNfDvGUmWUy20nuUhkREQyrpY4bEeIBuc=s0-w300';
check('MUST-BLOCK: exact prod shared tile url', () => {
assert.equal(isPlaceholderImage(TILE).blocked, true);
});
check('MUST-BLOCK: -rw resize variant', () => {
assert.equal(isPlaceholderImage(`${TILE}-rw`).blocked, true);
});
check('MUST-BLOCK: lh4 host rotation', () => {
assert.equal(isPlaceholderImage('https://lh4.googleusercontent.com/xyz123=s900').blocked, true);
});
check('MUST-BLOCK: lh2 host rotation', () => {
assert.equal(isPlaceholderImage('https://lh2.googleusercontent.com/xyz123').blocked, true);
});
check('MUST-BLOCK: uppercase host', () => {
assert.equal(isPlaceholderImage('https://LH3.GOOGLEUSERCONTENT.COM/J6_coFbo=s0').blocked, true);
});
check('MUST-BLOCK: extra query string / fragment', () => {
assert.equal(isPlaceholderImage(TILE + '&dummy=1#f').blocked, true);
});
check('MUST-BLOCK: unknown key (not the known tile) still blocked by host rule', () => {
assert.equal(
isPlaceholderImage('https://lh3.googleusercontent.com/someOtherKeyWq9aB=s64').blocked,
true,
);
});
check('MUST-BLOCK: whitespace-wrapped url', () => {
assert.equal(isPlaceholderImage(` ${TILE} `).blocked, true);
});
check('MUST-PASS: wired photo (publisher CDN)', () => {
assert.equal(
isPlaceholderImage('https://media.wired.com/photos/6a84a2c1/191:100/w_1280/c_limit/x.jpg').blocked,
false,
);
});
check('MUST-PASS: cnbcfm photo (publisher CDN)', () => {
assert.equal(isPlaceholderImage('https://media.cnbcfm.com/i/2026/08/robo.jpg').blocked, false);
});
check('MUST-PASS: google.com logo asset (NOT googleusercontent)', () => {
assert.equal(isPlaceholderImage('https://www.google.com/logos/2026/xx512.png').blocked, false);
});
check('MUST-PASS: example.com path segment merely containing the host string', () => {
assert.equal(
isPlaceholderImage('https://cdn.example.com/googleusercontent.com/img.png').blocked,
false,
);
});
check('MUST-PASS: null', () => {
assert.equal(isPlaceholderImage(null).blocked, false);
});
check('MUST-PASS: undefined', () => {
assert.equal(isPlaceholderImage(undefined).blocked, false);
});
check('MUST-PASS: empty string', () => {
assert.equal(isPlaceholderImage('').blocked, false);
});
check('MUST-PASS: not a URL', () => {
assert.equal(isPlaceholderImage('not a url').blocked, false);
});
console.log(`\nimage-guard: ${passed} passed, ${failed} failed`);
if (failed > 0) process.exit(1);
+143
View File
@@ -0,0 +1,143 @@
/**
* Image-proxy helper tests (offline — no network).
*
* Contract under test:
* - sniffImage: magic-byte → Content-Type for jpeg/png/gif/webp/avif/heic/bmp,
* null for HTML blockpages and junk (a hotlink guard returning 200 HTML
* must never be served as an image).
* - isSafeProxyTarget: public http(s) only — no loopback, RFC1918, link-local,
* IPv6 private, or non-web schemes; our own site origin rejected.
* - proxyImageUrl: remote http(s) → /images?url=... ; relative / data: /
* our-own-origin URLs pass through untouched.
*
* Runner: `tsx tests/image-proxy.test.ts` → TAP-style, exit 1 on failure.
*/
import assert from 'node:assert/strict';
process.env.NEXT_PUBLIC_SITE_URL = 'https://technews.krisforbes.ca';
import {
isSafeProxyTarget,
proxyImageUrl,
isPrivateHost,
} from '@/lib/image-mapping';
import { mediaKey, sniffImage } from '@/lib/images';
let passed = 0;
let failed = 0;
function check(name: string, fn: () => void): void {
try {
fn();
passed += 1;
console.log(`ok ${passed + failed} ${name}`);
} catch (err) {
failed += 1;
console.log(`not ok ${passed + failed} ${name}`);
console.log(` ${(err as Error).message}`);
}
}
const b = (b64: string) => Buffer.from(b64, 'base64');
// Minimal valid magic headers (payload irrelevant to the sniff).
const JPEG = b(
'/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoKBwYKDwMNDhgMEggRCwUNDAwTFBMSFBQUFxQVFRUUgAFMAAQHBgMCAwYHBgcKEA0HCAkKDw0NDhERCg0RHREKCA8VEg0RERoNDAwQGiYNDg8VIRUQNBMfISEYGRM0KhwjGhs0MioaIxwkIhgY',
);
const PNG = b('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAACklEQVR4nGMYAQCABQAB6V282gAAAABJRU5ErkJggg==');
const WEBP = b('UklGRlQAAABXRUJQVlA4IBoAAAAwAQCdASoBAAEAAUAmJaQAA3AA/vuUAAA=');
const AVIF = Buffer.concat([Buffer.alloc(4), Buffer.from('ftypavif', 'ascii')]);
const GIF = b('R0lGODlhAQABAIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==');
const HTML = Buffer.from('<!doctype html><html><body><h1>Access Denied</h1><p>The referenced entity does not exist</p>');
check('sniff: jpeg magic → image/jpeg', () => {
assert.equal(sniffImage(JPEG), 'image/jpeg');
});
check('sniff: png magic → image/png', () => {
assert.equal(sniffImage(PNG), 'image/png');
});
check('sniff: webp magic → image/webp', () => {
assert.equal(sniffImage(WEBP), 'image/webp');
});
check('sniff: avif ftyp → image/avif', () => {
assert.equal(sniffImage(AVIF), 'image/avif');
});
check('sniff: gif magic → image/gif', () => {
assert.equal(sniffImage(GIF), 'image/gif');
});
check('sniff: HTML blockpage → null', () => {
assert.equal(sniffImage(HTML), null);
});
check('sniff: short/garbage buffer → null', () => {
assert.equal(sniffImage(Buffer.from('ab')), null);
});
check('ssrf: public https ok', () => {
assert.ok(isSafeProxyTarget('https://ichef.bbci.co.uk/ace/branded_news/x.jpg'));
});
check('ssrf: public http ok', () => {
assert.ok(isSafeProxyTarget('http://example.com/a.webp'));
});
check('ssrf: localhost blocked', () => {
assert.ok(!isSafeProxyTarget('http://localhost:3000/images?url=x'));
});
check('ssrf: 127/10/192.168/169.254 blocked', () => {
assert.ok(!isSafeProxyTarget('http://127.0.0.1/a'));
assert.ok(!isSafeProxyTarget('http://10.0.0.5/a'));
assert.ok(!isSafeProxyTarget('http://192.168.1.2/a'));
assert.ok(!isSafeProxyTarget('http://169.254.169.254/latest'));
});
check('ssrf: 172.16-31 blocked, 172.15/172.32 ok', () => {
assert.ok(!isSafeProxyTarget('http://172.17.0.1:11434/'));
assert.ok(!isSafeProxyTarget('http://172.16.0.9/'));
assert.ok(!isSafeProxyTarget('http://172.31.9.9/'));
assert.ok(isSafeProxyTarget('http://172.15.255.1/')); // below private range
assert.ok(isSafeProxyTarget('http://172.32.0.1/')); // above private range
});
check('ssrf: ipv6 loopback/link-local/ULA blocked, mapped private blocked', () => {
assert.ok(!isSafeProxyTarget('http://[::1]/a'));
assert.ok(!isSafeProxyTarget('http://[fe80::1]/a'));
assert.ok(!isSafeProxyTarget('http://[fd00::1]/a'));
assert.ok(!isSafeProxyTarget('http://[::ffff:127.0.0.1]/a'));
assert.ok(!isSafeProxyTarget('http://[::ffff:192.168.1.2]/a'));
assert.ok(!isSafeProxyTarget('http://[::]/a'));
});
check('ssrf: non-web schemes blocked', () => {
assert.ok(!isSafeProxyTarget('file:///etc/passwd'));
assert.ok(!isSafeProxyTarget('gopher://example.com'));
assert.ok(!isSafeProxyTarget('blob:https://x.com/abc'));
});
check('ssrf: our own origin rejected (no self-proxy)', () => {
assert.ok(!isSafeProxyTarget('https://technews.krisforbes.ca/article/x'));
});
check('ssrf: unparseable url → false', () => {
assert.ok(!isSafeProxyTarget('not a url'));
});
check('proxy: remote url → /images?url=...', () => {
const u = 'https://data-api.investing.com/trkd-images/abc.jpg';
assert.equal(proxyImageUrl(u), `/images?url=${encodeURIComponent(u)}`);
});
check('proxy: relative path passes through', () => {
assert.equal(proxyImageUrl('/covers/tech.svg'), null);
});
check('proxy: data URI passes through', () => {
assert.equal(proxyImageUrl('data:image/png;base64,AAA='), null);
});
check('proxy: own-origin url passes through', () => {
assert.equal(proxyImageUrl('https://technews.krisforbes.ca/logo.png'), null);
});
check('proxy: empty/blank → null', () => {
assert.equal(proxyImageUrl(null), null);
assert.equal(proxyImageUrl(' '), null);
});
check('key: stable + 40 hex chars', () => {
const k1 = mediaKey('https://x.com/a.jpg');
const k2 = mediaKey('https://x.com/a.jpg');
assert.equal(k1, k2);
assert.match(k1, /^[0-9a-f]{40}$/);
assert.notEqual(k1, mediaKey('https://x.com/b.jpg'));
});
const total = passed + failed;
console.log(`\n# tests ${total}, pass ${passed}, fail ${failed}`);
process.exit(failed === 0 ? 0 : 1);
+164
View File
@@ -0,0 +1,164 @@
/**
* LLM synthesis parse-contract unit test (gate stage 3 / Flux 3).
*
* Pinned contract — if a change breaks any of these, the secret→build→test
* gate hard-blocks the commit. Covers the pipeline choke point that EVERY
* provider funnels through: src/lib/llm/parse.ts (pure, zero-dep, zero-net).
*
* Notable pins (2026-08-15 "redo it" cut):
* - body survives up to 6 paragraphs (3→6 cut in the prompt)
* - headline capped at 160 chars
* - takeaways ≤5, tags ≤6, tags sanitized to [a-z0-9-]
* - determinism: same input ⇒ same output (the gate's own principle)
*
* Runner: no framework. `tsx tests/llm-parse.test.ts` → TAP-style, exit 1 on
* any failure. Keeps node deps at zero so the gate never needs a test litany.
*/
import assert from 'node:assert/strict';
process.env.TS_NODE_PROJECT = process.cwd() + '/tsconfig.json';
import {
extractJson,
parseSynthesis,
primaryTag,
safeParse,
type ParsedSynthesis,
} from '@/lib/llm/parse';
let passed = 0;
let failed = 0;
function check(name: string, fn: () => void): void {
try {
fn();
passed += 1;
console.log(`ok ${passed + failed} ${name}`);
} catch (err) {
failed += 1;
console.log(`not ok ${passed + failed} ${name}`);
console.log(` ${(err as Error).message}`);
}
}
const GOOD_JSON = JSON.stringify({
headline: 'Toronto MSTP extension debate heats up',
body: ['P1 fact.', 'P2 context.', 'P3 detail.', 'P4 quote.', 'P5 view.', 'P6 outlook.']
.join('\n\n'),
takeaways: ['a', 'b', 'c', 'd', 'e', 'f'],
tags: ['Transit / Big City!', 'CanAdA', 'overly-long-tag-that-exceeds-character-limit'],
});
check('extractJson: passthrough of strict JSON', () => {
assert.equal(extractJson(GOOD_JSON), GOOD_JSON);
});
check('extractJson: strips ```json fences', () => {
assert.equal(extractJson('```json\n{"headline":"H","body":"B"}\n```'), '{"headline":"H","body":"B"}');
});
check('extractJson: recovers object from verbose prose', () => {
const raw = 'Sure! Here is the synthesis you requested:\n{"headline":"H","body":"B"}\nLet me know if you need anything else!';
assert.equal(extractJson(raw), '{"headline":"H","body":"B"}');
});
check('parseSynthesis: canonical 6-paragraph shape preserved', () => {
const r: ParsedSynthesis = parseSynthesis(GOOD_JSON);
assert.equal(r.headline, 'Toronto MSTP extension debate heats up');
assert.equal(r.body.split(/\n{2,}/).length, 6);
});
check('parseSynthesis: 7 paragraphs trimmed to 6 (cut contract)', () => {
const seven = JSON.stringify({
headline: 'H',
body: Array.from({ length: 7 }, (_, i) => `paragraph ${i}`).join('\n\n'),
});
const r = parseSynthesis(seven);
assert.equal(r.body.split(/\n{2,}/).length, 6, 'body must cap at 6 paragraphs');
assert.ok(!r.body.includes('paragraph 6'), 'seventh paragraph dropped');
});
check('parseSynthesis: takeaways capped at 5', () => {
assert.equal(parseSynthesis(GOOD_JSON).takeaways.length, 5);
});
check('parseSynthesis: tags sanitized, capped at 6, no junk', () => {
const tags = parseSynthesis(GOOD_JSON).tags;
assert.ok(tags.length <= 6);
for (const t of tags) {
assert.match(t, /^[a-z0-9-]{1,24}$/);
}
assert.ok(tags.includes('transit-big-city'));
assert.ok(tags.includes('canada'));
});
check('parseSynthesis: headline capped at 160 chars', () => {
const long = JSON.stringify({ headline: 'x'.repeat(400), body: 'b' });
assert.equal(parseSynthesis(long).headline.length, 160);
});
check('parseSynthesis: falls back to primaryTag when no tags', () => {
const r = parseSynthesis(JSON.stringify({ headline: 'Ottawa Auto Recall Notice', body: 'b' }));
assert.deepEqual(r.tags, ['ottawa']);
});
check('parseSynthesis: missing headline throws (fail before persist)', () => {
assert.throws(() => parseSynthesis(JSON.stringify({ body: 'b' })), /missing headline or body/i);
});
check('parseSynthesis: missing body throws', () => {
assert.throws(() => parseSynthesis(JSON.stringify({ headline: 'h' })), /missing headline or body/i);
});
check('parseSynthesis: array input rejected (array ≠ synthesis object)', () => {
// JSON.parse of "[1,2,3]" is an array (typeof === 'object' in JS), so it
// flows past the object guard and is rejected as missing headline/body.
// Either way it must THROW — garbage in, article marked failed.
assert.throws(() => parseSynthesis('[1, 2, 3]'));
});
check('parseSynthesis: scalar (not-object) JSON throws', () => {
// "42" parses to a number → hits the explicit not-an-object guard.
assert.throws(() => parseSynthesis('42'), /not an object/i);
});
check('parseSynthesis: unparseable JSON throws (caller marks failed)', () => {
assert.throws(() => parseSynthesis('definitely not json'), /Unparseable synthesis JSON/i);
});
check('parseSynthesis: null/empty-string headline treated as missing', () => {
assert.throws(() => parseSynthesis(JSON.stringify({ headline: null, body: 'b' })));
});
check('primaryTag: strips stopwords, keeps first content word', () => {
assert.equal(primaryTag('The announcement of a big fuel increase'), 'announcement');
});
check('primaryTag: all-stopwords/short headline falls back to canada', () => {
// every token is either a stopword or ≤3 chars → no content word → canada
assert.equal(primaryTag('The of and'), 'canada');
});
check('safeParse: valid JSON passes through', () => {
assert.deepEqual(safeParse('[1, 2, 3]'), [1, 2, 3]);
});
check('safeParse: malformed JSON returns fallback', () => {
const fb = { default: 'x' };
assert.equal(safeParse('nope', fb), fb);
});
check('determinism: same input ⇒ byte-identical output (gate principle)', () => {
const a = parseSynthesis(GOOD_JSON);
const c = parseSynthesis(GOOD_JSON);
assert.deepEqual(a, c);
});
check('fenced+prose end-to-end still parses (small-model degradation path)', () => {
const r = parseSynthesis(extractJson('```json\n' + GOOD_JSON + '\n```'));
assert.equal(r.body.split(/\n{2,}/).length, 6);
});
const total = passed + failed;
console.log(`\n# tests ${total}, pass ${passed}, fail ${failed}`);
process.exit(failed === 0 ? 0 : 1);
+21
View File
@@ -0,0 +1,21 @@
{
"compilerOptions": {
"target": "es2020",
"lib": ["dom", "dom.iterable", "esnext"],
"allowJs": true,
"skipLibCheck": true,
"strict": true,
"noEmit": true,
"esModuleInterop": true,
"module": "esnext",
"moduleResolution": "bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "preserve",
"incremental": true,
"plugins": [{ "name": "next" }],
"paths": { "@/*": ["./src/*"] }
},
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
"exclude": ["node_modules"]
}