# BTS Apps Dashboard

Internal dashboard that monitors the production Health API of all BTS applications.
Requirements: `../BTS Apps Dashboard — Development Requirements Prompt.md`.

## Stack

Next.js 16 (App Router) · React 19 · TypeScript · Tailwind CSS 4 · PostgreSQL · Prisma 7

## Local setup

Requires Node 20.19+ (see `.nvmrc`; Prisma does not support Node 23) and a local PostgreSQL.

```bash
nvm use
npm install
cp .env.example .env          # then set DATABASE_URL
createdb bts_apps_dashboard
npm run db:deploy             # apply migrations
npm run user:create -- admin  # prompts for a password
npm run dev                   # http://localhost:3000
```

Other machines on the office network can open `http://<your LAN IP>:3000` (the IP is printed as "Network" when the
dev server starts). The dev server only serves its assets to hostnames listed in `allowedDevOrigins` in `next.config.ts` —
private-network IP ranges and `*.local` names are already listed; add yours if your network uses a different range.
A production build (`npm run build && npm start`) has no such restriction.

## Scripts

| Script | Purpose |
|---|---|
| `npm run dev` / `build` / `start` | Next.js |
| `npm run lint` / `typecheck` | ESLint / `tsc --noEmit` |
| `npm test` | Unit tests (vitest) — pure logic, no database |
| `npm run test:e2e` | End-to-end tests against a dedicated `…_test` database (see Testing) |
| `npm run check` | typecheck + lint + unit tests |
| `npm run db:migrate -- --name <name>` | Create + apply a migration (dev), then run `db:generate` |
| `npm run db:deploy` | Apply pending migrations |
| `npm run db:generate` | Regenerate the Prisma client into `src/generated/prisma` |
| `npm run user:create -- <username> [--role viewer]` | Create a user (Super Admin by default, or view-only) or reset their password |

> Scripts call binaries as `node node_modules/...` on purpose: the parent folder
> `Web:Backend` contains a colon, which breaks npm's `node_modules/.bin` PATH lookup.

## Layout

```
prisma/                schema + migrations
scripts/               CLI utilities (create-user)
src/proxy.ts           optimistic "no cookie → /login" redirect
src/server/            backend modules (no React) — db, env, logger, auth/, projects/, health/, import/, dashboard/, domains/
src/lib/validation/    zod schemas shared by browser forms and server actions
src/lib/import/        pure sheet → import-candidate logic (header detection, new/existing/duplicate/invalid)
src/lib/domains/       pure domain-expiry logic (URL → registered domain, RDAP parsing, expiry status, ordering)
src/app/(auth)/        login
src/app/(dashboard)/   authenticated pages, all guarded by requireUser()
src/components/        ui primitives + layout
```

Backend logic lives in `src/server` and is only called from server actions / route
handlers, so it can be lifted into a standalone service later.

## Auth model

- Argon2id password hashes; users are created via `npm run user:create`.
- Two roles (`users.role`):
  - **Super Admin** (`ADMIN`, the default): everything.
  - **View only** (`VIEWER`, `--role viewer`): sees the dashboard, apps, app details and domains.
    Cannot add, edit, delete, import, enable/disable monitoring, start checks (Check Now / Check All / Check Domains Now),
    or open Monitoring Settings (including run history).
  - Every mutating server action checks `canEdit()` (`src/lib/auth/roles.ts`) and returns a "view-only" error, so the
    rule holds even if a request bypasses the UI. Admin-only pages call `requireAdmin()` and are a 404 for viewers;
    the shared pages just leave the buttons out.
- Opaque random session token in an HTTP-only, SameSite=Lax cookie; only its SHA-256 is stored (`sessions`).
- Absolute session lifetime `SESSION_TTL_HOURS` (default 12).
- Login rate limit: 5 failures per username / 20 per IP per 15 minutes (`login_attempts`).
- CSRF: mutations go through Server Actions (built-in Origin check) plus SameSite cookies.

## Applications

- Managed under **Apps**: add, edit, enable/disable monitoring, delete.
- Validation runs in the browser and again on the server with the same zod schema (`src/lib/validation/project.ts`).
- Delete is a soft delete (`deleted_at`); history is kept and every read filters `deleted_at IS NULL`.
- Project Code is unique (case-insensitive) among active apps only, enforced by the partial index
  `projects_project_code_active_key` (raw SQL in the `add_projects` migration), so a deleted app's code can be reused.
- The per-app timeout defaults to 60 s (max 180 s). It must stay above the slow-response threshold (below), otherwise a slow API is
  cut off as Not Working (timeout) before it can be observed as Slow.

### Import from a sheet

**Import Apps** (Apps page and dashboard) uploads a `.csv` / `.tsv` or `.xlsx` export of the sheet. The first worksheet is read;
the header row (searched within the first 20 rows, so title rows are fine) must contain **Project Code**, **Project Name** and
**API Notifier URL** — matched loosely and by name, so column order and spelling like `api_notifier_url` or `URL` also work.

1. **Preview** (`previewImportAction` → `src/server/import/service.ts`): nothing is saved. Every row is validated with the same
   zod rules as the Add form and classified as *new*, *existing* (Project Code already on the dashboard, case-insensitive — with
   a "will be updated" / "no changes" note), *duplicate* (same code as an earlier row in the file) or *invalid* (with the reason).
2. **Import selected**: new rows are ticked by default, existing rows only when they differ. Existing apps get their name and
   URL updated in place (id and code casing kept); new apps are created with monitoring on and the defaults. Rows are
   re-validated and de-duplicated server-side, each row is its own unit of work, and the result reports added / updated /
   already up to date / failed. Created and re-targeted apps are health-checked in the background straight away.

The parsers (`src/server/import/spreadsheet.ts`) are dependency-free: RFC 4180 CSV with delimiter detection, and a minimal
`.xlsx` reader (zip + shared strings + inline/numeric/formula cells) — the published `xlsx` package carries unfixed advisories
and only text cells are needed. Files are capped at 4 MB / 2000 rows (`experimental.serverActions.bodySizeLimit` = 5 MB).

## Health monitoring

`src/server/health/` — every check, manual or scheduled, takes the same path:

| File | Role |
|---|---|
| `checker.ts` | Pure HTTP probe (no DB, no Next): GET, per-app timeout, `redirect: "manual"`, response time, error classification. Never throws. One retry after `HEALTH_CHECK_RETRY_DELAY_MS` (default 3000) before an app counts as Not Working. A correct answer slower than the slow-response threshold is `SLOW` (with `error_type = SLOW_RESPONSE` and a note) and is not retried. |
| `transition.ts` | Pure "Not Working Since" rules: `failure_started_at` is set by the first failed check and never overwritten; recovery clears it and sets `last_recovered_at`. Slow counts as up. |
| `recorder.ts` | One transaction: lock the project row, insert the `health_checks` row, update the project's current status. Results for apps deleted/disabled mid-flight are discarded. Uses raw SQL so `updated_at` only changes on config edits. |
| `runner.ts` | `checkProject` (Check Now), `checkAllEnabledProjects` (bulk, `HEALTH_CHECK_CONCURRENCY` at a time, each app isolated), `checkProjectQuietly` (background). |

- Working = response received **and** status equals the app's expected status. Anything else (4xx/5xx, 3xx, timeout, DNS, connection, TLS, other) is Not Working with `error_type` + `error_message` stored.
- **Slow Response** = Working, but the response took longer than the **slow-response threshold** in Monitoring Settings (default 30 s). It is a third stored status (`SLOW`): shown in amber everywhere, counted separately on the dashboard, filterable, but treated as *up* for outage tracking (`last_successful_check_at` is set, "Not Working Since" is not). Changing the threshold applies from the next check.
- Response time is time-to-headers in ms; stored for HTTP responses and timeouts, null for DNS/connection/TLS failures.
- Apps are checked automatically in the background when created (also by import), re-enabled, or when their URL / expected status changes.
- Displayed status (`src/lib/status.ts`): Working, Slow Response, Not Working, Monitoring Disabled (always wins), Not checked yet.

## Dashboard & history

- `/` — six summary cards (click to filter), then analytics: **Health overview** (share of checked apps responding + proportional status bar), **Check outcomes** (stacked columns per day for the last 14 days in the display timezone, with a screen-reader table), **Response times** (average / median / slowest of responding apps + buckets), **Needs attention** (outages, longest first) and **Slowest responses**. Below: status tabs, search, sortable columns, Refresh, Check All, per-row Check Now. All view state is in the URL (`?status=&q=&sort=&dir=&page=`, see `src/lib/dashboard-query.ts`). Default order puts problems first: Not Working → Slow → Not checked yet → Working → Monitoring Disabled.
- `/apps` — the same status tabs (`?status=working|slow|not-working|pending|disabled`) combined with search, plus Import Apps and Check All.
- A disabled app is counted and shown only as Monitoring Disabled; its last (stale) result is hidden in the table and labelled "Last Known Status" on its details page.
- `/apps/[id]` — current status (with a red outage banner or an amber "response time too long" banner), 30-day stats (availability, check counts by outcome, average / slowest response), a timeline strip of the latest 60 outcomes, configuration, and paginated health-check history (25 per page) filterable by Working / Slow / Not Working.
- `src/server/dashboard/service.ts` uses raw SQL for the custom status ordering; sort keys map to fixed SQL fragments, never to user input.

## Domain expiry

`/domains` lists the registered domain of every monitored app's health API URL (`api.calai.twilbp.com` → `twilbp.com`, via the public suffix list in `tldts`), with its registration, expiry and last-changed dates, nearest expiry first. Apps sharing a domain share one row; apps on an IP address or `localhost` are named below the table. Highlighting only — nothing is sent (spec §37).

- **Lookup** (`src/server/domains/rdap-client.ts`): RDAP, the registries' own JSON service. The server for each ending comes from IANA's bootstrap file (cached 7 days), with a small built-in list for endings IANA doesn't list (`.io`, `.me`, `.ch`, `.li` in `src/lib/domains/rdap.ts`). `GET <server>/domain/<name>`, 20 s timeout, one retry on 429 / 5xx / network errors. The `events` array gives `registration`, `expiration`, `last changed` and `last update of RDAP database`.
- **Outcomes** (`domains.last_lookup_result`): FOUND, NOT_FOUND (registry 404 → "Not Registered"), UNSUPPORTED (no RDAP server for the ending), FAILED. A failed lookup keeps the last good dates.
- **Status** (`src/lib/domains/status.ts`, derived on every read): Expired, Critical (≤ 7 days), Expiring Soon (≤ 30 days), OK, Unknown (no expiry available), Not checked yet. Registry statuses `redemption period`, `pending delete`, `pending restore`, `client hold`, `server hold` raise a domain to at least Critical, `pending transfer` to at least Expiring Soon; the usual `client … prohibited` registrar locks are ignored. Order: Not Registered → Expired → Critical → Unknown → Not checked yet → Expiring Soon → OK, then by nearest expiry.
- **Watched domains** = domains of monitored apps + domains added by hand (**Add Domain**, `/domains/new`), minus hidden ones. Typed input (`www.x.com`, a URL) is reduced to the registered domain.
- **Edit** (`/domains/[id]/edit`): registrar / account and a manual expiry date. `.ch` publishes no expiry date anywhere, so such rows link to **Set expiry date**; a registry date always wins over a manual one. The name is fixed.
- **Delete**: a domain added by hand is deleted; a domain an app still uses is *hidden* (`hidden_at`) — not listed, checked or counted on the dashboard — and listed under **Hidden domains** with **Restore**.
- **Filters** (URL state, `src/lib/domains/query.ts`): status tabs (All, Needs Attention, Expired / Not Registered, Critical, Expiring Soon, Unknown / Not checked, OK; the summary cards link to them too), source (used by apps / added manually) and search over domain, registrar and app code / name.
- **When**: daily at the check time from Monitoring Settings (`runDomainTick`, `src/server/domains/run.ts`), as its own run in `domain_runs` with the same lock design as `scheduler_runs`; **Check Domains Now** on the page; and right after an app on a never-checked domain is created, re-targeted, re-enabled or imported. 3 lookups at a time.
- The dashboard shows a red / amber callout when any domain is Expiring Soon or worse, and each app's page shows its domain and expiry.

## Time handling

- Everything is stored as `timestamptz` and the DB session is pinned to UTC in `src/server/db-client.ts` — always construct the client through `createPrismaClient()`. Without the pin, the pg driver adapter stores Dates shifted by the session's UTC offset.
- Display goes through `src/lib/format.ts` (Asia/Kolkata until Monitoring Settings lands), built from `formatToParts` so server and browser render identical strings.

## Scheduler & settings

- **Monitoring Settings** (`/settings`): frequency (Every 12 hours — the default — or Daily), check time, timezone and the slow-response threshold (seconds) — stored in the `monitoring_settings` singleton. The timezone is also the display timezone for every date. Saving only the threshold does not move the schedule anchor. The page also lists the last 10 check runs (the execution log, with working / slow / not working counts).
- **Slots** (`src/lib/schedule.ts`, pure): a schedule is "every N hours anchored at HH:mm in a timezone"; Daily is N = 24, every 12 hours is N = 12 (default 09:00 → runs at 09:00 and 21:00). Hourly / 6 h are already understood by the calculator and the DB enum — offering one in the UI means adding it to `ENABLED_FREQUENCIES` in `src/lib/validation/settings.ts`.
- **Tick** (`src/server/scheduler/run.ts`): `runSchedulerTick()` runs the most recent slot if it is after the last settings save (so saving never fires a run) and hasn't run yet — however late, so a slot missed during downtime is caught up. Safe to call from anywhere, any number of times.
- **Lock = `scheduler_runs`**: `slot_key` is unique (a slot runs once, even with several instances) and a partial unique index allows only one `RUNNING` row (scheduled runs and **Check All** never overlap). A run whose heartbeat is older than 5 minutes is treated as crashed and marked FAILED.
- **Who ticks** (`SCHEDULER_MODE`): `internal` (default) — `src/instrumentation.ts` starts an in-process timer every `SCHEDULER_TICK_SECONDS` (60); `external` — a cron service POSTs `/api/cron/tick` with `Authorization: Bearer $CRON_SECRET`; `off`.
- **Check All** (dashboard and Apps page): claims a manual run and returns immediately; both pages show a progress banner and refresh themselves every 3 s until the run ends.

## Testing

**Unit** — `npm test` (`tests/unit`, ~6 s): health checker against a real local HTTP server (200/4xx/5xx, redirect, timeout, slow response vs threshold, DNS, refused, reset, TLS, retry), the "Not Working Since" rules, slot arithmetic (timezones, DST, all frequencies, catch-up), validation, status derivation, URL state, date formatting, the CSV / `.xlsx` parsers (`tests/fixtures/import-sample.xlsx`) and import classification.

**End-to-end** — `npm run test:e2e` (`tests/e2e`, ~4 min). The runner creates/migrates/empties a separate database named `<dev database>_test` (it refuses any database whose name doesn't end in `_test`, because the suites TRUNCATE tables), builds the app, starts `next start` on :3100 with a 5 s scheduler tick plus a fake health-API server on :4545, and drives the real server over HTTP — pages, form posts and server actions — asserting on HTML and on the database. Your development data is never touched. Needs `psql` on PATH.

```bash
npm run test:e2e                          # everything
npm run test:e2e -- --skip-build          # reuse the last build
npm run test:e2e -- --skip-build health   # one suite: auth | apps | import | health | dashboard | scheduler | domains
```

| Spec §40 test | Covered in |
|---|---|
| Healthy API, HTTP 4xx, HTTP 5xx, Timeout, DNS / Connection / SSL failure | unit `checker.test.ts`, e2e `health` |
| API recovery, Multiple failed days | unit `transition.test.ts`, e2e `health`, `dashboard` |
| Monitoring disabled, Deleted application | e2e `apps`, `health`, `dashboard`, `scheduler` |
| Duplicate Project Code | e2e `apps` (app check, case-insensitive, and the DB index itself) |
| Authentication, Unauthorized access | e2e `auth` (+ anonymous checks in every suite) |
| Scheduler failures | e2e `scheduler` (crashed run, busy lock, duplicate slot, racing instances) |
| Multiple applications checked independently | e2e `scheduler` (failing + slow + corrupt app in one run, concurrency timing) |
| Slow response (threshold, no outage, history, filters) | unit `checker.test.ts`, `transition.test.ts`, e2e `health`, `dashboard` |
| Import from sheet (preview, selective import, updates, duplicates, background checks) | unit `import.test.ts`, e2e `import` |
| Domain expiry (lookup, retry, failures, status, ordering, manual date, add / edit / hide / restore / delete, filters, daily run, lock) | unit `domains.test.ts`, e2e `domains` (fake RDAP registry on the fake-API server) |

## Running in production

Deployment-specific packaging (Docker, a platform config) is intentionally not included yet. The app itself is a standard Node server:

```bash
npm ci
npm run db:deploy                 # apply migrations
npm run user:create -- <username> # first user
npm run build && npm start        # PORT=3000 by default
```

- **Environment**: see `.env.example`. Only `DATABASE_URL` is required. Invalid values stop the server at boot with a clear message (`src/server/env.ts`).
- **HTTPS**: in production the session cookie is `Secure` and HSTS + a Content-Security-Policy are sent, so serve it over HTTPS (browsers make an exception for `http://localhost`).
- **Reverse proxy**: login rate limiting reads the client IP from `X-Forwarded-For` / `X-Real-IP`. Make sure your proxy sets (and overwrites) that header, otherwise all users share one IP bucket or a client can spoof it.
- **Scheduler**: one long-running instance → keep `SCHEDULER_MODE=internal`. Serverless or several instances → `external` + a cron hitting `POST /api/cron/tick` (several instances in `internal` mode are also safe — the database lock guarantees one run per slot — just wasteful).
- **Self health probe**: `GET /api/health` → `{"status":"ok"}` (200) or 503 when the database is unreachable. Public, reveals nothing else.
- **Logs**: one JSON object per line on stdout/stderr (`event`, `level`, `time`, …); passwords, tokens, cookies and secrets are redacted by key. Point your log collector at the process output.
- **Data growth**: one `health_checks` row per app per check (100 apps daily ≈ 36 500 rows/year) — no pruning needed at this scale; the `(project_id, checked_at DESC)` index keeps history pages fast. `sessions` and `login_attempts` prune themselves on login.
- **Known audit note**: `npm audit` reports `deepmerge-ts` via Prisma's CLI config loader. It only ever merges this repo's own `prisma.config.ts` (no user input reaches it) and the only "fix" is downgrading Prisma, so it is accepted. `mysql2` (also CLI-only) is pinned to a patched version through `overrides`.