# Deploying BTS Apps Dashboard

Step-by-step production deployment on a single Linux server (Ubuntu 22.04/24.04) with PostgreSQL, a systemd
service and Nginx in front for HTTPS. This is the recommended setup: one long-running Node process that also runs the
daily health-check scheduler in-process. Other targets are covered at the end.

```
Browser ──HTTPS──▶ Nginx :443 ──▶ Node (next start) :3000 ──▶ PostgreSQL :5432
                                         │
                                         └──HTTP(S)──▶ health APIs of the monitored BTS apps
```

---

## 0. Before you start

- [ ] **Commit and push the code.** The repo has no commits yet and several new files are untracked
      (`prisma/migrations/20260925063320_add_slow_status/`, `src/server/import/`, `src/lib/import/`, …).
      Anything not committed will not reach the server — in particular a missing migration breaks the app.
      ```bash
      git add -A
      git status            # confirm .env is NOT listed (it is git-ignored)
      git commit -m "Initial version"
      git remote add origin <your-git-url>
      git push -u origin main
      ```
- [ ] Run the checks locally once: `npm run check` (and `npm run test:e2e` if you have a local Postgres).
- [ ] You have: a Linux server, a DNS name pointing to it (e.g. `bts-dashboard.example.com`), and SSH access with sudo.
- [ ] The server can reach the monitored apps' API Notifier URLs (outbound HTTP/HTTPS, port 80/443 or whatever the
      URLs use). If the apps are on an internal network, the server must be on it too.
- [ ] The server can reach the domain registries' RDAP servers over HTTPS (port 443) for the daily domain-expiry
      check: `data.iana.org`, `rdap.verisign.com`, `tld-rdap.verisign.com`, `rdap.identitydigital.services`, `rdap.nixiregistry.in`, `pubapi.registry.google`, `rdap.nic.ch`.
      A new domain ending may add another registry host (it is looked up in IANA's list).

---

## 1. Install system packages

```bash
sudo apt update
sudo apt install -y git curl build-essential nginx postgresql postgresql-contrib
```

### Node 20.20.2 (required — Prisma 7 refuses Node 23)

Install via nvm for the user that will run the app (here: a dedicated `bts` user):

```bash
sudo adduser --disabled-password --gecos "" bts
sudo -iu bts
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash
source ~/.bashrc
nvm install 20.20.2
nvm alias default 20.20.2
node -v                       # v20.20.2
which node                    # note this path, systemd needs it (e.g. /home/bts/.nvm/versions/node/v20.20.2/bin/node)
exit
```

---

## 2. Create the database

```bash
sudo -u postgres psql <<'SQL'
CREATE USER bts_dashboard WITH PASSWORD 'CHANGE_ME_STRONG_PASSWORD';
CREATE DATABASE bts_apps_dashboard OWNER bts_dashboard;
SQL
```

> If PostgreSQL runs on a separate host / managed service (RDS, Cloud SQL, …), just create the user + database there
> and use its host in `DATABASE_URL`. Add `?sslmode=require` if the provider requires TLS.

---

## 3. Get the code and install dependencies

```bash
sudo -iu bts
git clone <your-git-url> ~/bts-apps-dashboard
cd ~/bts-apps-dashboard
nvm use                       # picks up .nvmrc → 20.20.2
npm ci                        # full install — devDependencies are needed (prisma CLI, tsx, tailwind)
```

`npm ci` runs `postinstall`, which generates the Prisma client into `src/generated/prisma`.

> Always install on the server itself. Never copy `node_modules` from a Mac — `@node-rs/argon2` ships a
> platform-specific native binary.

---

## 4. Configure the environment

```bash
cp .env.example .env
chmod 600 .env
nano .env
```

Production values:

```dotenv
DATABASE_URL="postgresql://bts_dashboard:CHANGE_ME_STRONG_PASSWORD@localhost:5432/bts_apps_dashboard"
NODE_ENV=production

SESSION_TTL_HOURS=12
HEALTH_CHECK_RETRY_DELAY_MS=3000
HEALTH_CHECK_CONCURRENCY=10

# One long-running server → the scheduler runs inside the app process.
SCHEDULER_MODE=internal
SCHEDULER_TICK_SECONDS=60
# CRON_SECRET is only needed for SCHEDULER_MODE=external (see section 9).
```

| Variable | Required | Notes |
|---|---|---|
| `DATABASE_URL` | **yes** | URL-encode special characters in the password (`@` → `%40`, `#` → `%23`, …). |
| `NODE_ENV` | yes in prod | `production` turns on the `Secure` cookie, HSTS and CSP. `npm start` sets it too, but the scripts below (`db:deploy`, `user:create`) read `.env`. |
| `SESSION_TTL_HOURS` | no | 1–720, default 12. |
| `HEALTH_CHECK_RETRY_DELAY_MS` | no | 0–60000, default 3000. |
| `HEALTH_CHECK_CONCURRENCY` | no | 1–50, default 10. |
| `SCHEDULER_MODE` | no | `internal` (default) / `external` / `off`. |
| `SCHEDULER_TICK_SECONDS` | no | 5–3600, default 60. How often the app *looks* for a due run — not the run frequency. |
| `CRON_SECRET` | only for `external` | ≥ 16 characters. Generate with `openssl rand -hex 32`. |
| `RDAP_BOOTSTRAP_URL` | no | Leave unset (IANA's list of RDAP servers). Only the end-to-end tests change it. |

Invalid values stop the server at boot with a clear `Invalid environment configuration — …` message.

---

## 5. Create the tables and the first user, build

```bash
cd ~/bts-apps-dashboard
npm run db:deploy                    # creates all tables in the empty database
npm run user:create -- admin                   # Super Admin; prompts for the password (hidden)
npm run user:create -- viewer --role viewer    # optional: a view-only login
npm run build
```

- For a non-interactive user creation (e.g. in a script): `BTS_USER_PASSWORD='…' npm run user:create -- admin`.
  Running it again for an existing username resets that user's password (and signs them out); add `--role admin` or
  `--role viewer` to change their role at the same time, otherwise the role is kept.
- **Super Admin** can do everything. **View only** sees the dashboard, apps, app details and domains, but every add /
  edit / delete / import / check button is hidden and refused on the server, and Monitoring Settings is not available.
- Usernames: 3–50 chars, `a-z 0-9 . - _`.

Quick smoke test before wiring up systemd:

```bash
npm start                            # listens on :3000
# in another shell on the server:
curl -s http://localhost:3000/api/health      # → {"status":"ok"}
```

Stop it with `Ctrl+C`.

---

## 6. Run it as a systemd service

Create `/etc/systemd/system/bts-dashboard.service` (as root). Replace the `node` path with the output of `which node`
from step 1.

```ini
[Unit]
Description=BTS Apps Dashboard
After=network-online.target postgresql.service
Wants=network-online.target

[Service]
Type=simple
User=bts
WorkingDirectory=/home/bts/bts-apps-dashboard
EnvironmentFile=/home/bts/bts-apps-dashboard/.env
Environment=NODE_ENV=production
Environment=PORT=3000
Environment=HOSTNAME=127.0.0.1
ExecStart=/home/bts/.nvm/versions/node/v20.20.2/bin/node node_modules/next/dist/bin/next start
Restart=always
RestartSec=5
KillSignal=SIGTERM
TimeoutStopSec=30

[Install]
WantedBy=multi-user.target
```

- `HOSTNAME=127.0.0.1` makes Node listen only locally; Nginx is the only public entry point.
- Run **one** instance. Several instances in `internal` mode are safe (the database lock guarantees one run per slot)
  but wasteful.

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now bts-dashboard
sudo systemctl status bts-dashboard
journalctl -u bts-dashboard -f       # logs: one JSON object per line
```

---

## 7. Nginx + HTTPS

HTTPS is **mandatory**: in production the session cookie is `Secure`, so logging in over plain HTTP silently fails
(except on `localhost`).

Create `/etc/nginx/sites-available/bts-dashboard`:

```nginx
server {
    listen 80;
    server_name bts-dashboard.example.com;

    # Sheet imports are up to 4 MB; server actions accept 5 MB.
    client_max_body_size 6m;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;

        proxy_set_header Host              $host;
        # OVERWRITE (don't append) the client IP: the login rate limiter trusts the first
        # X-Forwarded-For entry, so appending would let clients spoof it.
        proxy_set_header X-Forwarded-For   $remote_addr;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host  $host;

        # "Check Now" waits for the probe: up to 180 s timeout + one retry.
        proxy_read_timeout 400s;
    }
}
```

```bash
sudo ln -s /etc/nginx/sites-available/bts-dashboard /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

# Free TLS certificate (adds the 443 server block and the HTTP→HTTPS redirect)
sudo apt install -y certbot python3-certbot-nginx
sudo certbot --nginx -d bts-dashboard.example.com
```

> Behind another load balancer / Cloudflare? Then `$remote_addr` is the balancer's IP. Use Nginx's `real_ip` module
> (`set_real_ip_from <lb range>; real_ip_header X-Forwarded-For;`) so `$remote_addr` becomes the real client again.
> Keep `X-Forwarded-Host` equal to the public host — Server Actions reject requests whose Origin doesn't match it.

Firewall:

```bash
sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw enable                      # port 3000 and 5432 stay closed to the outside
```

---

## 8. Verify the deployment

- [ ] `curl -s https://bts-dashboard.example.com/api/health` → `{"status":"ok"}`
- [ ] Open the site → redirected to `/login` → log in with the user from step 5 (sidebar shows **Super Admin**).
- [ ] If you created a view-only user: log in as it in a private window → sidebar shows **View only**, there is no
      Monitoring Settings link and no Add / Edit / Delete / Check buttons.
- [ ] **Settings** → confirm the schedule. A fresh install runs health checks **every 12 hours** at 09:00 and 21:00
      Asia/Kolkata; change the check time / **timezone** / frequency (Every 12 hours or Daily) / slow-response
      threshold if needed and save. (Saving never triggers a run; the next slot after the save will.)
- [ ] **Apps** → add an app (or **Import Apps** from the sheet) → it is checked in the background within seconds.
- [ ] Click **Check Now** on one app and **Check All** on the dashboard; the Settings page lists the run.
- [ ] **Domains** → click **Check Domains Now** → every domain gets an expiry date (or a lookup error explaining why
      not). `.ch` domains have no published expiry: click **Set expiry date** and copy it from the registrar's account.
- [ ] Response headers include `Strict-Transport-Security` and `Content-Security-Policy`
      (`curl -sI https://bts-dashboard.example.com/login`).
- [ ] `journalctl -u bts-dashboard` shows no `level":"error"` lines.

---

## 9. Scheduler options

**How often checks run** is not an env variable: it is the *Check Frequency* on the Settings page (stored in the
database). Default: **every 12 hours**, anchored at the check time (09:00 → 09:00 and 21:00). Daily is the other option.
The settings below only decide *who* triggers the scheduler.

The **domain-expiry check** rides on the same scheduler: once a day at the configured check time (e.g. 09:00
Asia/Kolkata), as its own run with its own lock, so it never delays health checks. `SCHEDULER_MODE` applies to both.

| Situation | Setting |
|---|---|
| One long-running server (this guide) | `SCHEDULER_MODE=internal` — nothing else to do. |
| Serverless / auto-scaled / several instances | `SCHEDULER_MODE=external` + `CRON_SECRET`, and a cron calls the tick endpoint. |
| Temporarily stop scheduled checks | `SCHEDULER_MODE=off` (manual Check Now / Check All still work). |

External cron example (every 5 minutes; the tick is idempotent, so calling it often is safe):

```bash
# crontab -e
*/5 * * * * curl -fsS -X POST -H "Authorization: Bearer <CRON_SECRET>" https://bts-dashboard.example.com/api/cron/tick > /dev/null
```

Without the correct secret the endpoint answers `404`.

---

## 10. Backups and maintenance

Daily database dump (as the `postgres` user, kept 14 days):

```bash
sudo mkdir -p /var/backups/bts-dashboard && sudo chown postgres /var/backups/bts-dashboard
sudo -u postgres crontab -e
```

```cron
30 2 * * * pg_dump -Fc bts_apps_dashboard > /var/backups/bts-dashboard/$(date +\%F).dump && find /var/backups/bts-dashboard -name '*.dump' -mtime +14 -delete
```

Restore: `pg_restore --clean --if-exists -d bts_apps_dashboard /var/backups/bts-dashboard/<date>.dump`

- Copy backups off the server too.
- No table pruning is needed: ~36 500 `health_checks` rows/year for 100 apps; sessions and login attempts prune
  themselves.
- Point any uptime monitor at `GET /api/health` (200 = up, 503 = database unreachable).
- Manage users any time with `npm run user:create -- <username> [--role admin|viewer]` from the app directory
  (creates or resets).

---

## 11. Other targets (short notes)

**Docker / container platforms (Render, Railway, Fly.io, ECS, …)** — same steps inside the image:
`node:20.20.2-bookworm-slim` base, `npm ci`, `npm run build`; on start run `npm run db:deploy` then
`node node_modules/next/dist/bin/next start`. Provide env vars through the platform, terminate HTTPS at the platform,
and use `SCHEDULER_MODE=external` if the platform may run more than one instance or sleeps idle containers.
Run `npm run user:create` once via the platform's shell/one-off job.

**Vercel / serverless** — works with caveats: there is no long-running process, so set `SCHEDULER_MODE=external`
and use a cron (Vercel Cron or any external one) to `POST /api/cron/tick` with the bearer secret; use a pooled
Postgres connection string; function time limits must cover "Check All" (bulk runs continue in the background
from the claiming request, which serverless platforms may cut off — a container/VM is the better fit).

---

## 12. Backend API surface (current)

The app has **no public REST API** apart from two route handlers. Everything the UI does goes through Next.js
**Server Actions**: `POST` requests to the page URL itself (identified by a `Next-Action` header), protected by the
session cookie + Next's built-in Origin check. They are not meant to be called by other clients.

### HTTP route handlers

| Method & path | Auth | Purpose | Responses |
|---|---|---|---|
| `GET /api/health` | Public | Self health probe for load balancers / uptime monitors. Runs `SELECT 1`. | `200 {"status":"ok"}` · `503 {"status":"unavailable"}` |
| `POST /api/cron/tick` | `Authorization: Bearer <CRON_SECRET>` | Runs the scheduler tick and the daily domain-check tick (only needed with `SCHEDULER_MODE=external`). Idempotent. | `200` health tick result, with the domain tick under `domains` · `404` wrong/missing secret or `CRON_SECRET` unset · `500` tick failed |

Any other `/api/*` path without a session cookie gets `401 {"error":"Unauthorized"}` (`src/proxy.ts`).

### Server Actions

All except login call `requireUser()` first; all except login and logout then refuse a **View only** user
(`canEdit()`, `src/lib/auth/roles.ts`).

| Action | File | Used by | What it does |
|---|---|---|---|
| `loginAction` | `src/app/(auth)/login/actions.ts` | `/login` | Validates credentials, rate-limits per username/IP, creates the session, redirects to `/`. |
| `logoutAction` | `src/app/(dashboard)/actions.ts` | Sidebar | Destroys the session, redirects to `/login`. |
| `checkAllAction` | `src/app/(dashboard)/actions.ts` | Dashboard, Apps | Claims a manual run (same lock as the scheduler) and checks all enabled apps in the background. |
| `createProjectAction` | `src/app/(dashboard)/apps/actions.ts` | `/apps/new` | Creates an app, then health-checks it in the background. |
| `updateProjectAction` | `src/app/(dashboard)/apps/actions.ts` | `/apps/[id]/edit` | Updates an app; re-checks it if the URL / expected status changed. |
| `setMonitoringAction` | `src/app/(dashboard)/apps/actions.ts` | Monitoring toggle | Enables/disables monitoring; re-checks on enable. |
| `deleteProjectAction` | `src/app/(dashboard)/apps/actions.ts` | Delete button | Soft-deletes an app (history kept). |
| `checkNowAction` | `src/app/(dashboard)/apps/actions.ts` | Check Now button | Runs one health check and waits for the result. |
| `previewImportAction` | `src/app/(dashboard)/apps/actions.ts` | Import Apps (step 1) | Parses an uploaded `.csv`/`.tsv`/`.xlsx` (≤ 4 MB) and classifies rows. Saves nothing. |
| `importProjectsAction` | `src/app/(dashboard)/apps/actions.ts` | Import Apps (step 2) | Creates/updates the selected rows (≤ 2000), then checks them in the background. |
| `updateSettingsAction` | `src/app/(dashboard)/settings/actions.ts` | `/settings` | Saves frequency, check time, timezone and slow-response threshold. |
| `checkDomainsAction` | `src/app/(dashboard)/domains/actions.ts` | `/domains` | Claims a manual domain run (same lock as the daily check) and looks every domain up in the background. |
| `createDomainAction` | `src/app/(dashboard)/domains/actions.ts` | `/domains/new` | Adds a domain no app uses (registrar, optional manual expiry date), then looks it up in the background. |
| `updateDomainAction` | `src/app/(dashboard)/domains/actions.ts` | `/domains/[id]/edit` | Saves a domain's registrar and hand-entered expiry date (for registries that publish none, e.g. `.ch`). |
| `deleteDomainAction` | `src/app/(dashboard)/domains/actions.ts` | Delete button | Deletes a domain added by hand; hides (restorably) a domain an app still uses. |
| `restoreDomainAction` | `src/app/(dashboard)/domains/actions.ts` | Hidden domains | Un-hides a domain so it is listed and checked again. |

Read-only data (dashboard, app lists, history, settings, run log) is loaded directly by server-rendered pages
(`/`, `/apps`, `/apps/[id]`, `/apps/[id]/edit`, `/apps/new`, `/settings`, `/settings/runs/[id]`, `/domains`, `/domains/new`, `/domains/[id]/edit`) from `src/server/**` — there are no
GET endpoints for it.

### Outbound calls

| Target | When |
|---|---|
| Each monitored app's **API Notifier URL** (`GET`, per-app timeout, no redirects followed, one retry on failure) | Scheduled runs, Check All, Check Now, and background checks after create / enable / URL change / import. |
| `https://data.iana.org/rdap/dns.json` (IANA's list of which registry serves which domain ending) | Before domain lookups; cached for 7 days. |
| Each domain's **registry RDAP server** (`GET <server>/domain/<name>`, 20 s timeout, one retry on 429 / 5xx / network error, 3 at a time) | Daily domain check, Check Domains Now, and after an app on a not-yet-seen domain is added / imported. |
| PostgreSQL (`DATABASE_URL`) | Everything else. |

No other third-party services are called.

---

## Troubleshooting

| Symptom | Cause / fix |
|---|---|
| `Invalid environment configuration — …` at start | Fix the named variable in `.env`, restart. |
| Login form returns to login, no error | Site served over plain HTTP → the `Secure` cookie is dropped. Use HTTPS. |
| Forms / buttons fail with an "invalid Server Actions request" error | The proxy changes the Host/Origin. Pass `Host` and `X-Forwarded-Host` = public domain (section 7). |
| Prisma errors about the Node version | Wrong Node in systemd `ExecStart` — point it to v20.20.2. |
| `Cannot find module '…/generated/prisma'` | Run `npm run db:generate` (or `npm ci`) then rebuild. |
| All apps show Not Working (DNS / connection) | The server can't reach the monitored URLs — check outbound firewall / VPN / internal DNS. |
| Import upload fails with 413 | Raise `client_max_body_size` in Nginx. |
| Every domain shows a lookup error such as `fetch failed (ENOTFOUND)` or `no answer within 20 s` | The server can't reach the RDAP hosts listed in section 0 — check the outbound firewall / proxy. |
| Everyone locked out after a few failed logins | Nginx isn't setting `X-Forwarded-For`, so all users share one IP bucket (section 7). |