# Project Engineering Instructions

## 1. Project Overview

Insta Save is a Python 3.9 Dockerized FastAPI backend for a mobile/web client. Its primary operation accepts an Instagram (or supported Threads) URL and a device identifier, resolves publicly accessible media through several third-party providers, and returns direct media URLs with available creator metadata. It also exposes AI-assisted caption, hashtag, transcription, and hook-extraction endpoints for uploaded media, plus IP-restricted secret-encryption endpoints.

The runtime is synchronous work inside `async` FastAPI handlers. `python -m app` starts Uvicorn; the Docker entrypoint additionally starts Tor and `python -m api.scheduler` for Apify quota rotation. MySQL is both configuration/state storage and analytics storage. External boundaries include Instagram, provider sites/APIs, Apify, Tor, Selenium/Chromium, Gemini, and Groq.

Key dependencies are FastAPI/Uvicorn, `requests`, Instaloader, yt-dlp, Selenium and webdriver-manager, MySQL Connector, Stem/Tor, Tenacity, Google GenAI, Cryptography/Fernet, and `schedule`. Dependencies are listed only in `requirements.txt`; there is no pyproject, lockfile, lint configuration, CI workflow, or committed automated test suite.

## 2. Repository Architecture

### Application and API

- `app/main.py` creates the FastAPI application, includes `app.api.v1.router.api_router`, registers a catch-all 500 handler, and registers a startup hook that seeds environment secrets into MySQL. `app/__main__.py` runs Uvicorn using `settings.host` and `settings.port`. `api/__main__.py` is an equivalent legacy launcher; `api.__getattr__('app')` preserves a legacy app import.
- `app/api/v1/endpoints/instagram.py` is the HTTP boundary. It defines unprefixed multipart POST routes: `/download_media`, `/frontend_success`, `/trendy_captions`, `/trendy_hashtags`, `/groq_caption`, `/groq_hashtags`, `/transcribe`, and `/extract_hook`. `download_media` and `frontend_success` accept optional `deviceType` form values `1` (iOS) or `2` (Android); omission remains compatible with installed legacy clients. It delegates business logic to `app.services.instagram_service`.
- `app/api/v1/endpoints/health.py` exposes unauthenticated `GET /api/health`. It delegates response construction to `app.services.health_service`, returning HTTP 200 only when MySQL accepts `SELECT 1`, otherwise HTTP 503. Its response intentionally contains app name/environment, per-check status and elapsed milliseconds, and a UTC timestamp; it must never return database credentials or raw connection errors.
- `app/api/v1/endpoints/crypto.py` exposes JSON `/crypto/encrypt` and `/crypto/decrypt`. The entire router uses `require_crypto_allowed_ip`; response shape is `ApiResponse` (`code`, optional `data`, optional `message`). `ip_allowlist.py` accepts either the direct client IP or the first `X-Forwarded-For` IP, so proxy deployment must ensure that header cannot be spoofed by untrusted clients.
- `app/api/v1/endpoints/auth.py` exposes encrypted `POST /auth/guest-login` (`deviceId`, `deviceType?`), `/auth/google` and `/auth/apple` (`token`, `deviceType?`), `/auth/auto-login` and `/auth/logout`; `app/api/v1/endpoints/history.py` exposes plain-JSON `GET /history`, `POST /history` (`url`, `item` object ≤ 64 KB), `POST /history/delete`, `POST /history/import` (up to 200 `{url, item}` oldest first; keeps order, an existing URL keeps its place so retries add nothing) and `DELETE /history` (`?scope=web` deletes only entries whose `item.source` is not `app`, keeping those the mobile app uploaded). Sessions (`app.services.session_service`, `app.repositories.account_repository`): every client holds an opaque `Authorization: Bearer` token (only its SHA-256 is stored in `app_sessions`). A guest is one `app_users` row per device (`guest_device_id`, unique); an account is keyed by the auth service user id, with its `auth_key` stored Fernet-encrypted (`APP_ENCRYPTION_KEY`). Google/Apple login sent with a guest token moves that guest's `user_download_history` rows to the account (newest item wins per URL), clears `guest_device_id`, sets `merged_into_user_id` and revokes the guest's sessions, so the device's next guest login creates a new, empty guest. Auto login re-checks an account's `auth_key` with the auth service `GET /api/v2/user` (rejected → session revoked, 401; auth service unreachable → still logged in). Logout revokes only that session. History is capped at the newest 200 items per user. `app.dependencies.auth.get_current_session` protects endpoints (401 `{code, data: null, message}` via `NotLoggedIn`). App ↔ backend traffic on `/auth/*` is AES-256-GCM encrypted by `app.core.client_crypto` (key = SHA-256 of `CLIENT_ENCRYPTION_KEY`, data = base64(IV 12 | tag 16 | ciphertext), same as the AiNoteTracker Flutter `CryptoUtils`/PHP `aesEncrypt`): the request body (`text/plain`, a JSON string, or multipart field `encrypted_data`) decrypts to the fields, and every response body is the encrypted JSON (bad/undecryptable request → 400; only a missing `CLIENT_ENCRYPTION_KEY` gives a plain-JSON 500). `CLIENT_ENCRYPTION_KEY` ships inside the apps, so it must never equal the auth service key. `app.services.auth_service` forwards the login with header `App: {AUTH_SERVICE_APP}` (and the client's optional `Lang` header) to the central auth service's encrypted v2 API `POST {AUTH_SERVICE_BASE_URL}/api/v2/social-login/{provider}`; v2 uses RNCryptor v3 (`app.core.rncryptor`, compatible with PHP `rncryptor/rncryptor`) with password `AUTH_SERVICE_RNCRYPTOR_KEY[offset:offset+length]` (default 8/32): the request body is the encrypted JSON sent as a JSON string, `Auth-Key` is sent encrypted, success bodies arrive encrypted and errors as plain JSON. Auth service 4xx answers are passed through as `{code, data: null, message}`; 5xx, timeouts or unreadable answers become HTTP 500 `Something went wrong. Please try again.`. Never log provider tokens, session tokens or `auth_key`.
- `app/api/v1/endpoints/payments.py` sells the AI plans (auto-renewing Week / Month / Year) through the external payment microservice (`PAYMENT_URL`, which holds the Razorpay secret; `app.services.payment_gateway` posts plain JSON to `/encryption`, the ciphertext to `/payment`, then `/decryption`; cancel uses multipart `/rnencryption` + `/cancel-subscription` with `PAYMENT_PROJECT_SLUG`). Plain-JSON routes under `/payment`: public `GET /plans`, `GET /status/{token}`; account-only (`Authorization: Bearer`, guests get 401) `POST /process/{slug}` (body `{email}` only when the account has none → returns `checkout {token, key: RAZORPAY_KEY_ID, subscription_id, url, prefill}`), `GET /latest-status`, `GET /my-subscription`, `POST /cancel-subscription` (stops auto-renew, access stays until `expires_date`); and `POST /payment/webhook/{PAYMENT_WEBHOOK_KEY}` for the microservice (wrong key → 404; raw body stored in `payment_webhooks` first; if that store fails → 500 so it is retried, otherwise always 200). `app.services.payment_service` follows the Razorpay flow doc: `payment_links.payment_id` (gateway token) = webhook `customer_payment_id` = `subscriptions.origin_transaction_id`; one `subscriptions` row per charge with unique `(origin_transaction_id, expires_date)` (first purchase, renewal when the period end moves later, otherwise update; null webhook fields never overwrite; a late older period end never shortens access); paid statuses gate creation, `subscription_status` CANCELLED/EXPIRED/HALTED only updates existing rows; one active plan at a time (`has_active_subscription`); `TEST_PAYMENT_USER_EMAILS` accounts are charged the inactive `recurring-test` plan's Razorpay id while keeping the chosen plan (the account's own email is read before a request email can be added). Active = `expires_date > UTC now`; all payment times are UTC. Plans live in `subscription_plans` (`scripts/migrate-payment-tables.py` seeds `ai-weekly`/`ai-monthly`/`ai-yearly` with placeholder prices and no Razorpay `price_id`, so they show as unavailable); set real values with `scripts/set-subscription-plan.py`. `app.dependencies.auth.require_ai_access` guards the six AI endpoints only when `AI_REQUIRES_SUBSCRIPTION=true` (401 no login, 402 `{requires_subscription: true}` without a plan); it is off by default so apps that send no session token keep working. Coin packs from the flow doc are not implemented.
- `app/api/v1/endpoints/store.py` binds App Store / Google Play purchases to our users (encrypted, needs a session): `POST /store/verify` (`{platform: apple, receipt}` = base64 app receipt, or `{platform: google, purchase_token}`) and `GET /store/entitlements` → `{premium, plus, sources, owned_by_other_account?}`. `app.services.store_verification` asks Apple `verifyReceipt` with `APPLE_SHARED_SECRET` (production, then sandbox on 21007; receipt `bundle_id` must be in `APPLE_BUNDLE_IDS`) or Google `purchases.subscriptionsv2` with the service account in `GOOGLE_PLAY_SERVICE_ACCOUNT_FILE` (package `GOOGLE_PLAY_PACKAGE_NAME`); not configured / unreachable → 503 (apps keep their current premium). `app.services.store_service` keeps one owner per store subscription in `store_purchases` (created with the auth tables; guest rows move to the account in `merge_guest_into_account`): owned by another *account* → not granted (`owned_by_other_account`, e.g. a guest after logout); owned by a guest → taken over. Tiers come from `STORE_PREMIUM_PRODUCT_IDS` / `STORE_PLUS_PRODUCT_IDS`. Entitlements include an active web plan as Plus; `payment_service.has_active_subscription` (AI gating, web purchase block) and `my_subscription` count store Plus (`platform` apple/google, cancel in the store). Lapsed rows are re-checked with the stored receipt / token after 6 hours.
- `app/middleware/legacy_encryption.py` (`LegacyEncryptionMiddleware`) wraps the old POST APIs `/download_media`, `/frontend_success`, `/transcribe`, `/extract_hook`, `/trendy_captions`, `/trendy_hashtags`, `/groq_caption`, `/groq_hashtags`. An encrypted request (same AES-GCM / `CLIENT_ENCRYPTION_KEY` as `/auth/*`) is either a form with field `encrypted_data` = encrypt(JSON of the fields), files staying plain parts, or a `text/plain` body = encrypt(JSON of the fields); the middleware rebuilds it as a normal multipart form, so endpoint validation is unchanged, and encrypts the whole response (errors too). `ENCRYPTION_REQUIRED=1` rejects plain requests to these paths with plain-JSON 400 `Encrypted request required.`; `0` (default) accepts both. Undecryptable → encrypted 400; missing key → plain 500. The same middleware makes encryption compulsory on the new APIs (`is_new_api`: `/history...` and `/payment/...` except `/payment/webhook/...`, which the payment microservice calls in plain JSON): a request body must be encrypt(JSON) as `text/plain` (plain JSON body → encrypted 400 `Encrypted request required.`) and reaches the endpoint as `application/json`; bodyless requests pass; every response is encrypted. `/auth/*` encrypts in its endpoints. Tests call these through `tests/encrypted_client.py`. Other paths are untouched.
- `app/exceptions/handlers.py` turns unhandled exceptions into `{"code": 500, "data": null, "message": ...}`. Many service paths instead catch their own errors and return their established response dictionaries.

### Configuration, database, and secrets

- `app/core/config.py` loads `.env` at import time and caches a Pydantic `Settings`. Configuration includes host/port, MySQL connection values, encryption key, auth service URL/App/timeout/RNCryptor key, app encryption key (`CLIENT_ENCRYPTION_KEY`), crypto allowlist, Tor password, scheduler log/interval, RapidAPI host/key, and an environment fallback for the opt-in `INSTAGRAM_OEMBED_METADATA_ENRICHMENT_ENABLED` flag. The active oEmbed enrichment value is read from non-encrypted `my_settings` on demand, so it can be toggled without a restart when the database is available. Changing environment variables after import will not change the singleton without process restart/cache reset.
- `app/db/session.py:get_connection` opens a new MySQL connection per operation and returns `None` after logging connector errors. There is no ORM or application-owned migration framework; repositories lazily create/alter some tables.
- `app/core/crypto.py` prefixes encrypted values with `enc:` and encrypts with Fernet. A non-Fernet `APP_ENCRYPTION_KEY` is deterministically SHA-256-derived to a Fernet key for backward compatibility. Never expose plaintext secrets, encrypted tokens, or the encryption key in logs or responses.
- `app/repositories/settings_repository.py` stores settings in `my_settings`. It selects `production_value` only for `APP_ENV` values `prod`/`production`, otherwise `development_value`; secret environment values are seeded only when absent unless forced. `scripts/migrate-secrets.py` migrates/verifies secrets, and `scripts/set-encrypted-setting.py` interactively or explicitly stores one plaintext value encrypted.
- `app/repositories/apify_key_repository.py` incrementally adds `apify_keys.token_encrypted`, encrypts any legacy plaintext `token`, then clears that plaintext column. Call `ensure_apify_token_encryption` before relying on an encrypted token.
- `app/repositories/instagram_service_repository.py` manages `download_media_service_config`. Although its API receives a context, it currently overwrites it with `all`; it seeds default services, optionally copies legacy `post` settings into `all`, and merges database sort/order/enabled flags with the in-code service definitions. This is shared mutable configuration for all post/profile fallback paths.

### Instagram service and state

`app/services/instagram_service.py` is the integration layer and contains both provider parsers and all Instagram/AI business logic. It produces provider results shaped as:

```python
{
    "postData": [{"type": "GraphImage" | "GraphVideo", "thumbnail": str, "link": str}],
    "username": str,
    "profilePic": str,
    "caption": str,
    # optional: "hashtags": list[str]
}
```

Important sections are:

- URL/metadata helpers: `normalize_instagram_url`, `_InstagramMetaParser`, `fetch_instagram_page_metadata`, `fetch_instagram_og_metadata`, `fetch_instagram_oembed_metadata`, `fetch_instagram_profile_picture`, `enrich_instagram_metadata`, and photo/video/Threads predicates. Successful media results retain provider-supplied fields. When the opt-in oEmbed enrichment flag is enabled, missing username/caption/hashtags are filled from oEmbed for any successful post type without changing provider media links; the legacy page fallback then remains best effort. A missing profile image is resolved only from a validated username's public profile `og:image`. Metadata requests have bounded timeouts and success-only TTL caches; metadata failure must never discard media links.
- Provider adapters/parsers: `fetch_instagram_rapidapi_provider`, DownloadGram (`fetch_instagram_downloadgram`), GrabGram (`fetch_instagram_snapdownloader`), SnapInsta Selenium interception (`fetch_instagram_data`), `fetch_instagram_oembed_post`, yt-dlp, Instaloader, GlobalSource, SSSInstagram, `fetch_apify_instagram_post`, an indown/GraphQL proxy chain, and SaveClip through ProxyOrb. DownloadGram posts form data directly to `api.downloadgram.org/media`, parses its escaped HTML response, accepts only HTTPS `cdn.downloadgram.org` wrapper links, then returns the token's direct HTTPS `*.cdninstagram.com` or `*.fbcdn.net` media URL rather than the forced-download wrapper. Several providers are retained but disabled by default in the workflow definitions.
- Fallback orchestration: `_instagram_service`, `_run_instagram_service`, `_run_instagram_services`, and `_instagram_failure_response`. DB configuration controls active order; success is the first handler that does not fail (and, when specified, returns non-empty `postData`).
- MySQL side effects: `update_download_history`, `update_frontend_success`, `log_analytics`, and `log_platform_request`. `device_type` is nullable in `insta_download_history` (`1` iOS, `2` Android); `ios_requests` and `android_requests` count valid platform-supplied `/download_media` requests once before provider fallback. `ANDROID_DOWNLOADGRAM_FIRST` is a non-encrypted `my_settings` feature flag, defaulting to false. When explicitly true, it forces exactly one DownloadGram attempt for Android requests after validation/privacy checks, then skips DownloadGram in the normal fallback to avoid a duplicate request. Analytics dynamically adds provider-specific success/failure columns; provider names must therefore remain safe/stable.
- AI handlers: `_get_gemini` caches a Gemini client and retrieves its key from DB then environment; `_vision_gemini` uploads a temporary file and attempts remote cleanup; `_query_groq` makes OpenAI-compatible Groq calls. Upload handlers read the whole file into memory, use an untrusted temporary file name, and must clean it on every failure path.
- Tor and browser utilities: `get_tor_session`, `change_tor_ip`, `reset_instagram_identity`, and `setup_driver`. Tor state (`last_ip_change_time`) and the unbounded page-metadata cache are module globals. Selenium requires Chromium/Chrome and a matching driver; Docker uses `/usr/bin/chromium` and `/usr/bin/chromedriver`.

### Background operation and deployment

- `api/apify_key_rotator.py` obtains a MySQL advisory lock, decrypts/migrates tokens, polls Apify usage, updates `apify_keys`, disables a near-exhausted active key, selects the lowest-usage eligible key, and optionally sends SMTP alerts. It writes action records to `apify_rotation_logs`.
- `api/scheduler.py` runs that rotator in a subprocess at startup and every `ROTATOR_INTERVAL_MIN` minutes, with a 10-minute subprocess timeout, filtering selected warnings and appending to the configured log file.
- `Dockerfile` uses Python 3.9 slim and installs Chromium, Tor, curl, and required system libraries. `scripts/docker-entrypoint.sh` rewrites Tor configuration, waits for a Tor exit, starts the API and scheduler, then exits when either process exits. `scripts/deploy-refactor.sh` builds/deploys an image, migrates secrets, removes selected plaintext secrets from the environment file, and rolls back to the old image only when startup fails.
- `docs/tor_setup.txt` is an older manual Tor guide. Verify it against the Docker cookie-auth Tor configuration before changing or relying on it.

## 3. Runtime Flow

### Download media

`POST /download_media` receives multipart `instagramURL` and non-empty `deviceId` -> endpoint delegates to `instagram_service.download_media` -> `normalize_instagram_url` resolves supported `/share/` redirects, tries to decode `/s/` highlight links, removes query/fragment/trailing slash, and validates regexes -> non-Threads URLs run `check_instagram_privacy` through Instagram oEmbed (it fails open on request errors) -> profile URLs use a legacy `{code: 200, data: profile_url}` sentinel, all other valid URLs remain strings -> service definitions are merged with `download_media_service_config` -> each enabled service is attempted in configured order until one succeeds -> optional metadata enrichment scrapes Instagram page OpenGraph tags -> a success updates `insta_download_history` and `insta_analytics` -> response is `{code: 200, data: provider_result}`.

When supplied, `deviceType` is validated as `1` (iOS) or `2` (Android) before any network/provider work. Platform counters record one request at entry and are deliberately independent from provider retries; omitted legacy-client values are not assigned to either platform. History updates preserve a known platform when a later request omits `deviceType`.

For a temporary Android-only DownloadGram override, set non-encrypted `my_settings.ANDROID_DOWNLOADGRAM_FIRST` to a truthy value (`true`, `1`, `yes`, or `on`). This does not enable DownloadGram in `download_media_service_config`, does not affect iOS or legacy clients, and does not mutate the normal configured service order. A failed forced attempt records a DownloadGram provider failure and then continues with the remaining normal fallback providers.

For ordinary post-like URLs, current in-code enabled services are RapidAPI, DownloadGram, GrabGram/SnapDownloader, SnapInsta, conditional oEmbed for confirmed photo posts, and Apify as final fallback. DownloadGram's default position is after RapidAPI and before SnapDownloader; the database still controls its enabled state and final order. For profiles, current enabled services are DownloadGram, SnapDownloader, SnapInsta, conditional oEmbed, and Apify. Database configuration can disable/reorder these. The other listed services are disabled in code unless a future code change enables them. On total provider failure, the API intentionally returns HTTP-success-shaped `{code: 200, data: None, message: ...}` and records a failed history/analytics event. An invalid URL returns `{code: 400, message: ...}`; it is not an HTTP 400 response.

Provider implementations are sync and can block the FastAPI event loop for network/Selenium delays. Network-level failures usually become a fallback attempt, while privacy detection exits early with the generic unavailable response. No provider downloads media to local disk; it returns direct provider/CDN links. Provider result schemas vary, so the orchestrator only enforces non-empty media for services declared `require_post_data=True`.

### AI uploads

Caption/hashtag/transcription/hook endpoints accept multipart `UploadFile`, read its full content, write it to `NamedTemporaryFile(delete=False)`, call Gemini or Groq, parse a JSON fragment from model text with regex, and return `{code: 200, data: ...}`. Gemini remote files are deleted in `_vision_gemini`'s `finally`; local cleanup is inconsistent (`trendy_captions` only unlinks after success, while the other handlers clean up in their exception paths). Do not assume model output is valid JSON or that cleanup occurred unless a test proves it.

### Secret and scheduler flows

Startup calls `seed_env_settings`; it is best-effort so database unavailability does not prevent API startup. The `/crypto` routes use the same encryption implementation as repository migration. Separately, the entrypoint starts the scheduler, which executes `api.apify_key_rotator` as an isolated process to avoid sharing database/circuit state with the API.

### Health readiness

`GET /api/health` -> `health.readiness_payload` -> `health_service.check_database` -> a newly opened MySQL connection executes `SELECT 1` -> cursor/connection are closed -> the endpoint returns the payload with HTTP 200 when the database check is true or HTTP 503 when it is false. The endpoint is intended for load-balancer/readiness checks and does not alter database state.

## 4. Instagram URL Handling

`normalize_instagram_url` is the only request-entry URL validator. It accepts `http`/`https` hosts `instagram.com` and `www.instagram.com` for:

- standard `/p/<shortcode>` posts, including profile-prefixed variants;
- `/reel/<shortcode>` and legacy `/tv/<shortcode>`, including profile-prefixed variants;
- stories shaped `/stories/<username>/<numeric-id>` and the alternate profile-prefixed pattern;
- highlights shaped `/stories/highlights/<numeric-id>` and the alternate profile-prefixed pattern;
- bare Instagram profiles; and
- `threads.com`/`threads.net` `@<user>/post/<id>` URLs.

It strips query/fragment parameters, so tracking parameters and a trailing slash do not affect the normalized result. `/share/` requests follow redirects before regex validation; `/s/` links only become a supported highlight URL when their decoded payload has `highlight:<digits>`. It does not accept bare `instagram.com` without scheme, lookalike domains, `/reels/`, generic story-profile URLs, arbitrary Instagram routes, or a current separate IGTV product beyond legacy `/tv/` syntax. Regex validation protects the initial URL, but external provider redirects and provider-returned CDN links are trusted by the existing code; do not broaden accepted hosts or redirect behavior without an SSRF/security review.

Stories/highlights pass validation but are not routed to `fetch_story_or_highlight` by the current fallback definitions. Profile routes are supported as "return latest post through a provider" rather than as a profile metadata API; output depends on the provider. `_is_instagram_photo_post_url` may make an additional Instagram HTML request to distinguish a photo post from an ambiguous `/p/` video before giving oEmbed priority.

## 5. Important Invariants

- Preserve route names, multipart field names (`instagramURL`, `deviceId`, `video_file`, etc.), and established dictionary response shapes unless a requested API change explicitly permits a breaking change.
- Preserve `postData` media item keys and `GraphImage`/`GraphVideo` values. Clients consume direct links; no local downloader/file lifecycle exists in this API.
- Preserve the profile sentinel returned by `normalize_instagram_url` and handled by `download_media`, unless both callers and tests are deliberately updated.
- Do not turn the generic all-providers-failed response into an HTTP error without coordinating a client-contract change.
- Keep service identifiers stable: they drive DB configuration and dynamically created analytics columns. Understand the global `all` context behavior before changing profile/post ordering.
- Treat cookies, API keys, tokens, SMTP credentials, proxy data, and encryption values as secrets. Never log them; do not commit `.env` or modify existing secret configuration unnecessarily.
- Preserve retry/timeout/Tor semantics of a provider unless its callers and operational impact are understood. Provider sites and Instagram are unstable; deterministic parser tests should isolate that instability.
- The application relies on pre-existing MySQL tables `insta_download_history`, `insta_analytics`, `apify_keys`, and `apify_rotation_logs`; only some repository tables/columns are self-created. Run `scripts/migrate-device-platform.py` and `scripts/migrate-android-downloadgram-setting.py` `scripts/migrate-auth-tables.py` (creates `app_users`, `app_sessions`, `user_download_history`) and `scripts/migrate-payment-tables.py` (plans, payment links, subscriptions, payment webhook log) before application startup when deploying these features; `scripts/deploy-refactor.sh` invokes them automatically. Schema changes need an explicit migration/compatibility plan.

## 6. Coding Conventions

- Target Python 3.9, matching `Dockerfile`; use standard-library typing (`Dict`, `List`, `Optional`) as the existing service does. Most code is PEP-8-ish but not formatter-enforced.
- Use FastAPI `Form`, `File`, and `UploadFile` at HTTP boundaries; route modules should delegate to services. Keep synchronous provider logic out of route modules.
- Repository functions open and close their own MySQL connection. Use parameterized SQL values; only construct identifiers after strict normalization/known constants.
- Existing logging uses `print` with useful provider/context labels. Keep diagnostic messages secret-free. Existing exception handling favors provider fallback and generic user output over propagating provider-specific errors.
- Add a focused helper/parser instead of duplicating provider parsing. Follow the normalized result schema and set `require_post_data=True` for handlers whose empty response must not be treated as success.
- Do not add dependencies casually: deployment depends on `requirements.txt`, Chromium, Tor, and shell tooling installed by Docker.

## 7. Change Protocol

Before editing, read the affected source and all callers, inspect the service definitions and their DB configuration implications, and search for tests/fixtures (none currently exist). Make the smallest coherent change. Update this document whenever architecture, routes, configuration, supported URLs, provider order, data shape, or testing requirements change.

For meaningful changes, add deterministic tests first where practical. Test URL normalization/classification and provider parsers with mocked HTTP/Selenium payloads; do not make CI depend on a live Instagram post. Run the targeted tests, then the full suite when one exists, and record every command actually executed. For changes affecting URL handling, providers, metadata, network behavior, or output formatting, exercise post/image, carousel, reel, story, highlight, profile, legacy `/tv/`, query variants, invalid/lookalike URLs, and restricted/unavailable behavior as applicable. Mark unexecuted live cases explicitly rather than claiming support.

## 8. Mandatory Automated QA

The repository currently provides no configured formatter, linter, type checker, or CI workflow. Deterministic DownloadGram coverage is in `tests/test_downloadgram.py`; social-login protocol coverage is in `tests/test_auth.py` (auth service faked); guest/account/session/history coverage is in `tests/test_sessions.py` and AI plan payment coverage in `tests/test_payments.py` (payment microservice faked); both need a throwaway MySQL/MariaDB (`TEST_DB_HOST/PORT/USER/PASSWORD/NAME`; their tables are dropped, so never point them at a database in use) and are skipped without `TEST_DB_NAME`; platform-tracking coverage is in `tests/test_platform_tracking.py`. Run the affected test module and `python3 -m unittest discover -s tests` when the suite is available, plus a read-only AST parse when bytecode-cache permissions prevent `compileall`. Before changing Docker/deployment scripts, validate shell syntax and build/run behavior where environment access permits. Live Instagram, DownloadGram, provider, Tor, MySQL, Apify, Gemini, and Groq checks require their real credentials/services and must be reported as environment-dependent smoke tests, not deterministic regression proof.