# Google Search Console Indexing - Design Spec
> **Dated artifact (2026-05-12) - one correction:** the deferred "future work" bullets below mention "Cloudflare for SaaS provisioning events." Cloudflare for SaaS was never adopted; custom domains use certbot/Let's Encrypt with A records to the origin server IP (see `docs/DEALER_DASHBOARD.md`). The design itself is unaffected.
**Status:** Approved through brainstorming, ready for implementation plan
**Date:** 2026-05-12
**Author:** Tim (timothy@aimclear.com), with Claude
## Problem
AMSOIL DLP serves ~1,140 dealer landing pages across four parent domains (`myamsoil.com`, `myamsoil.ca`, `shopamsoil.com`, `shopamsoil.ca`) plus an unbounded set of custom dealer domains. Subdomain dealer pages are not being indexed by Google reliably, and there is no programmatic mechanism to submit new dealers, content updates, or custom-domain dealers to Google Search Console (GSC). The current sitemap and `robots.txt` infrastructure is correct but passive - Google has to discover everything on its own.
## Goals
1. Every active dealer's canonical URL is known to Google via a submitted sitemap.
2. New dealer activations, custom-domain activations, and content publishes proactively notify Google.
3. We can observe per-dealer index status (indexed / not indexed) and detect indexing health regressions.
4. Custom dealer domains automatically become verified GSC properties.
5. Admins can force a reindex of any dealer from the existing admin UI.
6. Errors are tracked and visible. Failed jobs surface in admin UI and trigger ops alerts.
7. The system respects Google API quotas without manual intervention.
## Non-goals
- Dealer-facing UI for GSC status (deferred - see "Future work").
- Cert expiry monitoring (related but separate - see "Future work"; a separate GitHub issue will be created based on the SSL audit findings).
- Broader custom-domain activity auto-logging (DNS verification events, Cloudflare for SaaS provisioning events) - deferred to its own initiative.
- Indexing API usage for general content beyond the admin force-reindex button and audit-flagged stragglers - Google officially restricts the Indexing API to `JobPosting` and `BroadcastEvent` content; we use it sparingly.
- Tier-based logic (`subscriptionTier` is NOT read by GSC code; we operate on actual `Page`/blog records to handle grandfathered dealers correctly).
## Key constraints and decisions
### API quotas (verified against Google's current docs, 2026-05-12)
- **Indexing API:** 200 publish requests/day **per Google Cloud project**. Officially restricted to `JobPosting` / `BroadcastEvent`. We treat this as a scarce, best-effort resource - not load-bearing.
- **URL Inspection API:** 2,000 queries/day **per GSC property**. Across 4 main domains = 8,000/day. Generous.
- **Sitemap submission:** No documented daily quota. The workhorse for getting URLs in front of Google.
- **Site Verification API + Sites API:** No meaningful quota concerns at our scale.
### GSC property strategy
- **Four main AMSOIL domains:** Domain properties (`sc-domain:myamsoil.com` etc.), already manually created by the user. Cover all subdomains automatically. DNS-verified by the user.
- **Custom dealer domains:** URL-prefix properties (`https://customdomain.com/`), auto-created by our worker. Verified via `` rendered in `` (we control rendering; we do not control the dealer's DNS, so DNS-based verification is not viable). **Single property per custom domain** using `getCanonicalUrl(dealer)` (whatever the dealer stored as `customDomain`).
Production Apache `VirtualHost` config (`scripts/ssl-manager.sh:166-168`) serves both `example.com` AND `www.example.com` from the **same VirtualHost** with no redirect, via `ServerAlias`. Identical content is returned for both forms. The dealer page already renders `` pointing to the stored `customDomain` form, so Google consolidates indexing on the canonical form regardless of which URL it crawls first. Registering both forms in GSC would duplicate every operation (verify, sitemap, inspect, delete) for zero indexing benefit beyond what Google's canonical-tag resolution already provides.
### Property resolution per job (worker behavior)
The worker MUST resolve the correct GSC property for each job before calling Google APIs. For dealers with no active custom domain, all jobs target `sc-domain:{parent}` (e.g. `sc-domain:myamsoil.com`). For dealers with `customDomainStatus = 'active'`, the resolution depends on job type:
- `submit_sitemap`, `delete_sitemap`, `inspect_url`, `notify_indexing` for the canonical URL: use the URL-prefix property `https://{customDomain}/` (NOT the subdomain Domain property).
- `verify_property`, `register_property`, `delete_property`: target the host carried in `payload.propertyHost` (these jobs are always custom-domain-scoped).
Implementation: a single helper `resolvePropertyForJob(dealer, job)` returns the `siteUrl` to pass to Google API calls. Job payloads carry the canonical URL (e.g. `payload.url` for inspect, `payload.sitemapUrl` for submit), so the worker has the URL it needs without re-deriving from dealer state.
### Architectural pattern
DB-backed job queue (Postgres) drained by HTTP-triggered worker on server crontab. Triggers (activation hooks, publish hooks, daily audit, admin button) only `INSERT` rows. The worker is the only code that calls Google APIs. This keeps request handlers thin and untestable-API-calls isolated.
### Logging
Pino is in use (`lib/logger.ts`). Create `gscLogger = logger.child({ module: 'gsc' })` to match existing convention (`authLogger`, `webhookLogger`, etc. all use `module:`). Discipline: one log line per job completion, one summary per worker invocation, no per-API-call info logs in production. Compatible with existing PM2 + pm2-logrotate setup.
## Architecture
```
┌──────────────────────────────────────────────────────────────────────┐
│ Trigger sources (INSERT rows into GscJob) │
│ │
│ • app/api/admin/dealers/[id]/status (admin activation) │
│ • app/api/webhooks/stripe (subscription becomes active) │
│ • app/api/dealer/custom-domain/activate (custom domain → active) │
│ • lib/isr-revalidation.ts (CMS / blog publish) │
│ • app/api/cron/gsc-audit (daily homepage inspections) │
│ • app/api/admin/dealers/[id]/reindex-gsc (admin button) │
│ • Cancellation paths (status → cancelled, customDomainStatus ≠ active)│
└──────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ GscJob (Postgres) │
│ type, dealerId, payload, status, │
│ attempts, lastError, priority, runAt│
└──────────────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ /api/cron/gsc-drain │
│ (server crontab, every 5 min) │
│ - SELECT FOR UPDATE SKIP LOCKED │
│ - Preflight gate (SSL/reachability) │
│ - Quota gate (Indexing API) │
│ - Backoff on retryable errors │
└──────────────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ lib/google/* (thin API wrappers) │
│ auth.ts (JWT) │
│ search-console.ts (sitemaps) │
│ indexing.ts (urlNotif) │
│ site-verification.ts │
│ url-inspection.ts │
│ sites.ts (properties) │
└──────────────────────────────────────┘
```
### Module boundaries (each with single purpose, isolated, testable)
- **`lib/google/auth.ts`** - Service account JWT. Existing.
- **`lib/google/indexing.ts`** - Indexing API `urlNotifications:publish`. Existing (will be extended).
- **`lib/google/search-console.ts`** - Sitemaps submit/delete. Existing (will be extended).
- **`lib/google/url-inspection.ts`** - New. `urlInspection.index` queries.
- **`lib/google/site-verification.ts`** - New. Get token, verify, delete.
- **`lib/google/sites.ts`** - New. Add/delete GSC properties.
- **`lib/google/gsc-domains.ts`** - Domain → property mapping. Existing.
- **`lib/gsc-preflight.ts`** - New. HTTPS HEAD check with cached result.
- **`lib/gsc-jobs.ts`** - New. Enqueue functions, one per trigger event.
- **`lib/gsc-worker.ts`** - New. Job execution loop with backoff, error classification, quota gating.
- **`lib/gsc-submission.ts`** - Existing; will be deleted (its single function is superseded by `enqueueDealerActivation` in `gsc-jobs.ts`).
### Why DB-as-queue, not Redis/SQS
We're already on Postgres. `SELECT ... FOR UPDATE SKIP LOCKED` gives atomic claim-and-run with no race condition. Scale is comfortably below where Postgres queueing breaks down (~hundreds of jobs/day peak, low concurrency). Operational surface stays minimal - no new infrastructure to deploy or monitor.
## Data model
### New table: `GscJob`
```prisma
model GscJob {
id String @id @default(cuid())
type String // see job types below
dealerId String
payload Json // type-specific args
status String @default("pending") // pending | running | completed | failed
priority Int @default(100) // lower = sooner. Admin=10, publish=50, audit=200
attempts Int @default(0)
maxAttempts Int @default(5)
lastError String? @db.Text
runAt DateTime @default(now())
startedAt DateTime?
completedAt DateTime?
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
dealer Dealer @relation(fields: [dealerId], references: [id], onDelete: Cascade)
@@index([status, runAt, priority])
@@index([dealerId])
@@index([type, status])
}
```
**Job types:**
| Type | Triggered by | Calls |
|---|---|---|
| `verify_property` | custom-domain activation (pass 1) | Site Verification API getToken; writes `gscVerificationToken`; calls `revalidateAllDealerPages()`; enqueues `register_property` (delayed 60s) |
| `register_property` | follow-up after `verify_property` | Site Verification verify + Sites `add`; enqueues `submit_sitemap` |
| `submit_sitemap` | activation, publish, audit, admin | Sitemaps `submit` |
| `inspect_url` | daily audit, publish, admin deep-audit | `urlInspection.index`; updates `gscLastIndexStatus`; may enqueue `notify_indexing` if `not_indexed` |
| `notify_indexing` | admin force-reindex; audit straggler | Indexing API (quota-gated) |
| `delete_sitemap` | dealer cancellation; custom-domain deactivation | Sitemaps `delete` |
| `delete_property` | custom-domain deactivation | Sites `delete` + Site Verification delete |
### New columns on `Dealer`
```prisma
gscVerificationToken String? // meta tag content for custom-domain verification
gscPropertyRegisteredAt DateTime? // last successful Sites.add for this dealer's custom domain
gscLastInspectedAt DateTime? // last successful URL inspection of canonical homepage
gscLastIndexStatus String? // indexed | not_indexed | error
gscLastSubmittedAt DateTime? // last successful sitemap submission
gscPreflightHealthyAt DateTime? // last successful preflight check (TTL cache + display)
gscPreflightError String? // last preflight failure reason (null when healthy)
```
(The existing `gscSubmittedAt` column from prior work gets renamed to `gscLastSubmittedAt` in this migration for naming consistency. The existing prototype migration `20260511120000_add_gsc_submitted_at` will be merged into a single new migration covering all the above columns + the new tables.)
### Quota ledger: `GscQuotaUsage`
```prisma
model GscQuotaUsage {
date DateTime @id @db.Date // one row per day, UTC
indexingApiCalls Int @default(0)
}
```
Worker increments before each Indexing API call and refuses to make the call if today's count >= 180 (10% headroom under the 200 hard limit). Deferred jobs are rescheduled to `tomorrow 00:00 UTC`, not failed.
## Trigger hooks
| Event | Source location | Jobs enqueued (priority) |
|---|---|---|
| Dealer status → `active` | `admin/dealers/[id]/status`, Stripe webhook | `submit_sitemap` (50) + `inspect_url` (200, runAt +1h) |
| Custom domain → `active` | `dealer/custom-domain/activate` | `verify_property` (50) - chains to `register_property` → `submit_sitemap` |
| CMS / blog publish | `lib/isr-revalidation.ts` after `revalidateAllDealerPages()` | `submit_sitemap` (50) + `inspect_url` (50) per changed path |
| Daily audit cron | new `app/api/cron/gsc-audit` (server crontab) | `inspect_url` (200) for every active dealer's homepage |
| Admin "Reindex" button | new `app/api/admin/dealers/[id]/reindex-gsc` | `submit_sitemap` (10) + `inspect_url` (10) + `notify_indexing` (10, force=true) |
| Dealer status → `cancelled` | status route, Stripe webhook | `delete_sitemap` (50) |
| Custom domain → ≠ active | activate route, deactivation paths | `delete_sitemap` (50) + `delete_property` (50) |
All triggers route through `lib/gsc-jobs.ts`. Request handlers don't import Google clients directly.
## Worker behavior
`/api/cron/gsc-drain` called every 5 minutes by server crontab with Bearer `CRON_SECRET`.
Per invocation:
1. Verify Bearer auth (timing-safe).
2. Load/create today's `GscQuotaUsage`.
3. `SELECT * FROM GscJob WHERE status='pending' AND runAt <= NOW() ORDER BY priority ASC, runAt ASC LIMIT 50 FOR UPDATE SKIP LOCKED`. Concurrent invocations are safe - locked rows are skipped.
4. Mark each as `running`, set `startedAt`.
5. For each job:
a. **Quota gate** (only for `notify_indexing`): if today's count >= 180, set `runAt = tomorrow 00:00 UTC`, status `pending`, continue.
b. **Preflight gate** (only for `submit_sitemap` and `notify_indexing`): call `preflightCheck(dealer)`. If unhealthy, mark `gscPreflightError`, set `runAt = NOW + 1h`, attempts++, continue. **The first transition from healthy → unhealthy** writes an automatic `DealerNote` (`isAutomatic: true`, `noteType: 'internal'`, `adminId = GSC_SYSTEM_ADMIN_USER_ID`) so it appears in Activity & Notes. The transition is detected by `gscPreflightError` being null before the check and non-null after. Subsequent failures of the same kind do NOT re-log (to avoid flooding the activity feed). The reverse transition (unhealthy → healthy) writes a recovery note.
c. **Execute** via the appropriate `lib/google/*` client.
d. **Success:** status `completed`, `completedAt = NOW`, update relevant `Dealer.gsc*` column, enqueue any follow-up job (e.g. `verify_property` chains to `register_property`).
e. **Retryable error** (5xx, 429, network): attempts++, `runAt = NOW + backoff`, `lastError`. If attempts >= maxAttempts, status `failed`.
f. **Permanent error** (400, 401, 403): status `failed` immediately.
6. Log one info line per job + one summary line per invocation.
7. Return `{ processed, completed, deferred, failed, durationMs }`.
### Backoff schedule
`1min → 5min → 15min → 1hr → 6hr → fail` (5 attempts).
### Error classification
| HTTP status | Treatment |
|---|---|
| 200/201/204 | success |
| 400 (validation) | permanent fail |
| 401, 403 | permanent fail (config/auth) |
| 404 | permanent for delete jobs; retryable for submit jobs (cache lag) |
| 429 | retry with backoff |
| 5xx, network errors | retry with backoff |
### Preflight check
`lib/gsc-preflight.ts`:
```ts
async function preflight(dealer): Promise<{ healthy: boolean, reason?: string }>
```
- Returns cached result if `gscPreflightHealthyAt` is within 6 hours.
- Otherwise: `fetch(canonicalUrl, { method: 'HEAD', signal: AbortSignal.timeout(10_000), redirect: 'manual' })`. Node's `https` rejects invalid certs by default - SSL failures throw.
- Healthy = status in `[200, 301, 302, 304]` AND no thrown error.
- Side effect: writes `gscPreflightHealthyAt = NOW` and clears `gscPreflightError` on success; writes `gscPreflightError = reason` on failure.
- Used by `submit_sitemap` and `notify_indexing` jobs. (Inspection and delete jobs do not require preflight - we always want to know if a URL is indexed; deletes target GSC, not the dealer's site.)
### Concurrency safety
`FOR UPDATE SKIP LOCKED` is the only concurrency primitive needed. Cron invocations may overlap without coordination. No leader election.
### Idempotency
All seven Google endpoints used (Sitemaps.submit, Sitemaps.delete, urlNotifications.publish, urlInspection.index, siteVerification.token, siteVerification.delete, Sites.add, Sites.delete) are idempotent - re-running a completed job has no harmful effect, so retry-after-partial-failure is safe.
## Admin UI
### Dealer-list API exposes `hasPendingReindex`
To support the context-menu disable rule across page reloads (not just session-local state), the dealer-list API (used by `DealerTable`) must include a `hasPendingReindex: boolean` flag per row, set by joining against `GscJob` for `type='notify_indexing'` rows with `status IN ('pending', 'running')`. Without this, the disable signal evaporates on refresh.
### Per-dealer context menu (existing `QuickActionsMenu.tsx`)
This is the three-dot context menu rendered for each dealer row in the admin dealer table. Currently has: Support, Stripe, Edit DNS, ─── divider ───, Regenerate Pages. Add a new menu item **"Reindex with Google"** directly below "Regenerate Pages".
The menu item should be rendered **disabled (grayed out, non-clickable, with a tooltip explaining why)** when any of the following are true:
- The dealer's status is neither `active` nor `cancelled_pending` (those two are the statuses where the dealer's site still serves traffic).
- The dealer has no `subdomain`.
- A `notify_indexing` job for this dealer is already in `pending` or `running` status - prevents queueing duplicate admin force-pings. Other GSC job types for this dealer do not gate the button.
When clicked: POSTs to `/api/admin/dealers/[id]/reindex-gsc`, shows a toast indicating queued state, and updates the row's reindex-state for the disable check.
### Per-dealer detail modal (existing `DealerDetailModal.tsx`)
Add **"Indexing & SEO"** section after "Site Information":
- Canonical URL display
- Index status badge (with last-inspected relative time)
- Sitemap submission state + "Resubmit" button
- Custom domain property state (if applicable)
- SSL/reachability state (under "Site Information" per user request) showing `gscPreflightHealthyAt` or `gscPreflightError`
- Recent jobs (last 10) - type, status, lastError, completedAt
- Deep audit button (queues inspections for all of this dealer's `Page` + blog post records - query-driven, not tier-based)
- Force reindex button (same as menu action)
- Link to global GSC dashboard
### Auto-logging to Activity & Notes
When preflight transitions from healthy → unhealthy (or after 3 consecutive failures), worker writes:
```ts
await prisma.dealerNote.create({
data: {
dealerId,
adminId: ,
content: "Preflight check failed: SSL certificate expired (canonical: https://...). GSC submissions paused.",
noteType: 'internal',
isAutomatic: true,
}
});
```
When preflight transitions back to healthy, a follow-up note is written. Only state transitions are logged - individual check runs are not, to avoid flooding the activity feed.
### Global dashboard at `/admin/gsc`
Reached via a new tile on the main `/admin` page (the admin layout has no sidebar - sub-pages are linked from the dashboard tiles).
Sections:
- **Today's quota usage:** Indexing API count with progress bar (yellow at 75%, red at 90%)
- **Queue health:** counts by status (pending, running, completed-24h, failed)
- **Failed jobs table:** type, dealer, lastError, attempts, "Retry" / "Dismiss" buttons
- **Daily audit status:** last successful audit timestamp, "Run now" button
- **Backfill control:** "Enqueue submit_sitemap for all active dealers without recent submission" (idempotent; skips dealers with `gscLastSubmittedAt` within last 30 days)
Auth: `requireAdmin()` guard, same pattern as other admin routes.
## Observability
- **Structured logs** via `gscLogger = logger.child({ module: 'gsc' })`. One info line per job completion. One info summary per worker invocation. Warn/error for failures.
- **Daily anomaly-only email.** Folded into the existing `/api/cron/gsc-audit` daily run (not a separate cron). After enqueuing inspection jobs, the audit checks for anomalies: (a) any `GscJob` rows with `status='failed'` in the last 24h, (b) zero successful `gsc-drain` runs in the last 24h (worker stopped), (c) Indexing API quota >90% used. If any anomaly present, sends ONE summary email via the existing `sendEmail({to, subject, html})` helper at `lib/email/index.ts`. If no anomalies, sends nothing - quiet days are quiet days. Recipient address comes from a new `GSC_ALERT_EMAIL` env var (no existing ops alias to reuse). Email content links to `/admin/gsc` for triage.
- **Health endpoint:** `/api/cron/gsc-drain` returns `{ processed, completed, deferred, failed, durationMs }` JSON. Cron monitor can alarm on `failed > 0` without a separate ping.
## Testing strategy
### Red-Green-Refactor discipline (per project requirement)
**Every test must be run against the empty implementation first and confirmed to fail with the actual assertion failing - not a syntax error, not a wrong-import error, not a missing-module error.** Only after a meaningful red is observed do we write the minimal code to pass (green), then refactor.
Anti-patterns to actively guard against:
- Tests that pass against any implementation (no real assertion).
- Tests that only fail on errors the implementation is structurally forced to produce (e.g. asserting an exact error string the implementation literally throws).
- Implementation written to satisfy the test rather than the requirement (e.g. hardcoded return value matching the test's expected output).
- Mocking the system under test (mocking `lib/google/*` is correct; mocking `lib/gsc-jobs.ts` in a test of `lib/gsc-jobs.ts` is not).
Each test gets a one-line comment recording **what it would falsely pass** so reviewers can verify the assertion is meaningful.
### Unit tests
| Module | What gets tested |
|---|---|
| `lib/google/auth.ts` | Token parsing; missing/malformed env var |
| `lib/google/indexing.ts` | URL payload shape; error classification |
| `lib/google/search-console.ts` | Sitemap URL encoding; `sc-domain:` vs URL prefix encoding; delete vs submit |
| `lib/google/site-verification.ts` | Token retrieval; verify call payload; delete call |
| `lib/google/url-inspection.ts` | Request payload; response parsing for indexed/not_indexed/error |
| `lib/google/sites.ts` | Add/delete with both `sc-domain:` and URL-prefix forms |
| `lib/google/gsc-domains.ts` | Domain mapping; CA domains; unknown domain returns null |
| `lib/gsc-preflight.ts` | HTTP HEAD success cases; SSL failure; timeout; cache TTL behavior |
| `lib/gsc-jobs.ts` | Each enqueue function inserts correct row(s) with correct priority and payload |
| `lib/gsc-worker.ts` | Job dispatch; backoff math; error classification; quota gate; preflight gate; concurrent claim safety |
### Integration tests (alongside existing `__tests__` patterns)
| Endpoint | Coverage |
|---|---|
| `app/api/cron/gsc-drain/__tests__/route.test.ts` | Bearer auth; full enqueue→drain with mocked Google clients; quota gate behavior |
| `app/api/cron/gsc-audit/__tests__/route.test.ts` | Enqueues `inspect_url` for all active dealers once per day; idempotent re-run |
| `app/api/admin/dealers/[id]/reindex-gsc/__tests__/route.test.ts` | Admin-only auth; enqueues 3 jobs at priority 10 |
| `app/api/admin/dealers/[id]/status/__tests__/route.test.ts` (existing) | Adds: enqueue happens on activation transition only |
| `app/api/webhooks/stripe/__tests__/route.test.ts` (existing) | Adds: enqueue on activation transition |
| `app/api/dealer/custom-domain/activate/__tests__/route.test.ts` (existing) | Adds: enqueues `verify_property` chain on activation; cleanup on deactivation |
### Mocking strategy
The four Google API endpoints live in `lib/google/*.ts` modules. Tests inject mock implementations via Jest module mocks. **No real network calls in CI.**
### Implementation order
1. Schema + migration (data-only; no behavior to test)
2. `lib/google/*.ts` clients - easiest in isolation
3. `lib/gsc-preflight.ts`
4. `lib/gsc-jobs.ts` enqueue functions
5. `lib/gsc-worker.ts` - full execution loop with all gates
6. Cron routes (`gsc-drain`, `gsc-audit`)
7. Trigger hooks in existing routes (publish, status, custom domain activate, Stripe webhook)
8. Admin UI (menu item, detail-modal section, dashboard page)
Each item: write test (red), implement (green), refactor.
## Rollout plan
**Phase 1 - Code merged, feature dark.** `GOOGLE_SERVICE_ACCOUNT_JSON` not set → all triggers no-op via `isGscConfigured()` guard. Migration applies new columns/tables. Zero behavioral change.
**Phase 2 - Staging dry-run.** Env var on staging. Service account added as Owner on one test property. Manually trigger admin reindex on 1–2 test dealers. Verify queue drains, sitemap appears in GSC, URL inspection works end-to-end.
**Phase 3 - Production enable, narrow.** Env var on production. Service account added as Owner on all four main GSC Domain properties. Server crontab entries for `/api/cron/gsc-drain` (every 5 min) and `/api/cron/gsc-audit` (daily 03:00). Backfill is **gated by the admin "Enqueue submit_sitemap for all active dealers" button** - not auto-triggered.
**Phase 4 - Full backfill + custom-domain rollout.** Click backfill → queue gets ~1,140 `submit_sitemap` jobs at priority 200, drains over hours. Daily audit populates index-status data over following days. Once subdomain submissions look healthy, enable custom-domain auto-property creation. Backfill existing active custom domains via the same admin batch button.
**Kill switch:** removing `GOOGLE_SERVICE_ACCOUNT_JSON` stops everything. Pending jobs remain in queue; resume on env var restore. No data loss.
## Out of scope / future work
The following items were discussed and explicitly deferred:
1. **Cert expiry monitoring.** Audit confirmed `Dealer.certExpiresAt` is written but never monitored. `ssl-manager.sh` has no `renew` command. Renewal implicitly depends on certbot's systemd timer. Tracked separately as [#848](https://github.com/aimclear/amsoil-dlp/issues/848): health-check route, monthly cron audit, Resend alerts at 30/14/7/1 days to expiry, docs update. **Partial mitigation in this work:** preflight check will detect expired-cert symptoms and pause GSC submissions for affected dealers.
2. **Dealer-facing GSC status view.** Add `/dashboard/indexing` for dealers showing their own indexing health, with rate-limited self-service "Request reindex" button (priority 200, 1/day/dealer). Data model already supports this - no schema change needed.
3. **Custom-domain activity auto-logging.** Currently only status transitions are logged. Future work: log DNS verification events, Cloudflare for SaaS provisioning events, etc.
4. **Indexing API expanded usage.** Only if Google opens the API to general web content, or we negotiate quota.
## Configuration
- **Env var:** `GOOGLE_SERVICE_ACCOUNT_JSON` (single-line JSON of service account key). Documented in `.env.example`.
- **System user (auto-created by migration):** `DealerNote.adminId` is non-nullable, so automated notes need a user to attribute to. A data migration upserts a designated system user with `email = 'gsc-system@aimclear.internal'`, `role = 'admin'`, `password = null`, and **no linked `Account` rows** - meaning no auth path can succeed for it (credentials provider rejects null password; OAuth providers have nothing to match). The worker looks up this user by known email (cached in module memory) when writing auto-notes. **No env var or operator step required.** As belt-and-suspenders, add an `isSystem Boolean @default(false)` column to `User` plus a guard in NextAuth's `signIn` callback that rejects `isSystem: true` - protects against future auth-path refactors. Existing auto-logging sites (e.g. status-change notes) are unaffected; they continue to attribute to the acting admin.
- **GCP project setup (one-time):**
- Create service account.
- Enable: Google Search Console API, Web Search Indexing API, Site Verification API.
- Download JSON key → encode → set env var.
- **GSC ownership (one-time per property):** add service account email as Owner in each of the four Domain properties.
- **Crontab entries (server):**
- `*/5 * * * * curl -fsS -X POST -H "Authorization: Bearer $(cat /etc/amsoil/cron-secret)" https://amsoil.aimclear.com/api/cron/gsc-drain`
- `0 3 * * * curl -fsS -X POST -H "Authorization: Bearer $(cat /etc/amsoil/cron-secret)" https://amsoil.aimclear.com/api/cron/gsc-audit`