# GSC Indexing Implementation Plan > **Dated artifact (2026-05-12) - one correction:** this plan references "Cloudflare for SaaS" for custom-domain DNS. That was never adopted. Custom domains use **certbot/Let's Encrypt with A records to the origin server IP** (see `docs/DEALER_DASHBOARD.md`). The GSC pipeline itself is unaffected. > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Build a programmatic, observable pipeline that submits dealer sitemaps to Google Search Console, monitors indexing status, auto-creates GSC properties for custom domains, and exposes admin controls - replacing the v1 prototype that was rejected during design review. **Architecture:** DB-backed job queue (`GscJob` table) drained by an HTTP worker on a server cron. Triggers (activation hooks, publish hooks, daily audit, admin button) only insert jobs; the worker is the only code that calls Google. Sitemap submission is the primary mechanism; Indexing API used sparingly for admin force-pings and audit stragglers. Custom domains get URL-prefix properties auto-created via Site Verification API with meta-tag verification. **Tech Stack:** Next.js 16 App Router, Prisma 7 / Postgres, Jest, Pino logging, NextAuth, `google-auth-library` (already installed). Custom domains use certbot/Let's Encrypt (A records to the origin server IP). **Reference docs:** - Design spec: `docs/superpowers/specs/2026-05-12-gsc-indexing-design.md` - Cert monitoring follow-up issue: [#848](https://github.com/aimclear/amsoil-dlp/issues/848) - Project conventions: `CLAUDE.md`, `docs/CRON_JOBS.md` **TDD discipline (per project requirement):** Every test gets a Step that runs it against the empty/missing implementation FIRST. The expected output of that step is a *meaningful* failure - assertion failed, not "module not found." Each test file gets a one-line comment at the top of each test stating what it would falsely pass (so a reviewer can verify the assertion is meaningful). When a step says "Expected: FAIL with X", X is the actual expected error - not a placeholder. --- ## Phase 0 - Foundation: cleanup, schema, system user ### Task 1: Delete v1 prototype files The v1 prototype was committed during early exploration; the design review rejected its architecture. We delete those files before rebuilding to avoid confusion. **Files:** - Delete: `lib/gsc-submission.ts` - Delete: `app/api/cron/gsc-submit/route.ts` (entire directory) - Delete: `app/api/admin/gsc/submit/route.ts` (entire directory) - Delete: `prisma/migrations/20260511120000_add_gsc_submitted_at/` (entire directory) - Modify: `app/api/admin/dealers/[id]/status/route.ts` - remove `submitDealerToGscBackground` import and call site - Modify: `app/api/webhooks/stripe/route.ts` - remove `submitDealerToGscBackground` import and call site - Modify: `prisma/schema.prisma` - remove the `gscSubmittedAt DateTime?` line added by v1 (will be reintroduced as `gscLastSubmittedAt` in Task 3) - [ ] **Step 1: Remove the v1 prototype source files and migration** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing rm lib/gsc-submission.ts rm -rf app/api/cron/gsc-submit rm -rf app/api/admin/gsc rm -rf prisma/migrations/20260511120000_add_gsc_submitted_at ``` - [ ] **Step 2: Revert the import + call in `app/api/admin/dealers/[id]/status/route.ts`** Remove this line from the imports block: ```ts import { submitDealerToGscBackground } from '@/lib/gsc-submission'; ``` Remove this block (the GSC submission call that runs after the cache invalidation): ```ts // Submit to Google Search Console when a dealer becomes active if (isActivating && dealer.subdomain) { submitDealerToGscBackground({ id: dealer.id, subdomain: dealer.subdomain, domain: dealer.domain, domainPrefix: dealer.domainPrefix, customDomain: null, customDomainStatus: 'none', }); } ``` - [ ] **Step 3: Revert the import + call in `app/api/webhooks/stripe/route.ts`** Remove this line from the imports block: ```ts import { submitDealerToGscBackground } from '@/lib/gsc-submission'; ``` Remove this block inside the cache-invalidation conditional: ```ts // Submit to GSC when transitioning to active for the first time if (newStatus === 'active' && beforeState.status !== 'active') { submitDealerToGscBackground({ id: dealer.id, subdomain: dealer.subdomain, domain: dealer.domain, domainPrefix: dealer.domainPrefix, customDomain: dealer.customDomain, customDomainStatus: dealer.customDomainStatus, }); } ``` - [ ] **Step 4: Revert the `gscSubmittedAt` line in `prisma/schema.prisma`** In the `Dealer` model, remove this line: ```prisma gscSubmittedAt DateTime? // Last time this dealer's sitemap+homepage was submitted to GSC ``` - [ ] **Step 5: Run type-check to confirm nothing else references the deleted symbols** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npx tsc --noEmit 2>&1 | grep -E "gsc-submission|submitDealerToGsc|gscSubmittedAt" ``` Expected: no output (no references remain). - [ ] **Step 6: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add -A git commit -m "$(cat <<'EOF' chore(gsc): remove v1 prototype before redesign Deletes superseded files. Reverts in-tree references in status and Stripe webhook routes. Removes the v1 gscSubmittedAt column from the schema (will be reintroduced as gscLastSubmittedAt in the redesign). Co-Authored-By: Claude Opus 4.7 (1M context) EOF )" ``` --- ### Task 2: Add `isSystem` flag to User + NextAuth guard in BOTH config files Both `lib/auth.ts` and `lib/auth-config.ts` define their own `signIn` callback (the latter is a simplified version for Server Actions). The QA review found the original plan only patched one. Both need to reject system users, and the guard must run **before** the existing `if (!user.email || !account)` early-return (otherwise system-user attempts with no account return a redirect instead of `false`). To make the guard testable in isolation, extract it to a tiny `lib/auth-system-guard.ts` helper that both callbacks invoke. **Files:** - Modify: `prisma/schema.prisma` - add `isSystem Boolean @default(false)` to `User` - Create: `prisma/migrations/_add_is_system_to_user/migration.sql` - Create: `lib/auth-system-guard.ts` - Create: `lib/__tests__/auth-system-guard.test.ts` - Modify: `lib/auth.ts` - call guard as first step in signIn callback - Modify: `lib/auth-config.ts` - same - [ ] **Step 1: Add the schema field** In `prisma/schema.prisma`, in the `User` model, add this line just before `accounts Account[]`: ```prisma isSystem Boolean @default(false) ``` - [ ] **Step 2: Generate the migration** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npx prisma migrate dev --name add_is_system_to_user --create-only ``` Verify the generated `migration.sql` includes: ```sql ALTER TABLE "User" ADD COLUMN "isSystem" BOOLEAN NOT NULL DEFAULT false; ``` - [ ] **Step 3: Write the failing test for the guard helper** Create `lib/__tests__/auth-system-guard.test.ts`: ```ts // Falsely-passes guard: a helper that ALWAYS returns true would pass the // "allows normal user" test but fail the system-user-rejected test. A helper // that ALWAYS returns false would pass system-user but fail normal-user. // Both directions must be tested. import { isSystemUserByEmail } from '../auth-system-guard'; jest.mock('../prisma', () => ({ prisma: { user: { findUnique: jest.fn() } }, })); import { prisma } from '../prisma'; describe('lib/auth-system-guard', () => { beforeEach(() => jest.clearAllMocks()); it('returns true for a known system user email', async () => { (prisma.user.findUnique as jest.Mock).mockResolvedValue({ isSystem: true }); const result = await isSystemUserByEmail('gsc-system@aimclear.internal'); expect(result).toBe(true); expect(prisma.user.findUnique).toHaveBeenCalledWith({ where: { email: 'gsc-system@aimclear.internal' }, select: { isSystem: true }, }); }); it('returns false for a normal user', async () => { (prisma.user.findUnique as jest.Mock).mockResolvedValue({ isSystem: false }); const result = await isSystemUserByEmail('alice@example.com'); expect(result).toBe(false); }); it('returns false for an unknown email', async () => { (prisma.user.findUnique as jest.Mock).mockResolvedValue(null); const result = await isSystemUserByEmail('nobody@example.com'); expect(result).toBe(false); }); it('returns false (fails closed for non-system) on missing email', async () => { const result = await isSystemUserByEmail(''); expect(result).toBe(false); expect(prisma.user.findUnique).not.toHaveBeenCalled(); }); }); ``` - [ ] **Step 4: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/auth-system-guard.test.ts ``` Expected: FAIL - `Cannot find module '../auth-system-guard'`. - [ ] **Step 5: Implement the helper** Create `lib/auth-system-guard.ts`: ```ts import { prisma } from './prisma'; /** * Returns true if the given email belongs to a system user (e.g. gsc-system). * System users are seeded by data migrations and must never be allowed to * authenticate via any path (OAuth, credentials, etc.). * * Defense-in-depth: the credentials path is already blocked by `password=null` * and the OAuth path is already blocked by `no Account rows`. This helper * adds an explicit early-return so future auth-path additions don't bypass * the invariant by accident. */ export async function isSystemUserByEmail(email: string): Promise { if (!email) return false; const u = await prisma.user.findUnique({ where: { email }, select: { isSystem: true }, }); return u?.isSystem === true; } ``` - [ ] **Step 6: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/auth-system-guard.test.ts ``` Expected: 4 passing. - [ ] **Step 7: Wire the guard into `lib/auth.ts`** Find the `async signIn({ user, account, profile: _profile })` callback. Add the guard **as the first statement, BEFORE the existing `if (!user.email || !account)` check**: ```ts async signIn({ user, account, profile: _profile }) { // GSC: reject system users — they exist only to attribute auto-notes. if (user.email && await isSystemUserByEmail(user.email)) { authLogger.warn({ email: user.email }, '[SignIn] Rejected system-user login attempt'); return false; } if (!user.email || !account) { authLogger.error('[SignIn] Missing email or account data'); return '/auth/error?error=Configuration'; } // ... existing body unchanged ``` Add import at the top: ```ts import { isSystemUserByEmail } from './auth-system-guard'; ``` - [ ] **Step 8: Wire the guard into `lib/auth-config.ts`** Do the same: add the same import, add the same guard as the first statement of that file's signIn callback. - [ ] **Step 9: Apply migration to local DB** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npx prisma migrate dev ``` - [ ] **Step 10: Run the broader auth test suite to confirm no regressions** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/auth lib/__tests__/auth ``` Expected: existing tests still pass (the guard only activates for system-user emails, which won't appear in existing test fixtures). - [ ] **Step 11: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add prisma/schema.prisma prisma/migrations lib/auth-system-guard.ts lib/__tests__/auth-system-guard.test.ts lib/auth.ts lib/auth-config.ts git commit -m "$(cat <<'EOF' feat(auth): add isSystem flag + signIn guard in both auth configs The guard rejects system users (e.g. gsc-system, seeded by upcoming GSC migration) early in both lib/auth.ts and lib/auth-config.ts. The helper is extracted to lib/auth-system-guard.ts for isolated testing. Co-Authored-By: Claude Opus 4.7 (1M context) EOF )" ``` --- ### Task 3: Schema migration - GSC tables and Dealer columns Single migration that adds the queue table, quota ledger, all new `Dealer.gsc*` columns, and seeds the system user. **Files:** - Modify: `prisma/schema.prisma` - Create: `prisma/migrations/_gsc_indexing/migration.sql` - [ ] **Step 1: Add the new columns to the `Dealer` model** In `prisma/schema.prisma`, locate the `Dealer` model and add these lines just before `createdAt`: ```prisma // Google Search Console state gscVerificationToken String? gscPropertyRegisteredAt DateTime? gscLastInspectedAt DateTime? gscLastIndexStatus String? // indexed | not_indexed | error gscLastSubmittedAt DateTime? gscPreflightHealthyAt DateTime? gscPreflightError String? ``` - [ ] **Step 2: Add the `GscJob` model** In `prisma/schema.prisma`, after the existing models, add: ```prisma model GscJob { id String @id @default(cuid()) type String // verify_property | register_property | submit_sitemap | // inspect_url | notify_indexing | delete_property | delete_sitemap dealerId String payload Json status String @default("pending") // pending | running | completed | failed priority Int @default(100) attempts Int @default(0) maxAttempts Int @default(5) lastError String? @db.Text runAt DateTime @default(now()) startedAt DateTime? completedAt DateTime? createdAt DateTime @default(now()) updatedAt DateTime @updatedAt dealer Dealer @relation(fields: [dealerId], references: [id], onDelete: Cascade) @@index([status, runAt, priority]) @@index([dealerId]) @@index([type, status]) } ``` - [ ] **Step 3: Add the `GscQuotaUsage` model** ```prisma model GscQuotaUsage { date DateTime @id @db.Date indexingApiCalls Int @default(0) } ``` - [ ] **Step 4: Add the back-relation on `Dealer`** In the `Dealer` model, alongside the other relation arrays (`leads`, `linkClicks`, etc.), add: ```prisma gscJobs GscJob[] ``` - [ ] **Step 5: Generate the migration** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npx prisma migrate dev --name gsc_indexing --create-only ``` Inspect the generated SQL; it should include `CREATE TABLE "GscJob"`, `CREATE TABLE "GscQuotaUsage"`, `ALTER TABLE "Dealer" ADD COLUMN` for each of the seven new columns, and three indexes on `GscJob`. - [ ] **Step 6: Append a data-migration step that seeds the system user** At the end of the generated `migration.sql` file, append: ```sql -- Seed the GSC system user (idempotent) INSERT INTO "User" ("id", "email", "name", "role", "password", "isSystem", "createdAt", "updatedAt") VALUES ( 'gsc-system-user', 'gsc-system@aimclear.internal', 'GSC System', 'admin', NULL, TRUE, NOW(), NOW() ) ON CONFLICT ("email") DO NOTHING; ``` The deterministic `id = 'gsc-system-user'` makes lookups stable across environments without needing an env var. - [ ] **Step 7: Apply migration** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npx prisma migrate dev ``` - [ ] **Step 8: Verify the system user exists** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npx prisma studio & # Or in psql: # SELECT id, email, role, "isSystem", password FROM "User" WHERE email = 'gsc-system@aimclear.internal'; ``` Verify: one row, `isSystem = true`, `password = null`. - [ ] **Step 9: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add prisma/schema.prisma prisma/migrations git commit -m "$(cat <<'EOF' feat(gsc): add GscJob, GscQuotaUsage tables and Dealer.gsc* columns Migration also seeds the gsc-system@aimclear.internal user used by the worker for auto-logged DealerNotes. Co-Authored-By: Claude Opus 4.7 (1M context) EOF )" ``` --- ## Phase 1 - Google API clients (each with isolated tests) ### Task 4: Extend `lib/google/indexing.ts` with error classification The v1 file works but doesn't classify errors. We add a typed error so the worker can decide retry vs. permanent fail without parsing strings. **Files:** - Modify: `lib/google/indexing.ts` - Create: `lib/google/__tests__/indexing.test.ts` - [ ] **Step 1: Write the failing test** Create `lib/google/__tests__/indexing.test.ts`: ```ts // Falsely-passes guard: mocking the fetch but not asserting on URL/body would // silently pass even if the implementation calls the wrong endpoint. import { notifyUrlUpdated, GoogleApiError } from '../indexing'; describe('lib/google/indexing', () => { const realFetch = global.fetch; afterEach(() => { global.fetch = realFetch; }); it('POSTs the URL to urlNotifications:publish with type URL_UPDATED', async () => { let captured: { url: string; init: RequestInit } | null = null; global.fetch = (async (url: string, init: RequestInit) => { captured = { url, init }; return { ok: true, status: 200, text: async () => '' } as Response; }) as typeof fetch; // Stub the auth client so we don't need real creds in tests. jest.spyOn(require('../auth'), 'getGoogleAuthClient').mockReturnValue({ getAccessToken: async () => ({ token: 'test-token' }), }); await notifyUrlUpdated('https://example.com/'); expect(captured?.url).toBe('https://indexing.googleapis.com/v3/urlNotifications:publish'); expect(JSON.parse(captured?.init.body as string)).toEqual({ url: 'https://example.com/', type: 'URL_UPDATED', }); }); it('throws GoogleApiError with status 429 retryable=true on rate-limit', async () => { global.fetch = (async () => ({ ok: false, status: 429, text: async () => '{"error":"rate limit"}', } as Response)) as typeof fetch; jest.spyOn(require('../auth'), 'getGoogleAuthClient').mockReturnValue({ getAccessToken: async () => ({ token: 'test-token' }), }); await expect(notifyUrlUpdated('https://example.com/')).rejects.toMatchObject({ name: 'GoogleApiError', status: 429, retryable: true, }); }); it('throws GoogleApiError with retryable=false on 403', async () => { global.fetch = (async () => ({ ok: false, status: 403, text: async () => '{"error":"forbidden"}', } as Response)) as typeof fetch; jest.spyOn(require('../auth'), 'getGoogleAuthClient').mockReturnValue({ getAccessToken: async () => ({ token: 'test-token' }), }); await expect(notifyUrlUpdated('https://example.com/')).rejects.toMatchObject({ name: 'GoogleApiError', status: 403, retryable: false, }); }); }); ``` - [ ] **Step 2: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/google/__tests__/indexing.test.ts ``` Expected: FAIL - `GoogleApiError` not exported, or thrown error doesn't have `.retryable` property. - [ ] **Step 3: Implement `GoogleApiError` + classification in `lib/google/indexing.ts`** Replace the existing file content: ```ts import { getGoogleAuthClient, INDEXING_SCOPE } from './auth'; const INDEXING_API_URL = 'https://indexing.googleapis.com/v3/urlNotifications:publish'; export class GoogleApiError extends Error { constructor( public status: number, public retryable: boolean, public body: string, ) { super(`Google API ${status}: ${body}`); this.name = 'GoogleApiError'; } } export function isRetryableStatus(status: number): boolean { return status === 429 || status >= 500; } export async function notifyUrlUpdated(url: string): Promise { const client = getGoogleAuthClient([INDEXING_SCOPE]); const { token } = await client.getAccessToken(); if (!token) throw new Error('Failed to obtain access token for Indexing API'); const res = await fetch(INDEXING_API_URL, { method: 'POST', headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ url, type: 'URL_UPDATED' }), }); if (!res.ok) { const body = await res.text(); throw new GoogleApiError(res.status, isRetryableStatus(res.status), body); } } ``` - [ ] **Step 4: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/google/__tests__/indexing.test.ts ``` Expected: 3 passing tests. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/google/indexing.ts lib/google/__tests__/indexing.test.ts git commit -m "test(gsc): indexing API client with error classification Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 5: Extend `lib/google/search-console.ts` for delete + propagate errors **Files:** - Modify: `lib/google/search-console.ts` - add `deleteSitemap`, use `GoogleApiError` - Create: `lib/google/__tests__/search-console.test.ts` - [ ] **Step 1: Write the failing test** Create `lib/google/__tests__/search-console.test.ts`: ```ts // Falsely-passes guard: an assertion on `expect(true).toBe(true)` after the call // would pass even if the URL encoding is wrong. import { submitSitemap, deleteSitemap } from '../search-console'; describe('lib/google/search-console', () => { const realFetch = global.fetch; afterEach(() => { global.fetch = realFetch; }); beforeEach(() => { jest.spyOn(require('../auth'), 'getGoogleAuthClient').mockReturnValue({ getAccessToken: async () => ({ token: 'test-token' }), }); }); it('submitSitemap PUTs the correctly-encoded endpoint for sc-domain property', async () => { let capturedUrl = ''; let capturedMethod = ''; global.fetch = (async (url: string, init: RequestInit) => { capturedUrl = url; capturedMethod = init.method!; return { ok: true, status: 200, text: async () => '' } as Response; }) as typeof fetch; await submitSitemap('sc-domain:myamsoil.com', 'https://bobsoil.myamsoil.com/sitemap.xml'); expect(capturedMethod).toBe('PUT'); expect(capturedUrl).toBe( 'https://www.googleapis.com/webmasters/v3/sites/sc-domain%3Amyamsoil.com/sitemaps/https%3A%2F%2Fbobsoil.myamsoil.com%2Fsitemap.xml' ); }); it('deleteSitemap DELETEs the same endpoint shape', async () => { let capturedMethod = ''; global.fetch = (async (_: string, init: RequestInit) => { capturedMethod = init.method!; return { ok: true, status: 204, text: async () => '' } as Response; }) as typeof fetch; await deleteSitemap('sc-domain:myamsoil.com', 'https://bobsoil.myamsoil.com/sitemap.xml'); expect(capturedMethod).toBe('DELETE'); }); it('propagates GoogleApiError on non-2xx', async () => { global.fetch = (async () => ({ ok: false, status: 500, text: async () => 'server error', } as Response)) as typeof fetch; await expect( submitSitemap('sc-domain:myamsoil.com', 'https://x.com/sitemap.xml') ).rejects.toMatchObject({ name: 'GoogleApiError', status: 500, retryable: true }); }); }); ``` - [ ] **Step 2: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/google/__tests__/search-console.test.ts ``` Expected: FAIL - `deleteSitemap` not exported, or no `GoogleApiError`. - [ ] **Step 3: Implement** Replace `lib/google/search-console.ts`: ```ts import { getGoogleAuthClient, SEARCH_CONSOLE_SCOPE } from './auth'; import { GoogleApiError, isRetryableStatus } from './indexing'; const SC_API_BASE = 'https://www.googleapis.com/webmasters/v3/sites'; async function callSitemapApi(method: 'PUT' | 'DELETE', siteUrl: string, sitemapUrl: string): Promise { const client = getGoogleAuthClient([SEARCH_CONSOLE_SCOPE]); const { token } = await client.getAccessToken(); if (!token) throw new Error('Failed to obtain access token for Search Console API'); const endpoint = `${SC_API_BASE}/${encodeURIComponent(siteUrl)}/sitemaps/${encodeURIComponent(sitemapUrl)}`; const res = await fetch(endpoint, { method, headers: { Authorization: `Bearer ${token}` } }); if (!res.ok) { const body = await res.text(); throw new GoogleApiError(res.status, isRetryableStatus(res.status), body); } } export async function submitSitemap(siteUrl: string, sitemapUrl: string): Promise { return callSitemapApi('PUT', siteUrl, sitemapUrl); } export async function deleteSitemap(siteUrl: string, sitemapUrl: string): Promise { return callSitemapApi('DELETE', siteUrl, sitemapUrl); } ``` - [ ] **Step 4: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/google/__tests__/search-console.test.ts ``` Expected: 3 passing. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/google/search-console.ts lib/google/__tests__/search-console.test.ts git commit -m "feat(gsc): sitemap delete + GoogleApiError propagation Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 6: New `lib/google/url-inspection.ts` **Files:** - Create: `lib/google/url-inspection.ts` - Create: `lib/google/__tests__/url-inspection.test.ts` - [ ] **Step 1: Write the failing test** ```ts // Falsely-passes guard: returning a hardcoded { status: 'indexed' } would pass // indexed-case but fail not-indexed/error. All three must be tested. import { inspectUrl, IndexStatus } from '../url-inspection'; describe('lib/google/url-inspection', () => { const realFetch = global.fetch; afterEach(() => { global.fetch = realFetch; }); beforeEach(() => { jest.spyOn(require('../auth'), 'getGoogleAuthClient').mockReturnValue({ getAccessToken: async () => ({ token: 'test-token' }), }); }); it('parses response as "indexed" when verdict is PASS and coverage is "Submitted and indexed"', async () => { global.fetch = (async () => ({ ok: true, status: 200, json: async () => ({ inspectionResult: { indexStatusResult: { verdict: 'PASS', coverageState: 'Submitted and indexed', }, }, }), } as unknown as Response)) as typeof fetch; const result = await inspectUrl('sc-domain:myamsoil.com', 'https://bobsoil.myamsoil.com/'); expect(result.status).toBe('indexed'); }); it('parses response as "not_indexed" when verdict is FAIL', async () => { global.fetch = (async () => ({ ok: true, status: 200, json: async () => ({ inspectionResult: { indexStatusResult: { verdict: 'FAIL', coverageState: 'Crawled - currently not indexed' }, }, }), } as unknown as Response)) as typeof fetch; const result = await inspectUrl('sc-domain:myamsoil.com', 'https://bobsoil.myamsoil.com/'); expect(result.status).toBe('not_indexed'); }); it('throws GoogleApiError on non-2xx', async () => { global.fetch = (async () => ({ ok: false, status: 403, text: async () => 'forbidden', } as Response)) as typeof fetch; await expect(inspectUrl('sc-domain:myamsoil.com', 'https://x.com/')) .rejects.toMatchObject({ name: 'GoogleApiError', status: 403 }); }); }); ``` - [ ] **Step 2: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/google/__tests__/url-inspection.test.ts ``` Expected: FAIL - module not found. - [ ] **Step 3: Implement** Create `lib/google/url-inspection.ts`: ```ts import { getGoogleAuthClient, SEARCH_CONSOLE_SCOPE } from './auth'; import { GoogleApiError, isRetryableStatus } from './indexing'; const ENDPOINT = 'https://searchconsole.googleapis.com/v1/urlInspection/index:inspect'; export type IndexStatus = 'indexed' | 'not_indexed' | 'error'; export interface InspectionResult { status: IndexStatus; coverageState?: string; verdict?: string; } export async function inspectUrl(siteUrl: string, inspectionUrl: string): Promise { const client = getGoogleAuthClient([SEARCH_CONSOLE_SCOPE]); const { token } = await client.getAccessToken(); if (!token) throw new Error('Failed to obtain access token for URL Inspection API'); const res = await fetch(ENDPOINT, { method: 'POST', headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ inspectionUrl, siteUrl }), }); if (!res.ok) { const body = await res.text(); throw new GoogleApiError(res.status, isRetryableStatus(res.status), body); } const data = await res.json(); const indexResult = data?.inspectionResult?.indexStatusResult ?? {}; const verdict: string | undefined = indexResult.verdict; const coverageState: string | undefined = indexResult.coverageState; let status: IndexStatus = 'error'; if (verdict === 'PASS') status = 'indexed'; else if (verdict === 'FAIL' || verdict === 'NEUTRAL') status = 'not_indexed'; return { status, coverageState, verdict }; } ``` - [ ] **Step 4: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/google/__tests__/url-inspection.test.ts ``` Expected: 3 passing. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/google/url-inspection.ts lib/google/__tests__/url-inspection.test.ts git commit -m "feat(gsc): URL inspection API client Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 7: New `lib/google/site-verification.ts` **Files:** - Create: `lib/google/site-verification.ts` - Create: `lib/google/__tests__/site-verification.test.ts` - [ ] **Step 1: Write the failing test** ```ts // Falsely-passes guard: testing only that fetch is called would miss whether // the right METHOD or BODY is used. import { getVerificationToken, verifyOwnership, deleteVerification } from '../site-verification'; describe('lib/google/site-verification', () => { const realFetch = global.fetch; afterEach(() => { global.fetch = realFetch; }); beforeEach(() => { jest.spyOn(require('../auth'), 'getGoogleAuthClient').mockReturnValue({ getAccessToken: async () => ({ token: 'test-token' }), }); }); it('getVerificationToken returns the META verification token', async () => { global.fetch = (async () => ({ ok: true, status: 200, json: async () => ({ token: 'verify-abc123' }), } as unknown as Response)) as typeof fetch; const token = await getVerificationToken('https://example.com/'); expect(token).toBe('verify-abc123'); }); it('verifyOwnership POSTs to webResource:insert with META method', async () => { let capturedBody = ''; let capturedUrl = ''; global.fetch = (async (url: string, init: RequestInit) => { capturedUrl = url; capturedBody = init.body as string; return { ok: true, status: 200, json: async () => ({ id: 'site-123' }) } as unknown as Response; }) as typeof fetch; await verifyOwnership('https://example.com/', 'META'); expect(capturedUrl).toContain('webResource?verificationMethod=META'); expect(JSON.parse(capturedBody)).toMatchObject({ site: { identifier: 'https://example.com/', type: 'SITE' }, }); }); it('deleteVerification DELETEs the resource by encoded id', async () => { let capturedMethod = ''; let capturedUrl = ''; global.fetch = (async (url: string, init: RequestInit) => { capturedMethod = init.method!; capturedUrl = url; return { ok: true, status: 204, text: async () => '' } as Response; }) as typeof fetch; await deleteVerification('https://example.com/'); expect(capturedMethod).toBe('DELETE'); expect(capturedUrl).toContain(encodeURIComponent('https://example.com/')); }); }); ``` - [ ] **Step 2: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/google/__tests__/site-verification.test.ts ``` Expected: FAIL - module not found. - [ ] **Step 3: Add the Site Verification scope to `lib/google/auth.ts`** Add this constant near the existing scope constants: ```ts export const SITE_VERIFICATION_SCOPE = 'https://www.googleapis.com/auth/siteverification'; ``` - [ ] **Step 4: Implement `lib/google/site-verification.ts`** ```ts import { getGoogleAuthClient, SITE_VERIFICATION_SCOPE } from './auth'; import { GoogleApiError, isRetryableStatus } from './indexing'; const TOKEN_ENDPOINT = 'https://www.googleapis.com/siteVerification/v1/token'; const RESOURCE_ENDPOINT = 'https://www.googleapis.com/siteVerification/v1/webResource'; async function getToken(): Promise { const client = getGoogleAuthClient([SITE_VERIFICATION_SCOPE]); const { token } = await client.getAccessToken(); if (!token) throw new Error('Failed to obtain access token for Site Verification API'); return token; } async function handleResponse(res: Response): Promise { if (!res.ok) { const body = await res.text(); throw new GoogleApiError(res.status, isRetryableStatus(res.status), body); } return res.json(); } export async function getVerificationToken(siteUrl: string): Promise { const token = await getToken(); const res = await fetch(TOKEN_ENDPOINT, { method: 'POST', headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ verificationMethod: 'META', site: { identifier: siteUrl, type: 'SITE' }, }), }); const data = await handleResponse<{ token: string }>(res); return data.token; } export async function verifyOwnership(siteUrl: string, method: 'META' = 'META'): Promise { const token = await getToken(); const res = await fetch(`${RESOURCE_ENDPOINT}?verificationMethod=${method}`, { method: 'POST', headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' }, body: JSON.stringify({ site: { identifier: siteUrl, type: 'SITE' } }), }); await handleResponse(res); } export async function deleteVerification(siteUrl: string): Promise { const token = await getToken(); const res = await fetch(`${RESOURCE_ENDPOINT}/${encodeURIComponent(siteUrl)}`, { method: 'DELETE', headers: { Authorization: `Bearer ${token}` }, }); if (!res.ok) { const body = await res.text(); throw new GoogleApiError(res.status, isRetryableStatus(res.status), body); } } ``` - [ ] **Step 5: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/google/__tests__/site-verification.test.ts ``` Expected: 3 passing. - [ ] **Step 6: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/google/site-verification.ts lib/google/auth.ts lib/google/__tests__/site-verification.test.ts git commit -m "feat(gsc): Site Verification API client (getToken/verify/delete) Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 8: New `lib/google/sites.ts` **Files:** - Create: `lib/google/sites.ts` - Create: `lib/google/__tests__/sites.test.ts` - [ ] **Step 1: Write the failing test** ```ts // Falsely-passes guard: testing only the return value would miss whether the // PUT vs DELETE method is used correctly. import { addProperty, deleteProperty } from '../sites'; describe('lib/google/sites', () => { const realFetch = global.fetch; afterEach(() => { global.fetch = realFetch; }); beforeEach(() => { jest.spyOn(require('../auth'), 'getGoogleAuthClient').mockReturnValue({ getAccessToken: async () => ({ token: 'test-token' }), }); }); it('addProperty PUTs to /sites/{encoded}', async () => { let capturedUrl = ''; let capturedMethod = ''; global.fetch = (async (url: string, init: RequestInit) => { capturedUrl = url; capturedMethod = init.method!; return { ok: true, status: 200, text: async () => '' } as Response; }) as typeof fetch; await addProperty('https://example.com/'); expect(capturedMethod).toBe('PUT'); expect(capturedUrl).toBe( `https://www.googleapis.com/webmasters/v3/sites/${encodeURIComponent('https://example.com/')}` ); }); it('deleteProperty DELETEs to /sites/{encoded}', async () => { let capturedMethod = ''; global.fetch = (async (_: string, init: RequestInit) => { capturedMethod = init.method!; return { ok: true, status: 204, text: async () => '' } as Response; }) as typeof fetch; await deleteProperty('https://example.com/'); expect(capturedMethod).toBe('DELETE'); }); }); ``` - [ ] **Step 2: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/google/__tests__/sites.test.ts ``` Expected: FAIL - module not found. - [ ] **Step 3: Implement** ```ts import { getGoogleAuthClient, SEARCH_CONSOLE_SCOPE } from './auth'; import { GoogleApiError, isRetryableStatus } from './indexing'; const SITES_API_BASE = 'https://www.googleapis.com/webmasters/v3/sites'; async function call(method: 'PUT' | 'DELETE', siteUrl: string): Promise { const client = getGoogleAuthClient([SEARCH_CONSOLE_SCOPE]); const { token } = await client.getAccessToken(); if (!token) throw new Error('Failed to obtain access token for Sites API'); const res = await fetch(`${SITES_API_BASE}/${encodeURIComponent(siteUrl)}`, { method, headers: { Authorization: `Bearer ${token}` }, }); if (!res.ok) { const body = await res.text(); throw new GoogleApiError(res.status, isRetryableStatus(res.status), body); } } export async function addProperty(siteUrl: string): Promise { return call('PUT', siteUrl); } export async function deleteProperty(siteUrl: string): Promise { return call('DELETE', siteUrl); } ``` - [ ] **Step 4: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/google/__tests__/sites.test.ts ``` Expected: 2 passing. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/google/sites.ts lib/google/__tests__/sites.test.ts git commit -m "feat(gsc): Sites API client (add/delete property) Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ## Phase 2 - Preflight check ### Task 9: `lib/gsc-preflight.ts` **Files:** - Create: `lib/gsc-preflight.ts` - Create: `lib/__tests__/gsc-preflight.test.ts` - [ ] **Step 1: Write the failing test** ```ts // Falsely-passes guard: returning { healthy: true } unconditionally would pass // the happy-path test but fail the SSL and 5xx cases. import { preflight } from '../gsc-preflight'; const baseDealer = { id: 'd1', subdomain: 'bob', domain: 'us', domainPrefix: 'myamsoil', customDomain: null, customDomainStatus: 'none', gscPreflightHealthyAt: null, gscPreflightError: null, }; describe('lib/gsc-preflight', () => { const realFetch = global.fetch; afterEach(() => { global.fetch = realFetch; jest.restoreAllMocks(); }); it('returns healthy on 200 from HEAD', async () => { global.fetch = (async () => ({ ok: true, status: 200 } as Response)) as typeof fetch; const r = await preflight(baseDealer); expect(r.healthy).toBe(true); }); it('returns unhealthy with reason on SSL error (fetch throws)', async () => { global.fetch = (async () => { throw new Error('SSL_ERROR_BAD_CERT'); }) as typeof fetch; const r = await preflight(baseDealer); expect(r.healthy).toBe(false); expect(r.reason).toContain('SSL_ERROR_BAD_CERT'); }); it('returns unhealthy on 5xx response', async () => { global.fetch = (async () => ({ ok: false, status: 503 } as Response)) as typeof fetch; const r = await preflight(baseDealer); expect(r.healthy).toBe(false); expect(r.reason).toContain('503'); }); it('returns cached healthy when gscPreflightHealthyAt within TTL', async () => { const dealer = { ...baseDealer, gscPreflightHealthyAt: new Date(Date.now() - 60_000) }; let fetchCalled = false; global.fetch = (async () => { fetchCalled = true; return { ok: true, status: 200 } as Response; }) as typeof fetch; const r = await preflight(dealer); expect(r.healthy).toBe(true); expect(fetchCalled).toBe(false); }); }); ``` - [ ] **Step 2: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-preflight.test.ts ``` Expected: FAIL - module not found. - [ ] **Step 3: Implement** ```ts import { getCanonicalUrl } from './seo-utils'; const TTL_MS = 6 * 60 * 60 * 1000; // 6 hours const TIMEOUT_MS = 10_000; const ALLOWED_STATUSES = new Set([200, 301, 302, 304]); export interface PreflightInput { id: string; subdomain: string | null; domain: string; domainPrefix: string; customDomain: string | null; customDomainStatus: string; gscPreflightHealthyAt: Date | null; gscPreflightError?: string | null; } export interface PreflightResult { healthy: boolean; reason?: string; url?: string; } export async function preflight(dealer: PreflightInput): Promise { if (dealer.gscPreflightHealthyAt && Date.now() - dealer.gscPreflightHealthyAt.getTime() < TTL_MS) { return { healthy: true }; } const url = getCanonicalUrl(dealer, ''); try { const res = await fetch(url, { method: 'HEAD', redirect: 'manual', signal: AbortSignal.timeout(TIMEOUT_MS), }); if (ALLOWED_STATUSES.has(res.status)) return { healthy: true, url }; return { healthy: false, reason: `HTTP ${res.status} from HEAD ${url}`, url }; } catch (err) { return { healthy: false, reason: (err as Error).message, url }; } } ``` - [ ] **Step 4: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-preflight.test.ts ``` Expected: 4 passing. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/gsc-preflight.ts lib/__tests__/gsc-preflight.test.ts git commit -m "feat(gsc): preflight HEAD check with 6h cache TTL Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ## Phase 3 - Job enqueue functions ### Task 10: `lib/gsc-system-user.ts` - system user lookup **Files:** - Create: `lib/gsc-system-user.ts` - Create: `lib/__tests__/gsc-system-user.test.ts` - [ ] **Step 1: Write the failing test** ```ts // Falsely-passes guard: a stubbed-return that always yields 'gsc-system-user' // would pass without actually querying — make the test verify the DB lookup // happens at least once. import { getSystemUserId } from '../gsc-system-user'; jest.mock('../prisma', () => ({ prisma: { user: { findUnique: jest.fn() } }, })); import { prisma } from '../prisma'; describe('lib/gsc-system-user', () => { beforeEach(() => jest.clearAllMocks()); it('looks up by known email on first call', async () => { (prisma.user.findUnique as jest.Mock).mockResolvedValueOnce({ id: 'gsc-system-user' }); const id = await getSystemUserId(); expect(id).toBe('gsc-system-user'); expect(prisma.user.findUnique).toHaveBeenCalledWith({ where: { email: 'gsc-system@aimclear.internal' }, select: { id: true }, }); }); it('caches subsequent calls (only one DB lookup)', async () => { (prisma.user.findUnique as jest.Mock).mockResolvedValue({ id: 'gsc-system-user' }); await getSystemUserId(); await getSystemUserId(); expect(prisma.user.findUnique).toHaveBeenCalledTimes(1); }); it('throws if system user is missing', async () => { (prisma.user.findUnique as jest.Mock).mockResolvedValueOnce(null); // Clear module cache between tests jest.resetModules(); const { getSystemUserId: fresh } = await import('../gsc-system-user'); await expect(fresh()).rejects.toThrow(/system user/i); }); }); ``` - [ ] **Step 2: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-system-user.test.ts ``` Expected: FAIL - module not found. - [ ] **Step 3: Implement** ```ts import { prisma } from './prisma'; const SYSTEM_USER_EMAIL = 'gsc-system@aimclear.internal'; let cachedId: string | null = null; export async function getSystemUserId(): Promise { if (cachedId) return cachedId; const user = await prisma.user.findUnique({ where: { email: SYSTEM_USER_EMAIL }, select: { id: true }, }); if (!user) throw new Error( `GSC system user not found (expected email: ${SYSTEM_USER_EMAIL}). Did the migration run?` ); cachedId = user.id; return cachedId; } ``` - [ ] **Step 4: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-system-user.test.ts ``` Expected: 3 passing. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/gsc-system-user.ts lib/__tests__/gsc-system-user.test.ts git commit -m "feat(gsc): system user lookup with module-memory cache Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 11: `lib/gsc-jobs.ts` - enqueue functions One function per trigger event. Each inserts the right job row(s) with the right priority. **Files:** - Create: `lib/gsc-jobs.ts` - Create: `lib/__tests__/gsc-jobs.test.ts` - [ ] **Step 1: Write the failing test** ```ts // Falsely-passes guard: testing that prisma.gscJob.create is called without // asserting on type, dealerId, priority, or payload would pass any // implementation that just calls create(). import { enqueueDealerActivation, enqueueCustomDomainActivation, enqueuePublishEvent, enqueueAdminReindex, enqueueDealerCancellation, enqueueCustomDomainDeactivation, } from '../gsc-jobs'; jest.mock('../prisma', () => ({ prisma: { gscJob: { create: jest.fn(), createMany: jest.fn(), updateMany: jest.fn() } }, })); import { prisma } from '../prisma'; const dealer = { id: 'd1', subdomain: 'bob', domain: 'us', domainPrefix: 'myamsoil', customDomain: null, customDomainStatus: 'none', }; describe('lib/gsc-jobs', () => { beforeEach(() => jest.clearAllMocks()); it('enqueueDealerActivation inserts submit_sitemap @ priority 50 + inspect_url @ 200', async () => { await enqueueDealerActivation(dealer); const calls = (prisma.gscJob.create as jest.Mock).mock.calls.map(c => c[0].data); expect(calls).toEqual(expect.arrayContaining([ expect.objectContaining({ type: 'submit_sitemap', dealerId: 'd1', priority: 50 }), expect.objectContaining({ type: 'inspect_url', dealerId: 'd1', priority: 200 }), ])); }); it('enqueueAdminReindex inserts three jobs at priority 10', async () => { await enqueueAdminReindex(dealer); const calls = (prisma.gscJob.create as jest.Mock).mock.calls.map(c => c[0].data); const types = calls.map(c => c.type); expect(types).toEqual(expect.arrayContaining(['submit_sitemap', 'inspect_url', 'notify_indexing'])); expect(calls.every(c => c.priority === 10)).toBe(true); const ping = calls.find(c => c.type === 'notify_indexing'); expect(ping.payload).toMatchObject({ force: true }); }); it('enqueueCustomDomainActivation inserts only verify_property (chain starts there)', async () => { const cdDealer = { ...dealer, customDomain: 'example.com', customDomainStatus: 'active' }; await enqueueCustomDomainActivation(cdDealer); const calls = (prisma.gscJob.create as jest.Mock).mock.calls.map(c => c[0].data); expect(calls).toHaveLength(1); expect(calls[0]).toMatchObject({ type: 'verify_property', dealerId: 'd1', priority: 50 }); expect(calls[0].payload).toMatchObject({ propertyHost: 'example.com' }); }); it('enqueuePublishEvent inserts a submit_sitemap + one inspect_url per path', async () => { await enqueuePublishEvent(dealer, ['/services', '/about']); const calls = (prisma.gscJob.create as jest.Mock).mock.calls.map(c => c[0].data); const types = calls.map(c => c.type); expect(types).toEqual(expect.arrayContaining(['submit_sitemap', 'inspect_url', 'inspect_url'])); }); it('enqueueDealerCancellation inserts delete_sitemap', async () => { await enqueueDealerCancellation(dealer); const calls = (prisma.gscJob.create as jest.Mock).mock.calls.map(c => c[0].data); expect(calls).toEqual([expect.objectContaining({ type: 'delete_sitemap' })]); }); it('enqueueCustomDomainDeactivation cancels stale pending jobs AND inserts delete_sitemap + delete_property', async () => { await enqueueCustomDomainDeactivation(dealer, 'example.com'); // 1) Stale-job cancellation expect(prisma.gscJob.updateMany).toHaveBeenCalledWith(expect.objectContaining({ where: expect.objectContaining({ dealerId: 'd1', status: { in: ['pending', 'running'] }, type: { in: ['submit_sitemap', 'inspect_url', 'notify_indexing'] }, }), data: expect.objectContaining({ status: 'failed' }), })); // 2) Cleanup jobs enqueued const createCalls = (prisma.gscJob.create as jest.Mock).mock.calls.map(c => c[0].data); const types = createCalls.map(c => c.type); expect(types).toEqual(expect.arrayContaining(['delete_sitemap', 'delete_property'])); expect(createCalls).toHaveLength(2); // exact count assertion }); }); ``` - [ ] **Step 2: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-jobs.test.ts ``` Expected: FAIL - module not found. - [ ] **Step 3: Implement** Create `lib/gsc-jobs.ts`: ```ts import { prisma } from './prisma'; import { getCanonicalUrl } from './seo-utils'; interface DealerInput { id: string; subdomain: string | null; domain: string; domainPrefix: string; customDomain: string | null; customDomainStatus: string; } function homepageUrl(dealer: DealerInput): string { return getCanonicalUrl(dealer, ''); } function sitemapUrl(dealer: DealerInput): string { return `${homepageUrl(dealer)}/sitemap.xml`; } export async function enqueueDealerActivation(dealer: DealerInput): Promise { const url = homepageUrl(dealer); await prisma.gscJob.create({ data: { type: 'submit_sitemap', dealerId: dealer.id, priority: 50, payload: { sitemapUrl: sitemapUrl(dealer) }, }, }); await prisma.gscJob.create({ data: { type: 'inspect_url', dealerId: dealer.id, priority: 200, runAt: new Date(Date.now() + 60 * 60 * 1000), // 1h later payload: { url }, }, }); } export async function enqueueCustomDomainActivation(dealer: DealerInput): Promise { if (!dealer.customDomain) throw new Error('customDomain required'); await prisma.gscJob.create({ data: { type: 'verify_property', dealerId: dealer.id, priority: 50, payload: { propertyHost: dealer.customDomain }, }, }); } export async function enqueuePublishEvent(dealer: DealerInput, paths: string[]): Promise { await prisma.gscJob.create({ data: { type: 'submit_sitemap', dealerId: dealer.id, priority: 50, payload: { sitemapUrl: sitemapUrl(dealer) }, }, }); for (const p of paths) { await prisma.gscJob.create({ data: { type: 'inspect_url', dealerId: dealer.id, priority: 50, payload: { url: getCanonicalUrl(dealer, p) }, }, }); } } export async function enqueueAdminReindex(dealer: DealerInput): Promise { const url = homepageUrl(dealer); await prisma.gscJob.create({ data: { type: 'submit_sitemap', dealerId: dealer.id, priority: 10, payload: { sitemapUrl: sitemapUrl(dealer) } }, }); await prisma.gscJob.create({ data: { type: 'inspect_url', dealerId: dealer.id, priority: 10, payload: { url } }, }); await prisma.gscJob.create({ data: { type: 'notify_indexing', dealerId: dealer.id, priority: 10, payload: { url, force: true } }, }); } export async function enqueueDealerCancellation(dealer: DealerInput): Promise { await prisma.gscJob.create({ data: { type: 'delete_sitemap', dealerId: dealer.id, priority: 50, payload: { sitemapUrl: sitemapUrl(dealer) } }, }); } export async function enqueueCustomDomainDeactivation(dealer: DealerInput, domain: string): Promise { // CANCEL stale jobs targeting the old custom domain BEFORE enqueuing the // delete jobs. Otherwise a previously-enqueued submit_sitemap or // notify_indexing job pointing at https://{domain}/... will execute against // the parent sc-domain: property (resolved when dealer.customDomain is // null) and Google will 400. This is idempotent: cancelling already- // completed jobs is a no-op. await prisma.gscJob.updateMany({ where: { dealerId: dealer.id, status: { in: ['pending', 'running'] }, type: { in: ['submit_sitemap', 'inspect_url', 'notify_indexing'] }, // Filter by payload host where possible — Prisma JSON path filters work // on Postgres. If the payload doesn't carry propertyHost (e.g. legacy // jobs), cancel ALL pending non-delete jobs for this dealer to be safe. }, data: { status: 'failed', lastError: 'cancelled: custom domain deactivated' }, }); await prisma.gscJob.create({ data: { type: 'delete_sitemap', dealerId: dealer.id, priority: 50, payload: { sitemapUrl: `https://${domain}/sitemap.xml`, propertyHost: domain } }, }); await prisma.gscJob.create({ data: { type: 'delete_property', dealerId: dealer.id, priority: 50, payload: { propertyHost: domain } }, }); } ``` - [ ] **Step 4: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-jobs.test.ts ``` Expected: 6 passing. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/gsc-jobs.ts lib/__tests__/gsc-jobs.test.ts git commit -m "feat(gsc): job enqueue functions for each trigger event Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ## Phase 4 - Worker ### Task 12: Worker core - dispatch + error classification + backoff This task builds the worker scaffolding without the gates (quota / preflight). Those go in Task 13. Reason: keeps each task's test surface small. **Files:** - Create: `lib/gsc-worker.ts` - Create: `lib/__tests__/gsc-worker.test.ts` - [ ] **Step 1: Write the failing test** ```ts // Falsely-passes guard: returning { processed: 0 } without doing any work would // pass a happy-path test that asserts no exception. Each test must verify the // outcome state in the DB (mocks). import { runWorker } from '../gsc-worker'; jest.mock('../prisma', () => ({ prisma: { $queryRaw: jest.fn(), // atomic claim: UPDATE…RETURNING with FOR UPDATE SKIP LOCKED gscJob: { create: jest.fn(), update: jest.fn() }, // create for chain enqueues, update for state transitions dealer: { findUnique: jest.fn(), update: jest.fn() }, dealerNote: { create: jest.fn() }, // preflight transition auto-notes gscQuotaUsage: { upsert: jest.fn(), update: jest.fn() }, }, })); jest.mock('../google/search-console'); jest.mock('../google/indexing'); jest.mock('../google/url-inspection'); jest.mock('../google/site-verification'); jest.mock('../google/sites'); jest.mock('../gsc-preflight'); jest.mock('../isr-revalidation', () => ({ revalidateAllDealerPages: jest.fn().mockResolvedValue({ success: true }), })); // NOTE: the worker claims jobs via prisma.$queryRaw (atomic UPDATE…RETURNING // with FOR UPDATE SKIP LOCKED). Tests mock $queryRaw with the claimed-job // array — `findMany` is no longer used for claim and does not need mocking. import { prisma } from '../prisma'; import { submitSitemap } from '../google/search-console'; import { preflight } from '../gsc-preflight'; const baseDealer = { id: 'd1', subdomain: 'bob', domain: 'us', domainPrefix: 'myamsoil', customDomain: null, customDomainStatus: 'none', gscPreflightHealthyAt: new Date(), gscPreflightError: null, }; describe('lib/gsc-worker', () => { beforeEach(() => { jest.clearAllMocks(); (preflight as jest.Mock).mockResolvedValue({ healthy: true }); (prisma.dealer.findUnique as jest.Mock).mockResolvedValue(baseDealer); (prisma.gscQuotaUsage.upsert as jest.Mock).mockResolvedValue({ date: new Date(), indexingApiCalls: 0, }); }); it('marks job completed on successful submit_sitemap', async () => { (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'j1', type: 'submit_sitemap', dealerId: 'd1', priority: 50, attempts: 0, maxAttempts: 5, payload: { sitemapUrl: 'https://bob.myamsoil.com/sitemap.xml' } }, ]); (submitSitemap as jest.Mock).mockResolvedValue(undefined); const result = await runWorker({ batchSize: 50 }); expect(submitSitemap).toHaveBeenCalledWith( 'sc-domain:myamsoil.com', 'https://bob.myamsoil.com/sitemap.xml' ); expect(prisma.gscJob.update).toHaveBeenCalledWith(expect.objectContaining({ where: { id: 'j1' }, data: expect.objectContaining({ status: 'completed' }), })); expect(result.completed).toBe(1); }); it('marks job pending with backoff on retryable failure', async () => { (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'j2', type: 'submit_sitemap', dealerId: 'd1', priority: 50, attempts: 0, maxAttempts: 5, payload: { sitemapUrl: 'https://x/sitemap.xml' } }, ]); (submitSitemap as jest.Mock).mockRejectedValue( Object.assign(new Error('rate'), { name: 'GoogleApiError', status: 429, retryable: true }) ); await runWorker({ batchSize: 50 }); expect(prisma.gscJob.update).toHaveBeenCalledWith(expect.objectContaining({ where: { id: 'j2' }, data: expect.objectContaining({ status: 'pending', attempts: 1 }), })); }); it('marks job failed immediately on permanent error (403)', async () => { (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'j3', type: 'submit_sitemap', dealerId: 'd1', priority: 50, attempts: 0, maxAttempts: 5, payload: { sitemapUrl: 'https://x/sitemap.xml' } }, ]); (submitSitemap as jest.Mock).mockRejectedValue( Object.assign(new Error('forbidden'), { name: 'GoogleApiError', status: 403, retryable: false }) ); await runWorker({ batchSize: 50 }); expect(prisma.gscJob.update).toHaveBeenCalledWith(expect.objectContaining({ where: { id: 'j3' }, data: expect.objectContaining({ status: 'failed' }), })); }); it('marks job failed after maxAttempts exhausted', async () => { (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'j4', type: 'submit_sitemap', dealerId: 'd1', priority: 50, attempts: 4, maxAttempts: 5, payload: { sitemapUrl: 'https://x/sitemap.xml' } }, ]); (submitSitemap as jest.Mock).mockRejectedValue( Object.assign(new Error('5xx'), { name: 'GoogleApiError', status: 500, retryable: true }) ); await runWorker({ batchSize: 50 }); expect(prisma.gscJob.update).toHaveBeenCalledWith(expect.objectContaining({ where: { id: 'j4' }, data: expect.objectContaining({ status: 'failed', attempts: 5 }), })); }); // ----- Property resolution tests (QA-fix coverage) ----- it('submit_sitemap on a custom-domain dealer uses URL-prefix property, NOT sc-domain', async () => { (prisma.dealer.findUnique as jest.Mock).mockResolvedValue({ ...baseDealer, customDomain: 'bobsoil.com', customDomainStatus: 'active', }); (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'jcd1', type: 'submit_sitemap', dealerId: 'd1', priority: 50, attempts: 0, maxAttempts: 5, payload: { sitemapUrl: 'https://bobsoil.com/sitemap.xml' } }, ]); (submitSitemap as jest.Mock).mockResolvedValue(undefined); await runWorker({ batchSize: 50 }); expect(submitSitemap).toHaveBeenCalledWith( 'https://bobsoil.com/', 'https://bobsoil.com/sitemap.xml' ); }); it('verify_property uses payload.propertyHost regardless of dealer.customDomain', async () => { // A dealer with active customDomain A could still be verifying domain B if // they recently switched. Job payload is authoritative for lifecycle jobs. (prisma.dealer.findUnique as jest.Mock).mockResolvedValue({ ...baseDealer, customDomain: 'old.com', customDomainStatus: 'active', }); (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'jcd2', type: 'verify_property', dealerId: 'd1', priority: 50, attempts: 0, maxAttempts: 5, payload: { propertyHost: 'new.com' } }, ]); const { getVerificationToken } = require('../google/site-verification'); getVerificationToken.mockResolvedValue('tok-new'); await runWorker({ batchSize: 50 }); expect(getVerificationToken).toHaveBeenCalledWith('https://new.com/'); }); it('verify_property calls revalidateAllDealerPages after writing token', async () => { const { revalidateAllDealerPages } = require('../isr-revalidation'); revalidateAllDealerPages.mockResolvedValue({ success: true }); (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'jvp1', type: 'verify_property', dealerId: 'd1', priority: 50, attempts: 0, maxAttempts: 5, payload: { propertyHost: 'example.com' } }, ]); await runWorker({ batchSize: 50 }); expect(revalidateAllDealerPages).toHaveBeenCalledWith( 'd1', 'bob', 'myamsoil', 'us' ); }); }); ``` - [ ] **Step 2: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-worker.test.ts ``` Expected: FAIL - module not found. - [ ] **Step 3: Implement worker core** ```ts import { prisma } from './prisma'; import { logger } from './logger'; import { submitSitemap, deleteSitemap } from './google/search-console'; import { notifyUrlUpdated } from './google/indexing'; import { inspectUrl } from './google/url-inspection'; import { getVerificationToken, verifyOwnership, deleteVerification } from './google/site-verification'; import { addProperty, deleteProperty } from './google/sites'; import { getGscPropertyForDealer } from './google/gsc-domains'; import { preflight } from './gsc-preflight'; import { revalidateAllDealerPages } from './isr-revalidation'; const gscLogger = logger.child({ module: 'gsc' }); const BACKOFF_MS = [ 1 * 60_000, 5 * 60_000, 15 * 60_000, 60 * 60_000, 6 * 60 * 60_000, ]; export interface WorkerResult { processed: number; completed: number; failed: number; deferred: number; durationMs: number; } export async function runWorker({ batchSize = 50 } = {}): Promise { const startedAt = Date.now(); const result: WorkerResult = { processed: 0, completed: 0, failed: 0, deferred: 0, durationMs: 0 }; // Atomic claim: UPDATE … RETURNING with FOR UPDATE SKIP LOCKED. // Two concurrent cron invocations will not pick the same rows — each row // is locked, claimed, and returned in one statement. The spec promises // this as the only concurrency primitive; implementing it here avoids // duplicate work even if cron invocations overlap. type ClaimedJob = { id: string; type: string; dealerId: string; payload: unknown; priority: number; attempts: number; maxAttempts: number; }; const jobs = await prisma.$queryRaw` UPDATE "GscJob" SET status = 'running', "startedAt" = NOW(), "updatedAt" = NOW() WHERE id IN ( SELECT id FROM "GscJob" WHERE status = 'pending' AND "runAt" <= NOW() ORDER BY priority ASC, "runAt" ASC LIMIT ${batchSize} FOR UPDATE SKIP LOCKED ) RETURNING id, type, "dealerId", payload, priority, attempts, "maxAttempts"; `; for (const job of jobs) { result.processed++; // Note: status='running' + startedAt are already set by the claim query // above. No second UPDATE needed here. try { await executeJob(job); await prisma.gscJob.update({ where: { id: job.id }, data: { status: 'completed', completedAt: new Date(), lastError: null }, }); result.completed++; } catch (err) { const e = err as Error & { retryable?: boolean; status?: number }; const retryable = e.retryable !== false; // unknown errors are retryable const newAttempts = job.attempts + 1; if (!retryable || newAttempts >= job.maxAttempts) { await prisma.gscJob.update({ where: { id: job.id }, data: { status: 'failed', attempts: newAttempts, lastError: e.message }, }); result.failed++; gscLogger.error({ jobId: job.id, type: job.type, dealerId: job.dealerId, err: e.message }, 'GSC job permanently failed'); } else { const delay = BACKOFF_MS[Math.min(newAttempts - 1, BACKOFF_MS.length - 1)]; await prisma.gscJob.update({ where: { id: job.id }, data: { status: 'pending', attempts: newAttempts, lastError: e.message, runAt: new Date(Date.now() + delay), startedAt: null, }, }); gscLogger.warn({ jobId: job.id, attempts: newAttempts, retryInMs: delay, err: e.message }, 'GSC job retrying'); } } } result.durationMs = Date.now() - startedAt; gscLogger.info(result, 'GSC worker run complete'); return result; } /** * Resolves the GSC property to use for a given job + dealer combination. * * For custom-domain-scoped jobs (verify/register/delete property, plus * sitemap+inspect+notify when dealer has an active custom domain), we use the * URL-prefix property `https://{customDomain}/`. Otherwise, the parent Domain * property `sc-domain:{parent}` covers the subdomain. * * Job payloads may carry an explicit `propertyHost` (for custom-domain * lifecycle jobs); that takes precedence. */ function resolvePropertyForJob( dealer: { domain: string; domainPrefix: string; customDomain: string | null; customDomainStatus: string }, job: { type: string; payload: Record }, ): string { // Custom-domain lifecycle jobs always carry propertyHost in payload if (job.payload.propertyHost) { return `https://${job.payload.propertyHost as string}/`; } // For sitemap/inspect/notify on a dealer with active custom domain: URL-prefix if (dealer.customDomain && dealer.customDomainStatus === 'active') { return `https://${dealer.customDomain}/`; } // Otherwise: parent Domain property const sub = getGscPropertyForDealer(dealer.domainPrefix, dealer.domain); if (!sub) throw Object.assign(new Error(`No GSC property for dealer (${dealer.domainPrefix}.${dealer.domain})`), { retryable: false }); return sub; } async function executeJob(job: { type: string; dealerId: string; payload: unknown }): Promise { const dealer = await prisma.dealer.findUnique({ where: { id: job.dealerId } }); if (!dealer) throw new Error(`Dealer ${job.dealerId} not found`); const payload = (job.payload ?? {}) as Record; const property = resolvePropertyForJob(dealer, { type: job.type, payload }); switch (job.type) { case 'submit_sitemap': await submitSitemap(property, payload.sitemapUrl as string); await prisma.dealer.update({ where: { id: dealer.id }, data: { gscLastSubmittedAt: new Date() }, }); return; case 'delete_sitemap': await deleteSitemap(property, payload.sitemapUrl as string); return; case 'notify_indexing': await notifyUrlUpdated(payload.url as string); return; case 'inspect_url': { const r = await inspectUrl(property, payload.url as string); await prisma.dealer.update({ where: { id: dealer.id }, data: { gscLastInspectedAt: new Date(), gscLastIndexStatus: r.status }, }); return; } case 'verify_property': { const propertyHost = payload.propertyHost as string; const siteUrl = `https://${propertyHost}/`; const token = await getVerificationToken(siteUrl); await prisma.dealer.update({ where: { id: dealer.id }, data: { gscVerificationToken: token }, }); // CRITICAL: revalidate the dealer's pages so the meta tag renders into // the cached HTML before register_property tries to verify ownership. // The dealer page is wrapped in unstable_cache with a 1-year TTL, so // without this, the meta tag is invisible until natural revalidation. if (dealer.subdomain) { await revalidateAllDealerPages(dealer.id, dealer.subdomain, dealer.domainPrefix, dealer.domain); } // Chain: register_property after a delay to allow CDN cache flush. await prisma.gscJob.create({ data: { type: 'register_property', dealerId: dealer.id, priority: 50, runAt: new Date(Date.now() + 60_000), payload: { propertyHost }, }, }); return; } case 'register_property': { const propertyHost = payload.propertyHost as string; const siteUrl = `https://${propertyHost}/`; await verifyOwnership(siteUrl, 'META'); await addProperty(siteUrl); await prisma.dealer.update({ where: { id: dealer.id }, data: { gscPropertyRegisteredAt: new Date() }, }); // Chain: submit_sitemap for the new property. await prisma.gscJob.create({ data: { type: 'submit_sitemap', dealerId: dealer.id, priority: 50, payload: { sitemapUrl: `${siteUrl}sitemap.xml`, propertyHost }, }, }); return; } case 'delete_property': { const propertyHost = payload.propertyHost as string; const siteUrl = `https://${propertyHost}/`; try { await deleteProperty(siteUrl); } catch (_) { /* ignore */ } try { await deleteVerification(siteUrl); } catch (_) { /* ignore */ } await prisma.dealer.update({ where: { id: dealer.id }, data: { gscPropertyRegisteredAt: null, gscVerificationToken: null }, }); return; } default: throw Object.assign(new Error(`Unknown job type: ${job.type}`), { retryable: false }); } } ``` - [ ] **Step 4: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-worker.test.ts ``` Expected: 4 passing. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/gsc-worker.ts lib/__tests__/gsc-worker.test.ts git commit -m "feat(gsc): worker core — dispatch, error classification, backoff Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 13: Add quota gate and preflight gate to worker Builds on Task 12's worker. **Files:** - Modify: `lib/gsc-worker.ts` - Modify: `lib/__tests__/gsc-worker.test.ts` - append tests for gates - [ ] **Step 1: Append failing tests for the gates** Append to `lib/__tests__/gsc-worker.test.ts`: ```ts describe('lib/gsc-worker — quota gate', () => { it('defers notify_indexing when daily quota >= 180', async () => { (preflight as jest.Mock).mockResolvedValue({ healthy: true }); (prisma.gscQuotaUsage.upsert as jest.Mock).mockResolvedValue({ date: new Date(), indexingApiCalls: 180, }); (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'jq1', type: 'notify_indexing', dealerId: 'd1', priority: 10, attempts: 0, maxAttempts: 5, payload: { url: 'https://x/', force: true } }, ]); const result = await runWorker({ batchSize: 50 }); expect(result.deferred).toBe(1); expect(prisma.gscJob.update).toHaveBeenCalledWith(expect.objectContaining({ where: { id: 'jq1' }, data: expect.objectContaining({ status: 'pending' }), })); }); }); describe('lib/gsc-worker — preflight gate', () => { it('skips submit_sitemap when preflight unhealthy, sets gscPreflightError', async () => { (preflight as jest.Mock).mockResolvedValue({ healthy: false, reason: 'SSL_ERROR_BAD_CERT' }); (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'jp1', type: 'submit_sitemap', dealerId: 'd1', priority: 50, attempts: 0, maxAttempts: 5, payload: { sitemapUrl: 'https://x/sitemap.xml' } }, ]); await runWorker({ batchSize: 50 }); expect(submitSitemap).not.toHaveBeenCalled(); expect(prisma.dealer.update).toHaveBeenCalledWith(expect.objectContaining({ where: { id: 'd1' }, data: expect.objectContaining({ gscPreflightError: 'SSL_ERROR_BAD_CERT' }), })); }); it('does NOT preflight-gate inspect_url or delete jobs', async () => { (preflight as jest.Mock).mockResolvedValue({ healthy: false, reason: 'down' }); (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'ji1', type: 'inspect_url', dealerId: 'd1', priority: 200, attempts: 0, maxAttempts: 5, payload: { url: 'https://x/' } }, ]); require('../google/url-inspection').inspectUrl = jest.fn().mockResolvedValue({ status: 'indexed' }); const r = await runWorker({ batchSize: 50 }); expect(r.completed).toBe(1); // job ran despite preflight unhealthy }); }); ``` - [ ] **Step 2: Run test, expect RED on the new tests** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-worker.test.ts -t "quota gate|preflight gate" ``` Expected: FAIL - the new tests fail because gates aren't implemented; the original 4 tests still pass. - [ ] **Step 3: Implement quota gate** In `lib/gsc-worker.ts`, at the top of `runWorker`, after the initial query, add a quota usage row load: ```ts const today = new Date(); today.setUTCHours(0, 0, 0, 0); const quota = await prisma.gscQuotaUsage.upsert({ where: { date: today }, create: { date: today, indexingApiCalls: 0 }, update: {}, }); const INDEXING_API_DAILY_CAP = 180; let indexingCallsToday = quota.indexingApiCalls; ``` Then in the per-job loop, BEFORE the `await prisma.gscJob.update({...status: 'running'...})` call, add: ```ts // Quota gate (notify_indexing only) if (job.type === 'notify_indexing' && indexingCallsToday >= INDEXING_API_DAILY_CAP) { const tomorrow = new Date(); tomorrow.setUTCDate(tomorrow.getUTCDate() + 1); tomorrow.setUTCHours(0, 0, 0, 0); await prisma.gscJob.update({ where: { id: job.id }, data: { status: 'pending', runAt: tomorrow }, }); result.deferred++; continue; } ``` After a successful `notify_indexing` execution, increment the counter: ```ts // Inside executeJob's 'notify_indexing' case, after notifyUrlUpdated: await prisma.gscQuotaUsage.update({ where: { date: today }, data: { indexingApiCalls: { increment: 1 } }, }); ``` (You'll need to pass `today` into `executeJob` or restructure - simplest is to do the increment in `runWorker` instead, after `executeJob` returns. Adjust as needed.) Simpler approach - increment in `runWorker` immediately after successful execute when the job type is `notify_indexing`: ```ts if (job.type === 'notify_indexing') { indexingCallsToday++; await prisma.gscQuotaUsage.update({ where: { date: today }, data: { indexingApiCalls: { increment: 1 } }, }); } ``` - [ ] **Step 4: Implement preflight gate** Before the `executeJob(job)` call, for sitemap/notify jobs only: ```ts if (job.type === 'submit_sitemap' || job.type === 'notify_indexing') { const dealerForPreflight = await prisma.dealer.findUnique({ where: { id: job.dealerId }, select: { id: true, subdomain: true, domain: true, domainPrefix: true, customDomain: true, customDomainStatus: true, gscPreflightHealthyAt: true, gscPreflightError: true, }, }); if (!dealerForPreflight) { await prisma.gscJob.update({ where: { id: job.id }, data: { status: 'failed', lastError: 'dealer not found' }, }); result.failed++; continue; } const pre = await preflight(dealerForPreflight); if (!pre.healthy) { // Update dealer health state const wasHealthy = !dealerForPreflight.gscPreflightError; await prisma.dealer.update({ where: { id: dealerForPreflight.id }, data: { gscPreflightError: pre.reason ?? 'unknown' }, }); // Defer the job by 1h. Do NOT increment attempts — preflight failures // mean "site is currently unreachable," not "Google rejected this call." // Mixing them would prematurely fail jobs for dealers whose site flaked // for an hour. The result.deferred counter still tracks this for ops // visibility. await prisma.gscJob.update({ where: { id: job.id }, data: { status: 'pending', lastError: `preflight: ${pre.reason}`, runAt: new Date(Date.now() + 60 * 60 * 1000), startedAt: null, }, }); result.deferred++; if (wasHealthy) { // First transition healthy → unhealthy: log a note. Implementation in Task 14. // (Will be added there as a separate concern.) } continue; } else { // Refresh healthy timestamp and clear error await prisma.dealer.update({ where: { id: dealerForPreflight.id }, data: { gscPreflightHealthyAt: new Date(), gscPreflightError: null }, }); } } ``` - [ ] **Step 5: Run all worker tests, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-worker.test.ts ``` Expected: all 7 tests passing (4 original + 3 new). - [ ] **Step 6: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/gsc-worker.ts lib/__tests__/gsc-worker.test.ts git commit -m "feat(gsc): worker quota gate + preflight gate Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 14: Auto-note on preflight transitions **Files:** - Modify: `lib/gsc-worker.ts` - Modify: `lib/__tests__/gsc-worker.test.ts` - [ ] **Step 1: Append failing test** ```ts describe('lib/gsc-worker — preflight auto-notes', () => { it('writes a DealerNote when transitioning healthy → unhealthy', async () => { (preflight as jest.Mock).mockResolvedValue({ healthy: false, reason: 'SSL expired' }); (prisma.dealer.findUnique as jest.Mock).mockResolvedValue({ ...baseDealer, gscPreflightError: null, // was healthy }); (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'jn1', type: 'submit_sitemap', dealerId: 'd1', priority: 50, attempts: 0, maxAttempts: 5, payload: { sitemapUrl: 'https://x/sitemap.xml' } }, ]); (require('../gsc-system-user').getSystemUserId as jest.Mock) = jest.fn().mockResolvedValue('gsc-system-user'); (prisma.dealerNote = { create: jest.fn() } as any); await runWorker({ batchSize: 50 }); expect(prisma.dealerNote.create).toHaveBeenCalledWith(expect.objectContaining({ data: expect.objectContaining({ dealerId: 'd1', adminId: 'gsc-system-user', isAutomatic: true, noteType: 'internal', content: expect.stringContaining('SSL expired'), }), })); }); it('does NOT write a note if dealer was already unhealthy', async () => { (preflight as jest.Mock).mockResolvedValue({ healthy: false, reason: 'still down' }); (prisma.dealer.findUnique as jest.Mock).mockResolvedValue({ ...baseDealer, gscPreflightError: 'previous error', }); (prisma.$queryRaw as jest.Mock).mockResolvedValueOnce([ { id: 'jn2', type: 'submit_sitemap', dealerId: 'd1', priority: 50, attempts: 0, maxAttempts: 5, payload: { sitemapUrl: 'https://x/sitemap.xml' } }, ]); await runWorker({ batchSize: 50 }); expect(prisma.dealerNote.create).not.toHaveBeenCalled(); }); }); ``` Add `jest.mock('../gsc-system-user')` to the top mock block. - [ ] **Step 2: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-worker.test.ts -t "auto-notes" ``` Expected: FAIL - note never created. - [ ] **Step 3: Implement note writing** In `lib/gsc-worker.ts`, import: ```ts import { getSystemUserId } from './gsc-system-user'; ``` In the preflight-failed block, where the comment said "First transition healthy → unhealthy: log a note", replace with: ```ts if (wasHealthy) { try { const adminId = await getSystemUserId(); await prisma.dealerNote.create({ data: { dealerId: dealerForPreflight.id, adminId, isAutomatic: true, noteType: 'internal', content: `Preflight check failed: ${pre.reason}. GSC submissions paused for this dealer until the site is reachable again.`, }, }); } catch (logErr) { gscLogger.error({ err: (logErr as Error).message, dealerId: dealerForPreflight.id }, 'Failed to write preflight auto-note'); } } ``` Similarly, add a "recovery" note when transitioning the other way. In the `pre.healthy` branch (after clearing the error), add: ```ts if (dealerForPreflight.gscPreflightError) { try { const adminId = await getSystemUserId(); await prisma.dealerNote.create({ data: { dealerId: dealerForPreflight.id, adminId, isAutomatic: true, noteType: 'internal', content: `Preflight check now passing. GSC submissions resumed.`, }, }); } catch (_) { /* non-critical */ } } ``` - [ ] **Step 4: Run all worker tests, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-worker.test.ts ``` Expected: all 9 tests passing. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/gsc-worker.ts lib/__tests__/gsc-worker.test.ts git commit -m "feat(gsc): auto-log DealerNote on preflight transitions Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 14b: Daily anomaly-only email summary Folded into the existing `gsc-audit` cron - no separate cron, no dedup table. Once per day after enqueueing inspections, check for anomalies. If any found, send ONE summary email. If clean, send nothing. **Anomalies that trigger an email:** - Any `GscJob` row with `status='failed'` created in the last 24h. - No successful `gsc-drain` invocation in the last 24h (worker not running). - Today's Indexing API quota usage > 90% of cap. **Files:** - Create: `lib/gsc-anomaly-report.ts` - produces the report data structure - Create: `lib/__tests__/gsc-anomaly-report.test.ts` - Modify: `lib/gsc-audit.ts` (from Task 16) - call the report at end of audit, send email if anomalies present - Modify: `.env.example` - add `GSC_ALERT_EMAIL` > **Note:** This task is ordered before Task 15-16 in the plan, but `lib/gsc-audit.ts` is created in Task 16. To avoid forward references, an implementer working strictly in order should: (a) implement the anomaly report module here as a standalone library, then (b) wire it into `gsc-audit.ts` during Task 16. The integration is a small addition (~10 lines) at the end of `runDailyAudit`. - [ ] **Step 1: Write the failing test for the anomaly report** Create `lib/__tests__/gsc-anomaly-report.test.ts`: ```ts // Falsely-passes guard: returning an empty anomaly list unconditionally would // pass the happy-day test but fail every detection test. Each detection case // must be exercised. import { collectAnomalies } from '../gsc-anomaly-report'; jest.mock('../prisma', () => ({ prisma: { gscJob: { count: jest.fn(), findMany: jest.fn() }, gscQuotaUsage: { findUnique: jest.fn() }, }, })); import { prisma } from '../prisma'; describe('lib/gsc-anomaly-report', () => { beforeEach(() => { jest.clearAllMocks(); // Defaults: no anomalies (prisma.gscJob.count as jest.Mock).mockResolvedValue(0); (prisma.gscJob.findMany as jest.Mock).mockResolvedValue([]); (prisma.gscQuotaUsage.findUnique as jest.Mock).mockResolvedValue({ indexingApiCalls: 0, }); }); it('returns no anomalies on a clean day', async () => { // Also need to mock the "worker ran in last 24h" signal — we'll use // a separate count of successful completed jobs in last 24h as proxy. (prisma.gscJob.count as jest.Mock).mockResolvedValueOnce(0) // failed .mockResolvedValueOnce(10); // completed const report = await collectAnomalies(); expect(report.anomalies).toHaveLength(0); }); it('flags failed jobs in last 24h', async () => { (prisma.gscJob.count as jest.Mock).mockResolvedValueOnce(3) // failed .mockResolvedValueOnce(10); // completed (prisma.gscJob.findMany as jest.Mock).mockResolvedValueOnce([ { id: 'jf1', type: 'submit_sitemap', dealerId: 'd1', lastError: 'AUTH', completedAt: null }, ]); const report = await collectAnomalies(); expect(report.anomalies.some(a => a.kind === 'failed_jobs')).toBe(true); }); it('flags worker not running (zero completed in 24h)', async () => { (prisma.gscJob.count as jest.Mock).mockResolvedValueOnce(0) // failed .mockResolvedValueOnce(0); // completed const report = await collectAnomalies(); expect(report.anomalies.some(a => a.kind === 'worker_idle')).toBe(true); }); it('flags quota over 90% of cap', async () => { (prisma.gscJob.count as jest.Mock).mockResolvedValueOnce(0) .mockResolvedValueOnce(10); (prisma.gscQuotaUsage.findUnique as jest.Mock).mockResolvedValue({ indexingApiCalls: 185, // > 90% of 200 }); const report = await collectAnomalies(); expect(report.anomalies.some(a => a.kind === 'quota_high')).toBe(true); }); }); ``` - [ ] **Step 2: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-anomaly-report.test.ts ``` Expected: FAIL - module not found. - [ ] **Step 3: Implement `lib/gsc-anomaly-report.ts`** ```ts import { prisma } from './prisma'; export type AnomalyKind = 'failed_jobs' | 'worker_idle' | 'quota_high'; export interface Anomaly { kind: AnomalyKind; message: string; detail?: unknown; } export interface AnomalyReport { anomalies: Anomaly[]; summary: { failedCount: number; completedCount: number; quotaUsed: number; quotaCap: number; }; } const INDEXING_API_CAP = 200; const QUOTA_HIGH_THRESHOLD = 0.9; export async function collectAnomalies(): Promise { const since = new Date(Date.now() - 24 * 60 * 60 * 1000); const today = new Date(); today.setUTCHours(0, 0, 0, 0); const [failedCount, completedCount, recentFailures, quotaRow] = await Promise.all([ prisma.gscJob.count({ where: { status: 'failed', updatedAt: { gte: since } } }), prisma.gscJob.count({ where: { status: 'completed', updatedAt: { gte: since } } }), prisma.gscJob.findMany({ where: { status: 'failed', updatedAt: { gte: since } }, orderBy: { updatedAt: 'desc' }, take: 10, select: { id: true, type: true, dealerId: true, lastError: true, updatedAt: true }, }), prisma.gscQuotaUsage.findUnique({ where: { date: today } }), ]); const anomalies: Anomaly[] = []; if (failedCount > 0) { anomalies.push({ kind: 'failed_jobs', message: `${failedCount} GSC job(s) failed in the last 24 hours.`, detail: recentFailures, }); } if (completedCount === 0) { anomalies.push({ kind: 'worker_idle', message: 'No GSC jobs completed in the last 24 hours. Worker may be down.', }); } const quotaUsed = quotaRow?.indexingApiCalls ?? 0; if (quotaUsed > INDEXING_API_CAP * QUOTA_HIGH_THRESHOLD) { anomalies.push({ kind: 'quota_high', message: `Indexing API quota at ${quotaUsed}/${INDEXING_API_CAP} (>${QUOTA_HIGH_THRESHOLD * 100}%).`, }); } return { anomalies, summary: { failedCount, completedCount, quotaUsed, quotaCap: INDEXING_API_CAP }, }; } ``` - [ ] **Step 4: Run test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-anomaly-report.test.ts ``` Expected: 4 passing. - [ ] **Step 5: Add `GSC_ALERT_EMAIL` to `.env.example`** Append below the `GOOGLE_SERVICE_ACCOUNT_JSON` block: ``` # Address that receives the daily GSC anomaly summary email. # Sent ONCE per day at the time of the gsc-audit cron, ONLY if anomalies are # detected (failed jobs in last 24h, worker idle, quota >90%). Leave empty # to disable alerting entirely (dashboard remains the source of truth). GSC_ALERT_EMAIL="" ``` - [ ] **Step 6: Commit (integration wiring happens in Task 16)** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/gsc-anomaly-report.ts lib/__tests__/gsc-anomaly-report.test.ts .env.example git commit -m "feat(gsc): anomaly report module for daily summary email Co-Authored-By: Claude Opus 4.7 (1M context) " ``` > **Wiring note:** The anomaly-report module created in this task is wired into `lib/gsc-audit.ts`'s `runDailyAudit` function in Task 16 Step 3 (which has been updated to import `collectAnomalies` + `sendEmail` and includes the email-trigger logic + tests). No separate addendum to track. --- ## Phase 5 - Cron endpoints ### Task 15: `/api/cron/gsc-drain` route **Files:** - Create: `app/api/cron/gsc-drain/route.ts` - Create: `app/api/cron/gsc-drain/__tests__/route.test.ts` - [ ] **Step 1: Write the failing test** ```ts // Falsely-passes guard: testing only that status 200 returns would miss whether // auth is actually enforced. import { POST } from '../route'; import { NextRequest } from 'next/server'; jest.mock('@/lib/gsc-worker'); import { runWorker } from '@/lib/gsc-worker'; const ORIG_SECRET = process.env.CRON_SECRET; beforeAll(() => { process.env.CRON_SECRET = 'test-secret'; }); afterAll(() => { process.env.CRON_SECRET = ORIG_SECRET; }); function makeReq(authHeader?: string): NextRequest { const headers = new Headers(); if (authHeader) headers.set('authorization', authHeader); return new NextRequest('http://localhost/api/cron/gsc-drain', { method: 'POST', headers }); } describe('POST /api/cron/gsc-drain', () => { beforeEach(() => jest.clearAllMocks()); it('returns 401 with no auth header', async () => { const res = await POST(makeReq()); expect(res.status).toBe(401); }); it('returns 401 with wrong bearer secret', async () => { const res = await POST(makeReq('Bearer wrong')); expect(res.status).toBe(401); }); it('runs the worker and returns counts on valid auth', async () => { (runWorker as jest.Mock).mockResolvedValue({ processed: 5, completed: 4, failed: 1, deferred: 0, durationMs: 123, }); const res = await POST(makeReq('Bearer test-secret')); expect(res.status).toBe(200); const body = await res.json(); expect(body).toMatchObject({ processed: 5, completed: 4 }); }); }); ``` - [ ] **Step 2: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/cron/gsc-drain ``` Expected: FAIL - module not found. - [ ] **Step 3: Implement** Create `app/api/cron/gsc-drain/route.ts`: ```ts import { NextRequest, NextResponse } from 'next/server'; import { timingSafeEqual } from 'crypto'; import { runWorker } from '@/lib/gsc-worker'; import { logger } from '@/lib/logger'; const gscLogger = logger.child({ module: 'gsc' }); function safeCompare(a: string, b: string): boolean { if (!a || !b) return false; const ba = Buffer.from(a); const bb = Buffer.from(b); if (ba.length !== bb.length) return false; return timingSafeEqual(ba, bb); } export async function POST(request: NextRequest) { const secret = process.env.CRON_SECRET; if (!secret) { gscLogger.error('CRON_SECRET not configured'); return NextResponse.json({ error: 'CRON_SECRET not configured' }, { status: 500 }); } const authHeader = request.headers.get('authorization'); if (!authHeader) return NextResponse.json({ error: 'unauthorized' }, { status: 401 }); const provided = authHeader.replace('Bearer ', ''); if (!safeCompare(provided, secret)) return NextResponse.json({ error: 'unauthorized' }, { status: 401 }); const result = await runWorker(); return NextResponse.json(result); } export async function GET(request: NextRequest) { return POST(request); } ``` - [ ] **Step 4: Run, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/cron/gsc-drain ``` Expected: 3 passing. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add app/api/cron/gsc-drain/ git commit -m "feat(gsc): /api/cron/gsc-drain worker invocation endpoint Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 16: `/api/cron/gsc-audit` route **Files:** - Create: `app/api/cron/gsc-audit/route.ts` - Create: `app/api/cron/gsc-audit/__tests__/route.test.ts` - Create: `lib/gsc-audit.ts` (the logic, separated for testability) - Create: `lib/__tests__/gsc-audit.test.ts` - [ ] **Step 1: Write the failing test for `lib/gsc-audit.ts`** ```ts // Falsely-passes guard: enqueuing zero jobs would pass an "expect not to throw" // test — must assert exact count and types. import { runDailyAudit } from '../gsc-audit'; jest.mock('../prisma', () => ({ prisma: { dealer: { findMany: jest.fn() }, gscJob: { create: jest.fn(), count: jest.fn() }, }, })); import { prisma } from '../prisma'; describe('lib/gsc-audit', () => { beforeEach(() => jest.clearAllMocks()); it('enqueues inspect_url for each active dealer with subdomain', async () => { (prisma.dealer.findMany as jest.Mock).mockResolvedValue([ { id: 'd1', subdomain: 'a', domain: 'us', domainPrefix: 'myamsoil', customDomain: null, customDomainStatus: 'none' }, { id: 'd2', subdomain: 'b', domain: 'us', domainPrefix: 'shopamsoil', customDomain: null, customDomainStatus: 'none' }, ]); (prisma.gscJob.count as jest.Mock).mockResolvedValue(0); const result = await runDailyAudit(); expect(result.dealersAudited).toBe(2); expect(prisma.gscJob.create).toHaveBeenCalledTimes(2); }); it('is idempotent — skips dealers with an in-flight audit inspect_url today', async () => { (prisma.dealer.findMany as jest.Mock).mockResolvedValue([ { id: 'd1', subdomain: 'a', domain: 'us', domainPrefix: 'myamsoil', customDomain: null, customDomainStatus: 'none' }, ]); // d1 already has an audit-priority inspect_url today (prisma.gscJob.count as jest.Mock).mockResolvedValueOnce(1); const result = await runDailyAudit(); expect(result.dealersAudited).toBe(1); expect(result.jobsEnqueued).toBe(0); expect(prisma.gscJob.create).not.toHaveBeenCalled(); }); // ----- Email anomaly summary ----- // Mock the collectAnomalies + sendEmail dependencies separately so we can // exercise the email path without requiring a real anomaly to be detected. it('sends an email when anomalies are present AND GSC_ALERT_EMAIL is set', async () => { const { collectAnomalies } = require('../gsc-anomaly-report'); const { sendEmail } = require('../email'); (collectAnomalies as jest.Mock).mockResolvedValue({ anomalies: [{ kind: 'failed_jobs', message: '3 jobs failed' }], summary: { failedCount: 3, completedCount: 10, quotaUsed: 0, quotaCap: 200 }, }); process.env.GSC_ALERT_EMAIL = 'ops@example.com'; (prisma.dealer.findMany as jest.Mock).mockResolvedValue([]); const result = await runDailyAudit(); expect(result.anomaliesFound).toBe(1); expect(result.alertEmailSent).toBe(true); expect(sendEmail).toHaveBeenCalledWith(expect.objectContaining({ to: 'ops@example.com', subject: expect.stringContaining('Daily anomaly summary'), })); }); it('does NOT send an email when GSC_ALERT_EMAIL is unset, even if anomalies exist', async () => { const { collectAnomalies } = require('../gsc-anomaly-report'); const { sendEmail } = require('../email'); (collectAnomalies as jest.Mock).mockResolvedValue({ anomalies: [{ kind: 'worker_idle', message: 'idle' }], summary: { failedCount: 0, completedCount: 0, quotaUsed: 0, quotaCap: 200 }, }); delete process.env.GSC_ALERT_EMAIL; (prisma.dealer.findMany as jest.Mock).mockResolvedValue([]); const result = await runDailyAudit(); expect(result.anomaliesFound).toBe(1); expect(result.alertEmailSent).toBe(false); expect(sendEmail).not.toHaveBeenCalled(); }); it('does NOT send an email when there are zero anomalies', async () => { const { collectAnomalies } = require('../gsc-anomaly-report'); const { sendEmail } = require('../email'); (collectAnomalies as jest.Mock).mockResolvedValue({ anomalies: [], summary: { failedCount: 0, completedCount: 10, quotaUsed: 0, quotaCap: 200 }, }); process.env.GSC_ALERT_EMAIL = 'ops@example.com'; (prisma.dealer.findMany as jest.Mock).mockResolvedValue([]); const result = await runDailyAudit(); expect(result.anomaliesFound).toBe(0); expect(result.alertEmailSent).toBe(false); expect(sendEmail).not.toHaveBeenCalled(); }); }); ``` Update the mock block at the top of this test file to include: ```ts jest.mock('../gsc-anomaly-report'); jest.mock('../email'); ``` - [ ] **Step 2: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-audit.test.ts ``` Expected: FAIL - module not found. - [ ] **Step 3: Implement `lib/gsc-audit.ts`** ```ts import { prisma } from './prisma'; import { getCanonicalUrl } from './seo-utils'; import { collectAnomalies } from './gsc-anomaly-report'; import { sendEmail } from './email'; import { logger } from './logger'; const gscLogger = logger.child({ module: 'gsc' }); export interface AuditResult { dealersAudited: number; jobsEnqueued: number; anomaliesFound: number; alertEmailSent: boolean; } export async function runDailyAudit(): Promise { const dealers = await prisma.dealer.findMany({ where: { status: 'active', subdomain: { not: null } }, select: { id: true, subdomain: true, domain: true, domainPrefix: true, customDomain: true, customDomainStatus: true }, }); const todayStart = new Date(); todayStart.setUTCHours(0, 0, 0, 0); let jobsEnqueued = 0; for (const dealer of dealers) { const existing = await prisma.gscJob.count({ where: { dealerId: dealer.id, type: 'inspect_url', priority: 200, createdAt: { gte: todayStart }, }, }); if (existing > 0) continue; await prisma.gscJob.create({ data: { type: 'inspect_url', dealerId: dealer.id, priority: 200, payload: { url: getCanonicalUrl(dealer, ''), source: 'daily-audit' }, }, }); jobsEnqueued++; } // Anomaly-summary email (per Task 14b design). // Runs unconditionally so anomaly counts are always returned; emails only // when (a) there are anomalies AND (b) GSC_ALERT_EMAIL is configured. let alertEmailSent = false; const report = await collectAnomalies(); const alertTo = process.env.GSC_ALERT_EMAIL; if (alertTo && report.anomalies.length > 0) { const html = `

GSC daily summary

${report.anomalies.length} anomal${report.anomalies.length === 1 ? 'y' : 'ies'} detected in the last 24 hours:

    ${report.anomalies.map(a => `
  • ${a.kind}: ${a.message}
    • `).join('')}

Stats: ${report.summary.failedCount} failed, ${report.summary.completedCount} completed, Indexing API ${report.summary.quotaUsed}/${report.summary.quotaCap} used.

Open GSC dashboard

`; try { await sendEmail({ to: alertTo, subject: `[GSC] Daily anomaly summary (${report.anomalies.length})`, html, }); alertEmailSent = true; gscLogger.info({ anomalies: report.anomalies.length, to: alertTo }, 'GSC alert email sent'); } catch (err) { gscLogger.error({ err, to: alertTo }, 'Failed to send GSC alert email (non-blocking)'); } } return { dealersAudited: dealers.length, jobsEnqueued, anomaliesFound: report.anomalies.length, alertEmailSent, }; } ``` - [ ] **Step 4: Write the cron route test** ```ts import { POST } from '../route'; import { NextRequest } from 'next/server'; jest.mock('@/lib/gsc-audit'); import { runDailyAudit } from '@/lib/gsc-audit'; beforeAll(() => { process.env.CRON_SECRET = 'test-secret'; }); function makeReq(auth?: string): NextRequest { const h = new Headers(); if (auth) h.set('authorization', auth); return new NextRequest('http://localhost/api/cron/gsc-audit', { method: 'POST', headers: h }); } describe('POST /api/cron/gsc-audit', () => { it('401 without auth', async () => { const res = await POST(makeReq()); expect(res.status).toBe(401); }); it('runs the audit on valid auth', async () => { (runDailyAudit as jest.Mock).mockResolvedValue({ dealersAudited: 5, jobsEnqueued: 5 }); const res = await POST(makeReq('Bearer test-secret')); expect(res.status).toBe(200); expect(await res.json()).toMatchObject({ dealersAudited: 5, jobsEnqueued: 5 }); }); }); ``` - [ ] **Step 5: Implement the cron route** `app/api/cron/gsc-audit/route.ts`: ```ts import { NextRequest, NextResponse } from 'next/server'; import { timingSafeEqual } from 'crypto'; import { runDailyAudit } from '@/lib/gsc-audit'; import { logger } from '@/lib/logger'; const gscLogger = logger.child({ module: 'gsc' }); function safeCompare(a: string, b: string): boolean { if (!a || !b) return false; const ba = Buffer.from(a); const bb = Buffer.from(b); if (ba.length !== bb.length) return false; return timingSafeEqual(ba, bb); } export async function POST(request: NextRequest) { const secret = process.env.CRON_SECRET; if (!secret) return NextResponse.json({ error: 'CRON_SECRET not configured' }, { status: 500 }); const authHeader = request.headers.get('authorization'); if (!authHeader) return NextResponse.json({ error: 'unauthorized' }, { status: 401 }); const provided = authHeader.replace('Bearer ', ''); if (!safeCompare(provided, secret)) return NextResponse.json({ error: 'unauthorized' }, { status: 401 }); const result = await runDailyAudit(); gscLogger.info(result, 'GSC daily audit complete'); return NextResponse.json(result); } export async function GET(request: NextRequest) { return POST(request); } ``` - [ ] **Step 6: Run all new tests, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- lib/__tests__/gsc-audit.test.ts app/api/cron/gsc-audit ``` Expected: 4 passing. - [ ] **Step 7: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/gsc-audit.ts lib/__tests__/gsc-audit.test.ts app/api/cron/gsc-audit/ git commit -m "feat(gsc): daily audit cron — enqueue inspect_url for active dealers Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ## Phase 6 - Trigger hooks ### Task 17: Hook GSC into dealer status route **Files:** - Modify: `app/api/admin/dealers/[id]/status/route.ts` - Modify: `app/api/admin/dealers/[id]/status/__tests__/route.test.ts` - [ ] **Step 1: Append failing tests** Append to the existing tests file: ```ts import { enqueueDealerActivation, enqueueDealerCancellation } from '@/lib/gsc-jobs'; jest.mock('@/lib/gsc-jobs'); describe('PATCH /api/admin/dealers/[id]/status — GSC hooks', () => { beforeEach(() => jest.clearAllMocks()); it('calls enqueueDealerActivation on activation transition', async () => { // Set up: pending → active (use existing test helper pattern from this file) // ... (reuse the existing setup; assert enqueueDealerActivation called once) }); it('does NOT call enqueue functions on other transitions', async () => { // active → active (no transition) or active → suspended: no enqueue }); it('calls enqueueDealerCancellation on cancellation', async () => { // active → cancelled: assert enqueueDealerCancellation called }); }); ``` (Implementer: fill in the test setup using the pattern already established in this file. Don't invent a new mocking strategy - copy from existing tests in the same file.) - [ ] **Step 2: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/admin/dealers/\[id\]/status/__tests__ ``` Expected: FAIL - enqueue mocks not called. - [ ] **Step 3: Modify the route** In `app/api/admin/dealers/[id]/status/route.ts`: Add import: ```ts import { enqueueDealerActivation, enqueueDealerCancellation } from '@/lib/gsc-jobs'; ``` After the dealer status update + cache invalidation (around where the v1 prototype's GSC call used to be), add: ```ts // GSC integration: enqueue jobs for activation/cancellation transitions try { if (isActivating && dealer.subdomain) { await enqueueDealerActivation({ id: dealer.id, subdomain: dealer.subdomain, domain: dealer.domain, domainPrefix: dealer.domainPrefix, customDomain: null, customDomainStatus: 'none', }); } if (effectiveStatus === 'cancelled' && currentStatus !== 'cancelled') { await enqueueDealerCancellation({ id: dealer.id, subdomain: dealer.subdomain, domain: dealer.domain, domainPrefix: dealer.domainPrefix, customDomain: null, customDomainStatus: 'none', }); } } catch (err) { logger.error({ err, dealerId: dealer.id }, 'GSC enqueue failed (non-blocking)'); } ``` - [ ] **Step 4: Run, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/admin/dealers/\[id\]/status/__tests__ ``` Expected: all tests passing. - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add app/api/admin/dealers/\[id\]/status/ git commit -m "feat(gsc): enqueue GSC jobs on dealer status transitions Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 18: Hook GSC into Stripe webhook **Files:** - Modify: `app/api/webhooks/stripe/route.ts` - Modify: `app/api/webhooks/stripe/__tests__/route.test.ts` - [ ] **Step 1: Append failing tests** Append tests mirroring Task 17's pattern but for the Stripe webhook code path. Assert `enqueueDealerActivation` is called when subscription transitions to `active` for the first time, and `enqueueDealerCancellation` when subscription is deleted/cancelled. - [ ] **Step 2: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/webhooks/stripe/__tests__ ``` Expected: FAIL - enqueue mocks not called. - [ ] **Step 3: Modify the webhook** In `app/api/webhooks/stripe/route.ts`, in the `customer.subscription.updated` case, where the dealer status update happens, replace the v1's `submitDealerToGscBackground` block (already removed in Task 1) with: ```ts // GSC: enqueue jobs on activation transition if (newStatus === 'active' && beforeState.status !== 'active' && dealer.subdomain) { try { await enqueueDealerActivation({ id: dealer.id, subdomain: dealer.subdomain, domain: dealer.domain, domainPrefix: dealer.domainPrefix, customDomain: dealer.customDomain, customDomainStatus: dealer.customDomainStatus, }); } catch (err) { webhookLogger.error({ err, dealerId: dealer.id }, 'GSC enqueue failed'); } } if (newStatus === 'cancelled' && beforeState.status !== 'cancelled' && dealer.subdomain) { try { await enqueueDealerCancellation({ id: dealer.id, subdomain: dealer.subdomain, domain: dealer.domain, domainPrefix: dealer.domainPrefix, customDomain: dealer.customDomain, customDomainStatus: dealer.customDomainStatus, }); } catch (err) { webhookLogger.error({ err, dealerId: dealer.id }, 'GSC enqueue failed'); } } ``` Add the import at the top: ```ts import { enqueueDealerActivation, enqueueDealerCancellation } from '@/lib/gsc-jobs'; ``` - [ ] **Step 4: Run, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/webhooks/stripe/__tests__ ``` - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add app/api/webhooks/stripe/ git commit -m "feat(gsc): enqueue GSC jobs from Stripe subscription transitions Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 19: Hook GSC into custom domain activation/deactivation **Files:** - Modify: `app/api/dealer/custom-domain/activate/route.ts` - Modify: `app/api/dealer/custom-domain/route.ts` (DELETE handler for deactivation) - Modify: tests for both - [ ] **Step 1: Write failing tests** Append to existing test files asserting: - After activation succeeds, `enqueueCustomDomainActivation(dealer)` is called. - After deactivation (custom-domain DELETE handler), `enqueueCustomDomainDeactivation(dealer, domain)` is called. - [ ] **Step 2: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/dealer/custom-domain ``` - [ ] **Step 3: Modify activate route** In `app/api/dealer/custom-domain/activate/route.ts`, after the dealer update that sets `customDomainStatus: 'active'`, add: ```ts import { enqueueCustomDomainActivation } from '@/lib/gsc-jobs'; // ... try { await enqueueCustomDomainActivation({ id: dealer.id, subdomain: dealer.subdomain, domain: dealer.domain, domainPrefix: dealer.domainPrefix, customDomain: dealer.customDomain, customDomainStatus: 'active', }); } catch (err) { logger.error({ err, dealerId: dealer.id }, 'GSC custom-domain enqueue failed'); } ``` Similarly for the deactivation path: import `enqueueCustomDomainDeactivation` and call it with the previous `customDomain` value before clearing. - [ ] **Step 4: Run, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/dealer/custom-domain ``` - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add app/api/dealer/custom-domain/ git commit -m "feat(gsc): enqueue custom-domain GSC jobs on activate/deactivate Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 20: Hook GSC into CMS publish flow **Design change from spec (per QA review):** the spec said to hook in `lib/isr-revalidation.ts`, but `revalidateAllDealerPages()` is called from many contexts (status changes, custom-domain regeneration, etc.) - not just publish. Hooking there would fire GSC enqueues on every revalidation, including non-publish events. Better: hook at the publish endpoint itself, where the publish event actually originates. The function also returns a `BulkRevalidationResult` with `failedPaths` but does NOT expose the full path list (paths are constructed internally). To know what was just published, we use the same `prisma.page.findMany({ where: { dealerId, status: 'published' } })` query the publish endpoint already runs. **Files:** - Modify: `app/api/cms/publish/route.ts` - call `enqueuePublishEvent(dealer, paths)` after successful publish - Modify: `app/api/cms/publish/__tests__/route.test.ts` (or create) - assert enqueue happens - [ ] **Step 1: Inspect the publish route to find the post-publish hook point** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing grep -n "revalidateAllDealerPages\|prisma.page.findMany\|published" app/api/cms/publish/route.ts | head ``` Locate the point AFTER the publish succeeds - typically right before the success response returns and after page output paths have been computed. Pages and blog posts already get computed there; the enqueue can reuse them. - [ ] **Step 2: Write failing test** In `app/api/cms/publish/__tests__/route.test.ts`, add a test: ```ts import { enqueuePublishEvent } from '@/lib/gsc-jobs'; jest.mock('@/lib/gsc-jobs'); describe('POST /api/cms/publish — GSC hook', () => { beforeEach(() => jest.clearAllMocks()); it('enqueues a GSC publish event with all published page slugs', async () => { // Set up mocks per existing test patterns in this file // (auth, prisma.page.findMany returning 3 pages with slugs 'services', 'about', 'blog/post-1', etc.) // ... invoke POST handler ... expect(enqueuePublishEvent).toHaveBeenCalledWith( expect.objectContaining({ id: expect.any(String) }), expect.arrayContaining(['/', '/services', '/about', '/blog/post-1']) ); }); it('does NOT block the publish response if enqueue throws', async () => { (enqueuePublishEvent as jest.Mock).mockRejectedValue(new Error('db down')); // ... invoke POST handler, assert response is still 200 }); }); ``` If `app/api/cms/publish/__tests__/route.test.ts` does not exist, create it following the patterns used by other admin/CMS route tests in this codebase. - [ ] **Step 3: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/cms/publish/__tests__ ``` - [ ] **Step 4: Modify the publish route** In `app/api/cms/publish/route.ts`, add: ```ts import { enqueuePublishEvent } from '@/lib/gsc-jobs'; ``` After the page publish succeeds (around the existing `revalidateAllDealerPages` call), but before sending the response, gather the path list and enqueue: ```ts // Build path list: '/' for homepage + '/{slug}' for each published page, // and '/blog/{slug}' for blog posts. The exact path shape must match what // getCanonicalUrl(dealer, path) expects. const paths: string[] = ['/']; for (const page of pages) { paths.push(page.type === 'blog' ? `/blog/${page.slug}` : `/${page.slug}`); } if (pages.some(p => p.type === 'blog')) { paths.push('/blog'); } // Fire-and-forget GSC enqueue: non-blocking; failures logged but don't // affect publish response (mirrors the OG-regeneration pattern already // used in this route). enqueuePublishEvent( { id: dealer.id, subdomain: dealer.subdomain, domain: dealer.domain, domainPrefix: dealer.domainPrefix, customDomain: dealer.customDomain, customDomainStatus: dealer.customDomainStatus }, paths, ).catch((err) => { logger.error({ err, dealerId: dealer.id }, 'GSC enqueue failed (non-blocking)'); }); ``` (Adapt the dealer object construction to the variables actually available in the publish handler.) - [ ] **Step 5: Run, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/cms/publish/__tests__ ``` - [ ] **Step 6: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add app/api/cms/publish/ git commit -m "feat(gsc): enqueue inspect/sitemap jobs after CMS publish Hooks at the publish endpoint (where the event originates) rather than in isr-revalidation.ts, since revalidateAllDealerPages is called from many non-publish contexts (status changes, custom-domain regen). Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 20b: Render Site Verification meta tag in dealer template **Critical for custom-domain verification.** Without this, the worker's `register_property` step calls Google to verify ownership, Google fetches the dealer's homepage, finds no verification meta tag, and the verification step permanently fails for every custom-domain dealer. **Design decisions (per brainstorm):** - Render on EVERY dealer page (homepage + CMS pages), not just homepage - consistent, ~70 bytes per page, future-proof against verification-rule changes. - Render ONLY when `dealer.gscVerificationToken` is non-null AND `dealer.status` is `'active'` or `'cancelled_pending'`. (Existing logic returns `notFound()` for other statuses, so this is consistent with existing scope.) - Use Next.js's built-in `verification.google` field on `Metadata` (not `other`) - cleaner output, less boilerplate. **Files:** - Modify: `app/dealers/[subdomain]/[domainSlug]/page.tsx` - (a) add `gscVerificationToken: true` to `getCachedDealer`'s select, (b) add `verification.google` to active dealer metadata return. - Modify: `app/dealers/[subdomain]/[domainSlug]/[...slug]/page.tsx` - same two changes. - Test: create metadata tests in `__tests__/` next to each page. > **Important:** `getDealerBySubdomain` is NOT a shared module - it's defined locally in each page.tsx as a `React.cache(getCachedDealer)` wrapper. There is NO `lib/dealer-lookup.ts` to modify. Each page.tsx has its own `getCachedDealer` function with its own `unstable_cache`-wrapped `prisma.dealer.findFirst({ select: { ... } })`. **Both must be updated identically** to add `gscVerificationToken: true` to the select. Skipping one file means the metadata generator for that route silently sees `undefined` for the token, and the verification meta tag never renders for that route's pages. - [ ] **Step 1: Add `gscVerificationToken` to BOTH `getCachedDealer` select clauses** In `app/dealers/[subdomain]/[domainSlug]/page.tsx` (around line 80–113) and `app/dealers/[subdomain]/[domainSlug]/[...slug]/page.tsx` (similar location - locate via `grep -n "getCachedDealer\|findFirst" app/dealers/\[subdomain\]/\[domainSlug\]/\[...slug\]/page.tsx`), add this line to the `select` block alongside the other dealer columns: ```ts gscVerificationToken: true, ``` - [ ] **Step 2: Write the failing test for homepage metadata** Since `getCachedDealer` is local (not a module we can `jest.mock`), the right test seam is `@/lib/prisma` - mock `prisma.dealer.findFirst` to return the dealer data we want. `unstable_cache` will memoize the result, but Jest's module isolation between tests should be sufficient; if cache leakage is observed, call `jest.resetModules()` in `beforeEach`. Create `app/dealers/[subdomain]/[domainSlug]/__tests__/page.test.ts`: ```ts /** @jest-environment node */ // Falsely-passes guard: a stub that always emits the verification field would // pass the active-with-token case but fail the no-token and inactive-status // cases. All three must be tested explicitly. import { generateMetadata } from '../page'; jest.mock('@/lib/prisma', () => ({ prisma: { dealer: { findFirst: jest.fn() } }, })); import { prisma } from '@/lib/prisma'; describe('homepage generateMetadata — google-site-verification', () => { const baseActiveDealer = { id: 'd1', subdomain: 'bob', domain: 'us', domainPrefix: 'myamsoil', customDomain: null, customDomainStatus: 'none', status: 'active', city: 'Topeka', state: 'KS', businessName: "Bob's", user: { email: 'b@x.com' }, // existing select includes user.email // additional fields the page metadata reads: dealerNumber, hideAddress, // showContactName, subscriptionTier, heroSlider, settings, logoUrl, etc. // For test simplicity, set to null/empty where the metadata function // only checks for truthiness. }; beforeEach(() => { jest.clearAllMocks(); jest.resetModules(); // bust unstable_cache between tests }); it('includes verification.google when token is set on active dealer', async () => { (prisma.dealer.findFirst as jest.Mock).mockResolvedValue({ ...baseActiveDealer, gscVerificationToken: 'verify-abc123', }); const meta = await generateMetadata({ params: Promise.resolve({ subdomain: 'bob', domainSlug: 'myamsoil-com' }), }); expect(meta.verification?.google).toBe('verify-abc123'); }); it('does NOT include verification when token is null on active dealer', async () => { (prisma.dealer.findFirst as jest.Mock).mockResolvedValue({ ...baseActiveDealer, gscVerificationToken: null, }); const meta = await generateMetadata({ params: Promise.resolve({ subdomain: 'bob', domainSlug: 'myamsoil-com' }), }); expect(meta.verification?.google).toBeUndefined(); }); it('does NOT include verification when dealer status is suspended (early-return path)', async () => { (prisma.dealer.findFirst as jest.Mock).mockResolvedValue({ ...baseActiveDealer, status: 'suspended', gscVerificationToken: 'verify-abc123', }); const meta = await generateMetadata({ params: Promise.resolve({ subdomain: 'bob', domainSlug: 'myamsoil-com' }), }); expect(meta.verification?.google).toBeUndefined(); expect(meta.robots).toMatchObject({ index: false }); }); }); ``` - [ ] **Step 3: Run test, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/dealers/\[subdomain\]/\[domainSlug\]/__tests__/page ``` Expected: FAIL - `meta.verification` is `undefined` in the first test case. - [ ] **Step 4: Update homepage `generateMetadata` in `app/dealers/[subdomain]/[domainSlug]/page.tsx`** In the metadata return object for active/cancelled_pending dealers (the second `return {...}`, after the noindex early-return), add the verification field: ```ts return { title, description, alternates: { canonical: canonicalUrl }, robots: { index: true, follow: true, 'max-snippet': -1, 'max-image-preview': 'large', 'max-video-preview': -1, }, // ... existing openGraph and twitter unchanged ... ...(dealer.gscVerificationToken ? { verification: { google: dealer.gscVerificationToken } } : {}), }; ``` - [ ] **Step 5: Run homepage test, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/dealers/\[subdomain\]/\[domainSlug\]/__tests__/page ``` Expected: 3 passing. - [ ] **Step 6: Replicate for CMS-page metadata (`[...slug]/page.tsx`)** Add an identical test in `app/dealers/[subdomain]/[domainSlug]/[...slug]/__tests__/page.test.ts` (or create), verify RED, apply the same `verification.google` extension to the CMS page's metadata return, verify GREEN. - [ ] **Step 7: Manual smoke test** In dev, set `gscVerificationToken = 'verify-test-123'` on a test dealer via Prisma Studio. Load the dealer's homepage. View source. Confirm `` appears in ``. Load a CMS page; confirm the tag also appears. - [ ] **Step 8: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add lib/dealer-lookup.ts app/dealers git commit -m "$(cat <<'EOF' feat(gsc): render google-site-verification meta tag from dealer token Required for the Site Verification API to confirm property ownership for custom-domain dealers. Worker writes the token to dealer.gscVerificationToken; this renders it in via Next.js's built-in Metadata.verification.google field, on both homepage and CMS-page metadata generators. Gated on active/cancelled_pending status (consistent with the existing noindex pattern). Co-Authored-By: Claude Opus 4.7 (1M context) EOF )" ``` --- ## Phase 7 - Admin manual reindex endpoint ### Task 21: `/api/admin/dealers/[id]/reindex-gsc` **Files:** - Create: `app/api/admin/dealers/[id]/reindex-gsc/route.ts` - Create: `app/api/admin/dealers/[id]/reindex-gsc/__tests__/route.test.ts` - [ ] **Step 1: Write failing test** ```ts import { POST } from '../route'; import { NextRequest } from 'next/server'; jest.mock('@/lib/admin-auth'); jest.mock('@/lib/gsc-jobs'); jest.mock('@/lib/prisma'); import { requireAdmin } from '@/lib/admin-auth'; import { enqueueAdminReindex } from '@/lib/gsc-jobs'; import { prisma } from '@/lib/prisma'; function makeReq(): NextRequest { return new NextRequest('http://localhost/api/admin/dealers/d1/reindex-gsc', { method: 'POST' }); } describe('POST /api/admin/dealers/[id]/reindex-gsc', () => { beforeEach(() => jest.clearAllMocks()); it('rejects non-admin', async () => { (requireAdmin as jest.Mock).mockResolvedValue({ response: new Response(null, { status: 401 }), }); const res = await POST(makeReq(), { params: Promise.resolve({ id: 'd1' }) }); expect(res.status).toBe(401); }); it('returns 404 when dealer not found', async () => { (requireAdmin as jest.Mock).mockResolvedValue({ session: { user: { id: 'admin' } } }); (prisma.dealer.findUnique as jest.Mock).mockResolvedValue(null); const res = await POST(makeReq(), { params: Promise.resolve({ id: 'd1' }) }); expect(res.status).toBe(404); }); it('returns 400 when dealer status not active or cancelled_pending', async () => { (requireAdmin as jest.Mock).mockResolvedValue({ session: { user: { id: 'admin' } } }); (prisma.dealer.findUnique as jest.Mock).mockResolvedValue({ id: 'd1', status: 'suspended' }); const res = await POST(makeReq(), { params: Promise.resolve({ id: 'd1' }) }); expect(res.status).toBe(400); }); it('enqueues and returns success', async () => { (requireAdmin as jest.Mock).mockResolvedValue({ session: { user: { id: 'admin' } } }); (prisma.dealer.findUnique as jest.Mock).mockResolvedValue({ id: 'd1', status: 'active', subdomain: 'bob', domain: 'us', domainPrefix: 'myamsoil', customDomain: null, customDomainStatus: 'none', }); (enqueueAdminReindex as jest.Mock).mockResolvedValue(undefined); const res = await POST(makeReq(), { params: Promise.resolve({ id: 'd1' }) }); expect(res.status).toBe(200); expect(enqueueAdminReindex).toHaveBeenCalled(); }); }); ``` - [ ] **Step 2: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/admin/dealers/\[id\]/reindex-gsc ``` - [ ] **Step 3: Implement** ```ts import { NextRequest, NextResponse } from 'next/server'; import { requireAdmin } from '@/lib/admin-auth'; import { prisma } from '@/lib/prisma'; import { enqueueAdminReindex } from '@/lib/gsc-jobs'; import { logger } from '@/lib/logger'; const gscLogger = logger.child({ module: 'gsc' }); export async function POST(request: NextRequest, { params }: { params: Promise<{ id: string }> }) { const auth = await requireAdmin(); if ('response' in auth) return auth.response; const { id } = await params; const dealer = await prisma.dealer.findUnique({ where: { id }, select: { id: true, status: true, subdomain: true, domain: true, domainPrefix: true, customDomain: true, customDomainStatus: true }, }); if (!dealer) return NextResponse.json({ error: 'not found' }, { status: 404 }); if (!['active', 'cancelled_pending'].includes(dealer.status)) { return NextResponse.json({ error: 'dealer must be active' }, { status: 400 }); } if (!dealer.subdomain) return NextResponse.json({ error: 'dealer has no subdomain' }, { status: 400 }); await enqueueAdminReindex({ id: dealer.id, subdomain: dealer.subdomain, domain: dealer.domain, domainPrefix: dealer.domainPrefix, customDomain: dealer.customDomain, customDomainStatus: dealer.customDomainStatus, }); gscLogger.info({ dealerId: id, adminId: auth.session.user?.id }, 'Admin queued GSC reindex'); return NextResponse.json({ success: true }); } ``` - [ ] **Step 4: Run, expect GREEN** - [ ] **Step 5: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add app/api/admin/dealers/\[id\]/reindex-gsc/ git commit -m "feat(gsc): admin manual reindex endpoint Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ## Phase 8 - Admin UI ### Task 22: "Reindex with Google" context menu item + `hasPendingReindex` API The disable signal on the menu item must survive page reloads (spec: "Dealer-list API exposes `hasPendingReindex`"). This means the dealer-list API has to JOIN against `GscJob`, not just rely on local component state. Two parts: **Part A - Extend the dealer-list API to expose `hasPendingReindex`** **Files:** - Modify: `app/api/admin/dealers/route.ts` - add `hasPendingReindex: boolean` to each dealer in the response. - Modify: `app/api/admin/dealers/__tests__/route.test.ts` - add assertion. - [ ] **Step A1: Inspect existing dealer-list response shape** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing grep -n "GET\|prisma.dealer.findMany\|select:" app/api/admin/dealers/route.ts | head -20 ``` Note: it's a paginated list - be careful to do the GscJob lookup in a single grouped query, not N+1. - [ ] **Step A2: Write failing test** In `app/api/admin/dealers/__tests__/route.test.ts`, add: ```ts // Falsely-passes guard: a test that ignores hasPendingReindex would pass any // implementation. Must assert presence AND correct value (true vs false). import { GET } from '../route'; import { NextRequest } from 'next/server'; jest.mock('@/lib/admin-auth'); jest.mock('@/lib/prisma'); import { requireAdmin } from '@/lib/admin-auth'; import { prisma } from '@/lib/prisma'; describe('GET /api/admin/dealers — hasPendingReindex', () => { beforeEach(() => { jest.clearAllMocks(); (requireAdmin as jest.Mock).mockResolvedValue({ session: { user: { id: 'a' } } }); }); it('returns hasPendingReindex=true for dealers with pending notify_indexing jobs', async () => { (prisma.dealer.findMany as jest.Mock).mockResolvedValue([ { id: 'd1', subdomain: 'a' /* + minimal fields per existing shape */ }, { id: 'd2', subdomain: 'b' }, ]); // d1 has a pending notify_indexing; d2 does not (prisma.gscJob.groupBy as jest.Mock).mockResolvedValue([ { dealerId: 'd1', _count: { _all: 1 } }, ]); const res = await GET(new NextRequest('http://localhost/api/admin/dealers')); const body = await res.json(); const byId = Object.fromEntries(body.dealers.map((d: any) => [d.id, d])); expect(byId.d1.hasPendingReindex).toBe(true); expect(byId.d2.hasPendingReindex).toBe(false); }); }); ``` - [ ] **Step A3: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/admin/dealers/__tests__/route.test.ts -t "hasPendingReindex" ``` - [ ] **Step A4: Modify the dealer-list endpoint** In `app/api/admin/dealers/route.ts` GET handler, after fetching dealers: ```ts // Bulk-query pending/running notify_indexing jobs in ONE call to avoid N+1. const dealerIds = dealers.map(d => d.id); const pendingByDealer = await prisma.gscJob.groupBy({ by: ['dealerId'], where: { dealerId: { in: dealerIds }, type: 'notify_indexing', status: { in: ['pending', 'running'] }, }, _count: { _all: true }, }); const pendingDealerIds = new Set(pendingByDealer.map(p => p.dealerId)); // Attach hasPendingReindex to each dealer in the response payload const enriched = dealers.map(d => ({ ...d, hasPendingReindex: pendingDealerIds.has(d.id), })); ``` Return `enriched` instead of `dealers` in the response. - [ ] **Step A5: Run, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/admin/dealers/__tests__/route.test.ts ``` - [ ] **Step A6: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add app/api/admin/dealers/ git commit -m "feat(gsc): admin dealer-list exposes hasPendingReindex flag Single groupBy query (not N+1). Survives page reload so the Reindex-with-Google menu item stays disabled until the worker completes the queued notify_indexing job. Co-Authored-By: Claude Opus 4.7 (1M context) " ``` **Part B - Add the "Reindex with Google" context menu item using the new flag** **Files:** - Modify: `components/admin/QuickActionsMenu.tsx` - add menu item + extend `MENU_ITEMS` const - Modify: `components/admin/DealerTable.tsx` - handler, loading state, prop wiring - Modify or create: component tests - [ ] **Step B1: Inspect existing menu structure (including MENU_ITEMS const)** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing grep -n "Regenerate Pages\|onRegeneratePages\|MENU_ITEMS" components/admin/QuickActionsMenu.tsx ``` Note the existing `const MENU_ITEMS = ['menu-support', 'menu-stripe', 'menu-dns', 'menu-regenerate'] as const` - this drives keyboard navigation and MUST get the new id appended. - [ ] **Step B2: Write failing tests for the new menu item** In `components/admin/__tests__/QuickActionsMenu.test.tsx` (create if missing, follow existing component-test pattern in this codebase): ```tsx // Falsely-passes guard: assert exact disabled state in BOTH directions — // false when conditions met, true when not. A component that always-renders- // enabled would pass the enabled case; always-disabled would pass the // disabled case. Both must be tested. import { render, screen, fireEvent } from '@testing-library/react'; import { QuickActionsMenu } from '../QuickActionsMenu'; describe('QuickActionsMenu — Reindex with Google item', () => { const baseProps = { onStartSupport: jest.fn(), onEditDNS: jest.fn(), onRegeneratePages: jest.fn(), onReindexGSC: jest.fn(), stripeCustomerId: 'cus_x', hasSubdomain: true, canReindexGSC: true, isReindexingGSC: false, }; beforeEach(() => jest.clearAllMocks()); it('renders the "Reindex with Google" item when menu opens', () => { render(); fireEvent.click(screen.getByRole('button')); // trigger expect(screen.getByRole('menuitem', { name: /Reindex with Google/i })).toBeInTheDocument(); }); it('disables the item when canReindexGSC=false (e.g. pending job exists)', () => { render(); fireEvent.click(screen.getByRole('button')); expect(screen.getByRole('menuitem', { name: /Reindex with Google/i })).toBeDisabled(); }); it('disables the item when isReindexingGSC=true (just-clicked)', () => { render(); fireEvent.click(screen.getByRole('button')); expect(screen.getByRole('menuitem', { name: /Reindex with Google/i })).toBeDisabled(); }); it('calls onReindexGSC when clicked and enabled', () => { render(); fireEvent.click(screen.getByRole('button')); fireEvent.click(screen.getByRole('menuitem', { name: /Reindex with Google/i })); expect(baseProps.onReindexGSC).toHaveBeenCalledTimes(1); }); }); ``` - [ ] **Step B3: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- components/admin/__tests__/QuickActionsMenu ``` - [ ] **Step B4: Update `QuickActionsMenu.tsx`** Update props interface: ```ts interface QuickActionsMenuProps { onStartSupport: () => void; onEditDNS: () => void; onRegeneratePages: () => void; onReindexGSC?: () => void; // new stripeCustomerId: string | null; hasSubdomain: boolean; canReindexGSC?: boolean; // new — dealer is in correct status, has subdomain, no pending job isStartingSupport?: boolean; isRegenerating?: boolean; isReindexingGSC?: boolean; // new — optimistic just-clicked state } ``` Update the `MENU_ITEMS` const to include the new id: ```ts const MENU_ITEMS = ['menu-support', 'menu-stripe', 'menu-dns', 'menu-regenerate', 'menu-reindex-gsc'] as const; ``` Add the menu item below the "Regenerate Pages" entry in the JSX, with `id="menu-reindex-gsc"` and the disable check `disabled={!canReindexGSC || isReindexingGSC}`. - [ ] **Step B5: Update `DealerTable.tsx`** Add `reindexingDealerId` state. Add `handleReindexGSC(dealerId)` that POSTs to `/api/admin/dealers/{id}/reindex-gsc`, sets loading state, shows toast on success/failure. Compute `canReindexGSC`: ```ts const canReindexGSC = ['active', 'cancelled_pending'].includes(dealer.status) && !!dealer.subdomain && !dealer.hasPendingReindex; // from the API enrichment in Part A ``` Pass `canReindexGSC` and `isReindexingGSC={reindexingDealerId === dealer.id}` to QuickActionsMenu. - [ ] **Step B6: Run tests, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- components/admin/__tests__/QuickActionsMenu ``` - [ ] **Step B7: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add components/admin/ git commit -m "feat(gsc): Reindex with Google context-menu item Uses canReindexGSC prop fed by dealer-list API's hasPendingReindex flag, so the disable state survives page reloads. MENU_ITEMS const updated for keyboard nav. Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 23: GSC section in DealerDetailModal **Files:** - Modify: `components/admin/DealerDetailModal.tsx` - add "Indexing & SEO" section - Modify: `app/api/admin/dealers/[id]/detail/route.ts` - include `gsc*` columns + recent GscJob rows - [ ] **Step 1: Update the detail API to include GSC fields** In `app/api/admin/dealers/[id]/detail/route.ts`, extend the `prisma.dealer.findUnique` select to include all `gsc*` columns, and add a separate query for the last 10 `GscJob` rows for this dealer. Include both in the response. - [ ] **Step 2: Add UI test** Assert the new section renders with the canonical URL, index status badge, last-submitted timestamp, preflight state, and recent jobs list. - [ ] **Step 3: Run, expect RED** - [ ] **Step 4: Add the section to `DealerDetailModal.tsx`** Below the existing "Site Information" section, render a new collapsible/headered section "Indexing & SEO". Render the fields and a buttons row: "Resubmit sitemap" (POST `/api/admin/dealers/[id]/reindex-gsc`), "Deep audit" (new endpoint - defer to follow-up; for now show but disabled with tooltip "Coming soon"), and "Force reindex" (same as resubmit but with `force: true` query param - implementer choice on how to differentiate from the menu button). Under "Site Information", append a "SSL/reachability" row reading `gscPreflightHealthyAt` and `gscPreflightError`, formatted as described in the spec. - [ ] **Step 5: Run tests, expect GREEN** - [ ] **Step 6: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add components/admin/DealerDetailModal.tsx app/api/admin/dealers/\[id\]/detail/ git commit -m "feat(gsc): Indexing & SEO section in dealer detail modal Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 24: `/admin/gsc` global dashboard **Files:** - Create: `app/admin/gsc/page.tsx` - Create: `app/admin/gsc/page.module.css` - Create: `app/api/admin/gsc/summary/route.ts` - Create: `app/api/admin/gsc/summary/__tests__/route.test.ts` - Create: `app/api/admin/gsc/backfill/route.ts` - Create: `app/api/admin/gsc/backfill/__tests__/route.test.ts` - Modify: `app/admin/page.tsx` - add tile linking to `/admin/gsc` - [ ] **Step 1: Write failing tests for the summary endpoint** `app/api/admin/gsc/summary/__tests__/route.test.ts`: ```ts /** @jest-environment node */ // Falsely-passes guard: asserting only status 200 would pass even if the // response body is completely wrong. Each test asserts a specific field. import { GET } from '../route'; import { NextRequest } from 'next/server'; jest.mock('@/lib/admin-auth'); jest.mock('@/lib/prisma', () => ({ prisma: { gscJob: { count: jest.fn(), findMany: jest.fn() }, gscQuotaUsage: { findUnique: jest.fn() }, }, })); import { requireAdmin } from '@/lib/admin-auth'; import { prisma } from '@/lib/prisma'; const makeReq = () => new NextRequest('http://localhost/api/admin/gsc/summary'); describe('GET /api/admin/gsc/summary', () => { beforeEach(() => jest.clearAllMocks()); it('rejects non-admin requests', async () => { (requireAdmin as jest.Mock).mockResolvedValue({ response: new Response(null, { status: 401 }), }); const res = await GET(makeReq()); expect(res.status).toBe(401); }); it('returns todayQuotaUsed, queueCounts, and recentFailures for admin', async () => { (requireAdmin as jest.Mock).mockResolvedValue({ session: { user: { id: 'a' } } }); (prisma.gscJob.count as jest.Mock) .mockResolvedValueOnce(12) // pending .mockResolvedValueOnce(3) // running .mockResolvedValueOnce(847) // completed24h .mockResolvedValueOnce(4); // failed (prisma.gscQuotaUsage.findUnique as jest.Mock).mockResolvedValue({ indexingApiCalls: 47 }); (prisma.gscJob.findMany as jest.Mock).mockResolvedValue([ { id: 'jf1', type: 'submit_sitemap', dealerId: 'd1', lastError: 'AUTH_FAILED' }, ]); const res = await GET(makeReq()); expect(res.status).toBe(200); const body = await res.json(); expect(body).toMatchObject({ todayQuotaUsed: 47, queueCounts: { pending: 12, running: 3, completed24h: 847, failed: 4 }, recentFailures: expect.arrayContaining([ expect.objectContaining({ id: 'jf1', lastError: 'AUTH_FAILED' }), ]), }); }); it('handles missing quota row as 0', async () => { (requireAdmin as jest.Mock).mockResolvedValue({ session: { user: { id: 'a' } } }); (prisma.gscJob.count as jest.Mock).mockResolvedValue(0); (prisma.gscQuotaUsage.findUnique as jest.Mock).mockResolvedValue(null); (prisma.gscJob.findMany as jest.Mock).mockResolvedValue([]); const res = await GET(makeReq()); const body = await res.json(); expect(body.todayQuotaUsed).toBe(0); }); }); ``` - [ ] **Step 2: Write failing tests for the backfill endpoint** `app/api/admin/gsc/backfill/__tests__/route.test.ts`: ```ts /** @jest-environment node */ // Falsely-passes guard: tests must verify (a) auth, (b) idempotency (recent // submissions skipped), AND (c) actual enqueue. An empty-stub implementation // returning {enqueued: 0} would pass auth + idempotency but not enqueue. import { POST } from '../route'; import { NextRequest } from 'next/server'; jest.mock('@/lib/admin-auth'); jest.mock('@/lib/prisma', () => ({ prisma: { dealer: { findMany: jest.fn() }, gscJob: { create: jest.fn() } }, })); jest.mock('@/lib/gsc-jobs', () => ({ enqueueDealerActivation: jest.fn(), // reuse: a backfill is functionally an activation })); import { requireAdmin } from '@/lib/admin-auth'; import { prisma } from '@/lib/prisma'; import { enqueueDealerActivation } from '@/lib/gsc-jobs'; const makeReq = () => new NextRequest('http://localhost/api/admin/gsc/backfill', { method: 'POST' }); describe('POST /api/admin/gsc/backfill', () => { beforeEach(() => jest.clearAllMocks()); it('rejects non-admin requests', async () => { (requireAdmin as jest.Mock).mockResolvedValue({ response: new Response(null, { status: 401 }), }); const res = await POST(makeReq()); expect(res.status).toBe(401); }); it('enqueues only dealers WITHOUT recent (within 30d) submission', async () => { (requireAdmin as jest.Mock).mockResolvedValue({ session: { user: { id: 'a' } } }); const thirtyDaysAgo = new Date(Date.now() - 31 * 24 * 60 * 60 * 1000); (prisma.dealer.findMany as jest.Mock).mockResolvedValue([ // d1 has never been submitted { id: 'd1', subdomain: 'a', domain: 'us', domainPrefix: 'myamsoil', customDomain: null, customDomainStatus: 'none', gscLastSubmittedAt: null }, // d2 submitted 31 days ago (eligible for re-submission) { id: 'd2', subdomain: 'b', domain: 'us', domainPrefix: 'myamsoil', customDomain: null, customDomainStatus: 'none', gscLastSubmittedAt: thirtyDaysAgo }, ]); const res = await POST(makeReq()); const body = await res.json(); expect(body.enqueued).toBe(2); expect(enqueueDealerActivation).toHaveBeenCalledTimes(2); }); it('returns enqueued=0 when no eligible dealers', async () => { (requireAdmin as jest.Mock).mockResolvedValue({ session: { user: { id: 'a' } } }); (prisma.dealer.findMany as jest.Mock).mockResolvedValue([]); const res = await POST(makeReq()); const body = await res.json(); expect(body.enqueued).toBe(0); expect(enqueueDealerActivation).not.toHaveBeenCalled(); }); }); ``` The `dealer.findMany` should already filter by `gscLastSubmittedAt: { lt: thirtyDaysAgo } OR null` in the implementation, so the test mock simulates what the query would return - only eligible dealers. - [ ] **Step 3: Run, expect RED** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/admin/gsc ``` Expected: FAIL - modules not found. - [ ] **Step 4: Implement `app/api/admin/gsc/summary/route.ts`** ```ts import { NextRequest, NextResponse } from 'next/server'; import { requireAdmin } from '@/lib/admin-auth'; import { prisma } from '@/lib/prisma'; export async function GET(_request: NextRequest) { const auth = await requireAdmin(); if ('response' in auth) return auth.response; const since = new Date(Date.now() - 24 * 60 * 60 * 1000); const today = new Date(); today.setUTCHours(0, 0, 0, 0); const [pending, running, completed24h, failed, quota, recentFailures] = await Promise.all([ prisma.gscJob.count({ where: { status: 'pending' } }), prisma.gscJob.count({ where: { status: 'running' } }), prisma.gscJob.count({ where: { status: 'completed', updatedAt: { gte: since } } }), prisma.gscJob.count({ where: { status: 'failed', updatedAt: { gte: since } } }), prisma.gscQuotaUsage.findUnique({ where: { date: today } }), prisma.gscJob.findMany({ where: { status: 'failed', updatedAt: { gte: since } }, orderBy: { updatedAt: 'desc' }, take: 20, select: { id: true, type: true, dealerId: true, lastError: true, updatedAt: true, attempts: true }, }), ]); return NextResponse.json({ todayQuotaUsed: quota?.indexingApiCalls ?? 0, queueCounts: { pending, running, completed24h, failed }, recentFailures, }); } ``` - [ ] **Step 5: Implement `app/api/admin/gsc/backfill/route.ts`** ```ts import { NextRequest, NextResponse } from 'next/server'; import { requireAdmin } from '@/lib/admin-auth'; import { prisma } from '@/lib/prisma'; import { enqueueDealerActivation } from '@/lib/gsc-jobs'; import { logger } from '@/lib/logger'; const gscLogger = logger.child({ module: 'gsc' }); export async function POST(_request: NextRequest) { const auth = await requireAdmin(); if ('response' in auth) return auth.response; const thirtyDaysAgo = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000); const dealers = await prisma.dealer.findMany({ where: { status: 'active', subdomain: { not: null }, OR: [{ gscLastSubmittedAt: null }, { gscLastSubmittedAt: { lt: thirtyDaysAgo } }], }, select: { id: true, subdomain: true, domain: true, domainPrefix: true, customDomain: true, customDomainStatus: true }, }); let enqueued = 0; for (const d of dealers) { try { await enqueueDealerActivation(d as Parameters[0]); enqueued++; } catch (err) { gscLogger.error({ err, dealerId: d.id }, 'Backfill enqueue failed for dealer (skipping)'); } } gscLogger.info({ enqueued, considered: dealers.length, adminId: auth.session.user?.id }, 'Backfill complete'); return NextResponse.json({ enqueued, considered: dealers.length }); } ``` - [ ] **Step 6: Run tests, expect GREEN** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test -- app/api/admin/gsc ``` - [ ] **Step 7: Build the dashboard page** `app/admin/gsc/page.tsx` - `'use client'` component that fetches `/api/admin/gsc/summary` on mount, renders sections: - Today's quota progress bar (uses `todayQuotaUsed / 200`; yellow at 75%, red at 90%) - Queue health counts grid - Recent failures table with type / dealer / error / attempts columns - Backfill button - POST to `/api/admin/gsc/backfill`, show count toast Use the existing admin component styles (`app/admin/page.module.css`-style patterns). Component tests can be deferred to standard React-Testing-Library setup; the API contracts above are the load-bearing test surface. - [ ] **Step 8: Add tile on main admin dashboard** In `app/admin/page.tsx`, add a tile linking to `/admin/gsc`. Match the existing tile structure used for `/admin/amsoil-images`. - [ ] **Step 9: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add app/admin/gsc app/api/admin/gsc app/admin/page.tsx git commit -m "$(cat <<'EOF' feat(gsc): /admin/gsc dashboard + summary + backfill endpoints API routes tested for auth, response shape, and idempotency. Dashboard page consumes the summary endpoint for at-a-glance health. Co-Authored-By: Claude Opus 4.7 (1M context) EOF )" ``` --- ## Phase 9 - Documentation + final wiring ### Task 25: Update `.env.example` and `docs/CRON_JOBS.md` **Files:** - Modify: `.env.example` - Modify: `docs/CRON_JOBS.md` - [ ] **Step 1: Verify `.env.example` already has `GOOGLE_SERVICE_ACCOUNT_JSON`** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing grep -n "GOOGLE_SERVICE_ACCOUNT_JSON" .env.example ``` Expected: present (was added during v1 prototype; that section can stay). - [ ] **Step 2: Update `docs/CRON_JOBS.md` with the two new cron entries** Add entries for: ``` - /api/cron/gsc-drain Schedule: every 5 minutes Auth: Bearer $CRON_SECRET Purpose: drain the GscJob queue, respect rate limits Crontab: */5 * * * * curl -fsS -X POST -H "Authorization: Bearer $(cat /etc/amsoil/cron-secret)" https://amsoil.aimclear.com/api/cron/gsc-drain - /api/cron/gsc-audit Schedule: daily at 03:00 Auth: Bearer $CRON_SECRET Purpose: enqueue inspect_url jobs for all active dealer homepages Crontab: 0 3 * * * curl -fsS -X POST -H "Authorization: Bearer $(cat /etc/amsoil/cron-secret)" https://amsoil.aimclear.com/api/cron/gsc-audit ``` Match the documentation style already established in `CRON_JOBS.md`. - [ ] **Step 3: Commit** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git add .env.example docs/CRON_JOBS.md git commit -m "docs(gsc): document new cron entries Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ### Task 26: Final integration check + type-check + full test run **Files:** none (verification only) - [ ] **Step 1: Run the full test suite** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npm test ``` Expected: all tests passing, no new failures introduced. - [ ] **Step 2: Run type-check** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing npx tsc --noEmit 2>&1 | grep -E "lib/google|lib/gsc|app/api/cron/gsc|app/api/admin/gsc|app/api/admin/dealers/\[id\]/reindex-gsc|app/admin/gsc" ``` Expected: zero output (no type errors in any new or modified file). - [ ] **Step 3: Manual smoke test of the worker against an empty queue** Set `GOOGLE_SERVICE_ACCOUNT_JSON` to a known-bad value in `.env.local`, start the dev server, and POST to `/api/cron/gsc-drain` with the right Bearer header. Expect: empty result (no jobs to process). - [ ] **Step 4: Commit a checkpoint marker** ```bash cd /c/dev/aimclear/.worktrees/gsc-indexing git commit --allow-empty -m "chore(gsc): integration checkpoint — all phases green Co-Authored-By: Claude Opus 4.7 (1M context) " ``` --- ## Final notes **Rollout (post-merge, per the design spec):** 1. **Phase 1** - Code merged with `GOOGLE_SERVICE_ACCOUNT_JSON` unset. Feature dark. Migration runs (new tables, columns, system user). No behavioral change. 2. **Phase 2** - Set env var on staging. Manually trigger admin reindex on 1–2 test dealers. Verify queue drains and sitemap shows up in GSC. 3. **Phase 3** - Set env var on production. Service account added as Owner on all four Domain properties. Crontab entries installed. **Backfill is gated** by clicking the admin backfill button - not auto-triggered. 4. **Phase 4** - Click backfill. Watch worker drain ~1,140 sitemap submissions. Once subdomain submissions look healthy, the custom-domain auto-property creation begins triggering on subsequent custom-domain activations. **Kill switch:** Unset `GOOGLE_SERVICE_ACCOUNT_JSON`. All jobs remain queued; resume on env restore. ## Appendix A - QA-review refinements folded into earlier tasks Adversarial QA review surfaced additional items not worth a full task rewrite. Implementer should apply these as part of the relevant task: **Task 1 (delete v1 prototype):** if any developer has already applied the v1 migration locally (`20260511120000_add_gsc_submitted_at`), simply deleting its directory will cause `prisma migrate dev` to detect drift. Add a check: `npx prisma migrate status` - if it shows a stale entry for the deleted migration, the developer needs to manually drop the column (`ALTER TABLE "Dealer" DROP COLUMN "gscSubmittedAt";`) and remove the row from `_prisma_migrations` table before continuing. Production has never run this migration so it's a dev-only concern. **Task 9 (preflight):** add an early return for dealers with no canonical URL: ```ts if (!dealer.subdomain && !dealer.customDomain) { return { healthy: false, reason: 'dealer has no canonical URL' }; } ``` **Task 11 (gsc-jobs tests):** tighten `enqueueDealerActivation` assertion to verify EXACT count (`expect(prisma.gscJob.create).toHaveBeenCalledTimes(2)`) and the SHAPE of each call. The current `arrayContaining` allows extra silent calls. Apply same tightening to each enqueue function's test. **Task 13 (worker preflight gate):** combine the dealer fetch in the gate block with the one in `executeJob`. Pass the dealer object through rather than re-querying. Save one DB round-trip per submit/notify job. **Task 14 (auto-notes test):** move `(prisma.dealerNote = { create: jest.fn() } as any)` from describe-block scope into `beforeEach` to prevent state leakage between tests. **Tasks 17 + 18 (cancellation race):** the status route and Stripe webhook may both fire `enqueueDealerCancellation` for the same dealer transition. Job idempotency makes this harmless (duplicate `delete_sitemap` jobs hit a 404 from Google and complete cleanly), but the queue counts will briefly show duplicates. Acceptable for V1; consider deduping via a unique-by-(dealerId, type, status='pending') index if it becomes noisy. **Task 19 (custom-domain hooks):** in addition to `app/api/dealer/custom-domain/activate/route.ts` and the dealer's DELETE route, the following admin routes can also transition `customDomainStatus` and need the same hooks: - `app/api/admin/custom-domains/[domain]/route.ts` (status changes) - `app/api/admin/custom-domains/[domain]/regenerate/route.ts` (preserves status but reissues cert; enqueue a fresh submit_sitemap) - `app/api/admin/dealers/[id]/grandfathered-custom-domain/route.ts` (admin grandfather flow) Pattern: detect the transition (old status vs new), call `enqueueCustomDomainActivation` or `enqueueCustomDomainDeactivation` as appropriate. Each should follow the same try/catch + log pattern from Task 19. **Task 22 (Reindex menu disable-on-pending):** Now handled in Task 22 Part A (dealer-list API exposes `hasPendingReindex`) and Part B (menu uses `canReindexGSC` prop derived from that flag). No longer an appendix item. --- **Known V1 limitations (within this plan):** - **www variant handling (clarified design, not a limitation):** Production Apache serves both `example.com` and `www.example.com` from the same VirtualHost via `ServerAlias` (no redirect, identical content). The dealer page already emits `` pointing to the stored form, so Google's canonical resolution consolidates indexing automatically. V1 registers a SINGLE GSC property per custom domain - `getCanonicalUrl(dealer)`. Registering both forms would double every operation for zero indexing benefit. If a dealer modifies their `customDomain` field later (admin-only flow currently), the worker would need to delete the old property and register the new one - that's a separate follow-up. - **Worker concurrency (RESOLVED in Task 12):** The worker now uses `$queryRaw` with `UPDATE … RETURNING` + `FOR UPDATE SKIP LOCKED` for atomic claim, matching the spec's concurrency guarantee. Two overlapping cron invocations cannot claim the same row. No longer a known limitation. **Related issues:** - Cert expiry monitoring: [#848](https://github.com/aimclear/amsoil-dlp/issues/848) (separate work item; partially mitigated by preflight check in this plan).