# SERP Dealer Redirect & Schema Fix

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Fix Google SERP misidentification of dealer pages by redirecting `/dealers/[subdomain]` paths to canonical subdomain URLs, fixing the sitemap, and adding `WebSite` structured data.

**Architecture:** Three-layer fix: (1) Add `WebSite` schema node to JSON-LD graph so Google has explicit site name signals per dealer, (2) 301 redirect `/dealers/[subdomain]` on parent domains to the dealer's canonical host (custom domain or subdomain URL), (3) Fix sitemap to list canonical subdomain URLs instead of `/dealers/` paths. A `?noredirect` kill switch preserves direct access for debugging.

**Tech Stack:** Next.js proxy (middleware), Prisma, JSON-LD structured data, sitemap generation

---

## Background

Google is indexing `amsoil.aimclear.com/dealers/hbr` instead of `hbr.myamsoil.com`. This happens because:
- The sitemap explicitly lists `/dealers/[subdomain]` paths
- No redirect exists on those paths - they serve a 200
- The canonical tag hints at the subdomain URL but Google treats canonical as advisory
- Without `WebSite` schema, Google infers (and confuses) site names across 900+ identical dealer templates

### Task 0: Add WebSite schema node (ALREADY DONE)

**Files:**
- Modified: `components/DealerTemplate.tsx:83-97` - added `WebSite` node, fixed `WebPage.isPartOf`
- Modified: `components/__tests__/DealerTemplate.test.tsx` - added 2 tests

Already implemented and passing (63/63 tests). The `WebSite` node provides `name: dealer.name` for Google's site name system, and `WebPage.isPartOf` now references the `WebSite` `@id` instead of a bare URL string.

---

### Task 1: Create dealer canonical host lookup

**Files:**
- Create: `lib/dealer-canonical-host-lookup.ts`
- Test: `lib/__tests__/dealer-canonical-host-lookup.test.ts`

**Why:** The proxy needs to construct the redirect URL for a given subdomain. We need the dealer's `domain` (com/ca), `domainPrefix` (myamsoil/shopamsoil), and custom domain status. This follows the same cached lookup pattern as `subdomain-lookup.ts` and `subdomain-redirect-lookup.ts`.

**Step 1: Write the failing test**

Create `lib/__tests__/dealer-canonical-host-lookup.test.ts`:

```typescript
import {
  lookupDealerCanonicalHost,
  clearDealerCanonicalHostCache,
} from '../dealer-canonical-host-lookup';
import { prisma } from '../prisma';

jest.mock('../prisma', () => ({
  prisma: {
    dealer: {
      findFirst: jest.fn(),
    },
  },
}));

describe('Dealer Canonical Host Lookup', () => {
  beforeEach(() => {
    jest.clearAllMocks();
    clearDealerCanonicalHostCache();
  });

  it('should return null for unknown subdomain', async () => {
    (prisma.dealer.findFirst as jest.Mock).mockResolvedValue(null);
    const result = await lookupDealerCanonicalHost('nonexistent');
    expect(result).toBeNull();
  });

  it('should return subdomain URL for US myamsoil dealer', async () => {
    (prisma.dealer.findFirst as jest.Mock).mockResolvedValue({
      domain: 'com',
      domainPrefix: 'myamsoil',
      customDomain: null,
      customDomainStatus: 'none',
    });
    const result = await lookupDealerCanonicalHost('hbr');
    expect(result).toBe('hbr.myamsoil.com');
  });

  it('should return subdomain URL for Canadian dealer', async () => {
    (prisma.dealer.findFirst as jest.Mock).mockResolvedValue({
      domain: 'ca',
      domainPrefix: 'myamsoil',
      customDomain: null,
      customDomainStatus: 'none',
    });
    const result = await lookupDealerCanonicalHost('maple');
    expect(result).toBe('maple.myamsoil.ca');
  });

  it('should return subdomain URL for shopamsoil dealer', async () => {
    (prisma.dealer.findFirst as jest.Mock).mockResolvedValue({
      domain: 'com',
      domainPrefix: 'shopamsoil',
      customDomain: null,
      customDomainStatus: 'none',
    });
    const result = await lookupDealerCanonicalHost('bobsoil');
    expect(result).toBe('bobsoil.shopamsoil.com');
  });

  it('should return custom domain when active', async () => {
    (prisma.dealer.findFirst as jest.Mock).mockResolvedValue({
      domain: 'com',
      domainPrefix: 'myamsoil',
      customDomain: 'www.bobsoil.com',
      customDomainStatus: 'active',
    });
    const result = await lookupDealerCanonicalHost('bobsoil');
    expect(result).toBe('www.bobsoil.com');
  });

  it('should return subdomain URL when custom domain is not active', async () => {
    (prisma.dealer.findFirst as jest.Mock).mockResolvedValue({
      domain: 'com',
      domainPrefix: 'myamsoil',
      customDomain: 'www.bobsoil.com',
      customDomainStatus: 'pending',
    });
    const result = await lookupDealerCanonicalHost('bobsoil');
    expect(result).toBe('bobsoil.myamsoil.com');
  });

  it('should cache results', async () => {
    (prisma.dealer.findFirst as jest.Mock).mockResolvedValue({
      domain: 'com',
      domainPrefix: 'myamsoil',
      customDomain: null,
      customDomainStatus: 'none',
    });

    await lookupDealerCanonicalHost('hbr');
    await lookupDealerCanonicalHost('hbr');

    expect(prisma.dealer.findFirst).toHaveBeenCalledTimes(1);
  });

  it('should normalize subdomain to lowercase', async () => {
    (prisma.dealer.findFirst as jest.Mock).mockResolvedValue({
      domain: 'com',
      domainPrefix: 'myamsoil',
      customDomain: null,
      customDomainStatus: 'none',
    });

    await lookupDealerCanonicalHost('HBR');

    expect(prisma.dealer.findFirst).toHaveBeenCalledWith({
      where: {
        subdomain: 'hbr',
        status: { in: ['active', 'cancelled_pending'] },
      },
      select: {
        domain: true,
        domainPrefix: true,
        customDomain: true,
        customDomainStatus: true,
      },
    });
  });
});
```

**Step 2: Run test to verify it fails**

Run: `npx jest lib/__tests__/dealer-canonical-host-lookup.test.ts --no-coverage`
Expected: FAIL - module not found

**Step 3: Write the implementation**

Create `lib/dealer-canonical-host-lookup.ts`:

```typescript
import { prisma } from './prisma';
import { logger } from './logger';

/**
 * Cache for subdomain → canonical host lookups.
 * Key: subdomain (lowercase), Value: { host, expiresAt }
 *
 * Returns the canonical host for a dealer:
 * - Custom domain if active (e.g., "www.bobsoil.com")
 * - Otherwise subdomain URL (e.g., "hbr.myamsoil.com")
 *
 * Used by proxy to 301 redirect /dealers/[subdomain] paths.
 */
const canonicalHostCache = new Map<string, { host: string | null; expiresAt: number }>();

const CACHE_TTL_MS = 5 * 60 * 1000;
const NEGATIVE_CACHE_TTL_MS = 60 * 1000;
const MAX_CACHE_SIZE = 10000;

function evictCacheEntries(): void {
  const now = Date.now();
  for (const [key, value] of canonicalHostCache.entries()) {
    if (value.expiresAt <= now) {
      canonicalHostCache.delete(key);
    }
  }
  if (canonicalHostCache.size > MAX_CACHE_SIZE) {
    const entries = Array.from(canonicalHostCache.entries()).sort(
      ([, a], [, b]) => a.expiresAt - b.expiresAt
    );
    const toRemove = entries.slice(0, canonicalHostCache.size - MAX_CACHE_SIZE + 100);
    for (const [key] of toRemove) {
      canonicalHostCache.delete(key);
    }
  }
}

/**
 * Look up the canonical host for a dealer by subdomain.
 *
 * Returns the host that should be used in URLs:
 * - Active custom domain takes priority (e.g., "www.bobsoil.com")
 * - Otherwise: "{subdomain}.{domainPrefix}.{tld}" (e.g., "hbr.myamsoil.com")
 * - Returns null if dealer not found or inactive
 */
export async function lookupDealerCanonicalHost(subdomain: string): Promise<string | null> {
  const normalized = subdomain.toLowerCase();

  const cached = canonicalHostCache.get(normalized);
  if (cached && Date.now() < cached.expiresAt) {
    return cached.host;
  }
  canonicalHostCache.delete(normalized);

  try {
    const dealer = await prisma.dealer.findFirst({
      where: {
        subdomain: normalized,
        status: { in: ['active', 'cancelled_pending'] },
      },
      select: {
        domain: true,
        domainPrefix: true,
        customDomain: true,
        customDomainStatus: true,
      },
    });

    if (canonicalHostCache.size >= MAX_CACHE_SIZE) {
      evictCacheEntries();
    }

    if (!dealer) {
      canonicalHostCache.set(normalized, {
        host: null,
        expiresAt: Date.now() + NEGATIVE_CACHE_TTL_MS,
      });
      return null;
    }

    // Custom domain takes priority if active
    const host =
      dealer.customDomain && dealer.customDomainStatus === 'active'
        ? dealer.customDomain
        : `${normalized}.${dealer.domainPrefix}.${dealer.domain}`;

    canonicalHostCache.set(normalized, {
      host,
      expiresAt: Date.now() + CACHE_TTL_MS,
    });

    return host;
  } catch (error) {
    logger.error({ err: error, subdomain }, 'Dealer canonical host lookup error');
    return null;
  }
}

export function invalidateDealerCanonicalHostCache(subdomain: string): void {
  canonicalHostCache.delete(subdomain.toLowerCase());
}

export function clearDealerCanonicalHostCache(): void {
  canonicalHostCache.clear();
}
```

**Step 4: Run test to verify it passes**

Run: `npx jest lib/__tests__/dealer-canonical-host-lookup.test.ts --no-coverage`
Expected: PASS - all 8 tests

**Step 5: Commit**

```bash
git add lib/dealer-canonical-host-lookup.ts lib/__tests__/dealer-canonical-host-lookup.test.ts
git commit -m "feat: add dealer canonical host lookup for /dealers/ redirect"
```

---

### Task 2: Add /dealers/ → subdomain 301 redirect in proxy

**Files:**
- Modify: `proxy.ts:437-458` (existing `/dealers/` bot probe block)
- Modify: `proxy.ts:9` (add import)
- Test: `__tests__/proxy.test.ts` (add new describe block)

**Why:** When someone (or Googlebot) visits `amsoil.aimclear.com/dealers/hbr`, the proxy currently serves the page with a 200. It should 301 redirect to `https://hbr.myamsoil.com/`. Internal rewrites from subdomain routing (`NextResponse.rewrite`) don't re-trigger middleware, so there's no loop risk.

**Step 1: Write the failing tests**

Add to `__tests__/proxy.test.ts`. First, add the mock at the top (alongside existing mocks):

```typescript
// Mock dealer-canonical-host-lookup for /dealers/ redirects
const mockLookupDealerCanonicalHost = jest.fn().mockResolvedValue(null);
jest.mock('../lib/dealer-canonical-host-lookup', () => ({
  lookupDealerCanonicalHost: (...args: unknown[]) => mockLookupDealerCanonicalHost(...args),
}));
```

Then add the test block:

```typescript
describe('Proxy - /dealers/ Path Redirect', () => {
  beforeEach(() => {
    mockLookupDealerCanonicalHost.mockReset();
  });

  it('should 301 redirect /dealers/hbr to subdomain URL on parent domain', async () => {
    mockLookupDealerCanonicalHost.mockResolvedValue('hbr.myamsoil.com');

    const request = new NextRequest('http://amsoil.aimclear.com/dealers/hbr', {
      headers: { host: 'amsoil.aimclear.com' },
    });
    const response = await proxy(request);

    expect(response.status).toBe(301);
    expect(response.headers.get('location')).toBe('https://hbr.myamsoil.com/');
  });

  it('should preserve CMS page path in redirect', async () => {
    mockLookupDealerCanonicalHost.mockResolvedValue('hbr.myamsoil.com');

    const request = new NextRequest('http://amsoil.aimclear.com/dealers/hbr/services', {
      headers: { host: 'amsoil.aimclear.com' },
    });
    const response = await proxy(request);

    expect(response.status).toBe(301);
    expect(response.headers.get('location')).toBe('https://hbr.myamsoil.com/services');
  });

  it('should redirect to custom domain when active', async () => {
    mockLookupDealerCanonicalHost.mockResolvedValue('www.bobsoil.com');

    const request = new NextRequest('http://amsoil.aimclear.com/dealers/bobsoil', {
      headers: { host: 'amsoil.aimclear.com' },
    });
    const response = await proxy(request);

    expect(response.status).toBe(301);
    expect(response.headers.get('location')).toBe('https://www.bobsoil.com/');
  });

  it('should skip redirect when ?noredirect param is present', async () => {
    mockLookupDealerCanonicalHost.mockResolvedValue('hbr.myamsoil.com');

    const request = new NextRequest('http://amsoil.aimclear.com/dealers/hbr?noredirect', {
      headers: { host: 'amsoil.aimclear.com' },
    });
    const response = await proxy(request);

    // Should NOT redirect — falls through to normal page rendering
    expect(response.status).not.toBe(301);
  });

  it('should not redirect on localhost (dev environment)', async () => {
    mockLookupDealerCanonicalHost.mockResolvedValue('hbr.myamsoil.com');

    const request = new NextRequest('http://localhost:3000/dealers/hbr', {
      headers: { host: 'localhost:3000' },
    });
    const response = await proxy(request);

    expect(response.status).not.toBe(301);
  });

  it('should fall through when dealer not found', async () => {
    mockLookupDealerCanonicalHost.mockResolvedValue(null);

    const request = new NextRequest('http://amsoil.aimclear.com/dealers/nonexistent', {
      headers: { host: 'amsoil.aimclear.com' },
    });
    const response = await proxy(request);

    // Should not redirect — let the page handle it (shows noindex for missing dealers)
    expect(response.status).not.toBe(301);
  });

  it('should preserve query params in redirect (except noredirect)', async () => {
    mockLookupDealerCanonicalHost.mockResolvedValue('hbr.myamsoil.com');

    const request = new NextRequest('http://amsoil.aimclear.com/dealers/hbr?ref=google&utm_source=search', {
      headers: { host: 'amsoil.aimclear.com' },
    });
    const response = await proxy(request);

    expect(response.status).toBe(301);
    const location = response.headers.get('location')!;
    expect(location).toContain('ref=google');
    expect(location).toContain('utm_source=search');
  });
});
```

**Step 2: Run tests to verify they fail**

Run: `npx jest __tests__/proxy.test.ts --no-coverage -t "/dealers/ Path Redirect"`
Expected: FAIL - no redirect logic exists yet

**Step 3: Implement the redirect**

In `proxy.ts`, add the import at line 9 (after existing subdomain-redirect-lookup import):

```typescript
import { lookupDealerCanonicalHost } from './lib/dealer-canonical-host-lookup';
```

Then modify the existing `/dealers/` block at lines 437-458. After the bot probe checks (line 457), add the redirect logic:

```typescript
  if (pathname.startsWith('/dealers/')) {
    // Dealer pages are static (force-static) — block non-GET/HEAD methods.
    if (request.method !== 'GET' && request.method !== 'HEAD') {
      return createMethodNotAllowedResponse();
    }

    const segments = pathname.split('/').filter(Boolean);
    if (segments.length >= 2) {
      const targetSubdomain = segments[1];
      if (isBotProbe(targetSubdomain)) {
        return createBotProbe404Response();
      }
      for (let i = 2; i < segments.length; i++) {
        if (isSlugBotProbe(segments[i])) {
          return createBotProbe404Response();
        }
      }

      // 301 REDIRECT: /dealers/[subdomain] on parent domains → canonical subdomain URL
      // This prevents Google from indexing /dealers/ paths instead of subdomain URLs.
      // Internal rewrites from subdomain routing (NextResponse.rewrite) don't re-trigger
      // middleware, so there is no redirect loop.
      // Skip on localhost/dev (not production) and when ?noredirect is present (debugging).
      const isParentDomain = isMainAppDomain(hostname);
      const hasNoRedirect = request.nextUrl.searchParams.has('noredirect');
      if (isParentDomain && !hasNoRedirect) {
        const canonicalHost = await lookupDealerCanonicalHost(targetSubdomain);
        if (canonicalHost) {
          // Build redirect URL: preserve path after /dealers/[subdomain] and query params
          const remainingPath = segments.slice(2).join('/');
          const pagePath = remainingPath ? `/${remainingPath}` : '/';
          const search = request.nextUrl.search;
          const redirectUrl = `https://${canonicalHost}${pagePath}${search}`;
          return NextResponse.redirect(redirectUrl, 301);
        }
      }
    }
  }
```

**Step 4: Run tests to verify they pass**

Run: `npx jest __tests__/proxy.test.ts --no-coverage`
Expected: PASS - all existing tests + 7 new tests

**Step 5: Commit**

```bash
git add proxy.ts __tests__/proxy.test.ts lib/dealer-canonical-host-lookup.ts lib/__tests__/dealer-canonical-host-lookup.test.ts
git commit -m "feat: 301 redirect /dealers/ paths to canonical subdomain URLs"
```

---

### Task 3: Fix sitemap to list canonical subdomain URLs

**Files:**
- Modify: `app/sitemap.ts`

**Why:** The sitemap currently lists `https://amsoil.aimclear.com/dealers/hbr` - this tells Google "please index this URL." It should instead list `https://hbr.myamsoil.com/` (or the custom domain if active). This is the primary signal that got Google to index the wrong URLs in the first place.

**Step 1: Update the Prisma query to select domain fields**

In `app/sitemap.ts`, modify the dealer query (lines 61-69) to include `domain`, `domainPrefix`, `customDomain`, and `customDomainStatus`:

```typescript
    const dealers = await prisma.dealer.findMany({
      where: {
        status: 'active',
      },
      select: {
        subdomain: true,
        domain: true,
        domainPrefix: true,
        customDomain: true,
        customDomainStatus: true,
        updatedAt: true,
      },
    });
```

**Step 2: Generate canonical URLs instead of /dealers/ paths**

Replace the dealer URL generation loop (lines 71-79):

```typescript
    for (const dealer of dealers) {
      if (!dealer.subdomain) continue;

      // Use canonical host: custom domain if active, otherwise subdomain URL
      const host =
        dealer.customDomain && dealer.customDomainStatus === 'active'
          ? dealer.customDomain
          : `${dealer.subdomain}.${dealer.domainPrefix}.${dealer.domain}`;

      sitemap.push({
        url: `https://${host}/`,
        lastModified: dealer.updatedAt,
        changeFrequency: 'weekly',
        priority: 0.8,
      });
    }
```

**Step 3: Run the build to verify sitemap compiles**

Run: `npx tsc --noEmit`
Expected: No type errors

**Step 4: Commit**

```bash
git add app/sitemap.ts
git commit -m "fix: sitemap lists canonical subdomain URLs instead of /dealers/ paths"
```

---

### Task 4: Update robots.txt to disallow /dealers/ paths

**Files:**
- Modify: `app/robots.ts`

**Why:** Belt and suspenders. Now that we're redirecting `/dealers/` paths, we should also tell well-behaved crawlers not to index them. The 301 redirect is the primary signal, but `Disallow: /dealers/` in robots.txt prevents future crawl budget waste.

**Step 1: Add /dealers/ to the disallow list**

In `app/robots.ts`, add `/dealers/` to the `disallow` array for the production rules.

**Step 2: Verify robots.ts compiles**

Run: `npx tsc --noEmit`

**Step 3: Commit**

```bash
git add app/robots.ts
git commit -m "fix: disallow /dealers/ paths in robots.txt (now redirected to subdomains)"
```

---

### Task 5: Verify all tests pass

Run: `npm test -- --no-coverage`

Expect all tests to pass including:
- `components/__tests__/DealerTemplate.test.tsx` (63 tests - includes WebSite schema)
- `lib/__tests__/dealer-canonical-host-lookup.test.ts` (8 tests - new)
- `__tests__/proxy.test.ts` (existing + 7 new redirect tests)

**Commit the WebSite schema changes from Task 0:**

```bash
git add components/DealerTemplate.tsx components/__tests__/DealerTemplate.test.tsx
git commit -m "feat: add WebSite schema node for Google site name disambiguation"
```

---

## Post-Deploy Checklist

After deploying to staging/production:

1. **Verify redirect:** `curl -I https://amsoil.aimclear.com/dealers/hbr` should return 301 → `https://hbr.myamsoil.com/`
2. **Verify kill switch:** `curl -I "https://amsoil.aimclear.com/dealers/hbr?noredirect"` should return 200
3. **Verify sitemap:** `curl https://amsoil.aimclear.com/sitemap.xml` should list `https://hbr.myamsoil.com/` not `/dealers/hbr`
4. **Verify subdomain still works:** `curl -I https://hbr.myamsoil.com/` should return 200 (no redirect loop)
5. **Verify JSON-LD:** Check `https://hbr.myamsoil.com/` page source for `"@type": "WebSite"` node with dealer name
6. **Request reindex:** In Google Search Console, use URL Inspection to request reindex of affected dealer pages
7. **Monitor:** Check Search Console over next 2-4 weeks for `/dealers/` URLs dropping out of the index