Non-JS Crawler Support for Product Listings¶

The problem¶

The Zwerfkei Product Listing Page (category pages, search results, special collections) renders its product grid exclusively client-side, through Vue InstantSearch inside a VueJS component. On page load:

1. The server returns HTML for the page shell (header, sidebar structure, breadcrumbs, static content).
2. The browser downloads and executes the Vue application bundle.
3. The Vue component mounts.
4. Only after mounting does Vue InstantSearch fire an asynchronous XHR request to Algolia.
5. Algolia responds; Vue renders the product cards into the DOM.

Any device or crawler that does not complete step 2–5 sees a raw HTML response with no product links at all:

The technique: a server-rendered fallback grid with a JS-driven swap¶

The pattern solves this with progressive enhancement: render a real, minimal product grid server-side, and let JavaScript replace it with the rich Algolia experience once (and only once) real results have actually arrived.

sequenceDiagram
    actor Crawler as Crawler / Browser
    participant Server as Backend (SSR)
    participant DOM as Page DOM
    participant Vue as VueJS component
    participant Algolia as Algolia (InstantSearch)

    Crawler->>Server: GET /category-page
    Server-->>Crawler: HTML incl. server-rendered product grid
(real hrefs + JSON-LD)
    Note over Crawler: Non-JS / non-waiting crawler stops here.
Product links are already in the DOM.

    opt Only for clients that execute JS
        DOM->>Vue: Vue mounts
        Vue->>Algolia: async XHR search request
        Algolia-->>Vue: search results
        Vue->>DOM: render real Algolia hit cards
        Vue->>DOM: detect real hit cards present
        Vue->>DOM: hide server-rendered grid (display: none)
    end

Key design decisions¶

Functional breakdown¶

1. Server-rendered grid¶

• Built from the same underlying dataset used to populate the Algolia index (same category resolution logic, same page size), so the crawlable snapshot matches what a JS-executing visitor eventually sees.
• Renders enough product information to be useful on its own: brand, name, size range, price, and colour variants — not just a bare name + link.
• Deliberately excludes product images to keep the fallback lightweight; images are not required for link discovery or for the structured data fields actually used.
• Every colour variant is its own real, individually crawlable link (?color=...), following the same URL pattern used elsewhere on the PDP for colour-variant links.

2. Visibility swap¶

• The fallback grid is not wrapped in a generic "hide until JS boots" cloak. It is deliberately kept visible until the actual Algolia results are present in the DOM.
• Detection happens by observing DOM mutations on the component's own root element and checking for the presence of real hit-card elements — not by relying on component refs that may not exist yet, since the search-results child component only populates its slot once the search engine has connected and returned a (possibly empty) initial state, which happens asynchronously after the parent component's own mount lifecycle.
• This ordering detail matters: a naive implementation that checks a ref once, on mount, will silently and permanently fail to ever hide the fallback grid's replacement trigger, because the ref doesn't exist yet at that point in time.

3. Pagination¶

• Uses a plain, 1-based ?page=N query-string parameter for the server-rendered grid, decoupled from whatever page-state routing the client-side search library manages internally.
• Each page carries the correct <link rel="canonical"> (self-referential, including ?page=N for pages beyond the first), plus <link rel="prev"> / <link rel="next"> where applicable — omitted on the first/last page respectively.
• A JS-executing user who lands on a deep, indexed ?page=N URL and then hydrates into the client-side experience may see the client-side search state reset to its own paging — an accepted trade-off, since the primary audience for this query parameter is non-JS crawlers, and the fallback grid is superseded the moment real client-side results render anyway.

4. Structured data¶

• Product structured data is nested inside the page's ItemList, one entry per listed item, sourced from the same dataset as the visible grid (not a separately maintained query) — so position numbers and content stay correct across pagination (e.g. page 2 correctly starts at position 33, not 1).
• Only fields that are genuinely visible on the fallback grid are included (name, offer price, currency). Fields for content that isn't rendered on the fallback card (e.g. an image, a long description) are deliberately omitted — adding them would violate the "don't mark up content that isn't visible to readers of the page" principle from Google's structured data policies.

SEO impact¶

Recommended validation once implemented¶

• Crawl the PLP templates with a non-JS user agent (e.g. Screaming Frog, JS rendering disabled) and confirm every listed product resolves to a real, crawlable PDP link, across all paginated pages.
• Confirm the raw HTML response contains no <noscript>-only product links — links must exist in the regular DOM.
• Validate the JSON-LD Product/ItemList output with Google's Rich Results Test.
• Confirm pagination tags (canonical, prev, next) are correct on the first page, an intermediate page, and the last page.
• Confirm the fallback grid is visually hidden (no layout regression, no flash of duplicate/empty content) once real Algolia results render for JS-executing visitors.
• Baseline Search Console index coverage and crawl frequency for the PLP/PDP templates before rollout, to measure impact after deploy.