Technical SEO as public data infrastructure.
Follow a public record from its address to the evidence behind a finding.
Sulayman Bowles · · Updated
Where should I look when a page is visible but downstream data is wrong?
Trace the publication pipeline one boundary at a time: route and HTTP response, browser rendering, semantic extraction, structured record, and exported result. Preserve the input, output, identity, and transformation at each boundary so you can locate where information changed or disappeared. Being visible in a browser does not prove an extractor or search system received the same record. Report unavailable measurements and freshness limits before presenting the downstream output as authoritative.
Evidence: Google JavaScript SEO guidance · Google structured-data introduction · SEC EDGAR data APIs
Read this alongside Build search systems you can verify.
“The page” may mean several records: a route addressed through DNS and routing, the server’s response, a browser’s executed state, extracted fields, or an exported report. Information can change or disappear between them. Treating them as interchangeable hides the failing stage.
Record inputs, outputs, timestamps, identifiers, and owners at every transformation. Then a vague visibility complaint becomes a question you can investigate: was the URL nominated, fetched, rendered, interpreted, or exported incorrectly—and can the claim be traced to its capture?
Scope and assumptions
The data-infrastructure comparison describes operating discipline. It does not make a website equivalent to a regulated filing system or guarantee crawling, indexing, ranking, citation, or traffic.
- Pipeline stages
- 05
- Observed states
- RAW + RENDERED
- Primary artifact
- DATA CONTRACT
Keep each transformation distinct
Nomination starts the inventory. Links, sitemap rows, external references, feeds, redirects, and earlier runs can place an address in scope. Save that origin to explain inclusion and omission. Sitemap nomination alone does not establish an internal path.
Capture requested and resolved addresses, status, headers, body, timing, and failure state. Browser execution produces a related artifact rather than replacing it. Assets, APIs, consent, and runtime capabilities may change that state.
When normalizing directives, links, headings, metadata, and visible claims, retain the originating artifact and rule. Carry them into exports so a review-table row can lead back to its captured input.
- 01URLNormalized address and discovery source
- 02CrawlStatus, headers, source body, timestamp, fetch state
- 03RenderRendered DOM, discovered links, asset and console state
- 04StructureDirectives, metadata, headings, schema, content fields
- 05Evidence exportObservation, derivation, confidence, route, run, artifact
| Stage | Minimum record | Failure question |
|---|---|---|
| URL | Normalized address and discovery source | Was the intended route ever nominated or linked? |
| Crawl | Status, headers, source body, timestamp, fetch state | Did the server return a usable public document? |
| Render | Rendered DOM, discovered links, asset and console state | Did client execution materially change the record? |
| Structure | Directives, metadata, headings, schema, content fields | Do machine-readable claims match visible content? |
| Evidence export | Observation, derivation, confidence, route, run, artifact | Can another reviewer reproduce the finding? |
Measure the source-to-browser difference
Source records transport at a given time: status, headers, redirects, markup, and server-provided content. Use it to diagnose delivery and fields present before execution.
Executed snapshots record the document after scripts and dependent requests in a named environment. Google treats rendering as a separate JavaScript processing stage. Compare additions and removals across links, headings, claims, directives, fields, and errors.
Screenshots do not contain that full comparison. Keep browser version, timeout, console state, failed dependencies, and completeness alongside the executed artifact. An unfinished render must not become a complete-record claim.
Generate typed outputs from visible facts
Structured data serializes selected fields. Dataset can describe formats, measurement dates, and licensing; SoftwareApplication can state operating context and category. Generate these claims from the fields behind visible tables and descriptions.
Typing requires explicit units, enumerations, IDs, and missing-value rules. SEC APIs illustrate the benefit: XBRL facts retain filing periods and units. Websites have no equivalent regulatory status, but their exports still need defined row grain and checkable fields.
Conflicting JSON-LD, tables, CSV, and metadata give consumers competing dates, counts, units, or identities. Fix the shared source and regenerate the outputs rather than maintaining independent patches.
Atlas: compare captured delivery states
Atlas’s open-corpus run quotes-to-scrape-2026-07-16 declares two seeds: static Quotes to Scrape and its JavaScript variant. The JSON manifest retains capture time, bounded direct-HTTPS method, claim limit, and row payload; CSV exposes address-level fields without replacing the manifest.
Both returned 200. Captured static markup held ten quote cards. JavaScript markup held zero source cards and ten embedded runtime records consumed by a client-side loop. Each row retains next-page address, canonical state, confidence, and the reason for its count.
That evidence compares delivery states, not search performance or corpus defects. Seed, time, and method locate the observation; later finding labels remain interpretations. Reviewers can challenge those labels without losing the underlying record.
| Layer | Example field | Allowed conclusion |
|---|---|---|
| Captured source | HTTP status, source body, response time | What the fetch returned at the recorded time |
| Rendered state | DOM delta, discovered paths, browser errors | What changed in the tested browser environment |
| Derived review | Finding label, confidence, affected route | A versioned interpretation of captured fields |
| Artifact | Run ID, CSV row, JSON manifest, capture method | How the reviewer can locate and reproduce the record |
| Gap state | Skipped provider, challenge page, failed render | Coverage is incomplete; no normal health claim follows |
Declare row grain, gaps, and freshness
Define whether a row means requested URL, final response, rendered page, or normalized canonical document before counting. Mixed grains create duplicate totals, bad joins, and misleading percentages. Specify requirements and distinguish not observed, not applicable, blocked, failed, and genuinely empty values.
Do not silently combine a newly captured sitemap and a month-old DOM as current site state. Timestamps, run IDs, model or renderer versions, and source hashes expose their collection windows. Reproducibility means naming the observed web version, not freezing the web.
| Quality dimension | Check | Unsafe shortcut |
|---|---|---|
| Completeness | Required capture fields and explicit gap states | Treating null as proof that a feature is absent |
| Uniqueness | One declared row grain with duplicate-key checks | Counting requested, redirected, and canonical URLs as peers |
| Consistency | Visible content, metadata, schema, and exports agree | Validating each representation in isolation |
| Freshness | Capture time, run ID, hashes, and tool version retained | Combining observations from different collection windows |
| Integrity | Every finding links back to supporting observations | Publishing labels without reproducible source rows |
Deliver evidence an engineer can rerun
A useful audit connects reachable priority pages, inspectable source and DOM, consistent visible and machine-readable fields, and recommendations derived from captures. Developers can then test the same conditions after implementing a repair instead of trusting an issue label.
A clean pipeline supports processing eligibility and review, not guaranteed distribution. It cannot compel indexes, ranking systems, or answer products to use a page. Its practical benefit is less operational ambiguity and more defensible claims about observed site state.
- Nominate important routes through crawlable internal links and a canonical sitemap.
- Capture transport and rendered states separately, with timestamps and failure reasons.
- Generate metadata and structured records from the same visible source fields.
- Keep observations, derived findings, confidence, and measurement gaps in separate columns.
- Ship CSV or JSON artifacts that retain route, run, source, and calculation lineage.
- Rerun the same checks after implementation before declaring the repair complete.
Make the public record inspectable
Connect every finding to discovery, transport, rendering, and extraction evidence that another reviewer can inspect. This improves operation and engineering handoff; crawling, indexation, ranking, and answer-system use remain external outcomes.
Sources
- Google crawling and indexing overview
Verified Jul 16, 2026
- Google JavaScript SEO basics
Verified Jul 16, 2026
- Google structured data introduction
Verified Jul 16, 2026
- Google sitemap construction guidance
Verified Jul 16, 2026
- SEC EDGAR public data APIs
Verified Jul 16, 2026
- Atlas open-corpus demonstration
Verified Jul 16, 2026