How can you check whether your website blocks ChatGPT, Claude or Perplexity?

Which kind of access are you testing?

A checker that asks whether “ChatGPT is allowed” has already compressed several decisions. Search discovery, training collection and a person asking an assistant to open a URL can follow different policies. Use the product token in the report.

The checked guidance supplies the following starting map. OpenAI says robots rules may not apply to ChatGPT-User; Perplexity says its user fetcher generally ignores them. Anthropic says its bots honor robots directives, including its user fetcher. OpenAI, Anthropic and Perplexity document the individual purposes.

What does the public host actually serve?

Use the final public URL, including its scheme and hostname. The policy on https://example.com does not cover a separate www or help hostname just because the pages share a logo. Google’s robots specification describes the host, protocol and port scope.

Save GET responses for the policy and the page. These commands retain headers and bodies even when the server returns an error, and print the final address after redirects. Run them from an empty working folder and replace both example URLs.

curl --connect-timeout 10 --max-time 30 --silent --show-error --location \
  --dump-header robots.headers --output robots.body \
  --write-out 'final_url=%{url_effective} status=%{http_code}\n' \
  https://example.com/robots.txt

curl --silent --show-error --location \
  --dump-header page.headers --output page.html \
  --write-out 'final_url=%{url_effective} status=%{http_code}\n' \
  https://example.com/services/

A failed fetch may come from Cloudflare or host controls. Google’s Search AI controls govern a separate decision from Google-Extended.

Does the matching group allow the intended path?

Feed the saved policy into a parser that accepts a product token and full URL. Keep the parser name and version. Google publishes a robots.txt parser and matcher with a single-URL command; its documented build instructions provide one reproducible option, although another provider’s implementation may differ.

Test every selected search token against both URLs. Then test the training token against a public article if your policy declines training collection. Check grouping before changing a line: the wildcard group is a fallback, and rules from repeated groups for the same product token combine. RFC 9309.

Did the request receive the article?

Open page.html as text and look for the page’s main heading and a distinctive sentence. A login form, challenge screen, empty application shell or error message explains why “HTTP 200” is an incomplete success criterion. Save the body that supports your classification.

If a browser displays the missing text after JavaScript runs, record that as a browser observation. It is an additional representation to inspect. The raw HTML and rendered DOM method explains how to capture a browser’s own document response without assuming a separate curl request received the same bytes.

For Cloudflare challenges, cf-mitigated: challenge identifies a Challenge Page response. Cloudflare’s header reference gives the documented signal. Preserve any request ID and time so you can find the corresponding security event.

What evidence comes from the provider itself?

Now inspect edge logs for the chosen path and time window. A request header naming a provider is a lead. Use the trusted client address and the provider’s current verification method before attributing it; OpenAI and Perplexity link product-specific IP feeds from their documentation. Preserve the evidence date instead of silently classifying historical requests against a new list.

Changing curl’s User-Agent tests whether your own request is treated differently when that string changes. It does not move the request to the provider’s network or reproduce its TLS fingerprint, request sequence or credentials. Keep synthetic checks out of the provider-visit count.

The server-log verification guide covers proxy addresses and missing attribution evidence. If there is no attributable request in the window, mark provider delivery unverified. Do not turn silence into either a pass or a block.

What should the final access report say?

A useful report lets the owner choose the next repair. “Policy disallowed” sends the work to robots.txt. “Policy allowed, edge denied” sends it to security rules. “Document delivered, no attributed provider request” calls for an observation window, not another policy rewrite.

I would finish with the exact URL, check time, token, policy result, saved response and attribution status. Put any answer or citation experiment in another column with its own conditions. The crawler policy guide maps the choices; this check tells you which layer needs attention.