When you [add a domain to Cloudflare](https://developers.cloudflare.com/fundamentals/setup/manage-domains/add-site/), Cloudflare adds a `/cdn-cgi/` endpoint (`www.example.com/cdn-cgi/`) to that domain.

This endpoint is managed and served by Cloudflare. It cannot be modified or customized. The endpoint is not used by every Cloudflare product, but you may find some products use the endpoint in its URL.

A few examples include (but are not limited to):

* [Identify the Cloudflare data center serving your request](https://developers.cloudflare.com/support/troubleshooting/general-troubleshooting/gathering-information-for-troubleshooting-sites/#identify-the-cloudflare-data-center-serving-your-request), which is helpful for troubleshooting (`https://<YOUR_DOMAIN>/cdn-cgi/trace`).
* [JavaScript detection](https://developers.cloudflare.com/bots/additional-configurations/javascript-detections/) used by Cloudflare bot products (`example.com/cdn-cgi/challenge-platform/`)
* [Image transformations](https://developers.cloudflare.com/images/transform-images) in the new URLs you would use for images (`example.com/cdn-cgi/image/`)
* [Email address obfuscation](https://developers.cloudflare.com/waf/tools/scrape-shield/email-address-obfuscation/) used to hide email addresses from malicious bots (`example.com/cdn-cgi/l/email-protection`)
* [Web analytics](https://developers.cloudflare.com/web-analytics/get-started/#sites-proxied-through-cloudflare) for a website proxied through Cloudflare (`example.com/cdn-cgi/rum`). This endpoint returns a `204` HTTP status code.
* [Speed Brain](https://developers.cloudflare.com/speed/optimization/content/speed-brain/) adds an HTTP header called `Speculation-Rules` to web page responses. This header contains a URL that hosts an opinionated Speculation-Rules configuration, which instructs the browser to initiate prefetch requests for anticipated future navigations.

## Recommended exclusions

### Exclude from security scanners

Some scanners may display an error because certain `/cdn-cgi/` endpoints do not have an [HSTS setting](https://developers.cloudflare.com/ssl/edge-certificates/additional-options/http-strict-transport-security/) applied to it or for similar reasons. Because the endpoint is managed by Cloudflare, you can ignore the error and do not need to worry about it.

To prevent scanner errors, omit the `/cdn-cgi/` endpoint from your security scans.

### Disallow using robots.txt

`/cdn-cgi/` also can cause issues with various web crawlers.

Search engine crawlers can encounter [errors when crawling these endpoints](https://developers.cloudflare.com/support/troubleshooting/general-troubleshooting/troubleshooting-crawl-errors/) and — though these errors do not impact site rankings — they may surface in your webmaster dashboard.

SEO and other web crawlers may also mistakenly crawl these endpoints, thinking that they are part of your site's content.

As a best practice, update your `robots.txt` file to include `Disallow: /cdn-cgi/`.
