Allow Vizzybl to index your site

Vizzybl crawls your website to build the content index behind brand defense and content optimization. When our crawler can't read your pages, indexing returns 0 pages and a "needs attention" warning appears in Brand Protection → Content Index. This is Vizzybl's own crawler — not the Google or Vertex AI bot — so allow it explicitly using the details below.

How to tell if Vizzybl is blocked

Open Brand Protection → Content Index. A domain that couldn't be indexed shows one of two states:

• Blocked — we reached your site but requests were refused. The site is likely blocking our crawler at the firewall or WAF.
• No readable content — we reached your pages but found nothing to index. This is usually thin content or a JavaScript-only page.
• Asks crawlers not to index — your robots.txt tells automated crawlers to stay out, so we didn't crawl. Nothing is broken; change the robots rule if you want us to index the site.

Note: A 0-page result is not always a block. An over-broad rule in your robots.txt can hide almost every URL from every crawler — check that section below before assuming a firewall problem.

When it's a competitor's site

You can't edit a competitor's robots.txt or firewall, so the guidance on this page won't help there. When a competitor's site refuses our crawler but its robots.txt does not ask crawlers to stay away, the warning offers Try another way — an alternative fetch route that reaches some sites our own crawler cannot.

It is not guaranteed, and it usually returns fewer pages, so a result from it is labelled Limited coverage. It is available on Pro and above, and it costs the same credits as a normal index (see below).

If the competitor's robots.txt does ask crawlers not to index the site, we don't offer it. That is a stated preference and we respect it.

What indexing costs

Indexing is metered on the pages we actually fetch, not the scan size you pick:

Pages fetched	Credits
1	1
20	10
100	50
1,000	275

The rate is 0.5 credits per page, dropping to 0.25 per page beyond 100. Three things are always free:

• a run that couldn't fetch anything — a blocked or robots-denied index costs nothing;
• pages we skip because your sitemap says they haven't changed since we last indexed them;
• the failed first attempt when you use Try another way — you're charged once, for the pages that were actually retrieved.

Re-indexing is therefore cheap: a weekly re-index of a 100-page site that changed three pages fetches three pages and costs 2 credits, not 50.

Allow Vizzybl by IP (recommended)

Vizzybl crawls from a single, stable egress IP. Allowlisting that IP in your firewall or WAF is the most reliable option, because our crawler falls back to a standard browser User-Agent for compatibility — so User-Agent matching alone may not catch every request.

Add this IP to your allowlist:

35.202.121.22

Important: Allowlist by IP as your primary rule, and add the User-Agent rule below as a second signal.

Allow the VizzyblBot User-Agent

Our crawler identifies itself on its first request to each page as:

Mozilla/5.0 (compatible; VizzyblBot/1.0; +https://vizzybl.ai/bot)

Allow it in two places:

• robots.txt — add an explicit group for our crawler:

User-agent: VizzyblBot
Allow: /

• WAF / firewall — add an allow (or skip-bot-check) rule that matches the User-Agent VizzyblBot. Managed bot-fighting rules often challenge unfamiliar crawlers, so name us explicitly.

Note: User-Agent allowlisting is best-effort — if a site blocks VizzyblBot, our crawler retries as a browser so indexing still works where it can. The IP rule above is the deterministic one.

Check your robots.txt

Vizzybl honors robots.txt on every request, whatever User-Agent we send. An over-broad rule silently removes your content from indexing.

Warning: A rule like Disallow: /*? blocks every URL that contains a query string. If your pages live on query-parameter URLs — common on booking, search, and catalog sites — this hides them from all crawlers. Add an Allow: for the paths Vizzybl should read, or scope the Disallow more narrowly.

When both a wildcard group (User-agent: *) and a User-agent: VizzyblBot group are present, the VizzyblBot group takes precedence for us — so a specific Allow: / for VizzyblBot overrides a broad wildcard Disallow.

Verify it worked

1. Go to Brand Protection → Content Index and click Index Site to re-run.
2. A healthy run reports the number of pages indexed — not 0.
3. In your server or WAF logs, confirm requests from VizzyblBot (and our egress IP) return 200.

Related

• Cloudflare Setup — track which AI crawlers fetch your content
• Audit scores — how AI bot access affects your Technical score