Added: Import Entire Websites into Your Knowledge Base

13 Feb 2026
Elizabeth Long
Import Entire Websites into Your Knowledge Base - Image

You can now add a full website to your Knowledge Base using a single root URL. The system automatically discovers pages and lets you control which ones are included using rule-based filtering.

How It Works

We added website-level crawling to the Knowledge Base inside Whippy.

Instead of adding individual URLs one by one, you now:

1. Enter a root domain or starting URL

2. The system automatically discovers pages across that site

3. You apply include and exclude rules to control which URLs are indexed

This update supports structured content ingestion, large documentation libraries, blog archives, help centers, and resource hubs.

Pattern-Based URL Filtering

You can filter discovered pages using path-based rules. This gives you granular control over which content enters your Knowledge Base.

Supported match types:

  • Starts with
  • Ends with
  • Contains
  • Exact match
  • Custom (regex pattern)

Examples

Starts with /docs

Matches:

  • /docs/getting-started
  • /docs/install
  • /docs/api/authentication

Contains help

Matches:

  • /product/help/reset-password
  • /support/help/account-access
  • /help/contact-us

Ends with /archive

Matches:

  • /blog/2022/archive
  • /releases/v1/archive

Exact match /pricing

Matches only:

  • /pricing

Custom rule /blog/.*/draft

Matches:

  • /blog/2024/draft/new-feature
  • /blog/october/draft/release-notes

Important Guidelines

  • Website discovery is limited to 500 total pages per import to prevent over-crawling.
  • Exclude rules always take precedence over include rules.
  • Once published, a website import does not auto-refresh. New pages added to your site are not automatically re-indexed. Manual updates are required. Automatic refresh is planned for a future release.

Why It Matters

This update simplifies Knowledge Base management.

You no longer need to manually add individual URLs. You can import structured website content in bulk and control visibility with filtering rules. This improves:

  • AI response accuracy
  • Knowledge source consistency
  • Content governance
  • Documentation indexing
  • Help center integration

Teams using Whippy for AI messaging, automated replies, and customer support workflows can now connect entire documentation portals or support sites in minutes.

If you are using Whippy’s AI-driven communication tools, this feature makes it easier to maintain an up-to-date, structured Knowledge Base that powers automated conversations.