Connect your knowledge, keep it in sync

Add a URL, Confluence space, SharePoint site, or Notion workspace. BeforeQuery discovers every page, extracts clean content, and automatically re-indexes when content changes. No code, no webhooks, no manual uploads.

Start Crawling FreeQuickstart Guide

How the knowledge crawler works

Built for the real-world complexity of enterprise knowledge sources

Sitemap Discovery

BeforeQuery fetches sitemap.xml first, then follows links to discover every page the crawler should index. No manual URL lists required.

JavaScript Rendering

Uses a headless browser to render client-side content before extraction. SPAs, Next.js sites, and dynamically rendered intranet pages all index accurately.

Clean Markdown Extraction

Strips nav, footers, ads, and boilerplate. Preserves code blocks, tables, and heading hierarchy — the signal that matters for answer-grounding retrieval.

Incremental Sync

Every source type re-syncs on its own cadence, from 5 minutes to 24 hours, and git repos reindex on every push. Only pages that changed are re-embedded, keeping costs and latency low.

Incremental sync

Your index stays current automatically

Stale knowledge is worse than no knowledge — teams get wrong answers. BeforeQuery polls your sources on your chosen schedule and re-indexes only the pages that changed, keeping every answer grounded in the latest content.

Sync frequency
Hourly, daily, or on-demand
Change detection
ETag + content hash
Webhook trigger
POST /knowledge-bases/:id/sync
Crawl depth
Configurable, default 10 hops
CRAWL STATUS
Pages indexed2,847 / 3,100
2,801
Indexed
46
Pending
12
Updated
Last sync: 4 minutes ago

Works with every knowledge platform

If it renders in a browser or exposes an API, BeforeQuery can index it

Confluence
SharePoint
Notion
Google Drive
Docusaurus
Mintlify
GitBook
GitHub
Zendesk
Salesforce
What teams say

Deployed in production.

“Deployed on our docs site in an afternoon. Every answer shows the source, and the abstention gate means we've never had a customer complain about a made-up answer.”
Priya Nair
Head of Customer Support · Supabase
“The knowledge base connected to our Slack, Confluence, and helpdesk in one setup. On-call teams get the same grounded answer whether they ask in chat, in the widget, or from Cursor.”
Tom Richter
IT Operations Manager · Grafana Labs
“The gap analytics turned into a real docs backlog. Deflection went up because we finally knew which pages were missing — the AI told us.”
Ana Castillo
VP of Customer Experience · Clerk

Frequently Asked Questions

Common questions about BeforeQuery knowledge source connectors

Yes. BeforeQuery respects robots.txt directives by default. You can also configure a custom crawl rate and explicit allow/deny URL patterns in your knowledge base settings.
Yes. Pro and Enterprise plans support authenticated crawling via session cookies or custom HTTP headers. For complex auth flows, use the API to upload content directly.
BeforeQuery renders JavaScript before extracting content. SPAs, Next.js sites, Confluence cloud, and any other JS-rendered platform are fully supported.
Most knowledge sources of 100–500 pages complete in under 2 minutes. Larger sources (1,000+ pages) typically finish within 10–15 minutes.
Yes. Configure URL patterns to exclude (e.g., /changelog/*, /blog/*) in knowledge base settings. You can also provide an explicit allow-list of URL prefixes to crawl.

Index your first knowledge source in minutes

Connect a source, click sync, and your knowledge base is ready.

Get Started Free