Web Researcher MCP

Web Researcher MCP

Web Researcher MCP is a local, production-grade MCP server that lets AI assistants search the web (optionally restricted to trusted “search lenses”), read full sources (including JS-rendered pages, PDFs, and YouTube transcripts), and return verifiable citations with multi-provider failover. [Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt]
https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
Web Researcher MCP

Product Information

Updated:Jun 16, 2026

What is Web Researcher MCP

Web Researcher MCP is an open-source Model Context Protocol (MCP) server designed to give AI assistants (e.g., Claude, Cursor, and other MCP-compatible clients) reliable, source-grounded web research capabilities. It focuses on “real sources you can trust” by letting you search either the entire web or only specific, curated domains, then extract and analyze full content rather than snippets—producing working links and formatted citations. The project ships as a single Go binary with multiple installation options (e.g., Homebrew, install scripts, Docker, or Python-friendly uvx) and can run in local STDIO mode or as an HTTP service for shared/team setups. [Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt]

Key Features of Web Researcher MCP

Web Researcher MCP is a production-grade MCP server that plugs AI assistants (e.g., Claude, Cursor, VS Code) into live, source-grounded web research. It can search the web (with optional “search lenses” that restrict results to trusted domains), read and extract full content from URLs (including JavaScript-heavy pages via an automated browser pipeline), and handle many research-specific workflows such as academic and patent lookup, news search, citation verification/auditing, and exporting reproducible research sessions with provenance. It is designed to prioritize verifiable sources and reliable citations, supports multiple interchangeable search providers with automatic failover, and runs locally for privacy-focused research.
Search lenses (trusted-source routing): Restrict searches to curated domain lists (built-in or custom) so results come only from sources you trust (e.g., PubMed, SEC, courts, official docs) rather than the open web’s spam and low-quality content.
Multi-provider search with failover: Works with multiple search providers (e.g., DuckDuckGo zero-config, Google PSE, Brave, Serper, SearXNG, Tavily, Exa) and can automatically fall back when one provider is rate-limited or unavailable.
Full-page extraction (incl. JS-rendered sites): Reads complete webpages and documents—not just snippets—using a tiered scraping pipeline and optional Chromium rendering for JavaScript-heavy pages; also supports PDFs/Office docs and YouTube transcript extraction.
Research-grade citation workflows: Provides tools to verify citations, audit entire bibliographies for dead links/retractions/unverifiable entries, and format bibliographies (APA/MLA/BibTeX/RIS/CSL-JSON) for downstream academic or professional use.
Deep, reproducible research sessions: Supports sequential multi-step research with session persistence and export (markdown/JSON) including per-step provenance; includes diagnostics and status resources for tools, sessions, providers, and rate limits.
Specialized vertical search tools: Includes purpose-built tools for academic papers (DOIs, authors, citations), patents (USPTO/EPO/Lens with fallbacks), SEC filings (EDGAR), legal opinions/dockets (CourtListener), economics (World Bank/FRED), and clinical trials (ClinicalTrials.gov).

Use Cases of Web Researcher MCP

Academic & R&D literature review: Researchers can run structured literature searches, collect DOI-backed sources, traverse citation neighborhoods, and export properly formatted bibliographies for papers, grants, or internal technical reviews.
Legal research & litigation support: Law firms and compliance teams can search real court opinions/dockets, verify case citations, and archive sources to ensure references remain verifiable for filings and client deliverables.
Finance & due diligence: Analysts can pull SEC EDGAR filings, triangulate news and primary sources, and produce auditable research reports with links that clients can click and verify.
Healthcare & clinical intelligence: Medical researchers and policy teams can focus searches on clinical and biomedical sources, retrieve full-text evidence where available, and track clinical trial registrations and statuses (discovery-oriented, not medical advice).
Competitive and market intelligence: Product and strategy teams can monitor news, scan patents, and synthesize multi-source competitive analysis while keeping a reproducible paper trail of how conclusions were reached.
Engineering documentation and incident research: Developers and SRE teams can use documentation-focused lenses to search official references, scrape full pages (including JS-heavy docs), and compile grounded summaries for troubleshooting and design decisions.

Pros

Source control via lenses: You can constrain research to vetted domains, improving reliability and reducing spam/SEO noise.
End-to-end research tooling: Combines search, full-content extraction, citation verification/auditing, and session export with provenance in one MCP server.
Provider flexibility and resilience: Multiple search backends with optional automatic failover; also offers a zero-config DuckDuckGo fallback.
Privacy-forward local execution: Queries go from your machine to your chosen providers rather than through the tool’s own servers.

Cons

Best capabilities often require API keys: Higher-quality search, news/images, and some advanced tools depend on configuring third-party providers and credentials.
Heavier scraping may download/run Chromium: JavaScript rendering can require a large browser download (~200MB) and may be slower than simple extraction.
Upstream limits and variability: Rate limits, coverage, and freshness depend on the selected search provider(s) and their free/paid tiers.

How to Use Web Researcher MCP

1) Install Web Researcher MCP (fastest: uvx): Install uv (one-time), then run the MCP server via uvx. macOS/Linux: - curl -LsSf https://astral.sh/uv/install.sh | sh Then: - claude mcp add --scope user web-researcher -- uvx web-researcher-mcp This fetches the correct prebuilt binary for your OS and runs it (no Go toolchain needed). Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
2) Alternative installs (choose one): macOS (Homebrew): - brew install zoharbabin/tap/web-researcher-mcp - claude mcp add --scope user web-researcher -- web-researcher-mcp macOS/Linux (curl installer): - curl -fsSL https://raw.githubusercontent.com/zoharbabin/web-researcher-mcp/main/install.sh | sh Windows (PowerShell installer): - powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/zoharbabin/web-researcher-mcp/main/install.ps1 | iex" Other options: - winget install zoharbabin.web-researcher-mcp - scoop bucket add zoharbabin https://github.com/zoharbabin/scoop-bucket && scoop install web-researcher-mcp - brew install --cask zoharbabin/tap/web-researcher-mcp - go install github.com/zoharbabin/web-researcher-mcp/cmd/web-researcher-mcp@latest - docker run -i --rm -e GOOGLE_CUSTOM_SEARCH_API_KEY=YOUR_KEY -e GOOGLE_CUSTOM_SEARCH_ID=YOUR_CX docker.io/zoharbabin/web-researcher-mcp:latest Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
3) Connect it to your MCP client (generic JSON config): If your app isn’t Claude Code’s CLI flow, add an MCP server entry to your client config. Example (Google PSE): { "mcpServers": { "web-researcher": { "command": "web-researcher-mcp", "env": { "SEARCH_PROVIDER": "google", "GOOGLE_CUSTOM_SEARCH_API_KEY": "YOUR_GOOGLE_API_KEY", "GOOGLE_CUSTOM_SEARCH_ID": "YOUR_SEARCH_ENGINE_ID" } } } } Example (Brave): { "mcpServers": { "web-researcher": { "command": "web-researcher-mcp", "env": { "SEARCH_PROVIDER": "brave", "BRAVE_API_KEY": "YOUR_BRAVE_API_KEY" } } } } Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
4) Run with zero config (DuckDuckGo fallback): If you don’t set SEARCH_PROVIDER or any API keys, Web Researcher MCP still works out of the box using DuckDuckGo as the built-in fallback (no API key required). This is useful to validate your setup before adding provider keys. Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
5) (Recommended) Add a search provider key to improve quality and unlock more features: Set SEARCH_PROVIDER and the matching key(s). Supported providers include DuckDuckGo (no key), Google PSE, Brave, Serper, SearchAPI.io, SearXNG, Tavily, Exa, and Hacker News. Example env vars: - SEARCH_PROVIDER=brave - BRAVE_API_KEY=... Or Google PSE: - SEARCH_PROVIDER=google - GOOGLE_CUSTOM_SEARCH_API_KEY=... - GOOGLE_CUSTOM_SEARCH_ID=... Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
6) (Optional) Configure multi-provider failover: To avoid downtime/rate limits, configure multiple providers so the server automatically falls back. Example: - SEARCH_ROUTING=brave,google,serper - BRAVE_API_KEY=... - GOOGLE_CUSTOM_SEARCH_API_KEY=... - GOOGLE_CUSTOM_SEARCH_ID=... - SERPER_API_KEY=... Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
7) Use “search lenses” to restrict results to trusted sources: When you want higher trust and less spam, use a built-in lens (e.g., medical, legal, academic, news) so searches are restricted to curated domains. Example prompt to your AI client: - “Search for recent findings on SGLT2 inhibitors using the clinical lens.” You can also create custom lenses by adding a JSON file under lenses/ with a domain allowlist. Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
8) Ask your AI to use the core tools (common workflows): Once connected, your AI can call tools such as: - web_search: search the web (optionally with lenses) - scrape_page: read a URL in full (web pages, PDFs, DOCX, PPTX, YouTube transcripts) - search_and_scrape: search then automatically read top results - news_search / image_search: when supported by your provider - academic_search / patent_search / legal_search / filing_search / econ_search / clinical_search: specialized databases - verify_citation / audit_bibliography / format_bibliography: citation workflows - sequential_search: multi-step deep research Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
9) Use guided prompt templates for repeatable research: Use the built-in prompt templates (where your client supports prompt selection) to run structured workflows: - comprehensive-research - fact-check - competitive-analysis - literature-review These templates guide multi-step research so you don’t have to specify every instruction. Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
10) Troubleshoot common issues (quick fixes): If tools fail with “API key” errors: set keys in your MCP client config env block (not only in shell profiles). If pages come back empty: JavaScript-heavy sites require Chromium; the binary auto-downloads Chromium on first use (~200MB). You can set CHROME_PATH to an existing Chrome install. Docker image includes Chromium. If cache is stale after upgrade: delete the cache directory (e.g., ~/Library/Caches/web-researcher-mcp/ on macOS) or set CACHE_DIR. If you hit provider rate limits (429): switch providers or configure SEARCH_ROUTING for failover. Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt
11) (Teams) Run in HTTP mode and connect clients to a shared endpoint: For shared/team setups, run the server as an HTTP service and connect MCP clients to http://localhost:3000/mcp/. Example: - PORT=3000 OAUTH_ISSUER_URL=https://auth.example.com OAUTH_AUDIENCE=https://api.example.com ./web-researcher-mcp Docker Compose example: services: web-researcher: image: zoharbabin/web-researcher-mcp ports: - "3000:3000" environment: PORT: "3000" SEARCH_PROVIDER: brave BRAVE_API_KEY: ${BRAVE_API_KEY} Source: https://github.com/zoharbabin/web-researcher-mcp?ref=producthunt

Web Researcher MCP FAQs

Web Researcher MCP is an open-source Model Context Protocol (MCP) server that gives AI assistants web research capabilities (web search, full-page extraction, and specialized searches like academic/patent/legal), with an emphasis on grounded answers and real, verifiable citations. It runs locally on your machine rather than as a hosted service.

Latest AI Tools Similar to Web Researcher MCP

Jorpex
Jorpex
Jorpex is a comprehensive tender notification platform that aggregates and delivers instant tender alerts from across European countries directly to Slack, helping businesses never miss opportunities.
Leadsmrt
Leadsmrt
Leadsmrt is a lead generation tool that helps businesses scrape, verify and personalize targeted business leads from Google Maps with AI-powered personalization capabilities.
Omnial AI
Omnial AI
Omnial AI is a data intelligence platform that leverages AI agents to transform web prompts into structured, actionable data insights backed by Afore Capital.
SERPrecon
SERPrecon
SERPrecon is an advanced SEO tool that leverages vectors, machine learning, and natural language processing to help users analyze and outrank competitors by using the same methods as modern search engines.