Commit Graph

72 Commits (a90f3ffdac8bda5b7ea00a3e69d7883b706fbf33)

Author SHA1 Message Date
Peter Steinberger de63a646d6
docs: refresh shared web search references 8 months ago
Peter Steinberger df4b5d2137
docs: refresh self-hosted web search references 8 months ago
Peter Steinberger 38ed8c355a
docs: refresh perplexity web search references 8 months ago
Peter Steinberger 49d962a82f
docs: refresh brave web search references 8 months ago
Peter Steinberger 0051a86b8f
docs: clarify synthesized web search count behavior 8 months ago
Peter Steinberger e5d03f734a
docs: refresh kimi web search setup 8 months ago
Bruce MacDonald 5ec53fff0c
feat(ollama): add bundled web search provider (#59318)
Merged via squash.

Prepared head SHA: 1ec105f356a8b17ec12cb639485d398de5c21fd2
Co-authored-by: BruceMacD <5853428+BruceMacD@users.noreply.github.com>
Co-authored-by: BruceMacD <5853428+BruceMacD@users.noreply.github.com>
Reviewed-by: @BruceMacD
8 months ago
Vincent Koc 0500b410c5 docs: update config paths for Firecrawl web_fetch and xAI x_search migrations, add Android assistant section, backfill PR numbers 8 months ago
Vincent Koc 5c8d9da749 docs: add SearXNG web search provider page and navigation 8 months ago
Christof Salis 797a70fd95 Codex: add native web search for embedded Pi runs 8 months ago
Peter Steinberger c9f1506d2f docs(xai): clarify x_search onboarding flow 8 months ago
Peter Steinberger 5872f860c9 feat(xai): add plugin-owned x_search onboarding 8 months ago
huntharo 1c9684608a Docs: guide x_search toward exact-post stats lookups 8 months ago
huntharo 38e4b77e60 Tools: add x_search via xAI Responses 8 months ago
Peter Steinberger d44a399ae0
fix: alphabetize web search provider listings 8 months ago
Vincent Koc df3f9bb555 docs(tools): add Exa Search page, align all search provider docs
New page: tools/exa-search.md
- Neural/keyword/hybrid search modes with content extraction
- Tool parameters including contents (highlights, text, summary)
- Search mode reference table

Rewritten: tools/duckduckgo-search.md
- Aligned to consistent template (Setup, Config, Tool parameters, Notes, Related)
- Simplified from previous version

Aligned across all providers:
- Every search page now ends with a consistent ## Related section
- Replaced 'See [Web tools]' with proper Related links
- Added Exa + DuckDuckGo to web.md overview CardGroup and comparison table
- Added Exa to docs.json nav and redirects
8 months ago
Vincent Koc 0a8eb7fdf0 docs(tools): add DuckDuckGo Search provider page
New page: tools/duckduckgo-search.md
- Key-free fallback provider, no API key needed
- Clear Warning about unofficial HTML-based integration
- Limitations section covering bot-challenge risk and reliability
- CardGroup showing good-for vs not-recommended-for use cases

Updated: tools/web.md with DuckDuckGo in CardGroup and comparison table
Updated: docs.json nav and redirect
8 months ago
Vincent Koc 196c07cac6 fix(docs): broken FAQ anchor #how-does-openclaw-load-environment-variables
Correct anchor is #env-vars-and-env-loading (matching the actual
heading '## Env vars and .env loading' in help/faq.md).

Fixed in: tools/web.md, tools/perplexity-search.md, perplexity.md
8 months ago
Vincent Koc bb3e565487 docs(tools): restructure web tools IA and rewrite web.md
Navigation restructure:
- "Browser" group -> "Web Browser"
- New "Web Tools" group containing Web Fetch, Web Search, and all
  7 search provider sub-pages
- Other tools (btw, diffs, etc.) stay at top level

New page:
- tools/web-fetch.md: dedicated web_fetch reference with Steps,
  config, Firecrawl fallback, limits

Rewritten page:
- tools/web.md: now "Web Search" -- focused search overview with
  Steps quick-start, CardGroup provider picker, Tabs for key storage,
  provider comparison table, auto-detection, parameters, examples.
  Removed all inline provider setup (lives in sub-pages) and web_fetch
  content (now in dedicated page).

Final sidebar:
  Tools
  ├── Web Browser (browser, login, troubleshooting)
  ├── Web Tools
  │   ├── Web Fetch
  │   ├── Web Search
  │   └── Brave / Firecrawl / Gemini / Grok / Kimi / Perplexity / Tavily
  ├── btw, diffs, exec, ...
8 months ago
Vincent Koc 78399c67e9 docs(tools): restructure web search into nested group with provider sub-pages
Restructure "Web and search" into a "Web Search" nested group
(matching Browser pattern) with provider sub-pages:

New pages:
- tools/gemini-search: Gemini with Google Search grounding setup
- tools/grok-search: Grok/xAI web-grounded search setup
- tools/kimi-search: Kimi/Moonshot web search setup

Existing pages (Brave, Firecrawl, Perplexity, Tavily) already exist.

Changes to tools/web.md (overview page):
- Simplify provider table with links to sub-pages
- Replace inline Gemini section with link to dedicated page
- Replace inline Brave/Perplexity setup with CardGroup linking all 7 providers
- Keep auto-detection, web_search, web_fetch reference sections

Navigation: "Web and search" -> "Web Search" with all 8 pages
(overview + 7 providers alphabetical)
8 months ago
Lakshya Agarwal b36e456b09
feat: add Tavily as a bundled web search plugin with search and extract tools (#49200)
Merged via squash.

Prepared head SHA: ece9226e886004f1e0536dd5de3ddc2946fc118c
Co-authored-by: lakshyaag-tavily <266572148+lakshyaag-tavily@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
8 months ago
Vincent Koc 089c8bc65e docs: Phase 3 IA restructure — move pi to Reference, merge Models groups, move install/node to Install, move prose to Skills, migrate brave-search/perplexity/tts into tools/ 8 months ago
Tak Hoffman 0354d49a82
docs update web search config guidance 8 months ago
Peter Steinberger aa28d1c711 feat: add firecrawl onboarding search plugin 8 months ago
Keelan Fadden-Hopper fc408bba37
Fix incorrect rendering of brave costs in docs (#44989)
Merged via squash.

Prepared head SHA: 8c69de822273938cb57cf73371101fcc6e10ece2
Co-authored-by: keelanfh <19519457+keelanfh@users.noreply.github.com>
Co-authored-by: altaywtf <9790196+altaywtf@users.noreply.github.com>
Reviewed-by: @altaywtf
8 months ago
Josh Avant f0eb67923c
fix(secrets): resolve web tool SecretRefs atomically at runtime 8 months ago
Kesku adec8b28bb
alphabetize web search providers (#40259)
Merged via squash.

Prepared head SHA: be6350e5ae1c0610f813483dca0a5c98813a7f8e
Co-authored-by: kesku <62210496+kesku@users.noreply.github.com>
Co-authored-by: obviyus <22031114+obviyus@users.noreply.github.com>
Reviewed-by: @obviyus
8 months ago
Vincent Koc 55465d86d9 Docs: use placeholder OpenRouter key in web tool docs 8 months ago
Rémi 2970d72554
docs: update Brave Search API docs for Feb 2026 plan restructuring (#40111)
Merged via squash.

Prepared head SHA: c651f07855a2e57bb2a25ee42ec62b4bad5b6d8c
Co-authored-by: remusao <1299873+remusao@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
8 months ago
Ayaan Zaidi 28e46d04e5
fix(web-search): restore OpenRouter compatibility for Perplexity (#39937) (#39937) 8 months ago
Peter Steinberger 6dadfaa18c docs: use alphabetical provider ordering 8 months ago
Peter Steinberger acac7e3132 fix: land Brave llm-context gaps (#33383) (thanks @thirumaleshp) 8 months ago
Vincent Koc f966dde476
tests: fix detect-secrets false positives (#39084)
* Tests: rename gateway status env token fixture

* Tests: allowlist feishu onboarding fixtures

* Tests: allowlist Google Chat private key fixture

* Docs: allowlist Brave API key example

* Tests: allowlist pairing password env fixtures

* Chore: refresh detect-secrets baseline
9 months ago
Vincent Koc 5290d97574 Docs: fix web tools MDX links 9 months ago
Vincent Koc 61273c072c Docs: remove MDX-breaking secret markers 9 months ago
Vincent Koc e4d80ed556
CI: restore main detect-secrets scan (#38438)
* Tests: stabilize detect-secrets fixtures

* Tests: fix rebased detect-secrets false positives

* Docs: keep snippets valid under detect-secrets

* Tests: finalize detect-secrets false-positive fixes

* Tests: reduce detect-secrets false positives

* Tests: keep detect-secrets pragmas inline

* Tests: remediate next detect-secrets batch

* Tests: tighten detect-secrets allowlists

* Tests: stabilize detect-secrets formatter drift
9 months ago
Kesku 230fea1ca6
feat(web-search): switch Perplexity to native Search API (#33822)
* feat: Add Perplexity Search API as web_search provider

* docs fixes

* domain_filter validation

* address comments

* provider-specific options in cache key

* add validation for unsupported date filters

* legacy fields

* unsupported_language guard

* cache key matches the request's precedence order

* conflicting_time_filters guard

* unsupported_country guard

* invalid_date_range guard

* pplx validate for ISO 639-1 format

* docs: add Perplexity Search API changelog entry

* unsupported_domain_filter guard

---------

Co-authored-by: Shadow <hi@shadowing.dev>
9 months ago
Henry Loenwind 75775f2fe6
chore: Updated Brave documentation (#26860)
Merged via squash.

Prepared head SHA: f8fc4bf01e0eacfb01f6ee58eea445680f7eeebd
Co-authored-by: HenryLoenwind <1485873+HenryLoenwind@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
9 months ago
Josh Avant 806803b7ef
feat(secrets): expand SecretRef coverage across user-supplied credentials (#29580)
* feat(secrets): expand secret target coverage and gateway tooling

* docs(secrets): align gateway and CLI secret docs

* chore(protocol): regenerate swift gateway models for secrets methods

* fix(config): restore talk apiKey fallback and stabilize runner test

* ci(windows): reduce test worker count for shard stability

* ci(windows): raise node heap for test shard stability

* test(feishu): make proxy env precedence assertion windows-safe

* fix(gateway): resolve auth password SecretInput refs for clients

* fix(gateway): resolve remote SecretInput credentials for clients

* fix(secrets): skip inactive refs in command snapshot assignments

* fix(secrets): scope gateway.remote refs to effective auth surfaces

* fix(secrets): ignore memory defaults when enabled agents disable search

* fix(secrets): honor Google Chat serviceAccountRef inheritance

* fix(secrets): address tsgo errors in command and gateway collectors

* fix(secrets): avoid auth-store load in providers-only configure

* fix(gateway): defer local password ref resolution by precedence

* fix(secrets): gate telegram webhook secret refs by webhook mode

* fix(secrets): gate slack signing secret refs to http mode

* fix(secrets): skip telegram botToken refs when tokenFile is set

* fix(secrets): gate discord pluralkit refs by enabled flag

* fix(secrets): gate discord voice tts refs by voice enabled

* test(secrets): make runtime fixture modes explicit

* fix(cli): resolve local qr password secret refs

* fix(cli): fail when gateway leaves command refs unresolved

* fix(gateway): fail when local password SecretRef is unresolved

* fix(gateway): fail when required remote SecretRefs are unresolved

* fix(gateway): resolve local password refs only when password can win

* fix(cli): skip local password SecretRef resolution on qr token override

* test(gateway): cast SecretRef fixtures to OpenClawConfig

* test(secrets): activate mode-gated targets in runtime coverage fixture

* fix(cron): support SecretInput webhook tokens safely

* fix(bluebubbles): support SecretInput passwords across config paths

* fix(msteams): make appPassword SecretInput-safe in onboarding/token paths

* fix(bluebubbles): align SecretInput schema helper typing

* fix(cli): clarify secrets.resolve version-skew errors

* refactor(secrets): return structured inactive paths from secrets.resolve

* refactor(gateway): type onboarding secret writes as SecretInput

* chore(protocol): regenerate swift models for secrets.resolve

* feat(secrets): expand extension credential secretref support

* fix(secrets): gate web-search refs by active provider

* fix(onboarding): detect SecretRef credentials in extension status

* fix(onboarding): allow keeping existing ref in secret prompt

* fix(onboarding): resolve gateway password SecretRefs for probe and tui

* fix(onboarding): honor secret-input-mode for local gateway auth

* fix(acp): resolve gateway SecretInput credentials

* fix(secrets): gate gateway.remote refs to remote surfaces

* test(secrets): cover pattern matching and inactive array refs

* docs(secrets): clarify secrets.resolve and remote active surfaces

* fix(bluebubbles): keep existing SecretRef during onboarding

* fix(tests): resolve CI type errors in new SecretRef coverage

* fix(extensions): replace raw fetch with SSRF-guarded fetch

* test(secrets): mark gateway remote targets active in runtime coverage

* test(infra): normalize home-prefix expectation across platforms

* fix(cli): only resolve local qr password refs in password mode

* test(cli): cover local qr token mode with unresolved password ref

* docs(cli): clarify local qr password ref resolution behavior

* refactor(extensions): reuse sdk SecretInput helpers

* fix(wizard): resolve onboarding env-template secrets before plaintext

* fix(cli): surface secrets.resolve diagnostics in memory and qr

* test(secrets): repair post-rebase runtime and fixtures

* fix(gateway): skip remote password ref resolution when token wins

* fix(secrets): treat tailscale remote gateway refs as active

* fix(gateway): allow remote password fallback when token ref is unresolved

* fix(gateway): ignore stale local password refs for none and trusted-proxy

* fix(gateway): skip remote secret ref resolution on local call paths

* test(cli): cover qr remote tailscale secret ref resolution

* fix(secrets): align gateway password active-surface with auth inference

* fix(cli): resolve inferred local gateway password refs in qr

* fix(gateway): prefer resolvable remote password over token ref pre-resolution

* test(gateway): cover none and trusted-proxy stale password refs

* docs(secrets): sync qr and gateway active-surface behavior

* fix: restore stability blockers from pre-release audit

* Secrets: fix collector/runtime precedence contradictions

* docs: align secrets and web credential docs

* fix(rebase): resolve integration regressions after main rebase

* fix(node-host): resolve gateway secret refs for auth

* fix(secrets): harden secretinput runtime readers

* gateway: skip inactive auth secretref resolution

* cli: avoid gateway preflight for inactive secret refs

* extensions: allow unresolved refs in onboarding status

* tests: fix qr-cli module mock hoist ordering

* Security: align audit checks with SecretInput resolution

* Gateway: resolve local-mode remote fallback secret refs

* Node host: avoid resolving inactive password secret refs

* Secrets runtime: mark Slack appToken inactive for HTTP mode

* secrets: keep inactive gateway remote refs non-blocking

* cli: include agent memory secret targets in runtime resolution

* docs(secrets): sync docs with active-surface and web search behavior

* fix(secrets): keep telegram top-level token refs active for blank account tokens

* fix(daemon): resolve gateway password secret refs for probe auth

* fix(secrets): skip IRC NickServ ref resolution when NickServ is disabled

* fix(secrets): align token inheritance and exec timeout defaults

* docs(secrets): clarify active-surface notes in cli docs

* cli: require secrets.resolve gateway capability

* gateway: log auth secret surface diagnostics

* secrets: remove dead provider resolver module

* fix(secrets): restore gateway auth precedence and fallback resolution

* fix(tests): align plugin runtime mock typings

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
9 months ago
Peter Steinberger 085c23ce5a fix(security): block private-network web_search citation redirects 9 months ago
Peter Steinberger 5eb72ab769 fix(security): harden browser SSRF defaults and migrate legacy key 9 months ago
AkosCz 3a3c2da916
[Feature]: Add Gemini (Google Search grounding) as web_search provider (#13075)
* feat: add Gemini (Google Search grounding) as web_search provider

Add Gemini as a fourth web search provider alongside Brave, Perplexity,
and Grok. Uses Gemini's built-in Google Search grounding tool to return
search results with citations.

- Add runGeminiSearch() with Google Search grounding via tools API
- Resolve Gemini's grounding redirect URLs to direct URLs via parallel
  HEAD requests (5s timeout, graceful fallback)
- Add Gemini config block (apiKey, model) with env var fallback
- Default model: gemini-2.5-flash (fast, cheap, grounding-capable)
- Strip API key from error messages for security
- Add config validation tests for Gemini provider
- Update docs/tools/web.md with Gemini provider documentation

Closes #13074

* feat: auto-detect search provider from available API keys

When no explicit provider is configured, resolveSearchProvider now
checks for available API keys in priority order (Brave → Gemini →
Perplexity → Grok) and selects the first provider with a valid key.

- Add auto-detection logic using existing resolve*ApiKey functions
- Export resolveSearchProvider via __testing_provider for tests
- Add 8 tests covering auto-detection, priority order, and explicit override
- Update docs/tools/web.md with auto-detection documentation

* fix: merge __testing exports, downgrade auto-detect log to debug

* fix: use defaultRuntime.log instead of .debug (not in RuntimeEnv type)

* fix: mark gemini apiKey as sensitive in zod schema

* fix: address Greptile review — add externalContent to Gemini payload, add Gemini/Grok entries to schema labels/help, remove dead schema-fields.ts

* fix(web-search): add JSON parse guard for Gemini API responses

Addresses Greptile review comment: add try/catch to handle non-JSON
responses from Gemini API gracefully, preventing runtime errors on
malformed responses.

Note: FIELD_HELP entries for gemini.apiKey and gemini.model were
already present in schema.help.ts, and gemini.apiKey was already
marked as sensitive in zod-schema.agent-runtime.ts (both fixed in
earlier commits).

* fix: use structured readResponseText result in Gemini error path

readResponseText returns { text, truncated, bytesRead }, not a string.
The Gemini error handler was using the result object directly, which
would always be truthy and never fall through to res.statusText.
Align with Perplexity/xAI/Brave error patterns.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix import order and formatting after rebase onto main

* Web search: send Gemini API key via header

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
9 months ago
Peter Steinberger 166cf6a3e0 fix(web_fetch): cap response body before parsing 9 months ago
青雲 89fa93ed75
feat: support freshness parameter for Perplexity web_search provider (#15343)
Merged via /review-pr -> /prepare-pr -> /merge-pr.

Prepared head SHA: 01aba2bfba053c028f62970dcd153b19d00c5e3b
Co-authored-by: echoVic <16428813+echoVic@users.noreply.github.com>
Co-authored-by: sebslight <19554889+sebslight@users.noreply.github.com>
Reviewed-by: @sebslight
9 months ago
Seb Slight 578a6e27aa
Docs: enable markdownlint autofixables except list numbering (#10476)
* docs(markdownlint): enable autofixable rules except list numbering

* docs(zalo): fix malformed bot platform link
9 months ago
Sebastian 0a1f4f666a revert(docs): undo markdownlint autofix churn 9 months ago
Sebastian c7aec0660e docs(markdownlint): enable autofixable rules and normalize links 9 months ago
Sebastian 1bf9f237f7 docs: linting 9 months ago
Peter Steinberger d3ba57b7d7 feat: add configurable web_fetch maxChars cap 10 months ago
Seb Slight abcaa8c7a9
Docs: add nav titles across docs (#5689) 10 months ago