free-claude-code/tests/application/test_web_tools.py
Ali Khokhar 4e99ddaa5c
Some checks are pending
Post merge / Warm dependency cache (Linux) (push) Waiting to run
Post merge / Warm dependency cache (Windows) (push) Waiting to run
Post merge / Warm dependency cache (macOS) (push) Waiting to run
Post merge / Publish to PyPI (push) Waiting to run
minor: select API keys for fallback models (#2058)
## Why

Fallback rows could choose a model and reasoning effort but always used
the provider's first API key. Users need to choose which numbered key
each fallback uses, including different keys for the same model.

## How

Add a Key N selector and persist a validated `api_key_number`,
defaulting existing entries to Key 1. Reject duplicate model/key pairs
and key removals that leave a saved fallback's selection unavailable.

Carry the number through routing and the request's retained runtime
generation. Cache and construct clients per provider/key pair so
concurrent attempts keep their selected credentials.

Share rate and concurrency capacity across a provider's keys, while
recovery belongs to the actual credential. A healthy selected fallback
key receives an attempt after another key exhausts rate-limit retries.
Reordering a key preserves its recovery state, and replacing it uses a
separate state. Keep the existing retry classification, attempt count,
and backoff.

Refresh key choices after provider edits while preserving unsaved model
drafts. Display removed numbers as unavailable and reset the choice to
Key 1 when a row changes provider. Numbers follow the current order in
the provider's settings. Providers with another authentication method
show a disabled default selection.

The Admin editor targets desktop viewports, as requested. Its existing
rows and add row keep the three selections aligned horizontally.

Local CI passes, including 9,049 Python tests and 277 browser tests.
Added regressions capture actual authentication headers after 403 and
exhausted 429 failures across both public APIs and all three custom
endpoint formats, and check shared capacity, client coalescing, retained
generations, credential reorder/replacement, and Admin edits.


<!-- greptile_comment -->

<!-- greptile_summary -->

<h2><a
href="https://app.greptile.com/api/retrigger?id=79105240"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/RetriggerDark.svg?v=2"><source
media="(prefers-color-scheme: light)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/Retrigger.svg?v=2"><img
alt="Retrigger"
src="https://greptile-static-assets.s3.amazonaws.com/badges/Retrigger.svg?v=2"
align="right"></picture></a>Confidence Score: 5/5</h2>

<!-- greptile-impact-verdict --> No outstanding blocking findings were
identified. The reviewed changes are safe to merge.

<details open><summary><h3>Summary</h3></summary>

Adds a numbered API-key choice to each fallback model and carries that
choice through routing and provider construction.

- Fallback rows can choose a numbered API key.
- Fallback attempts send requests with their selected API keys.
- Provider clients keep separate keys while sharing provider limits.
</details>

<!-- greptile_confidence_score:5 -->

<sub>Reviews (3) · Last reviewed commit: ["Keep credential recovery
separate from
p..."](c7c33ac61b)
· [Reviewed by
Greptile](https://www.greptile.com/?utm_source=greptile_expert&utm_medium=github&utm_campaign=code_reviews&utm_content=alishahryar1/free-claude-code)</sub>

<!-- /greptile_comment -->
2026-10-12 10:09:36 -07:00

131 lines
4.5 KiB
Python

"""Application web-tool behavior through its explicit outbound capability."""
import asyncio
from collections.abc import AsyncIterator
import pytest
from free_claude_code.application.errors import InvalidRequestError
from free_claude_code.application.execution import ProviderExecutor
from free_claude_code.application.routing import ModelRouter
from free_claude_code.application.web_tools.ports import WebFetchEgressPolicy
from free_claude_code.application.web_tools.service import WebToolService
from free_claude_code.config.settings import Settings
from free_claude_code.core.anthropic.models import Message, MessagesRequest, Tool
from free_claude_code.core.web_tools import WebFetchResult
from tests.web_tools_support import StubWebToolsClient
async def _no_provider(_provider_id, _api_key_number):
pytest.fail("This request must not resolve a provider")
def _service(settings, client, *, token_counter=lambda *_: 5):
return WebToolService(
settings=settings,
client=client,
executor=ProviderExecutor(_no_provider, progress_timeout_seconds=30),
token_counter=token_counter,
)
def _request(tool, *, typed=True):
return MessagesRequest(
model="claude-sonnet",
messages=[Message(role="user", content="https://example.com/")],
tools=[
Tool(
name=tool,
type=(
"web_fetch_20250910"
if tool == "web_fetch"
else "web_search_20250305"
)
if typed
else None,
)
],
tool_choice={"type": "tool", "name": tool},
)
async def _drain(body: AsyncIterator[str]) -> str:
return "".join([chunk async for chunk in body])
def test_ordinary_client_tool_needs_no_web_or_token_work():
settings = Settings()
service = _service(
settings,
StubWebToolsClient(),
token_counter=lambda *_: pytest.fail("Unhandled request counted tokens"),
)
routed = ModelRouter(settings).resolve_messages_request(
_request("web_search", typed=False)
)
assert service.try_stream_messages(routed, request_id="req_unhandled") is None
def test_disabled_tool_fails_during_preparation():
settings = Settings().model_copy(update={"enable_web_server_tools": False})
service = _service(settings, StubWebToolsClient())
routed = ModelRouter(settings).resolve_messages_request(_request("web_fetch"))
with pytest.raises(InvalidRequestError, match="disabled"):
service.try_stream_messages(routed, request_id="req_disabled")
@pytest.mark.asyncio
async def test_concurrent_requests_keep_their_own_egress_policy():
entered, release = asyncio.Event(), asyncio.Event()
class Client(StubWebToolsClient):
def __init__(self):
self.policies = []
async def fetch(
self, url: str, *, egress: WebFetchEgressPolicy
) -> WebFetchResult:
self.policies.append(egress)
if len(self.policies) == 2:
entered.set()
await release.wait()
return WebFetchResult(
url=url, title="page", media_type="text/plain", data="text"
)
client = Client()
settings = [
Settings().model_copy(
update={
"web_fetch_allow_private_networks": False,
"web_fetch_allowed_schemes": "https",
}
),
Settings().model_copy(
update={
"web_fetch_allow_private_networks": True,
"web_fetch_allowed_schemes": "http, https",
}
),
]
tasks = []
try:
for index, snapshot in enumerate(settings):
service = _service(snapshot, client)
body = service.try_stream_messages(
ModelRouter(snapshot).resolve_messages_request(_request("web_fetch")),
request_id=f"req_{index}",
)
assert body is not None
tasks.append(asyncio.create_task(_drain(body)))
await asyncio.wait_for(entered.wait(), 1)
assert client.policies == [
WebFetchEgressPolicy(False, frozenset({"https"})),
WebFetchEgressPolicy(True, frozenset({"http", "https"})),
]
release.set()
outputs = await asyncio.gather(*tasks)
assert all("web_fetch_tool_result" in output for output in outputs)
finally:
release.set()
await asyncio.gather(*tasks, return_exceptions=True)