agents.clients.generic

Module Contents

Classes

GenericHTTPClient

A generic client for interacting with OpenAI-compatible APIs, including vLLM, ms-swift, lmdeploy, Google Gemini etc. This client works with LLM multimodal LLM models and supports both standard and streaming responses. It is designed to be compatible with any API that follows the OpenAI standard. With a GenericDecisionModel, it asks typed questions of a decision model through the TypeSafe-compatible /v1/systemone API.

API

class agents.clients.generic.GenericHTTPClient(model: Union[agents.models.Model, Dict], host: str = '127.0.0.1', port: Optional[int] = 8000, inference_timeout: int = 30, api_key_env: Optional[str] = None, logging_level: str = 'info', ca_cert: Optional[str] = None, **kwargs)

Bases: agents.clients.model_base.ModelClient

A generic client for interacting with OpenAI-compatible APIs, including vLLM, ms-swift, lmdeploy, Google Gemini etc. This client works with LLM multimodal LLM models and supports both standard and streaming responses. It is designed to be compatible with any API that follows the OpenAI standard. With a GenericDecisionModel, it asks typed questions of a decision model through the TypeSafe-compatible /v1/systemone API.

serialize() → Dict

Get api key env variable serialized.

property supports_tool_calls: bool

Generic HTTP client (OpenAI compatible) supports tool calling, except with a decision model, which does not generate text.

Return type:

bool

property supports_decisions: bool

Generic HTTP client answers typed questions with a decision model.

Return type:

bool

property context_length: Optional[int]

The context length, in tokens, the model is served with. Defaults to None. Override in subclasses that can report it.

Return type:

Optional[int]

check_connection() → None

initialize.

Return type:

None

initialize() → None

initialize.

Return type:

None

inference(inference_input: Dict[str, Any]) → Optional[MutableMapping]

inference.

Parameters:

inference_input (dict[str, Any])

Return type:

dict | None

deinitialize()

deinitialize.