cohere
cohere is the official Python SDK for Cohere's models: Command for chat and tool use, Embed for embeddings, and Rerank for search result reordering. Its distinguishing feature is multi-cloud reach; the same SDK ships clients for the Cohere platform plus AWS Bedrock and SageMaker, Azure, GCP, and Oracle OCI, so enterprise teams can call Cohere models wherever their compliance rules keep them. The code is generated by Fern from Cohere's API spec, with two client generations (Client and ClientV2) living side by side.
A competent, typed SDK whose real selling points are Rerank and the multi-cloud client variants; if either matters to you, use it. If you are just picking a chat model vendor, weigh the smaller ecosystem and the generated-code contribution model before committing call sites to it.
Use it if
- You use Cohere's Rerank models, which are the practical reason many teams pull this SDK: a two-line quality boost on top of any existing search or RAG retrieval stack
- You need Cohere models through a specific cloud: BedrockClientV2, SageMaker, Azure, and OciClient variants ship in the box, with OCI auth handled down to instance principals
- You are on Cohere's Embed models for multilingual or int8-quantized embeddings and want typed access to embedding_types and input_type options
- You want first-party support for Command chat with tool use, streaming, and structured JSON output without writing raw HTTP against the v2 API
- You want to stay provider-flexible: this SDK speaks only Cohere's API, so if model choice is still open, litellm or direct openai/anthropic SDKs keep your call sites portable
- You expect to fix SDK bugs upstream: the code is Fern-generated and the README states PRs cannot be merged as-is, so fixes wait on regeneration by the Cohere team
- You are copying older tutorials: v1 Client and v2 ClientV2 have different chat signatures (message vs messages, different response shapes), and mixed-era examples are a consistent source of confusion
- Community help matters to you: the ecosystem around this SDK is far smaller than OpenAI's, so expect to lean on official docs rather than Stack Overflow when something misbehaves
Setup reality
pip install cohere is a reasonable dependency tree (httpx, pydantic, tokenizers, fastavro, requests) with no native build pain. Set CO_API_KEY in the environment and cohere.ClientV2() picks it up; hardcoding the key is the only alternative. The real friction is choosing the right client class: Client vs ClientV2, plus per-cloud variants like BedrockClientV2 and OciClient, each with its own auth setup, and the OCI path needs a pip install 'cohere[oci]' extra. Docs examples still mix v1 and v2 shapes, so check which client a snippet assumes before pasting.
Patterns
Create a v2 clientclient-setup
import cohere
co = cohere.ClientV2() # reads CO_API_KEY from the environment
# or explicitly:
co = cohere.ClientV2(api_key="...")Prefer the environment variable; the README recommends exporting CO_API_KEY. ClientV2 is the current generation, and v1 Client responses have a different shape, so do not mix snippets between them.
Basic chat completionchat-basic
import cohere
co = cohere.ClientV2()
response = co.chat(
model="command-r-plus-08-2024",
messages=[{"role": "user", "content": "hello world!"}],
)
print(response.message.content[0].text)v2 takes a messages list like other modern chat APIs; v1 took a single message string. The reply text lives at response.message.content[0].text, not response.text as in v1.
Stream chat tokens as they arrivechat-streaming
import cohere
co = cohere.ClientV2()
stream = co.chat_stream(
model="command-r-plus-08-2024",
messages=[{"role": "user", "content": "hello world!"}],
)
for event in stream:
if event.type == "content-delta":
print(event.delta.message.content.text, end="")The stream yields typed events, not raw text; filter on event.type == "content-delta" for tokens. Other event types mark message start, tool calls, and end, so a bare print of every event produces noise.
Keep conversation history across turnsmulti-turn-conversation
messages = [{"role": "system", "content": "Answer briefly."}]
messages.append({"role": "user", "content": "What is rerank?"})
res = co.chat(model="command-r-plus-08-2024", messages=messages)
messages.append({"role": "assistant",
"content": res.message.content[0].text})The API is stateless: you own the history list and resend it every call. Token costs grow with the transcript, so trim or summarize old turns in long sessions.
Embed documents for vector searchembed-texts
res = co.embed(
model="embed-english-v3.0",
texts=["printer drum replacement", "toner refill steps"],
input_type="search_document",
embedding_types=["float"],
)
vectors = res.embeddings.float_input_type matters: index documents with search_document and embed queries with search_query, or retrieval quality drops. In v2 the vectors are under embeddings.float_ (trailing underscore, since float is reserved).
Rerank search results by relevancererank-results
docs = ["toner smearing fix", "paper jam in tray 2", "drum unit lifespan"]
res = co.rerank(
model="rerank-english-v3.0",
query="why are my prints streaky",
documents=docs,
top_n=2,
)
for r in res.results:
print(r.index, r.relevance_score, docs[r.index])Results carry an index into your original list, not the document text, so keep the list around. This is the cheapest quality upgrade for an existing BM25 or vector search: retrieve wide, rerank narrow.
Let the model call your functionstool-use
tools = [{"type": "function", "function": {
"name": "get_stock",
"description": "Stock level for a part number",
"parameters": {"type": "object",
"properties": {"part": {"type": "string"}},
"required": ["part"]}}}]
res = co.chat(model="command-r-plus-08-2024",
messages=messages, tools=tools)
for tc in res.message.tool_calls or []:
print(tc.function.name, tc.function.arguments)arguments arrives as a JSON string to parse, and tool_calls can be None when the model answers directly. You execute the function yourself and send the result back as a tool-role message.
Force valid JSON outputjson-output
res = co.chat(
model="command-r-plus-08-2024",
messages=[{"role": "user",
"content": "List 3 copier brands as JSON with a brands array"}],
response_format={"type": "json_object"},
)
import json
data = json.loads(res.message.content[0].text)json_object guarantees parseable JSON, not your exact schema; still validate the keys. Prompting for the desired structure alongside the flag is what makes it usable.
Async calls for concurrent requestsasync-client
import asyncio, cohere
async def main():
co = cohere.AsyncClientV2()
res = await co.chat(
model="command-r-plus-08-2024",
messages=[{"role": "user", "content": "hi"}],
)
print(res.message.content[0].text)
asyncio.run(main())AsyncClientV2 mirrors the sync surface with await. Create one client and reuse it across tasks; per-request client creation wastes connection pools.
Call Cohere models on Oracle Cloudoci-client
# pip install 'cohere[oci]'
import cohere
co = cohere.OciClient(
oci_region="us-chicago-1",
oci_compartment_id="ocid1.compartment.oc1...",
) # auth from ~/.oci/config by default
res = co.embed(model="embed-english-v3.0",
texts=["Hello world"], input_type="search_document")OCI on-demand inference supports Embed and Chat but not Generate or Rerank, per the README; those need fine-tuned dedicated endpoints. OCI embed also allows only one embedding type per request.