groq review
groq 1.6.0 is the official Python client for Groq's hosted model API. Its synchronous Groq and asynchronous AsyncGroq clients wrap httpx, accept typed request dictionaries, and return Pydantic response objects for chat, streaming, tool calls, audio transcription, and other documented endpoints. Stainless generates the package, which explains its close resemblance to other generated AI SDKs. The 1.6.0 release changes Groq's CI workflow templates and CODEOWNERS rather than adding an application-facing endpoint, so there is no runtime feature to chase in this version.
groq 1.6.0 installed in 0.7 seconds, occupied 11 MB, and imported in 0.95 seconds in our sandbox, so its local cost is modest. Use the official SDK when Groq-specific typing and error handling are useful; use a provider-neutral layer when Groq is only one route among many.
We installed it
| Install | ✓ · 0.7s | 14 packages on disk · 11 MB |
| Import | ✓ | import groq in 0.95s · pure Python · py.typed · requires Python >=3.10 |
| Known vulns | 0 | (pip-audit) |
Answers from our run
Does groq install cleanly?
Yes. In a fresh container with an empty cache, pip install groq finished in 0.7s, leaving 14 packages and 11 MB on disk. pip-audit reported no known vulnerabilities.
What does groq need to run?
Python >=3.10, and nothing compiled: it is pure Python. In our run import groq succeeded in 0.95s, and the package ships py.typed for type checkers.
groq or openai: which should you use?
openai: Use it when your code already targets an OpenAI-compatible interface and Groq-specific response types add little. groq 1.6.0 installed in 0.7 seconds, occupied 11 MB, and imported in 0.95 seconds in our sandbox, so its local cost is modest.
When should you not use groq?
Groq is one of several providers; LiteLLM may keep routing, fallback, and provider-specific credentials behind one interface
Use it if
- Groq is a primary inference provider and you want its typed Python request and response models
- Your service needs both blocking and async clients with the same method layout
- You need built-in streaming, file upload handling, status-specific exceptions, retries, and raw response access
- You want to swap httpx for the optional aiohttp backend in a high-concurrency async service
- Groq is one of several providers; LiteLLM may keep routing, fallback, and provider-specific credentials behind one interface
- Your code already speaks the OpenAI-compatible Groq endpoint through another maintained client and does not need Groq's generated types
- You need models or endpoints absent from Groq's current catalog; installing the official SDK does not add provider-side capabilities
- You require a strict promise that minor releases never alter static types; the README permits some type-only incompatible changes in minors
- Python 3.9 or older is fixed in production; version 1.6.0 requires Python 3.10 or newer
Setup reality
Our Python 3.12 sandbox installed groq 1.6.0 in 0.7 seconds. The install left 14 packages and 11 MB on disk, with 8 direct dependencies, and import groq completed in 0.95 seconds. It is pure Python, carries Apache-2.0 licensing, includes py.typed, and pip-audit reported no known vulnerabilities. The package is small enough that provider coupling and request behavior matter more than local install cost.
Set GROQ_API_KEY in the process environment or pass api_key when constructing Groq or AsyncGroq. Keep the secret outside source control. The client defaults to Groq's API, while base_url and a custom httpx client can route through a proxy or test server. Model IDs come from the live provider catalog, so validate configured IDs during deployment rather than assuming an example model will stay available forever.
The SDK retries connection failures plus HTTP 408, 409, 429, and 5xx responses 2 times by default. Its default request timeout is 1 minute, and a timeout can therefore take longer than one minute once retries occur. Set explicit retry and timeout budgets for web requests and background jobs. Error subclasses separate authentication, rate limiting, status failures, and connection failures; catch only the failures your application can safely repeat.
AsyncGroq uses httpx unless you install groq[aiohttp] and pass DefaultAioHttpClient. Use an async context manager or close long-lived clients during shutdown so connection pools are released. Streaming response wrappers also require a context manager. Uploaded PathLike files are read asynchronously by the async client, while bytes and filename tuples are accepted by both clients. A response field set to JSON null and an omitted field both appear as None; inspect model_fields_set when that distinction changes behavior.
Patterns
Request one chat completion chat-completion
from groq import Groq
client = Groq()
response = client.chat.completions.create(
model='openai/gpt-oss-20b',
messages=[{'role': 'user', 'content': 'Summarize this incident.'}],
)
print(response.choices[0].message.content)Groq reads GROQ_API_KEY automatically. Confirm the model ID against the provider catalog before deployment.
Call Groq from async code async-completion
import asyncio
from groq import AsyncGroq
async def main():
async with AsyncGroq() as client:
response = await client.chat.completions.create(
model='openai/gpt-oss-20b',
messages=[{'role': 'user', 'content': 'Say hello'}],
)
print(response.choices[0].message.content)
asyncio.run(main())The context manager closes the HTTP connection pool. AsyncGroq otherwise mirrors the synchronous methods.
Print streamed text chunks stream-tokens
stream = client.chat.completions.create(
model='openai/gpt-oss-20b',
messages=[{'role': 'user', 'content': 'Write a short status update'}],
stream=True,
)
for chunk in stream:
text = chunk.choices[0].delta.content
if text is not None:
print(text, end='', flush=True)Some stream events have no content delta, so check for None before joining or printing text.
Read a requested function call call-tool
import json
tools = [{
'type': 'function',
'function': {
'name': 'lookup_order',
'description': 'Look up an order by ID',
'parameters': {
'type': 'object',
'properties': {'order_id': {'type': 'string'}},
'required': ['order_id'],
},
},
}]
response = client.chat.completions.create(
model='openai/gpt-oss-20b',
messages=[{'role': 'user', 'content': 'Check order A42'}],
tools=tools,
)
call = response.choices[0].message.tool_calls[0]
arguments = json.loads(call.function.arguments)tool_calls can be absent when the model answers directly. Function arguments arrive as JSON text and still need application validation.
Upload an audio file for transcription transcribe-audio
from pathlib import Path
transcript = client.audio.transcriptions.create(
model='whisper-large-v3-turbo',
file=Path('meeting.mp3'),
response_format='verbose_json',
)
print(transcript.text)The client accepts a PathLike object and handles the multipart upload. Available audio models remain provider-controlled.
Separate retryable API failures handle-rate-limit
import groq
try:
response = client.chat.completions.create(
model=model,
messages=messages,
)
except groq.RateLimitError:
queue_for_later(messages)
except groq.APIConnectionError as exc:
report_network_error(exc.__cause__)
except groq.APIStatusError as exc:
report_status(exc.status_code, exc.response)The client already retries 429 and connection failures 2 times unless max_retries changes, so another application retry needs its own cap.
Bound retries and network time set-timeout
import httpx
from groq import Groq
client = Groq(
max_retries=1,
timeout=httpx.Timeout(20.0, connect=3.0, read=15.0),
)Version 1.6.0 defaults to a 1-minute timeout and 2 retries. The total wall time can exceed one timeout interval.
Expose the first failure in a test disable-test-retries
response = client.with_options(
max_retries=0,
timeout=2.0,
).chat.completions.create(
model=model,
messages=messages,
)with_options returns a configured client view for that call; it does not mutate the original client.
Read headers before parsing a response inspect-headers
raw = client.chat.completions.with_raw_response.create(
model=model,
messages=messages,
)
print(raw.headers.get('x-request-id'))
completion = raw.parse()with_raw_response returns an APIResponse wrapper. Call parse() to obtain the normal typed completion.
Close a streamed HTTP response reliably stream-raw-response
with client.chat.completions.with_streaming_response.create(
model=model,
messages=messages,
) as response:
print(response.headers.get('x-request-id'))
for line in response.iter_lines():
consume(line)The streaming wrapper requires a context manager so the connection closes even when iteration stops early.
Switch the async transport to aiohttp use-aiohttp
from groq import AsyncGroq, DefaultAioHttpClient
async with AsyncGroq(
http_client=DefaultAioHttpClient(),
) as client:
response = await client.chat.completions.create(
model=model,
messages=messages,
)Install the groq[aiohttp] extra first. aiohttp is optional and is not part of the measured base install.
Tell an omitted field from JSON null distinguish-null
value = response.some_field
if value is None:
if 'some_field' in response.model_fields_set:
handle_explicit_null()
else:
handle_missing_field()Pydantic exposes both omitted values and explicit JSON null as None; model_fields_set records whether the key appeared.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| openai | PyPI | Use it when your code already targets an OpenAI-compatible interface and Groq-specific response types add little |
| anthropic | PyPI | Use it when Claude models and Anthropic's native message features are the actual requirement |
| litellm | PyPI | Use it to route requests across several model providers through one application-facing layer |
More ai / ml guides
openai · mcp · huggingface-hub · scikit-learn · tiktoken · @modelcontextprotocol/sdk · the whole shelf →
How this guide is made: grounded in the library's documentation, release notes, changelog, and issue history, on a fixed rubric — not a hands-on install of every release. The 50 most-downloaded entries are additionally install-verified in clean containers. Corrections: contact the desk.