PyPIDataupdated 08 Aug 2026

pandas-gbq

Item: pandas-gbq
Rating: 3.8
Author: Keyoor

pandas-gbq is a focused bridge between pandas DataFrames and Google BigQuery. Its read_gbq function runs GoogleSQL or reads a table into local memory, while to_gbq uploads a DataFrame and can create, append to, or replace a table. It handles Google authentication, schema inference, nullable BigQuery data types, optional Storage API downloads, and partitioning or clustering options. The package now lives under packages/pandas-gbq in Google's large google-cloud-python monorepo and is still classified as beta on PyPI.

pandas-gbq · v0.35.1 · 8.0M/wk · ★ 5,365 · pushed 08 Aug 2026 · googleapis/google-cloud-python · docs · reviewed by Keyoor

Verdict

Use pandas-gbq for a narrow DataFrame-to-BigQuery handoff, especially in notebooks and modest ETL steps. Skip it when the data should remain remote or the job needs more than read, upload, and a handful of configuration switches.

API stability3/5read_gbq and to_gbq are long-lived, but the package remains beta, contains deprecated parameters, and continues to add or redirect behavior through the underlying BigQuery clients.

Docs4/5The package README is concise and the dedicated docs cover authentication, reading, writing, schemas, and privacy; advanced behavior often requires following links into BigQuery client documentation.

Maintenance4/5Version 0.35.1 supports current Python releases and the Google monorepo was pushed today, though monorepo-level activity cannot be treated as package-specific activity.

Ecosystem4/5It sits directly between pandas, PyArrow, Google auth, BigQuery, and the optional Storage API, with familiar notebook behavior and Google-maintained dependencies.

Use it if

Your workflow starts or ends with a pandas DataFrame and you want a two-function BigQuery adapter
You need to upload a local DataFrame with inferred or partially overridden BigQuery schema
You run moderate BigQuery query results into local Python analysis and want nullable BigQuery dtypes mapped for pandas
You already use Google Application Default Credentials and prefer a small surface over the full BigQuery client API

Skip it if

The result is large enough that it should stay in BigQuery: read_gbq materializes data locally, so BigQuery DataFrames or Ibis is a better fit for server-side filtering and aggregation
You need full control over query jobs, retries, labels, destinations, extracts, routines, models, or table administration: use google-cloud-bigquery directly instead of forcing its API through configuration dictionaries
You expect a light install: pandas-gbq brings pandas, NumPy, PyArrow, db-dtypes, Google auth, the BigQuery client, psutil, and related Google API packages
You need a formally GA support promise: PyPI still marks 0.35.1 as beta, and package activity is mixed into a very large monorepo whose top-level stars and issue totals say little about this connector

Setup reality

Installation is pure pip but not small, and Python 3.10 or newer is required. Authentication is the first annoyance: local use may open an OAuth browser flow, while production should provide Application Default Credentials or an explicit service-account credential. Then you must align billing project, dataset location, API permissions, and table schema. Fast downloads need the bqstorage extra, the BigQuery Storage API enabled, and additional IAM permission, and they may cost more.

Patterns

Run GoogleSQL into a DataFramerun-query

import pandas_gbq

sql = '''
SELECT name, COUNT(*) AS uses
FROM `bigquery-public-data.usa_names.usa_1910_2013`
GROUP BY name
ORDER BY uses DESC
LIMIT 20
'''
df = pandas_gbq.read_gbq(sql, project_id='billing-project')

project_id is the project billed for the query; fully qualify tables that live in another project.

Read rows directly from a tableread-table

df = pandas_gbq.read_gbq(
    'analytics-project.events.daily',
    project_id='billing-project',
    max_results=10_000,
    progress_bar_type=None,
)

A table ID is accepted directly. max_results caps rows brought into memory, not bytes scanned by a SQL query.

Override pandas dtypesset-result-dtypes

df = pandas_gbq.read_gbq(
    'SELECT user_id, score FROM `project.dataset.scores`',
    project_id='billing-project',
    dtypes={'user_id': 'string', 'score': 'Float64'},
)

Use pandas nullable dtypes when BigQuery columns can contain NULL; plain int and bool dtypes cannot represent missing values.

Download with the BigQuery Storage APIuse-storage-api

# install first: pip install 'pandas-gbq[bqstorage]'
df = pandas_gbq.read_gbq(
    sql,
    project_id='billing-project',
    use_bqstorage_api=True,
)

The Storage API must be enabled and needs bigquery.readsessions.create permission; it can add cost and is ignored when max_results is set.

Pass a BigQuery job configurationdisable-query-cache

df = pandas_gbq.read_gbq(
    sql,
    project_id='billing-project',
    location='EU',
    configuration={'query': {'useQueryCache': False}},
)

configuration uses BigQuery REST job fields, not QueryJobConfig objects, and location must match every dataset referenced.

Estimate a query without executing itdry-run-query

stats = pandas_gbq.read_gbq(
    sql,
    project_id='billing-project',
    location='US',
    dry_run=True,
)
print(stats)

dry_run returns a pandas Series of job statistics rather than a result DataFrame.

Pass explicit service-account credentialsuse-service-account

from google.oauth2 import service_account
import pandas_gbq

credentials = service_account.Credentials.from_service_account_file(
    'service-account.json'
)
df = pandas_gbq.read_gbq(
    sql,
    project_id='billing-project',
    credentials=credentials,
)

Do not commit the key file. On Google Cloud, Application Default Credentials usually avoid long-lived keys entirely.

Upload a DataFrame to a new tableupload-new-table

pandas_gbq.to_gbq(
    dataframe=df,
    destination_table='analytics.daily_scores',
    project_id='data-project',
    location='US',
    if_exists='fail',
)

if_exists defaults to fail. Keep that safe default unless replacing or appending is an explicit data-management decision.

Append with schema overridesappend-with-schema

schema = [
    {'name': 'user_id', 'type': 'STRING'},
    {'name': 'event_time', 'type': 'TIMESTAMP'},
]

pandas_gbq.to_gbq(
    df,
    'analytics.events',
    project_id='data-project',
    if_exists='append',
    table_schema=schema,
    progress_bar=False,
)

Provided fields override inferred types, but the existing table schema still governs whether an append can succeed.

Create a partitioned and clustered tablecreate-partitioned-table

pandas_gbq.to_gbq(
    df,
    'analytics.events_partitioned',
    project_id='data-project',
    if_exists='fail',
    time_partitioning_column='event_time',
    time_partitioning_type='DAY',
    clustering_columns=['customer_id'],
)

These options matter when creating a table; they do not redesign an existing table during append.

Alternatives

Package	Registry	Pick it when
google-cloud-bigquery	PyPI	You need the complete BigQuery job, table, dataset, extract, and load API rather than a DataFrame-first wrapper
bigframes	PyPI	Your data is too large for local pandas and computations should execute in BigQuery
ibis-framework	PyPI	You want a dataframe-like expression layer that can target BigQuery and other analytical backends