> ## Documentation Index
> Fetch the complete documentation index at: https://elementary-devin-1782754750-bigquery-permissions-docs.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect to Bigquery

This guide contains the necessary steps to connect a BigQuery environment to your Elementary account.

## Choose an authentication method

Elementary supports two authentication methods for BigQuery. Pick the one that fits your security model:

* **Service account** — create a service account, download its JSON key, and upload the key to Elementary. Simplest to set up.
* **Workload Identity Federation (WIF)** — Elementary authenticates from its AWS role through a federated identity. No long-lived credentials are stored in Elementary.

Select a tab below and follow the steps for your chosen method.

<Tabs>
  <Tab title="Service account">
    ### Create service account:

    1. In the Cloud Console, go to: IAM & Admin > Service Accounts

    2. Click on 'CREATE SERVICE ACCOUNT'

    <img src="https://res.cloudinary.com/mintlify/image/upload/v1659304875/elementary/create-service-account_birda0.png" alt="Create service account" />

    3. Fill in the service account name ('elementary') and account description ('Elementary Data') and click 'CREATE AND
       CONTINUE':

    <img src="https://res.cloudinary.com/mintlify/image/upload/v1659304876/elementary/create-continue_ai6ta9.png" alt="Create and Continue" />

    4. Now we need to configure the relevant permissions for this new service account.
       Select the following role: BigQuery Job User (you will need to grant read access to specific datasets later).

    5. The last step is optional, skip it and press done.

    6) Press on the dots icon to the right of your screen for your new service account and select 'Manage keys':

    <img src="https://res.cloudinary.com/mintlify/image/upload/v1659304878/elementary/dots-icon_aafqkv.png" alt="Manage keys" />

    7. Press on 'ADD KEY' and select 'Create new key':

    <img src="https://res.cloudinary.com/mintlify/image/upload/v1659304875/elementary/add-key_v4hztr.png" alt="Add Key" />

    8. Use the 'JSON' option radio button and press 'CREATE':

    <img src="https://res.cloudinary.com/mintlify/image/upload/v1659304877/elementary/json_mybzda.png" alt="JSON" />

    9. This will automatically generate and download a JSON file with your private key information for this service account.
       This JSON file provides the credentials to programmatically connect and work with your BigQuery environment.

    ### Grant service user access

    In order for a service user to work with Elementary Cloud, it requires the following permissions:

    * Role **"BigQuery Data Viewer"** on your Elementary dataset.
    * Roles **"BigQuery Metadata Viewer"** and **"BigQuery Resource Viewer"** at the **project level** for the project containing your dbt models and any external projects it references.

    <Note>
      "BigQuery Metadata Viewer" and "BigQuery Resource Viewer" **must be granted at the project level** (via IAM & Admin), not at the dataset level. These roles include permissions such as `bigquery.jobs.listAll` which are required for reading query history and metadata from `INFORMATION_SCHEMA` — they cannot be granted on individual datasets.
    </Note>

    #### Step 1: Grant project-level roles

    These roles are required for Elementary to read query history (`INFORMATION_SCHEMA.JOBS`) and table metadata. Without them, BI integrations (such as Hex) that rely on query history will not function.

    1. Go to [IAM & Admin](https://console.cloud.google.com/iam-admin/iam) in the Google Cloud Console.

    2. Select the project that contains your dbt models.

    3. Click **"Grant Access"** at the top.

    4. Fill out the form:
       * In the "New principals" field, enter the **email address** of the Elementary service account.
       * Add the role **"BigQuery Metadata Viewer"** (`roles/bigquery.metadataViewer`).
       * Click **"Add another role"** and add **"BigQuery Resource Viewer"** (`roles/bigquery.resourceViewer`).
       * Click **"Save"**.

    5. **Repeat for any additional GCP projects** referenced in your dbt sources or used by BI tools (e.g., Hex, Looker) for query execution.

    #### Step 2: Grant dataset-level role on the Elementary dataset

    This role provides read access to your Elementary schema data.

    1. Go to your project in [BigQuery console](https://console.cloud.google.com/bigquery).

    2. In the "Explorer" tab, find your Elementary dataset.

    3. Click on the three dots icon next to the dataset name, then Share.

    <img src="https://res.cloudinary.com/diuctyblm/image/upload/f_auto,q_auto/v1/dwh/bigquery/dataset_share" alt="Dataset share" />

    4. Click the "ADD PRINCIPAL" button on the top right corner.

    <img src="https://res.cloudinary.com/diuctyblm/image/upload/f_auto,q_auto/v1/dwh/bigquery/add_principal_button" alt="Add principal" />

    5. Fill out the form:
       * In the "New principals" textbox, write the **email address** of your service account.
       * In the "Select a role" dropdown menu, choose **"BigQuery Data Viewer"**.
       * Click "Save".

    <img src="https://res.cloudinary.com/diuctyblm/image/upload/f_auto,q_auto/v1/dwh/bigquery/grant_access_form" alt="Grant access" />
  </Tab>

  <Tab title="Workload Identity Federation">
    ### Create a Workload Identity pool and provider:

    1. In the Cloud Console, go to: IAM & Admin > Workload Identity Federation. Click 'CREATE POOL' (or select an existing pool), give it a name (e.g. `elementary-identity-pool`), and continue.

    2. On the pool, click 'ADD A PROVIDER' and choose 'AWS'. Set the provider name to `Elementary AWS` and the AWS account ID to `743289191656` (Elementary's AWS account), then continue.

    <img src="https://mintcdn.com/elementary-devin-1782754750-bigquery-permissions-docs/zLPVq6zLYCQB7e8x/pics/bigquery-wif/add_aws_provider.png?fit=max&auto=format&n=zLPVq6zLYCQB7e8x&q=85&s=675af8d842c4944997a4b94d26e1d9db" alt="Add AWS provider" width="1338" height="1200" data-path="pics/bigquery-wif/add_aws_provider.png" />

    3. In the provider's attribute conditions, add the following condition. This is the Elementary role for GCP — only Elementary can authenticate through this provider. Then save the provider:

       ```
       attribute.aws_role == "arn:aws:sts::743289191656:assumed-role/elementary-gcp-wif-role-prod-elementary-cloud"
       ```

    <img src="https://mintcdn.com/elementary-devin-1782754750-bigquery-permissions-docs/zLPVq6zLYCQB7e8x/pics/bigquery-wif/attribute_condition.png?fit=max&auto=format&n=zLPVq6zLYCQB7e8x&q=85&s=fe19c6fc560c59d9212de9ccaa64d424" alt="Attribute condition" width="1338" height="1280" data-path="pics/bigquery-wif/attribute_condition.png" />

    ### Grant BigQuery permissions to the identity pool principal:

    4. Go to IAM & Admin > IAM and click 'GRANT ACCESS'. In 'New principals', paste the following principalSet. Replace `PROJECT_NUMBER` with your GCP project number and `POOL_NAME` with the pool you created above:

       ```
       principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_NAME/attribute.aws_role/arn:aws:sts::743289191656:assumed-role/elementary-gcp-wif-role-prod-elementary-cloud
       ```

    5. Assign the following project-level roles to the principalSet and save. You will grant `BigQuery Data Viewer` at the dataset level in the next step:
       * **BigQuery Job User**
       * **BigQuery Metadata Viewer**
       * **BigQuery Resource Viewer**

    ### Grant principal access to specific datasets:

    6. In the [BigQuery console](https://console.cloud.google.com/bigquery), find your Elementary dataset in the 'Explorer' tab. Click the three dots next to it > Share > ADD PRINCIPAL. Paste the same principalSet from step 4, select `BigQuery Data Viewer`, and save. If your dbt dataset is in a different project, also grant it `BigQuery Metadata Viewer` and `BigQuery Resource Viewer`.

    ### Download the credential configuration file:

    7. Back in IAM & Admin > Workload Identity Federation, select your pool and click 'GRANT ACCESS' at the top.

    8. Choose 'Grant access using federated identities (Recommended)' and click 'DOWNLOAD CONFIG'.

    <img src="https://mintcdn.com/elementary-devin-1782754750-bigquery-permissions-docs/zLPVq6zLYCQB7e8x/pics/bigquery-wif/grant_access_federated_identities.png?fit=max&auto=format&n=zLPVq6zLYCQB7e8x&q=85&s=284c3021ce7f1974f8efdca5b972e4d6" alt="Grant access using federated identities" width="1216" height="760" data-path="pics/bigquery-wif/grant_access_federated_identities.png" />

    9. In the 'Configure your application' panel, select the provider you created (e.g. `Elementary AWS`) from the Provider dropdown, then click 'DOWNLOAD CONFIG' to download the JSON configuration file.

    <img src="https://mintcdn.com/elementary-devin-1782754750-bigquery-permissions-docs/zLPVq6zLYCQB7e8x/pics/bigquery-wif/configure_application_download_config.png?fit=max&auto=format&n=zLPVq6zLYCQB7e8x&q=85&s=396a7b1c5e573d7d260cc568c453b9d2" alt="Select provider and download config" width="1190" height="1030" data-path="pics/bigquery-wif/configure_application_download_config.png" />

    10. Upload this JSON file to Elementary Cloud in the connection form below.
  </Tab>
</Tabs>

#### Permissions and security

Elementary Cloud doesn't require read permissions to your tables and schemas, but only the following:

* Read-only access to the Elementary schema.
* Access to read metadata in `INFORMATION_SCHEMA` (table metadata and query history), related to the tables in your dbt project.

It is recommended to create a user using the instructions specified above to avoid granting excess privileges.
For more details, refer to [security and privacy](/cloud/general/security-and-privacy).

For BigQuery, this translates to:

* **BigQuery Data Viewer** on the Elementary dataset (for reading Elementary schema data).
* **BigQuery Metadata Viewer** and **BigQuery Resource Viewer** at the **project level** (for reading `INFORMATION_SCHEMA.JOBS` query history and table metadata). These project-level roles are required for BI integrations like Hex that rely on query history.

### Fill the connection form

Use the **Authentication method** toggle at the top of the form to select either **Service account** or **Workload Identity Federation**, matching the method you set up above. The credentials upload field changes based on your selection:

* **Service account file** (Service account method): the service account JSON key file you downloaded.
* **WIF credential file** (Workload Identity Federation method): the external account JSON configuration file you downloaded from the identity pool.

Then provide the remaining fields:

* **Project**: The name of your BigQuery project.
* **Elementary dataset**: The name of your Elementary dataset. Usually `[dataset name]_elementary`.
* **Location**: Use this field to configure the location of BigQuery datasets as per [the BigQuery documentation](https://cloud.google.com/bigquery/docs/locations).

### Add the Elementary IP to allowlist

Elementary IP for allowlist: `3.126.156.226`

### Need help with onboarding?

We can provide [support on Slack](https://elementary-data.com/community) or hop on an [onboarding call](https://savvycal.com/MaayanSa/df29881c).