Configure Connection to Data Source

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Customer Managed Applies to customer-managed instances of Alation

After you install the Google BigQuery OCF connector, you must configure the connection to the Google BigQuery data source.

The various steps involved in configuring the Google BigQuery data source connection setting are:

Provide Access

Go to the Access tab on the Settings page of your Google BigQuery data source, set the data source visibility using these options:

  • Public Data Source — The data source is visible to all users of the catalog.

  • Private Data Source — The data source is visible to the users allowed access to the data source by Data Source Admins.

You can add new Data Source Admin users in the Data Source Admins section.

Connect to Data Source

To connect to the data source, you must perform these steps:

Important

The Alation user interface displays standard configuration settings for credentials and connection information stored in the Alation database. If your organization has configured Azure KeyVault or AWS Secrets Manager to hold such information, the user interface will change to include the following buttons adjacent to the respective fields:

../../../_images/SnowflakeOCF_New_Vault_Button.png

By default, you see the user interface for Standard. In the case of Vault, instead of the actual credential information, you must select the source and provide the corresponding key. For details, see Configure Secrets for OCF Connector Settings.

Note

For Vault, ensure that you encode the service account certificate content into base64. It is applicable for both .json and .p12 certificates.

Provide the JDBC URI

Before using the JDBC URI format, understand the JDBC URI format and parameters for the Google BigQuery OCF connector.

URI Format

bigquery://https://www.googleapis.com/bigquery/v2:<Port_Number>;ProjectId=<Project_Id>

Example

bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=alation-edw;

Note

  • The service account can be given multiple projects permissions. Only include the main project ID into this URI. See Extract from Multiple Projects for more details.

  • Legacy SQL is supported: QueryDialect=BIG_QUERY.

  • QueryDialect defaults to SQL if you don’t provide a value.

Provide the JDBC URI in Alation

To provide to the JDBC URI in the Alation UI, perform these steps:

From Alation version 2023.3.4 and connector version 1.3.0

  1. On the Settings page of your Google BigQuery data source, go to the General Settings tab.

  2. In the Provide the JDBC URI section, enter the JDBC URI.

    Follow the correct JDBC URI format (* bigquery://https://www.googleapis.com/bigquery/v2:<Port_Number>;ProjectId=<Project_Id>;*).

    Note

    Click the View history icon to view the history of recent URI values provided by users, if any. The History: Recent URI Values list displays 100 URI entries. You can copy a URI from the list to reuse.

  3. Click Save.

Configure Authentication

For Google BigQuery data sources, Alation supports authentication with a service account or with a user account and OAuth client.

Authentication with a service account supports metadata extraction, sampling and profiling, and query log ingestion. However, it will not allow you to use the functionality where a specific user initiates a connection to the database, such as Compose, query forms, dynamic sampling and profiling, and data upload. These features require a user account and will need to be configured additionally.

Authentication with a user account and an OAuth client supports all extraction features (metadata extraction, sampling and profiling, QLI), as well as Compose, query forms, dynamic sampling and profiling, and data upload.

Configure Service Account Authentication

Before configuring the service account authentication in Alation user interface, ensure that you create a service account and generate a certificate. For details, see the Create a Service Account section in Prerequisites.

To configure service account authentication, perform these steps:

From Alation version 2023.3.2.1 and connector version 2.2.9

  1. On the Settings page of Google BigQuery data source, go to the General Settings tab.

  2. In the Configure authentication step, select the Service account tab.

  3. Enter the service account ID in the following format: <SA_Name>@<Project_Id or Service_type>.gserviceaccount.com

    Example: alation-test@orbital-eon.iam.gserviceaccount.com

  4. Upload the service account certificate.

    Alation accepts the .json or .p12 extensions.

    Note

    Service account ID is mandatory for certificate type .p12 and optional for .json.

  5. Click Save.

Extract from Multiple Projects

When authenticating with a service account, you can extract metadata from multiple projects. To extract from multiple projects, cross-reference projects by adding the service account as a member on all the projects you want to extract. Assign it the required permissions on all these projects.

The connection URI in Alation only needs to include the project ID of the main project (the project on which this service account was created) as the value of the Database Name parameter. Alation will fetch all datasets from all projects the service account has access to.

Configure OAuth Authentication

This type of authentication supports all data source features, such as MDE, sampling and profiling, QLI, Compose, query forms, dynamic sampling and profiling, and data upload.

Before configuring OAuth authentication in Alation, ensure that you create a user account and an OAuth client for Alation. For details, see the Create a User Account and an OAuth Client section in Prerequisites.

To configure OAuth authentication, perform these steps:

From Alation version 2023.3.4 and connector version 1.3.0

  1. On the Settings page of your Google BigQuery data source, go to the General Settings tab.

  2. In the Configure authentication step, select the OAuth tab.

    If you have not created any configuration objects yet you will see a warning that none exist. Perform the steps in the Create an OAuth Configuration for Extraction section in Prerequisites to create the required authentication configuration.

  3. Select the Authentication profile.

    It’s the configuration object you created for this data source.

  4. Click Authenticate.

  5. Click Save.

Test the Connection

The connection test checks database connectivity. Alation uses the JDBC URI to connect to the database and to confirm when the connection is established.

After specifying the JDBC URI and configuring authentication, test the connection.

To validate the network connectivity, go to General Settings > Test Connection of the Settings page of your Google BigQuery data source and click Test.

A dialog box appears confirming the status of the connection test.

Configure Additional Connection Settings

Apart from the mandatory configurations that you perform to connect to the data source in the General Settings tab, you can configure the following additional settings:

  • Configure Additional Data Source Connections

  • Disable Obfuscate Literals

  • Disable automatic lineage generation

Configure Additional Data Source Connections

Alation can associate objects in a data source with objects in another source in the catalog through lineage. For example, you can show lineage between your data source and BI sources that use its data.

Provide additional connection information for the data source to see lineage across multiple sources on the Lineage chart.

From Alation version 2023.3.4 and connector version 1.3.0.

To enter additional data source connection details, go to General Settings > Advanced settings of the Settings page of your Google BigQuery connector and enter the connection URL.

Use the following format: <host>:<port>

You can provide multiple values as a comma-separated list:

<host1>:<port1>,<host2>:<port2>

For example:

10.13.71.216:1541,sever.com:1542

For more details, see Configure Cross-Source Lineage.

Enable or Disable Automatic Lineage Generation

You can enable or disable the lineage for the data source to be generated automatically during metadata extraction, query log ingestion, and from Data Definition Language queries run by users in Compose.

From Alation version 2023.3.4 and connector version 1.3.0.

Go to General Settings > Advanced settings of the Settings page of your Google BigQuery data source and turn on or off the Disable automatic lineage generation toggle.

Turn off the Disable automatic lineage generation toggle when you want to automatically generate the lineage.

Enable this option when you do not want lineage to be automatically generated and prefer to create lineage manually or using an API.

By default, automatic lineage generation is enabled.

Enable or Disable Obfuscate Literals

You can hide literal values from queries ingested with query log ingestion and displayed on the Queries tab of a schema and table catalog objects.

From Alation version 2023.3.4 and connector version 1.3.0.

Go to the General Settings tab and turn on or off the Obfuscate literals toggle under the Advanced settings section.

When enabled, literal values are substituted with placeholder values. Disable this option when you want literal values in queries to be visible to users.

By default, this option is disabled.

Configure Logging

To set the logging level for your Google BigQuery data source logs, perform these steps:

From Alation version 2023.3.4 and connector version 1.3.0.

  1. On the Settings page of your Google BigQuery data source, go to General Settings > Connector logs.

  2. Select a logging level for the connector logs and click Save.

    The available log levels are based on the Log4j framework.

You can view the connector logs in Admin Settings > Server Admin > Manage Connectors > Google BigQuery OCF connector.