OCF Connector Release Notes 2023¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Customer Managed Applies to customer-managed instances of Alation
This section contains release notes for Open Connector Framework (OCF) Connectors for the year 2023.
December 27, 2023¶
Azure Databricks OCF Connector: Version 2.0.5¶
Fixed Issues¶
Azure Databricks OCF Connector Misses Processing the Last Query¶
Azure Databricks runs queries on different threads. These queries are logged into log files that are processed by the OCF connector. Howevetr, while processing, the connector misses processing the last query. This results in missing lineage from the catalog.
With this fix, the missing lineage from the last query is now generated for the Azure Databricks OCF connector.
December 20, 2023¶
Azure Databricks OCF Connector: Version 2.0.4¶
Fixed Issues¶
QLI Extraction Fails to Ingest Queries¶
QLI extraction for Azure Databricks OCF connector fails with NA_LOCATION_INFO error and does not return any query statement to ingest.
You can now successfully run the QLI extraction.
Databricks Unity Catalog OCF Connector: Version 2.1.1¶
Fixed Issues¶
QLI Execution Fails for Databricks Unity Catalog OCF Connector¶
The issue related to QLI execution for Databricks Unity Catalog data source with ANSI Mode enabled that resulted in the CAST_INVALID_INPUT error is now fixed.
SQL Server OCF Connector: Version 1.3.8¶
Fixed Issues¶
SQL Server Connection Error Soft Deletes Cataloged Objects¶
In SQL Server OCF Connector, an error in the SQL server connection caused an empty stream of data, which was treated as a successful MDE and resulted in the soft deletion of all the extracted metadata.
Alation now ensures that such configuration and connection errors result in metadata extraction failure instead of an empty data stream.
December 18, 2023¶
Snowflake OCF Connector: Version 2.2.12¶
Fixed Issues¶
Unable to Complete Test Connection and Test Access and Fetch Actions Without System Views Access¶
You can now complete the Test Connection and Test Access and Fetch Schema actions on the General Settings and Metadata Extraction tabs, respectively, even when your configured service account cannot access system views but would like to use custom views instead.
In such cases, Alation now displays a warning message allowing you to proceed if you intend to use Custom MDE Queries to configure and run extraction.
December 13, 2023¶
Add-On OCF Connector for dbt: Version 3.0.7¶
Compatible Alation Version - 2023.1 or higher
Enhancements¶
Improved User Experience¶
The enhanced Add-On OCF connector for dbt Admin user interface provides better navigation and multiple visual cues, making it easy to configure and manage dbt connector.
Important
Add-On OCF connector for dbt version 3.0.7 is supported with the Alation version 2023.1 and higher, however, the enhanced user experience is available only from the Alation version 2023.3.2.1.
Databricks on AWS OCF Connector: Version 2.1.6¶
Compatible Alation Version - 2023.1 or higher
Enhancements¶
Dynamic Profiling and Dynamic Sampling Uses Respective Database Credentials¶
Dynamic profiling and sampling for Databricks on AWS OCF connector now use the credentials the user provides for their respective databases when retrieving table samples and column profiles for specific tables and columns.
Greenplum OCF Connector: Version 1.0.7¶
Compatible Alation Version - 2022.3 (Patch version: 13.4.7) and higher
Fixed Issues¶
Popularity and Top User Calculations Differ Between OCF and Native Connectors¶
There are discrepancies in the Popularity and Top Users values displayed on catalog pages of schema, table, and column objects between the Greenplum OCF and native connectors.
Impala CDP OCF Connector: Version 2.0.1¶
Compatible Alation Version - 2022.4 or later
Fixed Issues¶
Test Connection Fails on the Impala CDP Connector¶
Test Connection is successful for Kerberos-enabled Impala CDP connector.
SAP BO OCF Connector: Version 1.2.0¶
Compatible Alation Version - 2022.4 or later
Enhancements¶
Enhancements to Performance and Memory Utilization¶
Alation has now improved the SAP BO OCF connector to reduce the MDE time and memory utilization. This enhancement addresses all the existing or potential performance issues (longer MDE runs).
SingleStore OCF Connector (MemSQL): Version 1.1.3¶
Compatible Alation Version - 2022.3 or higher
Fixed Issues¶
Data Upload Fails¶
When you try to upload data for a table, upload and table creation, fail with an error.
The SingleStore OCF connector code is now updated that fixes the issue.
Snowflake OCF Connector: Version 2.2.11¶
Compatible Alation Version - 2022.4 or higher
Enhancements¶
The allowUnderscoresInHost Parameter is Automatically Set in the JDBC URI¶
If you miss setting the allowunderscore parameter in the JDBC URI field on the General Settings tab of the Snowflake data source, the Snowflake OCF connector automatically sets the allowunderscore parameter to true (allowunderscore=true).
With this enhancement, Snowflake OCF connector supports underscores in the account names included in the Privatelink JDBC URLs.
Fixed Issues¶
Metadata Extraction Fails In the Absence of ALLOWED_VALUES Column¶
In Snowflake OCF connector version 2.2.9 and earlier, Alation introduced a new column called ALLOWED_VALUES in the Tag extraction query to extract all the available values associated with the ALLOWED_VALUES parameter from the Snowflake data source. If this column is absent in the custom query for the tag, MDE fails.
This column has now been made optional to ensure that MDE runs successfully for users who have installed the latest Snowflake OCF Connector version (2.2.9 and higher) on an earlier Alation version.
Starburst Trino OCF Connector: Version 1.3.1¶
Compatible Alation Version - 2022.4 or higher
Enhancements¶
JDBC Driver Upgrade¶
The JDBC driver for Starburst OCF connector is upgraded from trino-jdbc-380.jar to trino-jdbc-430.jar.
Support for New Starburst Enterprise Version¶
Starburst OCF connector now supports the new Starburst Enterprise version 413-e LTS.
Note
Alation will continue to support the Starburst Enterprise (Trino) version 380-e LTS (EOL version).
December 06, 2023¶
Power BI Scanner OCF Connector: Version 2.1.3¶
Compatible Alation Version - 2023.3.2 or higher
Enhancements¶
Support for Plus Character (+) in Ingested SQL Queries¶
Azure Power BI Scanner OCF Connector supports a plus sign (+) in the ingested SQL queries for dataset expressions. This enables the extraction of connection information from dataset expressions with queries having ‘+’ (plus character) to generate lineage.
Fixed Issues¶
Metadata Extraction Fails for Parameters with Same Name As Table Name in Datasets¶
In Azure Power BI Scanner OCF Connector, MDE fails if a dataset expression has a parameter with the same name as the table name and value as an expression. This is because the length of the object identifier exceeds the limit set by Alation.
Google BigQuery OCF Connector: Version 1.2.3¶
Compatible Alation Version - 2022.3 (Patch version: 13.4.7) or higher
Enhancements¶
Google BigQuery Connector Uses Service Account During Sampling¶
Google BigQuery Connector now uses a service account while performing dynamic sampling with OAuth setup using the Compose tab.
Impala CDP OCF Connector: Version 2.0.0¶
Compatible Alation Version - 2022.4 or later
Enhancements¶
Support for Public Cloud CDP Distribution¶
The Impala on CDP OCF connector now supports Public Cloud CDP Distribution and earlier support for Private Cloud CDP Distribution (Base and CDW).
Support for New and Combination of Multiple Authentication Types¶
The Impala on CDP OCF connector now supports the following authentication types:
Datasource (Impala)
No Auth
Username
Username Password (LDAP)
Kerberos (Password or Keytab)
SSL over any of the above methods
Metastore (For MDE)
No Auth
Username Password
Kerberos (Password or Keytab)
HDFS (For QLI)
Username
Username Password
Kerberos (Password or Keytab)
SSL over any of the above methods
Additionally, you can use multiple combinations of these authentication types for the data source and metastore connections. For details, see Impala on CDP OCF Connector.
Change of JDBC Driver¶
The Impala on CDP OCF connector now uses the Impala JDBC driver over the Hive JDBC driver. This allows support for public cloud CDP distribution and leverages the JDBC’s request pool parameter feature.
Limitations¶
Support for Compose¶
Impala OCF Connector version 2.0.0 supports Compose only from Alation version 2023.3.3.2 or later.
Microstrategy OCF Connector: Version 1.0.2¶
Compatible Alation Version - 2023.1 or higher
Fixed Issues¶
Unable to Access the Microstrategy Server Using External Source Links on Alation¶
When navigating to the Microstrategy OCF connector from a link on a report or dashboard, the MicroStrategy Web Server displays an error if the user is not already logged in to the Microstrategy Server. Alation now allows you to configure a new optional setting (I-Server) that redirects users to the corresponding object page without an error.
Redshift OCF Connector: Version 1.6.0¶
Compatible Alation Version - 2022.4.11 or higher
Enhancements¶
JDBC Driver Upgrade¶
The custom JDBC driver for the Redshift OCF connector is upgraded to version 2.1.0.21. This upgrade addresses associated potential issues with the Redshift OCF connector. It will also help prevent problems that may arise when handling large volumes of metadata for a Redshift data source.
November 29, 2023¶
Databricks Unity Catalog OCF Connector: Version 2.1.0¶
Compatible Alation Version - 2023.1.7.1 or higher
Enhancements¶
Support for OAuth Authentication with Azure Active Directory¶
The Databricks Unity Catalog OCF connector now supports OAuth authentication for Compose, enabling you to securely access and utilize data assets within the Databricks environment.
November 22, 2023¶
Databricks on AWS OCF Connector: Version 2.1.4¶
Compatible Alation version: 2023.1 or higher
Fixed Issues¶
Query Log Ingestion Fails for AWS OCF Connector¶
The Query Log Ingestion (QLI) Preview and ingestion fails due to the Log4j library upgrade after successful datasource connection and metadata extraction.
Preview Error:
QLI Preview failed: The connector responded with the following error message: NA_LOCATION_INFO.
QLI Error:
No queries were found to Ingest.
This issue is resolved.
Power BI Scanner OCF Connector: Version 2.1.0¶
Compatible Alation version: 2023.3.2 or higher
Enhancements¶
Support for Cross-System Lineage Between PowerBI and Databricks Unity Catalog, Azure SQL, and Azure Synapse¶
Alation now supports cross-system lineage between PowerBI and the following data sources:
Databricks Unity Catalog
Azure SQL
Azure Synapse
To start using this feature, perform the following:
Upgrade to Alation version 2023.3.2 or later.
Set the following flag to True using alation_conf:
alation_conf alation.resolution.DEV_bi_connector_returns_db_for_two_part_schema -s TrueInstall the latest Azure Power BI Scanner OCF Connector.
For details, see Power BI Scanner OCF Connector.
PostgreSQL OCF Connector: Version 1.1.10¶
Compatible Alation version: 2023.1 or higher
Fixed Issues¶
Query Log Ingestion Fails If the Custom Query Does Not Include a transactionid Column¶
From PostgreSQL OCF Connector versions 1.1.9 and 1.1.10 onwards, QLI fails if the Custom QLI Query does not include a transactionid column.
This issue is fixed. Now, in PostgreSQL OCF Connector versions 1.1.9 and 1.1.10 or higher, QLI completes successfully even if the Custom QLI Query does not include a transactionid column. However, Alation recommends that you include the transactionid column in your query to obtain accurate durations for the query.
SAP BW OCF Connector: Version 1.2.0¶
Compatible Alation version: 2022.4 or higher
Enhancements¶
Improved Performance and Memory Utilization¶
Alation has now improved the metadata extraction logic for the SAP BW OCF connector to enhance performance and memory utilization.
This enhancement will address the existing or potential performance issues (longer MDE runs) or MDE failures due to out-of-memory errors in the Connector.
SAP ECC OCF Connector: Version 1.2.0¶
Compatible Alation version: 2022.4 or higher
Enhancements¶
Improved Performance and Memory Utilization¶
Alation has now improved the metadata extraction logic for the SAP ECC OCF connector to enhance performance and memory utilization.
This enhancement will address the existing or potential performance issues (longer MDE runs) or MDE failures due to out-of-memory errors in the Connector.
November 16, 2023¶
Snowflake OCF Connector Version 2.2.9¶
Compatible Alation version: 2022.4 or higher
Enhancements¶
With Alation version 2023.3.2.1 and Snowflake OCF connector version 2.2.9, Alation has introduced an enhanced user interface for configuring the Snowflake data source settings.
Note
The connector version is compatible with Alation versions prior to 2023.3.2.1. However, you will view the old user interface with Alation versions prior to 2023.3.2.1.
The documentation for the new Snowflake experience is available at Snowflake OCF Connector.
The enhanced user interface provides the following features:
Improved User Experience¶
The enhanced Snowflake Admin user interface provides better navigation and multiple visual cues, making it easy to configure and manage Snowflake data source.
Easy to Connect¶
The Snowflake Admin user interface now provides a step-by-step process to connect to your Snowflake data source easily and quickly.
Extra Error-Prevention Checks¶
The Snowflake Admin user interface now includes added checks to avoid obvious configuration mistakes. The user interface also includes text to assist and provide cues while configuring.
Actionable Error Messages and Detailed Error Reports¶
The Snowflake Admin user interface has made troubleshooting easier for metadata extraction and query log ingestion, as you can now have actionable error messages. We also support a detailed error report for Metadata Extraction, which contains a detailed error description and steps to resolve the issue.
November 8, 2023¶
SQL Server OCF Connector Version 1.3.7¶
Compatible Alation version: 2022.2 or higher
Fixed Issues¶
Connection Test Fails When Using SSL with NTLM Authentication¶
Fixed an issue when the connection test failed when SSL was enabled with NTLM authentication. The test resulted in the following error:
Caused by: sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target
To use SSL with NTLM, basic, or Kerberos authentication, use the following JDBC URL format:
sqlserver://<Host_Name>:<Port_Number>;encrypt=true;hostNameInCertificate=<certificate_name>;
This JDBC URI includes the following additional SSL related parameters:
encrypt=true
hostNameInCertificate=<certificate_name>
You can run the command below to verify your certificate name. In the result, check for the value after CN=. That’s the certificate name.
openssl x509 -noout -subject -in <certificate_name>.crt;
Example:
sqlserver://10.13.82.165:1433;encrypt=true;hostNameInCertificate=abc.corp.net;