Overview¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Customer Managed Applies to customer-managed instances of Alation
Enhanced Connector Enhanced connectors add extended capabilities and require a separate entitlement in addition to your Alation platform license.
The Azure Data Explorer enhanced connector is used to catalog Azure Data Explorer data sources in Alation.
The connector extracts Azure Data Explorer objects, such as:
Cluster and database (cataloged as schemas)
Tables
Materialized views
Functions
Columns and column data types
After extraction, search and find Azure Data Explorer objects, curate the corresponding catalog pages, and understand data transformations using lineage diagrams.
The connector is distributed as a .zip package that you upload and install in the Alation application. Download the latest connector package from the Alation Customer Portal (Customer Portal > Connectors).
Team¶
Work with the following administrators to install and configure this connector:
Azure Data Explorer administrator
Creates a service account for Alation.
Provides connection details, including the JDBC URI.
Provides authentication details and assists with authentication configuration.
Alation Server Admin
Ensures that Alation Connector Manager is installed and running (or installs it).
Installs the connector.
Creates and configures the Azure Data Explorer data source in Alation.
Performs initial extraction and prepares the data source for Alation users.
Scope¶
The table below lists core features supported by the Azure Data Explorer Enhanced connector.
Feature |
Scope |
Availability |
|---|---|---|
Authentication |
||
Azure AD token authentication |
Authentication using an Azure AD service account |
Yes |
Metadata extraction (MDE) |
||
Default MDE |
Extraction of metadata based on default queries built into the connector |
Yes |
Selective extraction |
Ability to limit extraction using filters |
Yes |
Full extraction |
Ability to run extraction across all accessible Azure Data Explorer databases |
Yes |
Extracted metadata objects |
||
Cluster and database |
List of clusters and databases |
Yes |
Table |
List of tables |
Yes |
Materialized view |
List of materialized views |
Yes |
Function |
List of functions |
Yes |
Column |
List of columns and their data types |
Yes |
Sampling and profiling |
||
Table sampling |
Retrieval of data samples from extracted tables |
Yes |
Column sampling |
Retrieval of data samples from extracted columns |
No |
Custom query-based table sampling |
Ability to use custom queries for sampling specific tables |
Yes |
Custom query-based column sampling |
Ability to use custom queries for sampling specific columns |
No |
Deep column profiling |
Value distribution statistics (profiling V2) |
No |
Dynamic profiling |
Individual users connect with their own accounts to retrieve samples and profiles |
Yes |
Query log ingestion (QLI) |
||
Table-based QLI |
Ingestion of query history from a table or view |
No |
Custom query-based QLI |
Ingestion of query history based on a custom query |
No |
JOIN and filter calculation |
Calculation of JOIN and filter information based on ingested query history |
No |
Lineage |
||
Table-level lineage |
Auto-calculation of lineage based on MDE, and Compose queries |
Yes |
External table file lineage |
Lineage for external tables |
Yes |
Column-level lineage |
Calculation of lineage data at the column level (requires the Column-level Lineage Parser add-on) |
Yes |
Compose |
||
Compose on customer-managed deployments |
Customer-managed Compose on on-premise Alation instances |
Yes |
Compose on Alation Cloud Service |
Compose on Alation Cloud Service instances (may require Alation Agent depending on network) |
Yes |
OAuth SSO in Compose |
SSO via OAuth |
No |
Object mapping¶
Azure Data Explorer metadata extracted by the connector maps onto the Alation object model for RDBMS sources:
Data Source → Schema → Table → Column
Source object |
Object in Alation |
|---|---|
Cluster and database |
Schema |
Table |
Table |
Materialized view |
Table |
Function |
Table |
Column |
Column |