Overview¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Customer Managed Applies to customer-managed instances of Alation
To get the enhanced connector for Dremio, contact the Forward Deployed Engineering team or ask your Alation Account Team.
The connector is available as a Zip file that can be uploaded and installed in the Alation application.
This connector should be used to catalog Dremio Software assets in Alation. The connector catalogs Dremio objects such as schemas, tables, columns, and views. It enables end-users to search and find Dremio objects from the Alation user interface.
Use the instructions given in this guide to install and manage the connector.
Supported Dremio Versions¶
Dremio Software versions 24.3.0 and later are supported. Dremio Cloud is not yet supported.
Dremio Version |
Connector Support |
|---|---|
Dremio Enterprise Edition |
24.3.0 onwards |
Dremio CE (Community Edition) |
24.3.0 onwards |
Dremio Cloud |
Unsupported |
Team¶
The following administrators are required to install this connector:
Alation Server Admin:
Ensures that Alation Connector Manager or Alation Agent is installed and running.
Installs the connector.
Creates and configures a Dremio data source in the Catalog.
Dremio Administrator:
Creates a service account with required permissions (detailed in Prerequisites section).
A username and password or Personal Access Token is required
Provides Dremio server and port information
Scope¶
The table below shows the connector’s feature support:
Feature |
Dremio Software |
|---|---|
Dremio Cloud Support |
On Roadmap |
Core Capabilities |
|
Automated Metadata Extraction (MDE) |
Yes |
Query-based MDE |
On Roadmap |
Search |
Yes |
Catalog page curation |
Yes |
Catalog sets |
Yes |
Propagation of Data Quality Flags |
Yes |
Popularity |
Yes |
Compose |
|
On-prem Alation instances |
Yes |
Alation Cloud Service instances |
Yes |
QLI |
|
Query based |
Yes |
Filters |
Yes |
Lineage |
On Roadmap |
Sampling and Profiling |
|
Table Sampling |
Yes |
Custom Query Based Table Sampling |
On Roadmap |
Custom Query Based Column Profiling |
On Roadmap |
Dynamic Profiling |
On Roadmap |
Authentication |
|
Username and Password |
Yes |
Personal Authentication Token |
Yes |
Technical Metadata |
|
Tables |
Yes |
Attributes |
Yes |
Views |
Yes |
Schemas |
Yes |
Tags |
On Roadmap |
Object Mapping¶
The table below describes which metadata objects are extracted by this connector and how they are mapped to objects in Alation.
Dremio Type |
Alation Type |
Description |
|---|---|---|
Space |
Root schema |
A space in Dremio is a directory in which views are saved. The name of a space is used as a root schema. Folders and views under the space are shown in Alation as being under a schema with the same name as the source. |
Object Storage |
Root Schema |
An Object Storage source in Dremio is a data lake source, such as AWS S3 buckets. The name of an Object Storage source is used as a root schema. Folders and tables in the Object Storage source are shown in Alation as being under a schema with the same name as the Object Storage source. |
Database |
Root Schema |
Examples of Database sources in Dremio are Postgres, Oracle, and SQL Server. The name of a Database source is used as a root schema. Folders and tables in the Database source are shown in Alation as being under a schema with the same name as the Database source. |
View |
View |
1-to-1 mapping. |
Table |
Table |
1-to-1 mapping. |
Folder |
Schema |
Nested folders in Dremio are shown as nested schemas in Alation. |
The following Dremio sources and objects are not yet validated. While Alation may show data in these sources, some unexpected results may occur. Validation and support of these sources and objects is expected in a future connector:
Nessie catalogs
Metastore sources
Reflections