Amazon DynamoDB OCF Connector (Public Preview): Overview¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Customer Managed Applies to customer-managed instances of Alation
Available from Alation version 2023.3
The Amazon DynamoDB OCF Connector is currently released as Public Preview or Limited Availability.
Overview¶
The latest OCF connector for Amazon DynamoDB is available for download from the Connector Hub on Alation Customer Portal. Ask an Alation admin with access to Customer Portal to download the connector from the Connectors section (Customer Portal > Connectors). You can upload and install the connector file in the Alation application. The connector is compiled together with the required database driver, so no additional effort is needed to procure and install the driver. Use this connector to catalog Amazon DynamoDB as a data source on Alation on-premises and Cloud Service instances. It extracts and catalogs such database objects as schemas, tables, views and columns. After the metadata is extracted, it is represented in the data catalog as a hierarchy of catalog pages under the parent data source. Alation users can leverage the full catalog functionality to search for and find the extracted metadata, curate the corresponding catalog pages, create documentation about the data source, and exchange information about it.
Team¶
You may need the assistance of your database administrator to configure this data source.
DynamoDB administrator:
Creates a service account for Alation.
Provides the connection information and the JDBC URI.
Provides the authentication information and assists in configuring the authentication.
Alation Server administrator:
Ensures that Alation Connector Manager is installed and running or installs it.
Install the connector.
Creates and configures the DynamoDB data source in the catalog.
Performs initial extraction and prepares the data source for Alation users.
Scope¶
The table below describes which metadata objects are extracted by the connector and which catalog functionality is supported.
Feature |
Scope |
Availability |
|---|---|---|
Authentication |
||
Basic |
Authentication with username, password, and security token |
Yes |
AWS Root Keys |
Authentication with Access Key and Secret Key. |
Yes |
SSL |
Connection over the TLS protocol |
No |
LDAP |
Authentication with a database service account that is an LDAP account in an organization’s network |
No |
OAuth |
Authentication with the OAuth 2.0 protocol. |
No |
SSO |
Authentication using an SSO flow through an IdP application |
No |
Metadata Extraction (MDE) |
||
Default MDE |
Extraction of metadata based on the JDBC driver methods in the connector code |
Yes |
Custom query-based MDE |
Extraction of metadata based on extraction queries provided by a user |
No |
Popularity |
Indicator of the popularity (intensity of use) of a data object, such as a table or a column |
No |
Extracted metadata objects |
||
Data Source |
Data source object in Alation that is parent to extracted metadata |
Yes |
Schemas |
List of databases |
Yes |
Tables |
List of tables |
Yes |
Columns |
List of columns |
Yes |
Column data types |
Column data types |
Yes |
Views |
List of views |
N/A |
Source comments |
Source comments |
N/A |
Primary keys |
Primary key information for extracted tables |
Yes |
Foreign keys |
Foreign key information for extracted tables |
N/A |
Functions |
Extraction of function metadata |
N/A |
Function definitions |
Extraction of function definition metadata |
N/A |
Sampling and Profiling |
||
Table sampling |
Retrieval of data samples from extracted tables |
Yes |
Column sampling |
Retrieval of data samples from extracted columns |
Yes |
Deep column profiling |
On-demand profiling of specific columns with the calculation of value distribution stats |
Yes |
Dynamic profiling |
On-demand table and column profiling by individual users who use their own database accounts to retrieve the profiles |
Yes |
Custom query-based table sampling |
Ability to use custom queries for sampling specific tables |
Yes |
Custom query-based column profiling |
Ability to use custom queries for profiling specific columns |
Yes |
Query Log Ingestion (QLI) |
||
Table-based QLI |
Ingestion of query history based on a table that contains query history data |
No |
Query-based QLI |
Ingestion of query history based on a custom query history extraction query |
No |
JOINs and filters |
Calculation of JOIN and filter information based on ingested query history |
No |
Predicates |
Ability to parse predicates in ingested queries |
No |
Lineage |
||
Automatic lineage generation |
Auto-calculation of lineage based on query history ingested from QLI, MDE, and Compose queries |
No |
Direct lineage |
Extraction of lineage from system tables during MDE |
No |
Column-level lineage |
Extraction of lineages on the column level |
No |
Compose |
||
Not supported |
||
Object Mapping¶
DynamoDB Object |
SQL Object |
|---|---|
Collection |
Table |
Document |
Rows or Records within Table |