Release Notes 2024.1 (General Availability)

Note

The release notes on this page reflect changes to the Alation application. If you’re looking for OCF connector release notes, jump to OCF Connector Release Notes.

PATCH RELEASE 2024.1.5.5

build 18.5.5.89300

Released October 14, 2024

Note

This patch release is available for customer-managed (on-premise) instances only. This patch release doesn’t apply to Alation Cloud Service instances.

  • Fixed an issue where users experienced performance degradation during metadata extraction and query log ingestion (QLI), characterized by backed-up task queues, QLI executions missing the schedule, and high CPU usage by Celery processes. The performance issues were traced to a specific method that was called frequently, each time taking 0.1-1 second per call and slowing the system. We implemented optimizations to decrease the frequency and impact of database calls made by this method, enhancing the QLI performance. (AL-174546)

PATCH RELEASE 2024.1.5.4

build 18.5.4.85494

Released September 6, 2024

Note

This patch release is available for customer-managed (on-premise) instances only. This patch release doesn’t apply to Alation Cloud Service instances.

  • Optimized several internal APIs to reduce loading times of tasks generated by workflows. (AL-161158)

PATCH RELEASE 2024.1.5.3

build 18.5.3.84439

Released August 22, 2024

Note

This patch release is available for Alation Cloud Service instances on the cloud-native architecture. This patch release doesn’t apply to customer-managed (on-premise) instances and Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

  • Platform improvements for utilization. (AL-168402)

PATCH RELEASE 2024.1.5.2

build 18.5.2.83447

Released August 14, 2024

Note

This patch release is available for customer-managed (on-premise) instances only. This patch release doesn’t apply to Alation Cloud Service instances.

  • This patch release addresses a security vulnerability. (AL-164410)

PATCH RELEASE 2024.1.5.1

build 18.5.1.82675

Released August 2, 2024

Note

This patch release is available for Alation Cloud Service instances on the cloud-native architecture. This patch release doesn’t apply to customer-managed (on-premise) instances and Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

  • Updated the helm chart to address an issue where non-string custom label values were not handled properly. The issue previously hindered the upgrade of cloud instances. With this fix, the helm chart now supports non-string values for custom labels. (AL-166922)

RELEASE 2024.1.5

build 18.5.0.82175

Released August 1, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

New User Experience

  • Curate & Govern

    • Bulk operations on related data assets with Catalog Sets (AL-153502, AL-153503)

    • Bulk Actions in the full-search experience to bulk-manage Data Policies, Business Policies, Domains, custom fields, Stewards, and People Sets.(AL-156924)

    • The Catalog Activity and Curation Progress reports (AL-153501, AL-153500)

    • Creation and configuration of Workflows (AL-151062)

    • Creation and management of Data and Business Policies (AL-137933)

  • Homepage

    • Dynamic layout: Admin users can change the layout of the homepage through a drag-and-drop experience. Items can be added in the page preview mode and moved around the screen. Page elements can be reorganized within a block. Admin users can also change the width of custom blocks to occupy either the full screen or half of it. (AL-153691)

  • Migration of RDBMS data source pages to OCF is available in New User Experience (AL-156576)

  • File system objects are integrated into the New User Experience (AL-160203, AL-160171)

  • Agent and connector dashboards are integrated into the New User Experience (AL-151872)

  • Multiple improvements to My Inbox, including accurate conversation counts and displaying resolved conversations by default (AL-158174)

Authentication & User Management

Auth Service on Agent: IAM User Profile Support

As part of the 2024.1.5 release, we’ve added the IAM user profile-based authentication in Auth Service on Agent, where customers would store their credentials as profiles in a credentials file on their instance. This authentication mechanism is compatible with agents operating in an AWS environment, other platforms, and on-premise servers. (AL-163038)

Restriction on the User Profile Upload

For every CSV file with user profiles that is uploaded, the total number of user updates and creations is now allowed to be no greater than 50. Whenever a Server Admin uploads a CSV file, irrespective of this threshold being exceeded or not, the preview will always be shown. When the 50 limit is exceeded, then users won’t be able to confirm the changes. The Confirm button appears disabled and an error message will be shown right below the preview table. (AL-158623)

Data Dictionary

Support for Custom Field IDs in the Dictionary Upload

We’ve improved the handling of custom fields in the user interface to address issues arising from duplicate custom field names. Alation users can create multiple custom fields of different types with the same name, previously leading to conflicts during the data dictionary upload where the system could not resolve a custom field name to a specific field record. To prevent such issues, we’ve now enabled the inclusion of custom field IDs alongside names in the dictionary file headers. The allowed formats for entering custom fields IDs are:

  • field_id

  • field_id | field_name

For example, if there are two custom fields named status, one with ID 10001 and another with ID 10002, users can specify the ID in the header to distinguish between them, like 10001|status or 10002|status. This update ensures that custom fields are correctly identified during the upload. (AL-160329)

Document Hubs

Customize Document Tables

We have enhanced the column organization experience of Document tables on Document Hub and Folder pages. Document tables now have a standard (out-of-the-box) column configuration with capabilities for Admin users to assemble and save a custom configuration for specific Document Hubs. Admins can also apply the custom configuration to all Folder pages in the Document Hub. The Reset behavior has been enhanced to restore settings to the standard configuration or the Document-hub specific configuration. (AL-141394)

Lineage

Business Lineage: General Availability

This release marks a significant milestone in our multi-month effort to enhance, declutter, and simplify lineage charts. Alation Lineage now provides a comprehensive set of features that deliver a balanced and engaging chart experience for all users, including data analysts, data engineers, business users, and viewers. Starting with version 2024.1.5, features that were previously in Public Preview and labeled as Beta are now generally available. Users have multiple interaction options with the chart, allowing them to adjust the view and layout to meet their specific analytical needs. Below are the main highlights of the Lineage improvements you can now take advantage of. To access these features below, the Compound Layout must be enabled.

Data Quality Metadata Layer

Added support to viewing Data Quality flags on the lineage graph and to view additional information in a popup pane for a specific table or column. (AL-155339)

Lineage Chart Position Focus Improvement

This improvement ensures that the cursor focus remains consistent within the lineage chart when users toggle display options, like business titles, or navigate between tabs, such as switching from full lineage to column-level lineage tracing. This ensures a seamless and consistent viewing experience across different aspects of lineage. (AL-145290)

Lineage Minimap

Alation Cloud Service on the cloud-native architecture, Classic UI and New User Experience

The lineage chart will now show a small minimap with bi-directional drag for easy navigation to help users orient on large lineage charts. Currently, the minimap is behind a feature flag (alation.feature_flags.dev_enable_lineage_minimap) and requires enabling. (AL-142151)

Note

On customer-managed instances, you can modify the values of backend feature flags using the alation_conf tool.

Alation Cloud Service customers can request server configuration changes through Alation Support.

Lineage Chart Legend

Alation Cloud Service on the cloud-native architecture, Classic UI and New User Experience

Added a legend layer to lineage charts that shows all the icons on the chart with clarifications. The legend layer will be updated based on icons present at the time of viewing. If users filter dataflows or data sources the legend will also be filtered. This feature is currently behind a feature flag (alation.feature_flags.dev_enable_lineage_legend) and requires enabling. (AL-142152)

Note

On customer-managed instances, you can modify the values of backend feature flags using the alation_conf tool.

Alation Cloud Service customers can request server configuration changes through Alation Support.

Deprecation Warning Icon Change

Alation Cloud Service on the cloud-native architecture, New User Experience

The approach to Trust Flags in the New User Interface (Endorse - Add Important Info - Deprecate) is different from Classic (Endorse - Warn - Deprecate). Instead of the warning (yellow) icon, the propagated deprecation alerts will appear with the blue Important info label on lineage charts to be consistent with the new Trust Flag paradigm. (AL-142158)

Remove the Dataflow for BI Sources Feature Flag

The configuration flag alation.feature_flags.enable_dataflow_for_bi_lineage has been removed. By default, dataflow objects will be created for all BI sources and displayed on lineage charts. (AL-163393)

“Disable automatic lineage generation” Renamed to “Enable automatic lineage generation”

The Disable automatic lineage generation checkbox on the General Settings tab of data sources has been renamed to Enable automatic lineage generation. The default behavior of this setting remains unchanged (automatic lineage generation is enabled). (AL-152080)

The Compound Layout Excludes Dataflows for Viewer Users

The Dataflow filter in Compound Layout is now set to Hide by default for users with the Viewer role. (AL-162532)

GSP Upgrade

Upgraded the General SQL Parser (GSP) jar to version 2.9.x.x. The upgrade enables support for stored procedure lineage and for specific naming conventions, such as the use of the at sign @ in Db2 table names and numeric values at the end of catalog names for Google BigQuery (GBQ) target tables. (AL-159884)

Open Connector Framework (OCF)

OCF Connectors

Updates to the Open Connector Framework (OCF) connectors are released every week. See OCF Connector Release Notes for information on the updates in the month of July 2024. A few July highlights are:

  • Certification of query log ingestion (QLI) from Redshift Serverless (AL-156721)

    • Certified the Redshift OCF connector for Redshift Serverless instances on customer-managed and Alation Cloud service instances. Provided support for QLI using the SYS tables for Serverless, Provisioned, and Spectrum.

  • Column-level lineage from SAP HANA data sources (AL-143424)

    • The OCF Connector for SAP HANA now supports column-level lineage. The General SQL Parser (GSP) will use the HANA grammar to generate lineage at the column level. This feature is behind a flag Automatically extracts Column Level Lineage for SAP data source in Admin Settings > Feature Configuration.

  • New user experience for Power BI Scanner OCF connector (AL-136714)

    • The Power BI Scanner OCF connector now has an improved user experience with better navigation, multiple visual cues, and step-by-step guidance making it easier to configure and manage Power BI sources. The user interface now includes connection and access checks to help prevent configuration errors. Troubleshooting is made simpler as the error messages include actionable recommendations.

OCF Platform

Removed Feature Flags for Add-on Connectors

Alation now allows installation and usage of add-on connectors without the requirement to enable any feature flags. This change removes the flags for Fivetran (Enable Fivetran Connector for OCF RDBMS sources (Paid Feature)) and dbt (Enable DBT Connector for OCF RDBMS sources (Paid Feature)) from the user interface in Admin Settings > Feature Configuration. With this change, Data Source Admins will always see the add-on connector configuration tabs in the settings of the main connector. (AL-157919)

Removed Feature Flags for BI Column-Level Lineage

The configuration flags alation.feature_flags.enable_auto_cll_tableau and alation.feature_flags.enable_auto_cll_powerbi_scanner and the respective feature flags Automatically extracts Column Level Lineage for Tableau BI data source and Automatically extracts Column Level Lineage from Power BI Scanner data source are removed. By default all BI connectors will generate column-level lineage along with the table-level lineage for all the applicable data, if the respective RDBMS data source has its column-level lineage flag enabled. (AL-159784)

Bug Fixes

API

  • For the Relational Integration API, we have fixed the documentation on the response type for the GET methods. They are now properly indicated as arrays of objects. (AL-153910)

  • For both the Native Data Sources API and OCF Data Sources API, the unresolved_mention_fingerprint_method parameter is now correctly identified as an integer rather than a string. (AL-153910)

  • Fixed a bug that was preventing the BI Server API from appearing in the Swagger UI in an Alation instance. (AL-161394)

  • Fixed an issue with the Document API (GET integration/v2/document), where it might return a 504 timeout error for certain documents. The performance of the GET endpoint has been optimized by limiting the number of rows scanned to fetch the templates for objects. (AL-163608)

  • Fixed an issue where the GET all articles API returned a 504 timeout error after one minute, even with a limit set to 1. The query process for fetching article visibility information has been optimized to retrieve visibility information in a single query. (AL-162485)

Data Dictionary

  • Fixed an issue in the data dictionary download where the downloaded file included an extra description column, leading to a header error during the upload process (Error at header 13, repeated header found: ‘description’.’) (AL-160318)

Governance

  • Fixed an issue with extracting Dynamic Masking policies from Snowflake that affected instances with multiple Snowflake data sources using a connection to the same database. Previously, when running metadata extraction (MDE) from one data source, the policy links in the Data Managed by Policy field were incorrectly deleted for other Snowflake data sources. To apply the fix, metadata extraction should be rerun on affected Snowflake data sources after updating Alation to 2024.1.5. (AL-161825)

  • Classic UI: Previously, the flag score values for child RDBMS objects (schemas of a data source, tables of a schema, columns of a table) reflected only flags that are directly set on the child objects. Now, they will also reflect the flags set via catalog sets if a child object is a member of those catalog sets. Previously, the flag score column in the user interface was sortable, but now the sorting is disabled due to performance considerations. Users can navigate through all pages for viewing the flag score of child objects but cannot sort by this column anymore. (AL-147639)

    Note

    The flag score is the sum of all the trust flags set applied to an object. For example, if a schema has two endorsements and one deprecation, the flag score is +1 (+2-1). Warnings don’t have any impact on the score.

  • Resolved a synchronization issue in manual catalog sets that occurred during the data dictionary upload process. Previously, changes made through uploads did not reflect in the manual catalog sets. To address this, we’ve introduced support for synchronizing the metadata of manual catalog sets’ children during the upload flow. This functionality is controlled by a new feature flag, alation.feature_flags.DEV_update_equivalent_objects_in_upload_data_dictionary, which is disabled by default. Users looking to test or utilize this feature must explicitly enable the flag. (AL-142458)

  • Fixed an issue with conditional catalog sets where changes to Shared Fields were not reflecting on newly added objects that met the membership criteria. Previously, intermittently, updates to descriptions or other shared values in a catalog set were not automatically applied to existing or new members unless the membership expression was manually refreshed. The problem has been fixed, ensuring that any updates to catalog sets are now correctly and immediately applied to all relevant objects. (AL-115297)

Document Hubs, Glossaries, & Articles

  • Fixed an issue in the New UI where changes to the names of custom fields were not reflected in the table with customized Column Organization on the Folder page displaying documents with those altered fields. After this fix, if an admin has saved a table configuration and subsequently updates a custom field’s name, the table configuration will unselect this field but will display the updated name correctly in the Column Organization widget. An admin will need to re-select the updated field. (AL-160664)

  • Resolved an issue occurring when the feature flag Document Hubs 3: Enable Hub Creation (Public Preview) was activated. Previously, the Edit Columns widget did not display custom fields. This issue has now been fixed. (AL-154994)

  • Resolved an issue in the Bulk Utility where the download process would become unresponsive if a user attempted to download terms assigned to a titled domain and an untitled subdomain. This fix ensures that downloads proceed smoothly, regardless of the titling status of the domain or subdomain involved. (AL-160170)

OCF Connectors

  • Resolved an issue reported for the SQL Server OCF data sources where profiling failed for tables in schemas containing a period in their names. For example, while profiling was successful for tables in the ABC schema, it failed for tables in the ABC.ABC schema. The logic was modified to pass the schema name and catalog name separately in table profiling requests, which ensures successful profiling regardless of schema naming conventions. The changes have been done on both the OCF SDK 4.8.7 and the Alation application side. Specific connector updates will follow. (AL-141614)

  • Resolved an issue with the Elasticsearch OCF connector not functioning in Compose with Query Service disabled. Users previously encountered the error Credentials incorrect. Error connecting to the database when attempting to add credentials through the Connection Settings dialog. This has been fixed, ensuring the connector now operates correctly in Compose. (AL-157169)

  • Addressed security concerns reported for the OCF Oracle connector using JRE 1.8.0.x. In response, we have upgraded the JDK to a newer version where the identified vulnerabilities were fixed to ensure enhanced security and compliance of the Oracle connector. The fix is applied to connector version 1.2.2 and requires Alation version 2024.1.5. (AL-150899)

  • Fixed an issue with timezone normalization in the QLI results, which previously led to the partial success errors. The problem stemmed from the user-selected date range being converted to UTC for database queries, but not being similarly converted when checking if results fell outside this range. This normalization inconsistency has now been corrected, and the checking of dates in both the fetching and condition align correctly. (AL-152646, AL-154254)

Alation Analytics

  • Addressed a discrepancy between Alation Analytics on PostgreSQL and the Alation-managed Analytics Experience on Snowflake, regarding mismatched FP values in BI-related tables. The issue was traced to a loss of precision in larger integer values during the data transfer from PostgreSQL to Snowflake in the data copying framework. The problem has been corrected, ensuring that the FP values now consistently match across Alation Analytics and the Analytics Experience on Snowflake. (AL-157364)

Other Fixes

  • Resolved an issue where modifying OAuth Client Applications through the UI resulted in an “IAMException AlationIAMRequest - 400” error. This error occurred during attempts to create or update OAuth clients, particularly when adjusting user information such as username and activity status. The fix ensures that users can now successfully manage OAuth Client Applications under Authentication settings. (AL-163384)

  • Upgraded the Java component of the Alation server (Amazon Corretto) to version 11.0.23.x.x to take advantage of bug fixes and enhanced functionality. (AL-125882)

  • A recent penetration test identified a potential vulnerability related to file deletion. When a user uploaded and subsequently deleted an attachment, the file was not properly removed from the server. We have addressed this issue to ensure that attachments are securely removed from the server upon deletion. (AL-135046)

  • Addressed a discrepancy between the Classic UI and New User Experience: the Last Updated and Created On columns, previously only available in the Column Organization widget on Folder pages in the Classic UI, are now also included in the New UI. (AL-161955)

Known Issue

The data dictionary preview user interface issues an internal API error when a custom field name included into the data dictionary file contains the colon special character :. In this case, users can validate the preview by downloading the preview report. The upload to the catalog still works as expected and the updates from the dictionary will be saved accordingly. (AL-137006)

PATCH RELEASE 2024.1.4.2

build 18.4.2.82056

Released July 26, 2024

Note

This patch release is available for Alation Cloud Service instances on the cloud-native architecture. This patch release doesn’t apply to customer-managed (on-premise) instances and Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

  • Fixed an issue in the Celery task logic, originally designed to fix a race condition affecting the updating of task statuses in job tables. Previously, if a task was not found in the database, an exception would trigger a retry of the update. However, if retries were initiated for other reasons, they would block the task. Additionally, a call within the metrics module occasionally triggered the same exception for different tasks, leading to the blocking of the current one. This issue caused some disruptions, such as scheduled queries being randomly skipped, with a message in the user interface that they were skipped due to technical difficulties. This patch resolves the problem and ensures more reliable task management. (AL-165523)

PATCH RELEASE 2024.1.4.1

build 18.4.1.80840

Released July 11, 2024

Note

This patch release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

  • Fixed an issue where users encountered a pop-up message Sorry, there was an error in Alation in the Classic user interface on various pages after a recent update. While users could close this message to continue working, it was associated with a 500 internal server error in the internal API. After the fix, the error no longer occurs, ensuring uninterrupted access to Alation. (AL-163785)

RELEASE 2024.1.4

build 18.4.0.80242

Released July 4, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

Compose

Configurable Timestamp Format for Query Results

Introduced a new configuration option, Timestamp Format, for selecting a timestamp format in Compose Settings > Results. Users can now select a preferred format for displaying timestamp types: either Standardized (RFC 3339) or Legacy. This configuration applies to only newly stored data. Previously stored result data will not be affected by changes to this setting. (AL-157186)

Compose SSO with AWS Databricks Unity Catalog

Alation now supports an additional configuration of Compose OAuth for Databricks Unity Catalog on AWS. Users can connect to Databricks Unity Catalog data sources in Compose via partner applications, no longer relying on personal access tokens. To configure, you’ll need to create a new app connection in Databricks under Account Console > Settings > App Connections and provide its properties in the data source settings. (Admin privileges are required to create a new app connection.) (AL-155129)

Load Query

Compose users now have the ability to load a query from a file by clicking Load Query on the Compose dropdown menu (top left of the page), selecting a text file from the local machine, and loading the content of the file to the Compose query editor to create a new query. (AL-158717)

Disable Excel Live Reports by Default

New Alation installations and instances upgraded to 2024.1.4 will have their default values set to off for Excel Live Report. This affects both Cloud Service and customer-managed instances. (AL-158713)

Query Forms Support Object Names with Double Quotes

For Query Forms, the WHERE clause will now allow column names with double quotes for non-ASCII characters. (AL-116577)

Search

Alation Cloud Service instances on the cloud-native architecture

We’re beginning to deploy Intelligent Search (or Semantic Search) to qualified Alation Cloud Service instances after their search index is migrated to Elastic Cloud Service. Information about Intelligent Search is available on Alation Community (requires a login). (AL-119678)

Document Hubs

Delete Document Hubs

Users now have the ability to delete a document hub, which removes it from the user interface and prevents the creation of objects within it. Deleting is only allowed for hubs that contain soft-deleted folders and documents. (AL-152300)

Homepage

Documents and Folders on the Homepage

Alation Cloud Service instances on the cloud-native architecture

Documents and folders can now be embedded on a custom homepage. For documents, the Description field will be displayed. For folders, the Description field and a list of up to five of its documents will be displayed. Documents and folders can only be embedded using the homepage editor in the New User Experience, but they will be visible when viewing the homepage in both the New and Classic User Experiences. (AL-156127, AL-161045)

Free-Text Field on the Homepage

Alation Cloud Service instances on the cloud-native architecture

You now have the option to add a free-text RTF field to your custom homepages. RTF fields can only be embedded using the homepage editor in the New User Experience, but the RTF field will be visible when viewing the homepage in both the New and Classic User Experiences. (AL-161045)

Core Platform

Postgres Upgrade to 16.2

Customer-managed instances

The internal Alation database (Postgres) was upgraded to version 16.2 to take advantage of new features and security enhancements. (AL-151431)

Support for TLS 1.3 Connections

Alation Cloud Service instances on the cloud-native architecture

Added support for TLS 1.3 connections to all external non-FIPS/commercial Application Load Balancers. Customers will now see TLS 1.3 as the primary connection type, with fallback to TLS 1.2 for clients that do not support TLS 1.3. (AL-122273)

Alation Analytics

Alation Analytics Postgres Upgrade to 16.2

Customer-managed instances

The Alation Alatytics PostgreSQL database was upgraded to version 16.2 to take advantage of new features and security enhancements. (AL-151431)

Custom Field Scrubbing on Alation Analytics

A new configuration flag, alation_analytics-v2.extract.enable.custom_field_formatter, is now available for enabling custom field scrubbing on the Alation Analytics V2 database. This feature converts all empty or placeholder values (such as "", {""}, and {}) to NULL. To activate this functionality, enable the flag using alation_conf (requires a restart of the Alation Analytics Celery service (alation_supervisor restart celery:celery-alationanalytics_0)). By default, this flag is disabled. (AL-143652)

Note

Alation Cloud Service customers can request server configuration changes through Alation Support.

Open Connector Framework

Simplified Installation and Upgrade for Connectors

Alation Cloud Service instances on the cloud-native architecture

We’ve made changes to the user experience of administrators managing connectors, helping them streamline their actions. The experience is enabled by default for all Cloud instances on the cloud-native architecture.

  • Simplified installation—Server admins can now search for and install connectors directly from the Connector Dashboard with a few clicks.

  • Connector upgrade indicator—The Connector Dashboard now displays an Upgrade available indicator when a new version of the connector that you have installed becomes available.

  • Simplified connector upgrade—Server admins can upgrade a connector with a few clicks, without the need to download a newer connector locally first. (AL-146347)

Manage Access to BI Folders and Reports

Previously, access permissions could only be set at the BI server level, with all the child BI objects inheriting these settings. With this update, permissions can also be set individually on BI folder and report objects. If no explicit permissions are defined, folders and reports will inherit settings from their parent object (BI server or BI folder). BI Admins can enable granular permissions by selecting Enable explicit permission checkbox on a folder or report, allowing for specific user access control. This feature is disabled by default and can be activated by a Server Admin using alation_conf with the parameter alation.granular_object_privacy.enabled.

Note

Alation Cloud Service customers can request server configuration changes through Alation Support.

Alation Anywhere for Tableau

Upload Tag Metadata to Tableau

For Tableau BI sources, the Metadata Upload job now transfers tag information from Alation to Tableau for workbooks, dashboards, and view objects. (AL-158283)

Authentication

Retrieve Data Source Credentials from AWS Secrets Manager via Alation Agent

Alation Cloud Service instances on the cloud-native architecture

This feature allows customers to keep their data source credentials within their Virtual Private Cloud so that they never traverse the Alation Cloud Service infrastructure. To achieve this, the Authentication Service Add-on can now be installed on an Alation Agent. This release offers support for AWS Secret Manager vaults with instance-based IAM roles. With Auth Service installed on the Agent, credentials stored in the vault won’t leave the perimeter of the company network when being retrieved by Alation for authenticating with a data source for extraction or query log ingestion. Moreover, these credentials will not be persisted in any of the Alation components including Alation Agent. For more information, see Integration with AWS Secrets Manager for Data Source Authentication Using OCF Connector. (AL-149546)

Note

This feature requires customers to upgrade to the new Alation Agent (1.7.2.4360) and a connector version that supports the feature. To find out if a specific connector supports this feature, see the documentation for the specific connector in the Open Connector Framework section.

Lineage

New Lineage Chart Rendering Algorithm

With this release, we introduced a new chart rendering algorithm for the Compound Layout in lineage charts, enhancing both performance and user experience:

  • Node and Path Limits—The system now loads the entire graph until it reaches a hard limit of 2,000 nodes and 50,000 paths. If these limits are reached:

    • Use the Impact Analysis reports to export a complete list

    • Apply the Dataflow filter to hide certain elements

    • Navigate to another object within the same lineage graph to view its lineage

  • Deprecated Features—The features known as horizontal truncation (or node pagination) and vertical truncation (or collection nodes) have been deprecated. The collection node dialog has been removed from the user interface.

  • Bug Fixes—Ensured each data source group in the Compound Layout has one cluster on the chart and resolved an issue with lineage path highlighting in looping graphs. (AL-150882)

Governance

Enable De-duplication of Snowflake Tags

By default, Snowflake tags from a common database or schema that are used across other databases or schemas are duplicated in Alation for each database or schema where those tags are applied. This creates duplicate tag custom fields and necessitates admins to add all the duplicated fields to templates for usage. We’ve added a new feature flag alation.feature_flags.enable_centralized_tags_deduplication that enables de-duplication of common tags by creating only a single copy of the custom field for each, even though they are applied across different databases or schemas. The flag is disabled by default. (AL-157018)

Clear Values Using the Data Dictionary

While uploading a data dictionary in the user interface, users can now choose between replacing existing values with skipping empty values or with applying empty values from the dictionary. (AL-156670)

Alation API

Find a short summary of the Alation API 2024.1.4 changes below. More details are available in the 2024.1.4 API Release Notes on Alation’s Developer Portal.

Allie SDK

Alation has released an open-source Python SDK—Allie SDK—to better enable working with Alation APIs. Check out the SDK documentation and the Allie SDK public GitHub repository.

Documents API

You can now specify which fields should be returned when getting documents by providing the query parameter values and supplying a comma-delimited list of fields that should be returned. (AL-138823)

Folders API

You can now specify which fields should be returned when getting folders by providing the query parameter values and supplying a comma-delimited list of fields that should be returned. (AL-156335, AL-162492)

Members Permission and Privacy Settings APIs

We’ve added BI folders and BI reports as supported object types for the Members Permission API and Privacy Settings API, so you can now control access to these object types using Alation APIs. (AL-159617)

Improvements

  • The feature flag 3rd Party Metrics has been removed from the Admin Settings > Feature Configuration page. (AL-149013)

  • Upgraded the Java component of the Alation server to a newer version (amazon-corretto-11.0.23.x.x) to take advantage of bug fixes and improved security features. (AL-125882)

  • Upgraded the Redis Python package from version 3.x to 5.x.x. This update provides enhancements and new features that improve performance and compatibility. Due to compatibility issues, the django-redis-cache library, which requires a Redis version less than 4.0, has been replaced with django-redis. It offers similar functionality as django-redis-cache and ensures better support and updates. (AL-157423)

  • Addressed performance bottlenecks in the LMSTitleDescription serializer, which previously accounted for about 50% of the processing time when creating terms with a description that includes @-mentions. This optimization significantly speeds up the links endpoint of the internal server API, used for hydrating the Document tables on Folder pages. (AL-155303)

Bug Fixes

Articles, Terms, and Documents

  • Addressed an issue where users experienced a 504 Gateway Timeout error when accessing articles from the Classic user interface. Fixed by improving the performance of the internal article API, resulting in faster response times and enhanced efficiency when retrieving articles. (AL-160021)

  • Resolved an issue in the Classic user interface where attempting to remove or a selected template on the term or document catalog page(by selecting the topmost option in the template selector dropdown previously resulted in a 500 error. Although the error page was displayed, if the user refreshed the page, the template change was correctly persisted.This fix ensures that the template can be updated without the need to refresh the page for the change to take effect. (AL-158942)

  • Fixed an issue where article picker and multi-picker fields were not displayed on the catalog pages of article objects after the Document Hubs 1: Setup and Document Hubs 2: Show Glossary Hub feature flags were disabled. These fields are now consistently visible regardless of the status of these feature flags. (AL-160185)

  • Fixed an issue in the Classic user interface where Stewards and Composers could edit picker fields for terms, despite editing restrictions set at the glossary level. This fix ensures that editing restrictions are properly enforced. (AL-159727)

  • Addressed an issue where the download of glossary terms would stall when using the Bulk Utility if the terms were associated with a domain hierarchy. For instance, downloads previously halted if a term was linked to both a parent Domain and its child subdomain. This fix ensures that downloads proceed without interruption. (AL-152126)

  • Fixed a bug where articles failed to load properly due to an issue with the URI transformation in version 2023.3.4 (the / symbol was incorrectly converted into ASCII.) This fix ensures that all special characters in article titles are handled correctly. (AL-149344)

Bulk Utility

  • Fixed an issue where tags were not uploaded when using the Bulk Utility to add multiple tags to a term. This problem persisted regardless of whether users tried to upload a single tag or multiple tags, or whether they were creating or updating tags. After this fix, new tags can be added via the Bulk Utility. (AL-155181)

Governance Fixes

  • Fixed an issue where the option Reviewers of this step can bypass this entire workflow in workflow settings no longer worked. Now, a workflow can be bypassed if the user is a reviewer, and the workflow has this reviewer bypass flag enabled. (AL-156067)

  • Resolved a display issue on the main Policy Center page where suspended users appeared in the Stewards column. The Stewards column no longer shows suspended stewards, only displaying active stewards associated with policies and policy groups. (AL-153864)

  • Fixed an issue where users without the necessary permissions could view private data sources in the Titles & Descriptions Chart. Private data sources are now excluded from the list in this view, reinforcing data privacy and access controls. (AL-158729)

Other Fixes

  • Updated the New User Experience to match Classic, allowing all users to resolve or reopen conversations, not just the creators. This fix ensures the Resolve conversation or Reopen conversation button is available to everyone. (AL-157619)

  • Fixed a bug in the homepage editor (Admin Settings > Homepage Customization) where users could save the homepage after selecting Image as the banner background mode without actually uploading an image. Now, if a user chooses to set a banner image but does not upload one, an error message will appear, and saving the homepage will be blocked until an image is uploaded. (AL-141037)

PATCH RELEASE 2024.1.3.2

build 18.3.2.79136

Released June 20, 2024

Note

This patch release is available for customer-managed (on-premise) instances and Alation Cloud Service instances on the cloud-native architecture.

This patch isn’t available for Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

  • Fixed the issue of slow response times when checking job statuses. Previously, viewing the data source settings page was delayed by one to two seconds for each job in the Job History table due to checks on the status of Celery processes. To address this, we have removed the check for the Celery process from internal Celery tasks. This update is intended to improve response times and overall performance when accessing job statuses. Users should now experience faster loading times on the data source settings page. (AL-161486)

PATCH RELEASE 2024.1.3.1

build 18.3.1.78990

Released June 18, 2024

Note

This patch release is available for customer-managed (on-premise) instances and Alation Cloud Service instances on the cloud-native architecture.

This patch isn’t available for Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

  • Reverted the Django library update introduced in 2024.1.3 that had altered the handling of SAML attributes without an explicit NameFormat. The change in 2024.1.3 had caused login failures for customers with Microsoft Entra ID or Active Directory Federation Service (AD FS) as identity providers, where the authentication application settings did not include the NameFormat property (undefined by default). We have reverted this change to restore the previous behavior. This gives customers time to update their Entra ID or AD FS configurations accordingly. If your IdP is Entra ID or AD FS, please ensure that your IdP configurations are updated as recommended. (AL-161311)

  • Implemented a new response message for SAML replay attacks to enhance security. Any repeated use of a SAML response assertion will now trigger the message: Received SAMLResponse assertion has already been used. Additionally, following the upgrade to version 2023.1.3.1, all active user sessions that were established before the upgrade will be logged out upon the first attempt to access the application after the upgrade. This measure ensures that all sessions start under the latest security and application updates. (AL-161311)

  • Fixed a bug where the Custom Field Permissions page experienced significant performance issues, with the page becoming unresponsive or taking more than ten seconds to respond when attempting to edit permissions. The fix optimizes the loading process for the Custom Field Permissions page, reducing response times and resolving issues of unresponsiveness. (AL-157895)

  • Fixed an issue where Compose queries would continue running indefinitely in cases where the target table didn’t exist. Implemented a fix to terminate query service connections immediately if a table does not exist. This fix also resolves the issue of queries with typos continuing indefinitely. (AL-157061)

RELEASE 2024.1.3

build 18.3.0.76841

Released June 3, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

Core Platform

The Django Component Upgrade

We have upgraded our software infrastructure to Django 4.2, replacing the end-of-support Django 2.x. This new version contains enhancements that improve the robustness, security, and performance of the Alation application:

  • Improved WebSocket support—Django 4.2 enhances real-time communication capabilities for SaaS applications through improved WebSocket support.

  • Async views and tasks—This version allows for more efficient management of long-running operations through asynchronous views and tasks.

  • Async StreamingHttpResponse—With ASGI (Asynchronous Server Gateway Interface), this version enables the StreamingHttpResponse class to work with asynchronous iterators, improving the handling of large data streams. (AL-141005)

Important

ACTION REQUIRED

This upgrade impacts the SAML authentication configuration in Microsoft Entra ID (former Azure Active Directory) and Active Directory Federation Service (AD FS) as identity providers (IdPs). The default value of the Name Format property of SAML attributes on the Alation server has been changed to unspecified from the previous value of URI. This creates a requirement to explicitly set the Name Format property to URI or Basic in the SAML application in Microsoft Entra ID, as it also defaults to Omitted (default).

Customers who use Entra ID or AD FS as IdPs are asked to perform an additional configuration in their SAML application by July 30, 2024. Refer to the advisory Important Advisory - SAML authentication failure with Azure/Microsoft IDP and Microsoft ADFS after upgrade to 2024.1.3 on Alation Community for specific details.

New User Experience

Alation Cloud Service on Cloud-Native Architecture

  • In New User Experience, the column organization widget was added to the Folder and Documents tables on catalog pages of Document Hub objects and to the Documents table on catalog pages of Folder objects. (AL-141392)

  • The ALLIE AI Suggested Stewards feature is now available in New User Experience. (AL-132059)

  • Added more functionality to the alation_conf flag alation.feature_flags.hide_articles that can be used by customers who have completed the transition from articles to Document Hubs. Now, it also hides articles from the Recently Visited section of the homepage in New User Experience. The default value is False (no change in behavior). Alation Cloud Customers can request this change through Alation Support. (AL-146401)

Governance

Folder Object Type as a Field Option

Added the folder object type as a selectable option for Object Set and Reference custom field types. Note that it will appear as “Glossary” if the Document Hubs 3 feature flag is disabled. (AL-152523)

The Mentioned On Field

Added a new built-in field Mentions that lists all Documents and Folders that mention a specific catalog object. On the sidebar of a catalog page or when being added via Customize Catalog, it appears with a title of Mentions. When the field is located on the main panel of a template, it will display as two separate tables, one titled Mentioned on Folders and the other Mentioned on Documents. (AL-152316)

Data Dictionary

In release 2024.1.3, uploading a data dictionary file with empty values via the user interface will not reset the existing catalog field values. This is a change from the previous release (2024.1.2), where the catalog fields would be reset if the corresponding field in the uploaded data dictionary file was blank. (AL-155567)

Lineage

Support for Cross-Source Lineage with Direct Lineage Extraction

Cross-source lineage configuration now supports data sources with direct lineage extraction (Databricks Unity Catalog, SAP HANA). This addition follows the capabilities introduced in the 2023.1.6 release, which enabled cross-source lineage for data sources connected via the same host and port and extends these capabilities to include the direct lineage pipeline. (AL-148534)

Alation Public APIs

Find a short summary of the Alation API changes below. More details are available in the 2024.1.3 API Release Notes on Alation’s Dev Portal.

Folder API

We’ve added a new Folders API for creating, retrieving, updating, and deleting folder object types. (AL-152751)

Custom Field Values API

The Custom Field Values API now supports the glossary_v3 object type, which refers to both glossary and folder objects when the Document Hubs feature is enabled. (AL-155886)

Search API

The Search API now supports the glossary_v3 object type (can refer to both glossary and folder object types), and the glossary_term object type (can refer to both document and term object types). (AL-155886)

Data Dictionary API

The Data Dictionary API’s Upload a data dictionary endpoint now has parameters that control the behavior of blank values in the data dictionary. You can ignore blank values or cause them to reset the value of the field in the catalog. (AL-155567)

Documents API and Terms API

  • We updated the sample response for the GET method to be more accurate. The sample response is now an array of documents, and the otype and is_public items have been removed, as they are not returned when getting documents. (AL-149027)

  • Added throttling configurability for the Document API at the Celery component level to prevent API performance issues. The following alation_conf parameters were added:

    • alation.api.throttling.document_public_apis.read (defaults to null, which means “no rate limit”)

    • alation.api.throttling.document_public_apis.write (defaults to 100/minute) (AL-151897)

OCF Connectors

Updates to the Open Connector Framework (OCF) connectors are released every week. See OCF Connector Release Notes for information on the updates in the month of May 2024. A few May highlights are:

The Talend OCF Connector Public Preview

The OCF connector for Talend is now available on Connector Hub. The connector is compatible with Alation version 2023.3 and higher. (AL-143292)

Enhance the Parser to Handle Parameterized Values

Previously, users reported issues with extracting the connection and lineage information for Power BI datasets due to limitations in the connector logic in handling API responses that included parameterized values within expressions. We made enhancements to the connector to ensure it accurately handles parameterized values. Additionally, a fix was implemented to address an out-of-memory error caused by the repeated replacement of commonly used parameter names leading to excessive string expansion. Another fix ensures that table names in queries with special characters are correctly quoted, improving the resolution of nodes for cross-source lineage and enhancing the parser’s accuracy and reliability. (AL-153882)

Resolving Unqualified Column Names in Query Log Ingestion

Customer-managed (on-premise) instances and Alation Cloud Service instances that were not migrated to the cloud-native architecture

Query log ingestion (QLI) for Snowflake OCF data sources now supports the resolution of unqualified column names from query history. An unqualified column name is a reference to a column that does not include parent objects. Previously, joins and filters were not captured if the column names weren’t qualified with table aliases in the join and filter conditions. The change applies to customer-managed (on-premise) instances and Alation Cloud Service instances that were not migrated to the cloud-native architecture. It will be implemented on Alation Cloud Service instances on the cloud-native architecture in a future release. (AL-154123)

Known Issues

  • The 2024.1.3 release raised the default iteration count of the password hasher for user CSV files from 390,000 to 600,000. This adjustment has extended the processing time for password hashes in CSV files, resulting in UI request timeouts when bulk-uploading users from CSV. Despite the UI timeout, the process continues in the backend, and if the data is correct, the updates will still be completed. For information on the workaround, please refer to this Support Knowledge Base article. (AL-158623)

  • In Classic UI, removing the selected template by selecting the topmost option in the template selector dropdown on a term or document catalog pages results in the 500 error in the user interface. If the user refreshes the page, the change is saved and the error is resolved. (AL-158942)

  • Creating or updating an OAuth client application by Server Admins through the user interface in Admin Settings > Server Admin > Authentication > OAuth Client Applications fails with the error IAMException AlationIAMRequest - 400, preventing the creation or modification of any app used to generate authentication tokens. If you need to create or update an app, please contact Alation Support. The authentication tokens generated by existing applications remain unaffected. (AL-163384) (FIXED in version 2024.1.5)

Bug Fixes

Document Hubs

  • Previously, all custom fields from document templates, irrespective of the Hub, were displayed in the Column Organization widget for the Document table on a Document Hub’s catalog page. After this fix, the list of columns in Column Organization becomes specific to each Hub. (AL-148730)

  • Previously, documents and folders from unpublished Document Hubs appeared in Alation Search results. After this fix, documents and folders from unpublished Document Hubs won’t be discoverable through Search. (AL-154062)

Governance

  • Previously, the stewards listed on the Policy Center page didn’t match the actual stewards on policy group objects and suspended stewards were displayed. After this fix, suspended users are no longer visible in the list of policies, the row access policy links table, and the dynamic data masking policy links table. (AL-157793)

  • Fixed an issue where users with roles other than Server Admin couldn’t edit the Picker custom fields which had no value set. The issue occurred when updating a custom field on an object that a user had access to, but this object had a corresponding synced object established via a manual catalog set that the current user couldn’t access. As a result of this fix, when a user updates any field value on a catalog object that falls under manual catalog sets metadata sync settings, the system automatically updates the same field value on equivalent objects. This system action overrides any object-level permissions on these equivalent objects, ensuring the field value is updated across all related items. Additionally, the field edit history will record the details of the actual user who made the update, both on the edited object and its equivalent objects. (AL-142427)

  • Previously, the Bulk Utility encountered a 500 error and failed to open if there was an object template with the same name as one of the article templates. To fix the issue, we implemented an object type filter in Bulk Utility to ensure that only article templates are considered. (AL-155973)

  • When using Bulk Utility to download glossary terms, the download became unavailable if any of the terms had a domain hierarchy associated with them. For example, if a parent domain and a child subdomain were both associated with the term, then the download was unsuccessful. The issue has been fixed to ensure reliable functionality in downloading terms. (AL-152126)

Alation Analytics

  • An ETL sub-job for Alation Analytics, DatasourcesTransformJob, failed in version 2024.1.1 due to an error Current transaction is aborted…. This issue was traced back to handling null values for URIs of OCF data sources. The fix ensures that the URI field adheres to the NOT-NULL constraint, stabilizing the ETL process. (AL-155188)

  • Extended the consumer_timeout setting for RabbitMQ to six hours. This change ensures that longer-running tasks can complete without interruption due to premature timeouts. (AL-154677)

  • Fixed the error Duplicate key value violates unique constraint ‘bi_connection_column_fp_key’ reported in the Alation Analytics ETL process. The resolution involves removing the unique key constraint for the fp column in the bi_connection and bi_connection_column tables to prevent such conflicts during ETL operations. (AL-150501)

Other Fixes

  • Fixed an issue where for data sources supporting direct lineage extraction, the cross-source lineage configuration resulted in temporary objects on lineage charts. Extended the support for cross-source lineage to data sources with direct lineage extraction. (AL-148534)

  • For the Power BI OCF connector, fixed an issue where semantic models (renamed datasets) were missing connections after extraction. Fixed in connector version 2.2.1 (AL-147313)

  • Fixed an issue where query log ingestion (QLI) didn’t retrieve the join and filter information if column names were not qualified with parent table names. Fixed by extending QLI support to unqualified column names for on-premise instances and Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture. (AL-141950)

PATCH RELEASE 2024.1.2.2

build 18.2.2.78391

Released June 17, 2024

Note

This patch release is available for customer-managed (on-premise) instances and Alation Cloud Service instances on the cloud-native architecture. The patch will be deployed for specific customers as required.

(This patch release isn’t available for Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.)

  • Fixed a bug where the Custom Field Permissions page experienced significant performance issues, with the page becoming unresponsive or taking more than ten seconds to respond when attempting to edit permissions. The fix optimizes the loading process for the Custom Field Permissions page, reducing response times and resolving issues of unresponsiveness. (AL-157895)

  • Fixed an issue where Compose queries would continue running indefinitely in cases where the target table didn’t exist. Implemented a fix to terminate query service connections immediately if a table does not exist. This fix also resolves the issue of queries with typos continuing indefinitely. (AL-157061)

  • Introduced a new configuration option, Timestamp Format, for timestamp formatting in Compose settings. Users can now select the preferred format for displaying timestamp types: either Standardized (RFC 3339) or Legacy. This configuration applies to only newly stored data. Previously stored result data will not be affected by changes to this setting. (AL-153948, AL-153947, AL-157186)

PATCH RELEASE 2024.1.2.1

build 18.2.0.73651

Alation Analytics for Alation Cloud build 1.39.1.2516

Released May 9, 2024

Note

This patch release is available for Alation Cloud Service instances on cloud-native architecture.

  • Extended the consumer_timeout setting for RabbitMQ to six hours. This change ensures that longer-running tasks can complete without interruption due to premature timeouts. (AL-154677)

RELEASE 2024.1.2

build 18.2.0.73651

Released May 1, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

New User Experience (General Availability)

Alation Cloud Service on Cloud-Native Architecture

We’ve added more functionality to the General Availability (GA) release of the New User Experience arriving just prior to the upcoming 1.0 release. The following list captures a summary of additions since the Public Preview IV release. All additions are available on Alation Cloud Service on Cloud-Native Architecture, unless also specified for other instance types.

  • Query page: the Run Query feature

  • Policy object pages

  • Lineage: compound layout view

  • Manual lineage curation

  • Governance landing page: the Curate section (in Classic but linked from the new user interface)

  • My Inbox

  • User page

  • Alation add-ons: added Data Catalog Maturity Assessment (DCMA)

  • BI pages

    • Customizable BI layouts (also available on customer-managed (on-premise) instances and Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture)

    • Granular labels on BI report and folder catalog object types (site, project, report, dashboard, page, view, and others)

  • Customizable table layouts for data sources

Note

As we move into GA, you no longer need beta language in your contract or an addition to the whitelist to enable the New User Experience.

Alation Agent

Notifications for Alation Agent’s loss of connection and certificate expirations are now enabled by default for Server Admins. Existing preferences for Server Admins who have previously configured these notifications will remain unchanged. These notifications were previously opt-in by default. The change to opt-out by default aims to improve awareness and response to these important system events. Server Admins can adjust the notification settings under the Notifications tab on the Account Settings page. (AL-153656)

See Agent Release Notes for the full release notes for Alation Agent.

Authentication

Inactive User Suspension

Users who have not logged into the Alation catalog for an extended period can now be automatically suspended:

  • Non-FedRAMP instances—By default, this setting is disabled. To enable, navigate to Admin Settings > Server Admin > Authentication, find the Inactive User Suspension Setting section, and set a value between 30 and 365 days. To deactivate, reset the value to 0.

  • FedRAMP instances—Suspension is enabled by default at 90 days and cannot be altered or disabled. (AL-124754)

Lineage

Column-Level Lineage for Temporary Tables

Alation Cloud Service on Cloud-Native Architecture

Added support of column-level lineage for temporary (temp) tables. Temp tables are usually created with CREATE TABLE queries using either the TRANSIENT or TEMPORARY keyword. Such queries can also be generated by tools like dbt. Tables that are created and dropped in the same session are classified as temp tables as well. On lineage charts, column-level lineage will be available for a temp table if column names are explicitly mentioned in the query. If columns are not explicitly mentioned (queries like SELECT *), then column-level lineage will only be available for the CREATE TABLE AS SELECT * queries.

Column-level lineage from the ALTER TABLE queries is supported if these queries are executed during the same session as the CREATE TABLE queries. (AL-126794).

Column-Level Lineage for Custom SQL

Alation now generates column-level lineage between an RDBMS and a BI source when data was imported into the BI server with custom SQL queries. The feature applies to RDBMS sources with table and column-level lineage support. (Information about column-level lineage support is available in the Support Matrix.)

  • Custom SQL query content can be found on the DataSource Connection tab of BI objects. The feature has been currently certified for Tableau and Power BI OCF BI sources.

  • A limitation: column-level lineage is not displayed correctly for custom SQL in scenarios involving column name changes on the BI server side. (AL-147324)

Filtering Dataflows (Public Preview)

You now have the ability to filter dataflow objects out of lineage charts. This feature is intended to reduce clutter that may be caused by recurring dataflows due to automated intake from query log ingestion. To access dataflows when they are filtered out of the chart, you can trace the line between nodes on the chart and click to view dataflow content on the bottom and side panes. To access all dataflows for a particular node connection, you can alternatively use the Impact Analysis lineage reports filtering them by dataflow objects. This feature is currently in Public Preview with a Beta “chip” in the user interface. (AL-140791)

Lineage Settings Tab for BI Sources (Public Preview)

Introduced a new Lineage Settings tab in the settings of OCF BI sources. BI Source Admins can now configure cross-system lineage directly within BI source settings, improving usability and streamlining the setup process. The configuration doesn’t require manual entry of the host:port information. Instead, admins can select upstream RDBMS sources by name. This feature is currently in Public Preview with a corresponding Beta “chip”. The enhanced lineage settings have been certified with Tableau, Power BI, and MicroStrategy OCF BI sources in this release. (AL-145817)

Governance

Introduced a new feature flag alation.datalake_tags.centralized_tag_schemas to manage an allowed list of repositories for tag definitions in data lakes. Acceptable values for this list are the Fully Qualified Domain Names (FQDNs) of schemas that organize tags, such as SNOWFLAKE.CORE (default). When enabled in conjunction with the feature flag alation.feature_flags.enable_snowflake_tag_filtering_by_schema, this feature flag allows tags from specific schemas to be visible on catalog pages, regardless of whether or not the tag’s schema matches the object’s schema. (AL-145407)

Compose

Query Result Streaming

It is now possible to execute queries in Compose without storing query results on the Alation server or other Alation-managed architectures. This feature is intended to facilitate cloud migration strategies for customers with strict security policies that prohibit storage of query result data on Alation systems although the data does transit through these systems to be displayed in the user interface.

Enablement

This feature cannot be activated via the user interface and requires involvement from Alation Support for Alation Cloud Service instances or backend access for on-premise instances. Additionally, scripts must be executed to erase any previously stored query data.

With Query Results Streaming enabled:

  • Certain functionality will be disabled, such as query scheduling, result sharing and downloads, and preserving results indefinitely.

    Note

    In this release, the Run & Export feature for queries is also disabled, but support for enabling exports independently of enabling result streaming is planned for a future release.

  • Query results are accessible only within the Compose user interface or on query form pages immediately after query execution. They will be lost after refreshing the browser or after a re-login. (AL-148201)

Alation Analytics

Alation Cloud Service on Cloud-Native Architecture

Data Quality Improvements

  • As part of the Connect Third-Party BI Tools to Alation Analytics V2 feature, the ordering of custom fields for all object types will now be in ascending order of the field_id field to ensure that custom field creation happens in the same order on PostgreSQL and Snowflake. Previously, when multiple custom fields had the same name but different data types, they were created in different order on PostgreSQL and Snowflake, causing some data inconsistency issues. (AL-145364)

ETL Stability Improvements

The enhanced Alation Analytics pipeline now utilizes the system disk (STORAGE) in conjunction with System Memory (RAM) for efficient ETL operations. This reduces the likelihood of Out Of Memory (OOM) errors, particularly when handling vast amounts of data. As a result, you can expect significant improvement of ETL stability, especially when regularly ingesting large volumes of objects. (AL-152601)

Logical Metadata Services

Alation Cloud Service on Cloud-Native Architecture

For Alation Cloud Service customers in Asia Pacific, Canada, and Europe, we are upgrading the underlying infrastructure for Logical Metadata Services (LMS) to improve performance and reliability. LMS is the service that powers titles, descriptions, and custom field values in the Alation catalog. The upgrade will make Custom Field Value API response times six times faster, reduce page load times for glossary terms by over 50%, and decrease the occurrence of 504 gateway timeouts when loading glossary terms and catalog pages. This change also lays the groundwork for future performance and scalability improvements across the Alation platform. We will upgrade customers in the United States in the near future.

As part of this improvement, the Upload Logical Metadata API will be deprecated in the 2024.3 (September) release of Alation. After that, metadata updated with this API will not be displayed correctly in the catalog. Please visit the Alation Developer Portal for a detailed list of replacement APIs. (AL-148679)

Alation Public APIs

Find a short summary of the Alation API changes below. More details are available in the 2024.1.2 API Release Notes on Alation’s Dev Portal.

NoSQL API

The new NoSQL API allows creating, updating, and deleting NoSQL virtual data sources. The API relies on the optimized ingestion pipeline for NoSQL objects that enables uploading metadata of complex nested data types. (AL-148931)

  • Known Issue: Using object name, path, or ID as request parameters in the Delete endpoint incorrectly deletes data from attributes with the same name under different schemas. The deletion should only affect the specified object and its child objects. (AL-155752)

Homepage Preferences API

The new Homepage Preferences API lets Catalog Admins and Server Admins set the homepage preference for multiple groups that are already assigned to a specific homepage. (AL-143659)

BI Source API

Added a new endpoint for migrating curation data after migrating a BI server from a native connector to an OCF connector. (AL-146464)

Custom Field Values API

The way you can get the Job ID of the PUT endpoint of the Custom Field Values Async API has changed. When updating custom field values, this API now runs two background jobs. The API response returns the ID of the first job. Use that ID to query the Jobs API. The response from the Jobs API returns the ID of the second job. (AL-147696)

Queries API

When creating a query via API, you can now set additional fields:

  • Description

  • Domains

  • Tags (AL-139445)

Disable Public Unauthenticated APIs

Alation Cloud Service on Cloud-Native Architecture

Addressed a potential vulnerability in seven Alation Cloud Service monitoring APIs that allowed unauthenticated read access of the status information from the public internet. The following insecure monitoring endpoints are no longer publicly accessible:

  • monitor/num_active_users

  • monitor/backup_status

  • monitor/hostname

  • monitor/replication

  • monitor/aav2/status

  • integration/v1/monitor/services_healthcheck

  • integration/v1/monitor/admin_alert_checks (AL-148305)

OCF Connectors

Updates to the Open Connector Framework (OCF) connectors are released every week. See OCF Connector Release Notes for information on the updates in the month of April 2024. A few April highlights are:

New Connectors

Connector Settings User Experience Redesign

We’ve completely redesigned the user experience for Alation admins setting up OCF sources for the connectors listed below. Now, the configuration parameters are organized by steps, with new validation features added, such as testing connection and permissions to access the metadata. The extraction process has been fortified, and more clarifications and troubleshooting tips are now available in the user interface in case of extraction or ingestion errors.

Simplified Migration from Native to OCF Tableau BI Sources

For Tableau sources, the native to OCF migration will run as a downstream job for the first MDE after migration. Migration can also be done using API (BI Source API). (AL-146464)

Report Fields Extraction and Column-Level Lineage For Power BI Scanner OCF BI Sources

Starting in 2024.1.2 and connector version 2.2.0, the OCF connector for Power BI supports extraction and cataloging of report fields (dimensions) and column-level lineage. A Server Admin can enable this functionality with a dedicated feature flag Automatically extracts Column Level Lineage from Power BI Scanner data source in Admin Settings > Server Admin > Feature Configuration. (AL-132853)

Bug Fixes

  • Fixed an issue where exported term search results were missing custom fields. Now, exports will include all columns displayed in the search result table view. (AL-154073)

  • Resolved an intermittent issue with the Search functionality on the Queries > History tab for schema objects’ catalog pages. (AL-149787)

  • Previously, the Explain feature in Compose wasn’t functioning properly for multiple OCF connectors after enabling the Query Service feature flag. The issue was fixed and connector certification was performed to ensure that the Explain feature now functions as expected. (AL-143362)

  • Resolved an issue where data dictionary uploads were failing after upgrading to version 2024.1. A content validation error in the data dictionary preview UI was corrected, allowing the upload of files that contain specific Unicode characters. (AL-152621)

  • Resolved a bug in workflows where approving multiple changes for the same object led to only one change being applied. The system now supports multiple parallel updates to custom fields through workflow approvals. (AL-143259)

  • Resolved an issue where People Set custom field values were not included in files downloaded using the Bulk Utility. (AL-150153)

  • Resolved a Bulk Utility issue where a term’s Last Updated property was not updating correctly on upload. The ts_updated property for terms will now be correctly updated after the Bulk Utility upload. (AL-148881)

  • Resolved a bug affecting automated backups after upgrading to version 2023.3. Previously, backups did not execute when unfinished jobs persisted in jobs_job. Added logic now identifies and invalidates stuck jobs; if a job is active for over a day without a corresponding Linux process, it’s marked as failed, allowing new backup jobs to proceed. If these conditions aren’t met, existing jobs continue, and new jobs are postponed. (AL-149526)

  • Resolved a problem where notifications for failed jobs due to service or server outage were significantly delayed. Notifications will now be issued within 30 minutes of a job failure. (AL-148721)

  • Resolved a problem with the dbt add-on connector where the absence of the compiled_code field in the dbt manifest file caused lineage streaming failures for a Snowflake data source. The connector now handles cases where this field is empty. (AL-147412)

  • Resolved a bug in the Databricks Unity Catalog OCF connector where extraction failed due to the error A name’s original value can’t be null or the empty string. The lineage extraction queries have been updated to exclude empty column and table names from the result set. Fixed in connector version 2.2.2 (AL-141050, AL-140061)

  • Fixed an issue with the 3rd party BI access to Alation Analytics on Snowflake, where counts for schemas in PostgreSQL and Snowflake for the rdbms_datasources, rdbms_schemas, rdbms_tables, rdbms_columns tables were different. The bug was caused by the NULL records on join columns. We are now using the COALESCE function along with default values on join columns in Snowflake to prevent the NULL values. (AL-145893)

  • Now, during the installation of Alation Analytics, after installing the required Python packages using the Python Package Installer (pip), it’s removed from the base image to prevent a potential security vulnerability. (AL-145515)

  • Resolved a replication issue for the Alation Analytics table rdbms_schemas to Snowflake. This problem had multiple causes, including NULL values coexisting with other values in array column types and NULL records in join columns. All identified issues have been successfully addressed. (AL-140093)

  • Resolved an issue where the user’s display name during the LDAP login flow was not updated when first_name or last_name gets updated in the LDAP server (AL-87975).

RELEASE 2024.1.1

build 18.1.0.70857

Released April 4, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

New User Experience (Public Preview IV)

Alation Cloud Service on Cloud-Native Architecture

​The New User Experience has now moved into Public Preview IV. The following list captures a summary of the additional features and pages currently available in Public Preview IV:

  • Data source catalog details page

  • Schema catalog details page

  • Table catalog details page

  • Column catalog details page

  • Catalog navigation

  • Data source hub

  • Domains Hub and details

  • Homepage basic features and Admin guided user interface (GUI)

  • Query page (with functionality to run the query in the catalog page)

  • BI hub and all detail pages

  • The dynamic template layout in the Admin Settings > Customize Catalog > Custom Templates for the BI object types:

    • BI datasource column

    • BI report column

    • BI datasource

    • BI server

    • BI report

    Note

    The dynamic template layout will be available for the Classic User Experience and the New User Experience after the update to the 2024.1.1 release. All cloud-native environments (New User Experience environments) should be updated to this release by May 3, 2024.

  • The ability for Server Admins to make the New User Experience default if desired

  • Search (list and table view, saved search, export results)

    Note

    The More filter experience and other improvements are coming in the 1.0 release of the New User Experience.

  • Conversations

  • Tags object page

  • Backreference group move-ability in templates

  • Lineage (simple layout view)

    • The compound layout view and manual lineage are coming soon

  • Dataflow object page

  • Document hub basics

  • Alation add-ons

    Note

    To participate in the New User Experience Public Preview, you must have Beta language contained in your contract or have a Beta addendum signed. Reach out to your Alation account manager for more information if you would like to access the Preview.

Alation Agent

Alation Cloud Service on Cloud-Native Architecture

For BI sources using Alation Agent in polling mode, an informational message has been added to the Test Connection dialog to inform admin users that the operation may require additional time to complete. (AL-141782)

Domains

Filtering Catalog Content Based on Domain: Alation now allows for a more personalized catalog experience with filtering the left-hand navigation based on users’ domains. The feature is intended to provide quicker access to important information by filtering out data unrelated to domains. (AL-121118)

Governance

Create Multiple Table Templates

We have introduced the capability to create multiple templates for the table object type in Customize Catalog > Custom Templates. By default, this feature is disabled but can be activated with the feature flag Enable creation of multiple templates for table catalog pages in Admin Settings > Feature Configuration. Once enabled, both Catalog Admins and Server Admins can create multiple templates on the Table tab under Customize Catalog > Custom Templates and associate them with specific data sources. While a single template can be associated with multiple data sources, each data source is limited to one template. (AL-137912)

Redesign of the Govern Landing Page

In the Apps menu, the Govern tile was renamed to Curate & Govern. The landing page for Curate & Govern was redesigned and now includes tiles for each part of the Governance App. This change has a dedicated flag in alation_conf: enable_governance_home_page_redesign (enabled by default). (AL-147712)

Data Dictionary

Handling of blanks and invalid values: In a downloaded data dictionary, invalid cells are now represented by underscores, replacing the “N/A” and “null” values from the legacy process. In an uploaded data dictionary, blank values now trigger the “unset” operation for existing entries. The Replace Existing Values option also supports the “unset” operation—blank values in the upload will overwrite existing ones if selected; if not selected, existing values will be preserved. (AL-132472)

Lineage

Business Titles in the Compound Layout View: You can now choose to display business titles inside nodes on lineage charts instead of technical names. The option is located in the Filters dialog accessible on the top left of the chart. Business titles are sourced from the Title catalog field of an object. For objects with no Title, their technical names will be displayed. (AL-146600)

Alation Analytics

  • Added the table documenthubs to the Alation Analytics database. Users can now retrieve document hub information and categorize and analyze their document and folder objects by specific document hubs. (AL-150170)

  • Added a new document_hub_id property to the terms and glossaries tables in the Alation Analytics database. Alation Analytics users can leverage this field when retrieving information on a document hub (number of folders, documents, document templates, etc.) (AL-142164, AL-147417)

  • The column rdbms_datasources.uri in the Alation Analytics database will now be populated for OCF data sources. This change specifically accommodates the URI for the GBQ and Hive data sources. (AL-125234, AL-151676)

Alation Public APIs

Find a short summary of the Alation API changes in 2024.1.1 below. More details are available in the 2024.1.1 API Release Notes on Alation’s Dev Portal.

New My Domains API

The new My Domains API allows Catalog and Server Admins to assign users and groups to domains and more. The API is part of the Federated Catalog public preview. To use it, enable the Enable Federated Catalog Features flag on the Feature Configuration page in Admin Settings. (AL-121118)

Note

To participate in the Federated Catalog public preview, you must have Beta language contained in your contract or have a Beta addendum signed.

Domains API

  • Added endpoints to the Domains API to move a domain to a different parent and retrieve a single domain. (AL-121319)

  • The Domains API can now recursively assign domain membership to an object and all its children. (AL-141426)

Cross-System Lineage API

The Cross-System Lineage API endpoints are being merged into the existing Lineage API. The separate Cross-System Lineage API is being deprecated. (AL-145817)

OCF Data Sources API

  • You can now get and update all metadata extraction options for an OCF data source. The additional options are accessible through a new configuration_parameters body parameter. (AL-148936)

  • When updating metadata extraction options, you can now Base64 encode parameter values, enabling you to send multi-line SQL queries. (AL-149793, AL-148936)

  • For the Trigger metadata extraction job endpoint of the OCF Data Sources API, we will be deprecating the three body parameters limit_schemas, exclude_schemas, and remove_filtered_schemas. (AL-148936)

Relational Integration API

  • The Relational Integration API can now get and update STRUCT columns and child columns. (AL-152225)

  • The Relational Integration API documentation was updated to indicate that the POST methods should not be used to update objects. (AL-146270)

Data Dictionary API

Policy API

The List Policy Groups endpoint now works with a trailing slash. It will also continue to work without a trailing slash. (AL-145478)

Bug Fixes

Curate & Govern

  • Resolved a problem where removing values from Object Sets bypassed the workflow process. Object Set and People Set fields now fully adhere to workflows. (AL-144777)

  • Resolved a discrepancy in the display of Snowflake tag names between Customize Catalog > Custom Fields and Customize Catalog > Custom Field Permissions. Previously, tags shown with fully qualified names in the Custom Fields section were displayed with partial names in the Custom Field Permissions section. Now, tags are consistently displayed with their full names in both sections. (AL-144307)

  • Fixed an issue where email notifications for custom field modifications were sent without considering the custom field’s permission settings. Now, email notifications undergo a permission check so that only users with custom field access are notified of changes. (AL-147500)

Data Dictionary

  • Fixed an issue where updating a custom field through data dictionary uploads wasn’t reflected in field history. Field history will now be correctly updated when a custom field update is done through data dictionary upload. (AL-149122)

  • Fixed an issue preventing the user interface from accepting “N/A” as a valid option for Picker custom fields during data dictionary uploads. “N/A” values are now supported for uploading in Picker fields. (AL-146164)

  • Fixed an issue with the data dictionary upload where blank values led to inconsistent updates: Titles were updated while Descriptions were ignored when the Keep Existing Values option was selected. Now, both the Title and Description fields are updated. (AL-146220)

Alation Analytics

  • Fixed a bug causing the ETL process for Alation Analytics to fail with an error Must be a superuser to terminate superuser process. Now, Alation no longer aborts queries run by super users in Postgres, as it’s not allowed since version 13.13. (AL-148020)

  • Fixed an issue where the deletion of multiple terms using Bulk Utility wasn’t reflected in the Alation Analytics database by subsequent ETL jobs, as the ETL checkpoint failed to recognize the deletions. Now, objects deleted through the Term and Document public API will not alter the ts_updated field, ensuring accurate tracking of deletions. (AL-134334)

  • Resolved a bug where term IDs remained in the glossaries.glossary_links column of the Alation Analytics database after their removal from a glossary. This issue was addressed by modifying the term extraction query to consider the ts_deleted timestamp along with ts_updated. (AL-134089)

Terms, Glossaries, and Documents

  • Resolved a bug with the Terms API where using the PUT method removed all glossary associations within a glossary instead of just the specified terms. Now, only the intended terms are removed. (AL-147499)

  • Resolved an issue causing search facets to still display folders and document templates after disabling the Document Hubs 3 feature flag. With this fix, document hub-related search facets are no longer shown in search filters after disabling the Document Hubs feature. (AL-151800)

  • Fixed an issue causing the document templates to still be displayed in the list of term types in Customize Catalog > Custom Templates disabling the Document Hubs feature. Now, only term types for terms will be displayed in Customize Catalog > Custom Templates if the Document Hubs feature is disabled. (AL-149245)

Rich Text Fields Formatting

  • Fixed an issue that caused the loss of formatting when indented text, rows, or cells copied from MS Excel were pasted into the Rich Text editor for articles. Formatting is now preserved as expected. (AL-145495)

  • Fixed an issue where during the editing of a lengthy article, adding a carriage return caused the cursor to jump to the top of the article. (AL-129702)

  • Fixed an issue where formatting in Rich Text fields updated through a catalog set wasn’t rendered correctly on catalog pages of objects in the set. Resolved by adding the necessary classes to properly render the formatting. (AL-103510)

  • Resolved a rendering issue where the ampersand symbol (&) in object titles was not displayed correctly. (AL-71755)

The Connect Platform

  • Resolved a bug causing cross-system lineage between two SQL Server data sources to link to soft-deleted records after metadata extraction. Cross-system lineage will now create correct associations with existing objects in the catalog. (AL-148764)

  • Fixed an issue where partition keys (part of table metadata) were not extracted by the OCF connector for Impala on CDP. Updated table extraction logic in the connector code to include extraction of partition keys. (AL-147592)

  • Fixed an issue with extraction from DB2 sources. Previously, metadata extraction failed with an error MissingFormatArgumentException: Format specifier ‘%s’. To fix the issue, the OCF SDK for the DB2 OCF connector was upgraded to version 4.4.0. (AL-147543)

  • Fixed an issue where metadata extraction failed for an SSRS OCF BI source with the ArrayOutOfBoundException exception. (AL-146517)

  • Resolved an issue in the OCF SQL Server connector where the value title lookup was not functioning with dynamic profiling. The catalog attribute page’s title lookup feature has been updated to support value title lookup when dynamic profiling is enabled. (AL-131988)

Other Fixes

  • Fixed an issue where administrator users were unable to check the status of Alation services with the alation_action status_alation command after updating to version 2024.1. This command now requires execution as the alation user. (AL-150411)

  • Fixed an issue where after upgrading to 2023.3.5, search export containing glossary term objects failed with an error Failed to prepare download: ‘NoneType’ object is not iterable in the logs. (AL-148667)

  • Fixed an issue where switching object types while in the saved search view changed the column selection to default for the saved search. (AL-127725)

  • Resolved an issue where canceling a query for a Hive data source prevented subsequent queries from being executed due to concurrent query checks. This was fixed by adding a cancellation check alongside the concurrent queries check. (AL-142466)

  • Made a change to the internal server API forbidding direct POST requests to the URL /data/0/upload_docs. Now, valid otype and oid are required for this URL, which then enforces restrictions on file types. (AL-137774)

PATCH RELEASE 2024.1.0.2

Alation Cloud installer build 9.0.0.20165

Released April 5, 2024

Note

This release is available for Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

  • Corrected the Datadog volume name which results from chroot jail removal to ensure the upgrade process functions correctly (internal architecture fix). (AL-152926, AL-152845)

  • Added exception handling if Amazon RDS Custom Resource Definition (CRD) is missing (internal architecture fix). (AL-151792)

PATCH RELEASE 2024.1.0.1

Alation Cloud installer build 9.0.0.19815

Released March 26, 2024

Note

This release is available for Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

  • Resolved a bug in the Open Connector Framework affecting the Test access operation for metadata extraction and query log ingestion. This issue impacted OCF connectors for Snowflake, Oracle, and MicroStrategy, where Test access failed displaying a KeyError['podIP'] error in the user interface and an error Stream closed with status: Status{code=UNIMPLEMENTED, description=Method not found: grpc.health.v1.Health/Check, cause=null} in the logs. The problem was caused by an internal race condition when an operation was requested from OCF connectors. (AL-151828)

RELEASE 2024.1.0

build 18.0.0.66893

Released March 4, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

New User Experience (Public Preview III)

Alation Cloud Service on Cloud-Native Architecture

​​The New User Experience has now moved into Public Preview III. The list below captures a summary of the major features and pages currently available in the New User Experience:

  • Data Source Catalog Details page

  • Schema Catalog Details page

  • Table Catalog Details page

  • Column Catalog Details page

  • Catalog Navigation

  • Search List View

  • DataSource Hub

  • Domains Hub and Details

  • Homepage basic features + Admin Guided User Interface (GUI)

  • Query Page (without functionality to run the query in the catalog page)

  • BI Hub and all detail pages

Note

To participate in the New User Experience Public Preview, you must have Beta language contained in your contract or have a Beta addendum signed. Reach out to your Alation account manager for more information if you would like to access the Preview.

Document Hubs (Public Preview)

This release introduces document hubs, designed to enhance your organization’s ability to categorize and manage extensive, written content directly related to your data. Document hubs offer the flexibility to create customized documents, tailored to meet your organization’s data documentation needs.

Document Hubs are set to eventually replace the current articles functionality.

Check out the documentation for Document Hubs.

Key Features

  • Centralized data documentation: Each document hub serves as a dedicated space within the catalog for organizing your written content.

  • Structured organization: Document hubs consist of a hierarchical structure with hubs at the top level, followed by folders, and documents within those folders. This structure facilitates easy navigation and management of content.

  • Management of content: Administrators have the capability to create and publish document hubs. Users can configure folders, add or remove documents, modify titles and descriptions, and utilize search and sorting functionalities. (AL-143825, AL-142897)

Note

Document hubs are available as part of Alation’s public preview program. To participate in the Document Hubs Public Preview, you must have Beta language contained in your contract or have a Beta addendum signed. Reach out to your Alation account manager for more information if you would like to access the Preview.

Lineage

Lineage Compound Layout Is General Availability

The Compound Layout lineage view, initially launched into public preview in version 2022.3.0, is now a General Availability offering for all customers in all environments. The “Beta” label was removed from the Compound Layout view switch on lineage graphs.

Note

The Compound Layout view organizes objects on lineage graphs into groups under data sources, such as Snowflake or Tableau, enabling focused analysis of data movement across different platforms. Users have the capability to drill down from the data source to column level, tracing objects across data source groups and achieving a comprehensive end-to-end lineage view.

Limitation & Workarounds

There is still a performance limitation in the Compound Layout view feature, where lineage graphs with a large number of objects may experience slow initial loading times. We’ve prioritized efforts to resolve this issue in the upcoming releases.

If you run into this issue, you can use the following workarounds until the issue is resolved:

  • Adjust graph limits: Ask your Alation Server or Catalog Admin to adjust graph limits and try reloading the graph with the Compound Layout disabled.

  • Clean up the lineage graph: Disable the Compound Layout, and in the default legacy view, remove unnecessary lineage and dataflows either manually or via APIs. Enable the Compound Layout view and reload the graph.

  • Request a targeted node cleanup: Reach out to Alation Support for targeted node cleanup.

  • Revert to the default view: If a graph’s loading time exceeds 30 seconds or fails, refresh the page, revert to the default view, and attempt to reload with the Compound Layout view disabled.

  • Try out optimized graph display: Alation Cloud Service customers can request evaluation of a new beta feature for optimized graph displays. Contact Alation Support to have it enabled on your Alation instance.

Beta Features Within Compound Layout

Some elements of the Compound Layout view are still marked with a “Beta” label, as they remain in public preview:

These features have recently transitioned from private to public preview as we seek broader feedback before finalizing their release.

Manual Lineage in Compound Layout (Public Preview)

We’ve added an Edit button directly in the Compound Layout view of lineage graphs. The button enables users to manually create, publish, and, if necessary, delete lineage. The feature is still in public preview, as the user interface for previewing manual lineage and data source grouping is still under development. As a temporary measure, upon publishing manual lineage, users are redirected back to the compound layout to review their modifications within the established data source groups. Users also have the flexibility to refine their manual lineage as needed after reviewing the graph. (AL-145095)

Data Trust Metadata Layer (Public Preview)

We’ve enriched the lineage diagrams with the Data Trust Metadata Summary designed to provide users with quick access to key information about trust flags. By clicking the overlay icons like depreciation, endorsement, or warning symbols, users can view a pop-up summary containing asset descriptions, steward contacts, trust flag messages, and impact analysis. The feature is still in public preview (beta), with more details and additional metadata layers to come later based on user feedback. (AL-141586)

General SQL Parser Upgrade

The General SQL Parser’s JAR file was upgraded to version 2.8.5.8 to take advantage of the bug fixes and improvements in the newer version. (AL-143872)

Enterprise Readiness

Postgres Upgrade to 13.13

Upgraded Alation’s internal database (PostgreSQL) to version 13.13 to take advantage of new features and to fix vulnerabilities. (AL-143302)

Apache Kafka Upgraded to 3.6.1

Upgraded Alation’s internal event streaming platform (Apache Kafka) to version 3.6.1 to take advantage of new features and to fix vulnerabilities. (AL-143661)

Chroot Removal

Alation Cloud Service on Cloud-Native Architecture

The Alation Cloud Service instances no longer use the Chroot-restricted environment in their builds. (AL-105007)

Note

This change does not impact customer-managed installations, which will remain unchanged.

Authentication

Reauthentication Due to Inactivity

Alation Cloud Service on Cloud-Native Architecture

Starting with this release, when a user session remains inactive for a predetermined period, it will be automatically terminated. Users receive a warning two minutes prior to session expiration. For FedRAMP deployments, the inactive timeout is fixed at 15 minutes and cannot be adjusted. For non-FedRAMP deployments, the timeout interval can be configured as follows:

  • Default duration: 60 minutes

  • Minimum duration: 5 minutes

  • Maximum duration: 1440 minutes (24 hours) (AL-125476)

Cross-Account IAM Support for AWS Secrets Manager

The authentication support for AWS Secrets Manager has been expanded to include cross-account IAM roles, in addition to the existing methods of IAM user and IAM user+IAM role. The Authentication Configuration Methods for External Systems section of the user interface (Admin Settings > Server Admin > Authentication) has undergone some enhancements for AWS Secrets Manager. Only the relevant configuration fields are now displayed, ensuring a more intuitive user experience. (AL-119588)

Alation Analytics

Alation Analytics PostgreSQL Upgrade to 13.13

The Alation Analytics PostgreSQL database was upgraded to version 13.13 to take advantage of new features and to fix vulnerabilities. (AL-143302)

Information About Tables and Columns Linked to Data Policies Added to Alation Analytics

As of version 2023.1.7, Alation Analytics has supported querying data policies directly from the data_policy table. This enhancement introduces a new table, data_policy_link, which provides detailed associations between data policies and specific table or column objects. This table can be particularly useful for administrators and analysts looking to understand the application of data policies at the granular level of tables and columns.

Columns

  • dim_load_id, dim_ts_created, dim_checksum—ETL bookkeeping columns.

  • object_id, object_uuid—Object ID in numeric and UUID formats respectively.

  • object_type, object_type_id—Object type in the text (table, attribute) and number (27, 1) formats respectively.

  • id—Primary Key for the table in Alation Analytics.

  • policy_link_id—Internal ID for each policy link.

  • policy_id—Policy ID.

  • ts_created, ts_updated, ts_deleted—The created, updated, and deleted timestamps.

  • deleted—Boolean to denote if a link is soft-deleted. (AL-139218)

Expire Old WAL Archives

A new scheduler will now automatically remove WAL archives of the Alation Analytics database that are older than 48 hours. This change is designed to save disk space by ensuring that WAL archives are regularly rotated and outdated files are efficiently purged. (AL-139378)

Transition from Redis to Amazon Elasticache

Alation Cloud Service on Cloud-Native Architecture

Alation Analytics now utilizes cache from Amazon Elasticache instead of Redis within the Alation core container. This is a significant enhancement in the Alation Analytics architecture, aiming to provide a better and more reliable service to Alation Cloud customers. (AL-141201)

Governance

Resolution of Duplicate Tag Names

The display of Snowflake tags on catalog pages (including schema, table, and column pages) has been enhanced for better handling of tags with duplicate names across different schemas:

  • A new feature flag, alation.feature_flags.enable_snowflake_tag_filtering_by_schema, has been introduced to allow optional filtering of tags by schema within catalog pages (disabled by default).

  • The Custom Fields section under Customize Catalog now displays schema information for each Snowflake tag within picker fields. This addition is intended to help users differentiate between tags with identical names but belonging to different schemas. (AL-138710)

Auto-Enablement of Policy Center and Snowflake Tags Backend Parameters

Starting with this release, when the feature flags for Policy Center and Snowflake Tags are enabled in Feature Configuration, the corresponding set of backend flags is also automatically enabled. This enhancement eliminates the need for additional manual backend configuration when enabling the Policy Center and Snowflake Tag features in the Alation user interface. The following backend flags are auto-enabled:

  • alation.ocf.mde.policy.enable_extraction

  • alation.ocf.mde.policy.enable_ingestion

  • alation.ocf.mde.policy.enable_search

  • alation.ocf.mde.policy.enable_sync

  • alation.ocf.mde.custom_field.enable_sync

  • alation.ocf.mde.custom_field.enable_extraction

  • alation.ocf.mde.custom_field.enable_ingestion (AL-143847)

OCF Connectors

Updates to the Open Connector Framework (OCF) connectors are released every week. See OCF Connector Release Notes for information on the updates in the month of February 2024.

A few February highlights are:

Bug Fixes

Governance & Glossaries

  • Fixed an issue where uploading a data dictionary with custom field names in Japanese failed after upgrading to version 2023.3.3.1. The original parsing limitation to ASCII characters has been expanded to accept Unicode characters, ensuring successful uploads for Japanese and other Unicode characters. (AL-141572)

  • Fixed an issue where titles remained unchanged during a data dictionary file upload when the Keep Existing Values option was selected and the initial title and description values were blank. The fix ensures that titles are now updated correctly when the Keep Existing Values option is selected when the current titles are blank. (AL-146220)

  • Removal of field from an object set is now honored via workflows. (AL-144777)

  • Fixed an issue where manual catalog set synchronization failed with the error Job is not running. It might have been killed or crashed unexpectedly or there might have been a system outage. The fix is expected to lead to reduced job failures in catalog sets synchronization (AL-140909).

  • Resolved an issue where changing the term template via the Bulk Utility did not update the template. The Bulk Utility now supports template updates effectively. (AL-143246)

  • Fixed an issue in the Bulk Utility where the downloaded terms showed deleted objects in Object Sets. Specifically, when columns related to a term’s Object Set custom field Related Objects were dropped and subsequently soft-deleted during metadata extraction, they would still appear as [GONE] in the field history, despite being absent in the user interface. The Bulk Utility has been updated to exclude deleted objects from Object Set values. (AL-136475)

  • Resolved an issue introduced in version 2023.3.1.1 where HTML tags were visibly displayed in the glossary term pages within the Related Terms section. This fix ensures that HTML tags no longer appear on catalog pages for Object Sets. (AL-137238)

  • Resolved an access issue where users with the Steward roles were unable to use the data dictionary public API. Now, when the alation.feature_flags.disable_perm_check_on_upload_dd feature flag is set to False, users in the Steward, Composer, Source Admin, Catalog Admin, and Server Admin roles are allowed to make data dictionary public API calls, including uploading and retrieving task details. (AL-144523)

  • Resolved an issue where workflows reviewer selection was failing on specific instances when user tried to select group as reviewers (AL-145175)

Alation Analytics and Analytics Experience

  • Resolved a synchronization issue affecting column objects’ data between Alation and Alation Analytics, specifically for column titles and descriptions updated through data dictionary upload. To reload the title and description missing data, Reset ETL Checkpoint for the RDBMS Tables. (AL-143810, AL-146271)

  • Fixed an issue in the rosemeta_allvisits table where updates for conversation visits were not accurately reflected. Enhanced the logging mechanism for tracking visits to the conversation page in the public.visits table within Alation Analytics. This fix ensures that all user visits to conversations are properly logged and updated. (AL-142986)

  • Resolved ETL process failures that began following an upgrade to version 2023.3.2.1 and were manifested through the error TransformManager.transform(): Column name not supported in Snowflake: ACCOUNT. To address the incompatibility with Snowflake’s reserved keywords used as custom field names (ACCOUNT), we’ve implemented a solution to rename the affected columns in tables to the format <CUSTOM_FIELD_NAME>_<ID>, ensuring successful data loading into Snowflake without errors. (AL-142557)

  • Fixed an issue where transform containers failed to restart in a healthy state after an unattended server reboot. This fix ensures a coherent and reliable startup of all necessary containers post-reboot. (AL-136558)

Lineage

  • Resolved an issue where upstream column-level lineage links (CLL) failed to generate for certain tables with quoted column names, although downstream lineage remained unaffected. With this bug fix, we’re ensuring upstream lineage visibility for columns with quoted names. (AL-140987)

General SQL Parser

  • Fixed lineage parsing issues between dbt and Snowflake. The fix ensures lineage accuracy from constructs like the CONNECT BY subclause. (AL-144356)

  • Fixed an issue where Google BigQuery published queries were not associating with their respective tables, resulting in their absence from the Queries tab. This fix ensures that Google BigQuery published queries correctly link to tables. (AL-141819)

  • Resolved a parser error affecting Snowflake DDL views, where lineage for views defined and ingested by MDE was not displayed, although connections to related Tableau objects were visible. This update addresses parsing issues, including those related to the CONNECT BY subclause, ensuring accurate lineage representation for Snowflake DDL views. (AL-140906)

Connected Sheets

  • Addressed the Excel login error encountered on Chrome, which was caused by changes to third-party cookie handling. (ACS-562)

  • This fixes the issue where the connected sheets hub page redirects in loops because of chrome’s third-party cookie changes (ACS-589)

Connect Platform

  • Fixed an issue with Hive data sources where, after canceling a query within a session, subsequent queries failed to execute due to concurrent query checks. This fix introduces a cancellation check alongside the concurrent queries check, ensuring smooth execution of subsequent queries after a cancellation event. (AL-142466)

  • Fixed an issue where profiling failed for an AWS Databricks data source with the following error: The connector was provided with a list of columns to profile from a table, however the schema of the returned result set did not match those columns. The solution skips tables that cause failures and continues with the sampling process, ensuring profiling tasks complete successfully. (AL-141595)

  • Addressed an issue where multiple installed versions of a connector on an Alation instance were indistinguishable in the user interface due to the absence of version numbers. This led to confusion over which version was being selected. Now, in case multiple connector versions are installed, the dropdown shows the connector version next to its name. (AL-146195)

  • Resolved an issue where uploading data via XLSX files to native and OCF Redshift data sources failed with the error AttributeError: ‘NoneType’ object has no attribute ‘columns’. Users can now successfully use XLSX files to upload data to both new and existing tables. (AL-138753)

Other Fixes

  • Fixed an issue where content such as @-references, links, and photos included in user profile descriptions disappeared upon saving. The fix ensures that all content added to profile descriptions is saved. (AL-143451)

  • Resolved a deadlock issue in uWSGI worker threads introduced by the upgrade from Python 3.6 to 3.9. The deadlock occurred when threads loaded a Django application alongside the Python module traceback. From version 2024.1 onwards, this is addressed by disabling traceback by default. (AL-141594, AL-145955)