Configure the Data Contract

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

A data contract is an agreement between the data producer and the consumer that guarantees the data provided is high-quality, reliable, and predictable. As the Data Product Admin, configuring a data contract for your data product allows you to monitor for structural changes and quality in the underlying dataset.

Data contract checks provide confidence in data reliability and result in alerts within the data product configuration if a change in the relevant database schemas is detected. This allows you to take necessary corrective action, such as selecting more objects, updating join information, or removing objects from the data product.

A data contract check is a process that runs in the background, monitoring the schemas from which tables and columns are included in the data product.

By default, a data product does not have any schema checks configured.

The schema check is tied to metadata extraction (MDE) from the data source: it monitors the metadata extracted into the catalog, not the actual database. Schema checks run automatically when MDE completes for the relevant data source. If MDE is scheduled, the schema check runs on the same cadence after MDE completes. This process ensures consistency between the catalog and the data product.

The following structural changes trigger an alert:

  • A table removed from the schema.

  • A column removed from a table.

  • Column data type changed.

Note

If a schema is removed from metadata extraction, an alert will still appear, as the check monitors the extracted catalog metadata for consistency with the data product, regardless of whether the schema still exists in the physical database.

Enable the Schema Check

To add a schema check:

  1. Access the Data Product App.

  2. Click My Data Products in the left-side navigation.

  3. On the Manage My Data Products page, locate and click the name of the data product you want to configure. The data product’s settings page opens.

  4. Click the Data Contract tab.

  5. In the Schema checks on all tables and columns in data product block, click the Turn on check button.

  6. The check status changes from Inactive to Active. The check will now run whenever metadata extraction completes on the data source containing the included schema.

View Check Results

Upon completion, each schema check generates a record in the Check history table on the Data Contracts tab.

  • Pass Status: If no changes are detected in the schema, the check results in a green Pass status.

  • Alerts: If any structural change events are discovered, an alert is issued.

To view details for an alert, click Show details in the check history for the latest check. Note that Show details is only available for the latest alerts. You must determine whether to act on or ignore the alert.

Enable Data Contracts for Data Products Created in YAML

Data products can be created using the user interface (UI) or directly through a YAML specification. When a data product is created via the UI, and tables and columns are added from the Alation catalog, the system automatically includes the documentationURL property within the dataAccess property of the data product. This property is crucial as it allows Alation to resolve the data source information required for data contracts, enabling the data source to be automatically detected when you activate the schema check. (Use the URL https://<base_URL>/static/swagger/specs/data_products/product_schema.yaml for the detailed data product schema, substituting <base_URL> with your actual base URL.

If your data product was created using the YAML specification it may be missing the required documentationURL property. If this property is missing, Alation cannot resolve the data source because documentationURL is required for data contracts to work.

When you attempt to enable a schema check on such a data product, the following error message appears in the UI, despite previously adding objects from the catalog: No data sources are associated with this data product. Add cataloged tables to enable scheduled schema checks.

This message indicates that the necessary documentationURL property is absent.

Resolution: Add the documentationURL Property

To enable schema checks, you must update the data product’s YAML specification to include the documentationURL property. You should add this property to at least one table if all tables belong to the same data source.

  1. Open the Manage My Data Products page.

  2. Locate the data product and click Edit in the Data Product table (rightmost column) to open the editor.

  3. Click Edit YAML.

  4. Locate the recordSets property.

  5. Check the content under the dataAccess property. The documentationURL must point to the table page in the Alation catalog.

  6. Add documentationUrls to the recordSets in your data product. Ensure the configuration appears in YAML as follows:

    recordSets:
  HC_CLMS:
    name: HC_CLMS
    displayName: HC_CLMS
    description: ''
    dataAccess:
      - type: sql
      documentationUrl: https://my_data_catalog.alationcloud.com/table/485/
      qualifiedName:
        database: ALATION_EDW
        schema: HEALTHCARE
        table: HC_CLMS

  7. Save the changes.

  8. Try enabling the schema check again.