Best Practices for Curation Automation¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Curation Automation lets you create rules that automatically populate metadata across your catalog at scale. Several factors determine the quality of the AI-generated output: the inputs you provide directly when configuring a rule, and the richness of the metadata already present in your catalog.
This page covers how to maximize output quality across each factor.
Understand the Quality Levers¶
The three inputs you provide when configuring a rule are the primary quality levers. Each corresponds to a specific step in the Create a Curation Rule wizard. For step-by-step wizard instructions, see Manage Curation Rules.
Input |
Wizard step |
What it does |
|---|---|---|
Purpose |
Sets the overall intent of the rule. Gives the AI agent behind the Curation Automation feature context about your domain, data sources, company, and curation goal. Up to |
|
Field Instructions |
Written instructions per field (Description, Title, custom fields). Tells the AI agent what to write, for whom, and to what standard. Up to |
|
Reference Document |
An optional document attached to any field instruction, such as a data policy, glossary, or style guide. Gives the AI agent source material to draw from when generating values for that field. |
Beyond your rule inputs, curation quality is also shaped by the richness of the metadata already present in your catalog. Invest in the following signals before or alongside running curation rules:
Signal |
Why it matters |
|---|---|
Source comments |
Comments written by developers in the source database (column comments, table remarks) are ingested into Alation and passed to the AI agent as context. Richer source comments produce better output. |
Description quality |
When a rule curates multiple fields on the same object, Alation always generates the Description first and passes it as context when generating all remaining fields. A well-curated Description improves every other field on the same object. If your Descriptions aren’t yet high quality, include Description in your rule alongside your other fields. |
Query Log Ingestion (QLI) |
When QLI is enabled, Alation ingests the SQL queries run against a source, including joins, filters, and column usage patterns. This information is passed to the agent as behavioral context. Consider enabling and running QLI on every data source you plan to curate with Curation Automation. |
What the Agent Receives¶
When a rule runs, the AI agent receives an assembled prompt for each object in scope. Understanding what goes into that prompt helps you write better inputs and invest in the right catalog signals.
The prompt has three parts:
Part |
What it contains |
Where it comes from |
|---|---|---|
System prompt |
Core instructions and your rule Purpose |
You write it in Step 2: Enter Details. Applies to the entire rule. |
Field prompt |
Your Field Instruction, the field name, type, tooltip, and optional reference document content |
You write it in Step 4: Select Fields. One prompt per field selected. |
Object context |
Automatically assembled per object from your catalog |
Pulled from your catalog. Richer catalog means richer context means better output. |
Alation automatically assembles the following context for each object in scope and passes it to the agent:
Source type |
Object |
Context fields sent to agent |
|---|---|---|
RDBMS (Snowflake, Databricks, Redshift, and other supported data sources) |
Schema |
Title, description, source comment, source title, source description, fully qualified name |
RDBMS (Snowflake, Databricks, Redshift, and other supported data sources) |
Table |
Title, description, source comment, source title, source description, fully qualified name; top |
RDBMS (Snowflake, Databricks, Redshift, and other supported data sources) |
Column |
Title, description, source comment, data type, fully qualified name; top |
BI (Tableau, Power BI, and other supported BI tools) |
BI Report |
Name, external ID, source URL, BI object type, owner, popularity score, parent folder name, description at source, top |
BI (Tableau, Power BI, and other supported BI tools) |
BI Folder / Server |
Name, external ID, server type, description at source, parent name |
File System |
File |
Name, parent name, content type, top |
File System |
Directory |
Name, parent name, top |
Note
The built-in Description field on RDBMS objects (schema, table, and column) uses a significantly richer context payload than all other fields. In addition to the object context in the table above, the agent receives:
DDL-formatted schema: column names and data types in DDL format, from which the agent infers domain and business purpose
Whether the object is a database view
Parent titles across the full hierarchy (datasource, schema, and table titles)
Parent descriptions across the full hierarchy
Query titles from query history (requires QLI)
Join conditions between this table and related tables (requires QLI)
Schemas of commonly joined tables (requires QLI)
Domains assigned to the object in Alation
Tags applied to the object in Alation
Sibling columns, for column objects
Constraint flags (primary key, foreign key, nullable), for column objects
Because source comments, existing titles and descriptions, column names, and query titles are all sent to the agent automatically, you don’t need to repeat them in your field instructions. Focus your Purpose and Field Instructions on context the agent can’t see on its own: your industry, your company’s standards, your audience, your governance goals, and what good curation looks like for your organization.
The right framing: when writing your Purpose and Field Instructions, imagine you’ve just hired someone new. They’re smart and they understand data, but they know nothing about your domain, your data sources, or your standards. Write them a briefing so they can curate your catalog correctly. The more context you give the AI agent, the better it performs.
Write an Effective Purpose¶
The Purpose field is your primary briefing to the AI agent. You provide it in Step 2: Enter Details of the Create a Curation Rule wizard. It sets the strategic context for the entire rule: every asset in scope, every field it touches. You have up to 1,000 characters. Use as many as you can. A one-line purpose gives the AI almost no signal, while a well-elaborated purpose produces noticeably higher quality curation across every asset in scope. To draft or refine your Purpose with AI, use Suggest or Improve Instructions with AI.
A strong Purpose covers five dimensions:
Dimension |
What to write |
|---|---|
Industry and company |
The industry you operate in and your company’s context. For example: We are a healthcare company operating under HIPAA or a financial services firm subject to SOX. |
Data source |
Where the data comes from. Be specific: These tables are loaded from Snowflake, derived from SAP ERP exports, Adobe Analytics clickstream data, Salesforce CRM pipeline data. |
What the data represents |
What these assets actually contain. For example: customer transaction records, web session and event data, insurance claims and adjudication history, Gold-layer aggregated sales metrics. |
Domain and team |
The domain this rule applies to and who owns or uses it. For example: Finance domain, managed by the Data Governance team or Marketing Analytics, consumed by campaign managers. |
Goal and standard |
The outcome you are trying to achieve. For example: meet Silver certification requirements, improve discoverability for self-service analysts, comply with our data retention policy. |
Use the following template as a starting point. Replace every bracketed section with your organization’s specifics.
We are a [industry type] company [operating context: for example, ‘subject to HIPAA’, ‘a publicly traded company under SOX’, ‘serving customers across X region’]. This rule applies to tables in the [SCHEMA NAME] schema in [data platform: for example, Snowflake, Databricks, Redshift], which is [loaded / sourced] from [source system(s): for example, SAP S/4HANA, Salesforce CRM, Adobe Analytics, Epic EHR, Guidewire ClaimCenter] [on what schedule: for example, ‘nightly via ETL batch’, ‘in real time via Kafka’, ‘hourly incremental load’].
These tables contain [describe the data: for example, customer transaction records, web session and event data, clinical encounter records, insurance claims, production orders]. The data covers [scale or scope: for example, ‘3.5 million records per year’, ‘all transactions from the past 7 years’, ‘our North America operations across 12 facilities’].
The primary users of this data are [audience 1], [audience 2], and [audience 3]. [Describe their technical level: for example, ‘Most are business stakeholders with no SQL background.’]
The goal of this rule is to [describe the outcome]. Descriptions should [specific content requirements]. [Constraints: for example, ‘Avoid technical jargon.’ or ‘Do not reference internal system codes or database identifiers.’]
Examples by Industry¶
The following examples are written to the recommended standard. Each covers industry context, data source, what the data represents, who uses it, and the curation goal.
We are a healthcare organization operating under HIPAA compliance requirements, serving patients across 12 hospital facilities and 40 outpatient clinics in the Northeast region. This rule applies to tables in the CLINICAL schema in Snowflake, which is loaded nightly from our Epic EHR system via incremental ETL. These tables contain inpatient and outpatient encounter records, ICD-10 diagnosis codes, CPT procedure codes, provider and facility assignments, discharge summaries, and length-of-stay data covering approximately 3.5 million patient encounters per year.
This data is used by clinical data analysts, quality improvement teams, HEDIS reporting specialists, compliance officers, and product managers — most of whom do not have a SQL or database background. The goal of this rule is to ensure all tables in the CLINICAL schema meet our Gold certification standard, which requires a clear business description, documented data lineage, and an identified data steward. Descriptions must explain what each table contains, identify the Epic module it originates from, describe the clinical context in which it is used, and note any known data quality caveats such as late-arriving records or source system gaps. Never reference PHI or patient identifiers in any metadata. Write for a business audience. This catalog supports 300+ internal users across self-service analytics, regulatory reporting, and Joint Commission audit documentation.
We are a property and casualty insurance carrier offering residential and commercial coverage across 38 U.S. states. This rule applies to tables in the CLAIMS schema in Snowflake, populated daily via incremental ETL loads from our Guidewire ClaimCenter system. These tables represent first-notice-of-loss events, adjudication records, reserve movements, loss payments, salvage and subrogation recoveries, and litigation tracking across our auto, homeowners, and commercial property lines of business. The data covers approximately 1.2 million active and closed claims and reflects the full claims lifecycle from FNOL through closure.
This data is used by actuarial analysts, claims operations managers, financial reporting teams, loss control consultants, and external auditors. The goal of this rule is to ensure every table in the CLAIMS schema has a description that clearly explains its business meaning, its position in the claims lifecycle, and its relevance to our Statutory and GAAP financial reporting requirements. Our data catalog is actively reviewed during SOX compliance audits, state regulatory filings, and internal model governance reviews. Descriptions should be written in plain English, explain what the table tracks within the claims process, and note any special handling required for reserve calculations, coverage limits, or large-loss events. Avoid actuarial notation and Guidewire system-specific terminology where possible. Write for a business audience that understands insurance concepts but does not have database expertise.
We are a regional commercial bank operating in the United States, subject to BSA/AML regulations, Basel III capital requirements, and CECL accounting standards. This rule applies to tables in the LENDING schema in Snowflake, sourced from our Fiserv core banking platform via nightly batch extracts and supplemented with data from nCino for commercial loan origination workflows. These tables contain commercial and retail loan records including origination details, payment history, covenant tracking, risk ratings, days-past-due aging, charge-off records, and recovery data across our $8 billion loan portfolio.
This data is used by credit risk analysts, commercial portfolio managers, loan operations teams, regulatory reporting specialists, and model validation officers. Some users are technically sophisticated; many are not. The goal of this rule is to ensure all lending tables meet our Silver data certification standard, which requires accurate business descriptions, documented source system lineage, and identified data stewards. Descriptions should explain what each table contains within the lending lifecycle, reference the source system and batch schedule, and note any known data quality constraints such as nCino-to-Fiserv reconciliation timing, late-arriving wire payments, or known gaps in historical charge-off records. Write for a business audience. Avoid internal system codes, core banking jargon, and SQL terminology in descriptions.
We are an investor-owned electric and gas utility serving 2.4 million residential, commercial, and industrial customers across the Pacific Northwest, regulated by FERC and our state Public Utility Commission. This rule applies to tables in the OPERATIONS schema in Snowflake, which aggregates data from three source systems: our SAP IS-U customer information system, our GE ADMS grid management platform, and our Itron smart meter data platform (AMI). Data is loaded on varying schedules: SAP loads nightly, ADMS loads hourly, and AMI meter reads arrive every 15 minutes and are consolidated daily.
These tables contain customer account records, meter reads, outage events and restoration records, work orders, grid asset maintenance logs, and demand response program enrollments. This data is used by grid operations engineers, customer experience analysts, regulatory affairs teams, financial planning and analysis teams, and external rate case consultants. The goal of this rule is to curate all tables in the OPERATIONS schema so they can be certified for use in customer-facing analytics, FERC Form 1 and PUC regulatory filings, and internal operational dashboards. Descriptions should identify the source system, explain what the table represents in utility operations, state the data refresh cadence, and note known quality limitations such as AMI communication failures or SAP batch timing delays. Write in plain language for analysts who understand utility operations but are not database engineers.
We are a global discrete manufacturer of precision industrial components and assemblies, operating 14 production facilities across North America, Europe, and Asia, with annual revenue of approximately $2.4 billion. This rule applies to tables in the MANUFACTURING schema in Snowflake, populated via hourly ETL feeds from our SAP S/4HANA ERP system and supplemented with near-real-time production telemetry from our MES (Manufacturing Execution System) deployed at each plant. These tables contain production orders, bill of materials records, work center assignments and capacities, quality inspection results, defect and scrap records, inventory movements, and supplier delivery and lead time performance data.
This data is used by plant operations analysts, supply chain planners, quality engineers, plant controllers, and executive leadership for production reporting, OEE (Overall Equipment Effectiveness) tracking, and supplier scorecards. Most plant-level users access this data through Power BI dashboards and are not data specialists. The goal of this rule is to ensure all manufacturing data assets in this schema have business-friendly descriptions that enable self-service analytics without requiring users to consult database documentation. Descriptions should explain what each table tracks within the production and supply chain lifecycle, identify the source system and refresh frequency, and note any known data quality issues such as late-arriving MES events, SAP batch reconciliation delays, or plant-specific data gaps. Avoid ERP system codes, manufacturing system jargon, and technical identifiers in descriptions.
What to Avoid¶
A one-line purpose gives the AI almost no signal to work with. The following examples show weak purposes and what each is missing.
Weak purpose |
What is missing |
|---|---|
Provide analytics context |
No domain, data source, industry, or audience. The AI has no signal to differentiate this rule from any other rule in any other company. |
Add descriptions to BI tables |
Describes the mechanism, not the goal. Gives no context about what BI means here, who uses it, where the data comes from, or what quality looks like. |
Finance domain rule |
Just restates the scope. No source system, no industry context, no curation standard, no audience. |
Write Effective Field Instructions¶
Field instructions are written per field. You provide them in Step 4: Select Fields of the Create a Curation Rule wizard. Each field you choose to populate — Description, Title, or a custom field — gets its own instruction text. This is the most direct lever you have over the quality of what ends up in your catalog. You have up to 2,000 characters per field. Use as many as you can. To draft or refine an instruction with AI, use Suggest or Improve Instructions with AI.
Think of each field instruction as a writing brief for that specific field.
What to Include¶
Repeat the domain and data source context. Don’t assume the agent carries over context from the Purpose field. The agent reads each field instruction in isolation when generating values. Tell it again: what these tables are, where they come from, what data they contain.
Define what the output must contain. For a Description: include what the object represents, the type of data, and how it’s used in a business context. For a Title: specify length, capitalization rules, and what to avoid.
Name the audience. For example: Write for a business analyst with no SQL knowledge or Read by compliance auditors.
Use negative constraints. For example: Avoid technical jargon, Do not repeat the table name, Do not include SQL syntax or schema notation.
Set length and structure. For example: Write 2–3 sentences, Start with a verb, Use plain language and active voice.
The following table shows a strong instruction and a weak one for the same fields:
Field |
Strong instruction |
Weak instruction |
|---|---|---|
Description |
Write a clear, concise business description for this table. These are Snowflake tables in the CLINICAL schema loaded from our Epic EHR system, containing patient encounter records. Include what the table represents, what type of clinical data it contains, and how it is used by clinical analysts and compliance teams. Avoid technical jargon and do not reference PHI. Write for a business audience with no SQL background. |
Write a business friendly description. |
Title |
Generate a short, human-readable title ( |
Make a good title. |
Choose When to Overwrite Existing Values¶
Each field includes a Field Options setting that controls when the rule writes to the field. This is a quality and safety decision.
Setting |
When to use |
Risk to watch for |
|---|---|---|
Fill if empty (default) |
When fields may already contain manually curated content you want to preserve. Good for a first-pass automation on a catalog with existing metadata. |
Skips assets with any existing value, even outdated ones. |
Always replace |
When you are standardizing a domain with inconsistent or stale metadata. Good for governance enforcement rules. |
Overwrites manually curated content. Only use after validating instruction quality on a preview sample. |
Example Field Instructions¶
The following examples illustrate how to tailor instructions to specific field semantics.
Generate a clear, accurate description of this data object based on its name, metadata, relationships, and available context. Explain its purpose, the type of information it represents, and how it fits within the broader dataset or domain.
Create a clean, readable, business-friendly title for this object by interpreting its technical name, abbreviations, and conventions. Expand acronyms where appropriate and produce a meaningful title that helps users quickly understand what the object represents without altering the underlying intent.
Evaluate whether this object contains or relates to personally identifiable information by analyzing naming patterns, metadata, associated attributes, and contextual indicators. If the object depends on underlying components (such as child fields or attributes), incorporate those dependencies when determining whether PII is present. Classify only when there is clear, evidence-based justification.
Attach a Reference Document¶
In Step 4: Select Fields of the Create a Curation Rule wizard, each field instruction includes an optional Add document button. This lets you attach a document, such as a policy, standard, or reference guide, that the AI can draw from when generating values for that field.
This feature is useful for governance fields where correct values depend on formal organizational standards that would be difficult to encode in 2,000 characters of plain text. When you attach a document to a field instruction, the AI reads it alongside your written instruction and uses its contents when generating values for that field.
The following table shows example use cases:
Document to attach |
Field being populated |
What it enables |
|---|---|---|
Data Retention Policy |
Custom field: Retention Code |
The AI reads the policy and sets the correct retention code for each asset based on its domain, data type, or regulatory classification. |
Data Classification Standard |
Custom field: Classification Level |
The AI applies the correct sensitivity tier (for example, Confidential, Internal, Public) based on what the asset contains. |
Curation Style Guide |
Description |
The AI writes descriptions that follow your organization’s specific tone, terminology, and formatting standards. |
Domain Glossary |
Description or Title |
The AI uses your organization’s preferred terminology rather than generic language. |
Regulatory Framework (for example, HIPAA, SOX) |
Custom compliance fields |
The AI tags or classifies assets correctly against regulatory requirements without you encoding all the rules in plain text. |
Example: Data Retention Policy¶
Let’s assume you have a custom field called Retention Code with values such as R1 (1 year), R3 (3 years), R7 (7 years), and R-Perm (permanent). Rather than trying to encode your full retention policy in 2,000 characters of field instruction text:
Write a short instruction: “Set the Retention Code for this asset based on the attached data retention policy. Match the asset’s domain, data type, and regulatory classification to the correct retention tier.”
Attach your data retention policy document.
The AI reads the policy and applies the correct code to each asset consistently across thousands of objects.
Define Your Scope¶
Scope determines which data objects your rule targets. Start small and expand over time. Every time you run the rule, it processes only empty fields. Previously curated objects are not reprocessed unless you configure the rule to replace existing values. This means you can expand the scope and rerun the rule safely.
Start Small, Then Expand¶
Use an incremental approach:
Start with one schema, all tables, and all columns.
Run the rule.
Review the outcomes and execution report.
Edit the rule to expand the scope to additional schemas.
Repeat until you’ve covered the full target dataset.
Use a Bottom-Up Approach¶
Start with columns, then curate tables, then expand to schemas. Columns are the most granular layer of data. When column descriptions and classifications are accurate:
Table descriptions improve automatically, because the agent uses column-level context when generating table metadata.
PII classifications become more precise.
Trust signals become stronger across the hierarchy.
For file system sources, use the same logic: start with file attributes, then files, then directories.
Use Scope Filters for Large Catalogs¶
In large catalogs, curating every object in a source can be impractical and costly. Use scope filters to target only the objects that need governance attention, based on metadata your team has already applied.
For example, instead of curating all columns in a data source, add filters to include only columns where Sensitivity Level is High or Compliance Tag contains SOX or HIPAA. This narrows the scope to the relevant subset, reduces cost, and focuses curation where it matters most.
You can combine up to two filters per source using AND or OR logic. Filters support picker and multi-select picker custom fields.
Scope by Name with Pattern Mode¶
The Pattern mode matches data source objects by name at the schema, table, and column level, instead of selecting them manually. Pattern mode is available for data source assets only; file system sources support Search mode. Consider Pattern mode when:
Your assets follow naming conventions and you want to target a consistently named set, for example, all tables that start with
fact_.A manual selection would require a long include or exclude list to express the set you want.
You want the rule to cover future assets that match the conditions when you rerun it, without editing the asset selection.
You need precise or complex name matching, which you can express with the Matches regex operator.
Pattern conditions and scope filters are complementary, and you can use both on the same source:
Pattern conditions match on the object name. Use them when your scope follows a naming convention.
Scope filters match on custom field values (for example, Sensitivity Level is High). Use them when your scope depends on metadata your team has already applied.
For details, see Pattern Mode in the Create a Curation Rule wizard.
Use Preview to Validate and Tune¶
Step 5: Configure Agents of the Create a Curation Rule wizard is a tuning environment, not just a preview step. Use it to validate your assumptions and improve your instructions before committing a run.
Preview at least 3–5 diverse assets from across your scope. Don’t rely on the first five objects the dropdown suggests — choose objects that represent different schemas, object types, or edge cases you expect the rule to encounter.
For each object you preview:
Review the generated values and confidence scores.
If confidence is Medium or Low, read the AI’s reasoning field to understand what context it had and what it was guessing at.
Edit the field instruction, regenerate, and compare the output.
Repeat until results consistently reach High confidence.
Treat preview as an iteration space. Running the rule without previewing first risks consuming AI actions or ACU credits on poor-quality results applied at scale. Those results are difficult to correct after the fact.
Common Mistakes to Avoid¶
The following mistakes are common across deployments. Each has a predictable outcome and a straightforward fix.
Mistake |
What happens |
Fix |
|---|---|---|
Writing SQL-style or programmatic conditional logic in instructions |
The AI ignores SQL-style conditions. It doesn’t have access to a code interpreter and can’t evaluate programmatic logic. |
Use natural language conditionals instead: If the object type is a table, then… — the model understands this reliably. |
Writing a generic prompt with no domain context |
Vague, unhelpful metadata is generated. A poor first impression of AI curation can reduce trust and adoption. |
Always include your industry, company name, or data domain in both the Purpose and each Field Instruction. |
Not including Description in scope when curating custom fields |
Custom field quality is lower — the AI generates values without knowledge of the object’s overall business purpose. |
Include Description in the rule. The system generates it first and passes it as context to all other fields automatically. |
Running a full rule without previewing first |
AI actions or ACU credits are consumed and poor-quality results are applied at scale. |
Always preview |
Using domain-specific codes without a reference document |
The AI guesses codes randomly or draws on general knowledge that doesn’t match your classification scheme. |
Attach the policy or glossary document to any custom picker or multi-select picker field. Without it, the AI has no source of truth for valid values. |
Expecting @-mention auto-linking in generated content |
The AI can’t generate valid catalog @-mention links dynamically — it has no access to static object IDs at generation time. |
Use static hyperlinks with hard-coded catalog object IDs in the instruction, or add a constraint: Add this URL at the end: {url}. |
Not reviewing the execution report after a run |
Low-confidence objects are silently skipped. You won’t know which fields weren’t populated or why. |
Always download the CSV execution report. Filter by confidence tier to identify which objects need prompt iteration or additional catalog signals. |
Current Limitations¶
The following limitations are relevant when setting up Curation Automation rules.
Limitation |
Workaround |
|---|---|
Only one reference document can be attached per field instruction. |
Consolidate your reference material into a single well-structured document. Include the most critical information directly in the instruction text. |
Only Catalog Admins and Server Admins can create and run rules. |
Admins create and manage rules on behalf of data stewards. |
Rules only update blank fields by default. |
Select Replace existing values in Field Options to overwrite existing values. Use this deliberately and only after validating your instruction quality on a preview sample. |
No @-mention auto-linking in generated content. |
Use static hyperlinks with hard-coded catalog object IDs in the instruction. |
2,000 character limit per field instruction. |
Use conditional object-type language in your instruction (If this is a table… or If this is a column…) to share the character budget across object types. Move verbose reference material into an attached document. |
No bulk approval or rollback mechanism after a rule run. |
Use Preview mode and small pilot runs to validate results before applying at full scope. Download the execution report to review what was applied. |
Quick Reference Checklist¶
Use the following checklist before running a curation rule.
Purpose (Step 2: Enter Details)¶
Is the Purpose at least
3–5sentences? Are you using most of the1,000characters?Does it name your industry and company context?
Does it identify the data source (for example, Snowflake, SAP, Adobe Analytics, Salesforce)?
Does it describe what the data represents (for example, transactions, events, clinical records)?
Does it name the domain and the team or audience who uses it?
Does it reference the goal or standard being met?
Is it distinct from the rule title? Does it add real context beyond what the title says?
Field Instructions (Step 4: Select Fields)¶
Does each instruction repeat the domain and data source context?
Does it describe what the output must contain, not just that it should be “good”?
Does it name the intended audience?
Does it include at least one negative constraint (what to avoid)?
Does it specify length or structure?
Have you used as much of the
2,000character limit as the field warrants?For governance fields: have you attached a reference document (policy, standard, or glossary)?
If you’re using Replace existing values: have you previewed results before activating?