Configure Connection to File System Source¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Customer Managed Applies to customer-managed instances of Alation
After you install the Amazon S3 OCF connector, you must configure the connection to the Amazon S3 file system source.
The various steps involved in configuring the Amazon S3 OCF data source connection setting are:
Provide Access¶
On the Access tab, set the file system visibility as follows:
Public File System - The file system will be visible to all users of the catalog.
Private File System - The file system will be visible to the users allowed access to the file system by file system Admins.
Add new File System Admin users in the File System Admins section.
Connect to Data Source¶
To connect to the data source, you must perform these steps:
Configure Proxy Settings (Optional)
Configure Logging (Optional)
Important
The Alation user interface displays standard configuration settings for credentials and connection information stored in the Alation database. If your organization has configured Azure KeyVault or AWS Secrets Manager to hold such information, the user interface will change to include the following buttons adjacent to the respective fields:
By default, you see the user interface for Standard. In the case of Vault, instead of the actual credential information, you must select the source and provide the corresponding key. For details, see Configure Secrets for OCF Connector Settings.
Configure Authentication¶
Alation supports Basic authentication, STS authentication with an IAM user, and STS authentication with an IAM role. Before configuring authentication in Alation, ensure that you configure access and permissions for the selected authentication type. For information, see the Configure Permissions for Authentication section in Prerequisites.
Configure Basic Authentication¶
Note
Before configuring the Basic authentication in Alation, ensure that you assign the required permissions for IAM user account. For more information, see the Configure Permissions for Authentication section in Prerequisites.
To configure Basic authentication, perform these steps:
From Alation version 2023.3.5 and connector version 3.9.0
On the Settings page of your Amazon S3 file system source, go to the General Settings tab.
In the Step 1:Configure authentication section, specify a valid, active, and accessible AWS region in Region. For example,
us-east-1.
AWS GovCloud (US): Example:
us-gov-east-1.FIPS: If you have FIPS requirements, use the FIPS endpoints. Examples::
fips-us-east-1,fips-us-gov-east-1.The S3 endpoint uses the AWS region to connect to Amazon S3. To view the list of valid AWS regions, see Regional endpoints.
Select Basic and provide the following details:
Parameter
Description
AWS Access Key ID
Provide the AWS access key ID of the IAM user with basic authentication access. Ensure that the IAM user has access to the destination bucket.
AWS Access Key Secret
Provide the AWS secret access key.
Click Save.
On the Settings page of your Amazon S3 file system source, go to the General Settings tab.
Go to the Connector Settings > File System connection section.
Specify a valid, active, and accessible AWS region in Region. For example,
us-east-1.AWS GovCloud (US): Example:
us-gov-east-1.FIPS: If you have FIPS requirements, use the FIPS endpoints. Examples::
fips-us-east-1,fips-us-gov-east-1.
Select Basic Authentication and provide the following details:
Parameter
Description
AWS Access Key ID
Provide the AWS access key ID of the IAM user with basic authentication access. Ensure that the IAM user has access to the destination bucket.
AWS Access Key Secret
Provide the AWS secret access key.
Click Save.
Configure STS-IAM User Authentication¶
Note
Before configuring the STS-IAM User authentication in Alation, ensure that you assign the required permissions for IAM user account. For more information, see the Configure Permissions for Authentication section in Prerequisites.
To configure STS-IAM User authentication, perform these steps:
From Alation version 2023.3.5 and connector version 3.9.0
On the Settings page of your Amazon S3 file system source, go to the General Settings tab.
In the Step 1:Configure authentication section, specify a valid, active, and accessible AWS region in Region. For example,
us-east-1.
AWS GovCloud (US): Example:
us-gov-east-1.FIPS: If you have FIPS requirements, use the FIPS endpoints. Examples::
fips-us-east-1,fips-us-gov-east-1.The S3 endpoint uses the AWS region to connect to Amazon S3. To view the list of valid AWS regions, see Regional endpoints.
Select STS-IAM User and provide the following details:
Parameter
Description
AWS Access Key ID
Provide the AWS access key ID of the IAM user with STS authentication access. Make sure that the IAM user has access to the inventory bucket.
AWS Access Key Secret
Provide the AWS secret access key.
Role ARN
Provide the IAM role with the required permissions
Use Region-Specific Endpoint
Turn on the Use Region-Specific Endpoint toggle to use regional endpoints for STS request. When turned off, then the global endpoints will be used for STS request. For information on valid STS Endpoints, see STS Endpoints
STS Duration
- Provide the duration of the role session in seconds. Default
value: 3600 seconds.
Click Save.
On the Settings page of your Amazon S3 file system source, go to the General Settings tab.
Go to the Connector Settings > File System connection section.
Specify a valid, active, and accessible AWS region in Region. For example,
us-east-1.AWS GovCloud (US): Example:
us-gov-east-1.FIPS: If you have FIPS requirements, use the FIPS endpoints. Examples::
fips-us-east-1,fips-us-gov-east-1.
Select STS Authentication and provide the following details:
Parameter
Description
STS: AWS Access Key ID
Provide the AWS access key ID of the IAM user with STS authentication access. Make sure that the IAM user has access to the destination bucket.
STS: AWS Access Key Secret
Provide the AWS secret access key.
Role ARN
Provide the IAM role to assume with the required permissions
STS Duration
Provide the duration of the role session.
Region-Specific Endpoint
Select the Region-Specific Endpoint checkbox to use regional endpoints for STS request. If this checkbox is unselected, then the global endpoints will be used for STS request.
Click Save.
Configure STS Authentication with an AWS IAM Role¶
Note
Before configuring the STS-IAM Role authentication in Alation, ensure that you assign the required permissions for IAM user account. For more information, see the Configure Permissions for Authentication section in Prerequisites.
To configure STS-IAM Role authentication, perform these steps:
From Alation version 2023.3.5 and connector version 3.9.0
On the Settings page of your Amazon S3 file system source, go to the General Settings tab.
In the Step 1:Configure authentication section, specify a valid, active, and accessible AWS region in Region. For example,
us-east-1.
AWS GovCloud (US): Example:
us-gov-east-1.FIPS: If you have FIPS requirements, use the FIPS endpoints. Examples::
fips-us-east-1,fips-us-gov-east-1.The S3 endpoint uses the AWS region to connect to Amazon S3. To view the list of valid AWS regions, see Regional endpoints.
Select STS-IAM Role and provide the following details:
Parameter
Description
Authentication Profile
Select the authentication profile you created in Admin Settings.
Role ARN
Provide the ARN of the role that gives access to the Amazon resource.
External ID
Provide the External ID you added to the role that gives access to the amazon resource.
STS Duration
Provide the STS token duration in seconds. This value must be less than or equal to the Maximum session duration of the IAM role that provides access to the Amazon resources.
Click Save.
On the Settings page of your Amazon S3 file system source, go to the General Settings tab.
Go to the Connector Settings > File System connection section.
Specify a valid, active, and accessible AWS region in Region. For example,
us-east-1.AWS GovCloud (US): Example:
us-gov-east-1.FIPS: If you have FIPS requirements, use the FIPS endpoints. Examples::
fips-us-east-1,fips-us-gov-east-1.
Select IAM Role Authentication and provide the following details:
Parameter
Description
Auth Type
Select AWS IAM
Authentication Profile
Select the authentication profile you created in Admin Settings.
Role ARN
Provide the ARN of the role that gives access to the Amazon resource.
External ID
Provide the External ID you added to the role that gives access to the amazon resource.
STS Duration
Provide the STS token duration in seconds. This value must be less than or equal to the Maximum session duration of the IAM role that provides access to the Amazon resources.
Click Save.
Configure Proxy Settings¶
If you are using an HTTP proxy to access your S3, specify the proxy settings. These proxy fields support basic proxy and auth proxy modes.
To configure the proxy settings, perform these steps:
From Alation version 2023.3.5 and connector version 3.9.0
On the Settings page of your Amazon S3 file system source, go to the General Settings tab.
In the Proxy configuration section, provide the following details:
Parameter
Description
Proxy Host
Specify the proxy host to access S3 via the proxy server. This optional field should be used only if S3 is connected using a proxy.
This field is required for Basic Proxy and Auth Proxy modes.
Proxy Port
Specify the proxy port number. This optional field should be used only if S3 is connected using a proxy.
This field is required for Basic Proxy and Auth Proxy modes.
Proxy Username
Specify the proxy username. This optional field should be used only if S3 is connected using a proxy.
This field is required only for Auth Proxy mode.
Proxy Password
Specify the proxy username. This optional field should be used only if S3 is connected using a proxy.
This field is required only for Auth Proxy mode.
Click Save.
On the Settings page of your Amazon S3 file system source, go to the General Settings tab.
Go to the Connector Settings > File System connection section.
Provide the following details:
Parameter
Description
Proxy Host
Specify the proxy host to access S3 via the proxy server. This optional field should be used only if S3 is connected using a proxy.
This field is required for Basic Proxy and Auth Proxy modes.
Proxy Port
Specify the proxy port number. This optional field should be used only if S3 is connected using a proxy.
This field is required for Basic Proxy and Auth Proxy modes.
Proxy Username
Specify the proxy username. This optional field should be used only if S3 is connected using a proxy.
This field is required only for Auth Proxy mode.
Proxy Password
Specify the proxy username. This optional field should be used only if S3 is connected using a proxy.
This field is required only for Auth Proxy mode.
Click Save.
Test the Connection¶
The connection test checks Amazon S3 connectivity. Alation uses the connection information that you provide to confirm that a connection can be established.
After configuring authentication and providing the proxy details, if applicable, test the connection.
From Alation version 2023.3.5 and connector version 3.9.0
To validate the network connectivity, go to General Settings > Test connection of the Settings page of your file system source and click Test.
Alation performs the following checks before establishing the connection:
The AWS region provided is accessible, valid, and active in the user’s AWS account.
The AWS credentials provided are valid.
The S3 endpoint of the given region is accessible.
In case of STS based authentication, the STS endpoint is accessible.
To validate the network connectivity, go to General Settings > Test connection of the Settings page of your file system source and click Test.
A dialog box appears confirming the status of the connection test.
Configure Logging¶
To set the logging level for your Amazon S3 OCF file system source logs, perform these steps:
From Alation version 2023.3.5 and connector version 3.9.0
On the Settings page of your Amazon S3 file system source, go to the General Settings tab.
In the Connector logs section, select a logging level for the connector logs.
Click Save.
The available log levels are based on the Log4j framework.
On the Settings page of your Amazon S3 file system source, go to the General Settings tab.
In the Logging Configuration section, select a logging level for the connector logs.
Click Save.
The available log levels are based on the Log4j framework.
You can view the connector logs in Admin Settings > Server Admin > Manage Connectors > Amazon S3 OCF connector.