Update Alation Analytics Database to 16.2 or 16.4 and a Compatible Release¶
Customer Managed Applies to customer-managed instances of Alation
Use the steps on this page to update Alation Analytics to version 2024.3.1, 2024.3, 2024.1.5, or 2024.1.4 from versions requiring the Alation Analytics PostgreSQL database upgrade to a newer major version. This includes the following updates:
From 2023.3.x to 2024.3.1 (upgrade to version 16.4)
From 2023.3.x to 2024.3 (upgrade to version 16.2)
From 2023.3.x to 2024.1.5 (upgrade to version 16.2)
From 2023.3.x to 2024.1.4 (upgrade to version 16.2)
From 2024.1.0, 2024.1.1, 2024.1.2.x, or 2024.1.3.x to 2024.3.1 (upgrade to version 16.4)
From 2024.1.0, 2024.1.1, 2024.1.2.x, or 2024.1.3.x to 2024.3 (upgrade to version 16.2)
From 2024.1.0, 2024.1.1, 2024.1.2.x, or 2024.1.3.x to 2024.1.5 (upgrade to version 16.2)
From 2024.1.0, 2024.1.1, 2024.1.2.x, or 2024.1.3.x to 2024.1.4 (upgrade to version 16.2)
These steps require the role of the Server Admin.
To update Alation Analytics:
Step 1: Verify Backup Availability¶
Ensure you have a valid latest Alation Analytics backup. Refer to Alation Analytics V2 Backup and Restore to take a backup of Alation Analytics.
Step 2: Prepare for the PostgreSQL Upgrade¶
As part of the update, the Alation Analytics PostgreSQL database will be upgraded to a newer version. To ensure a successful update:
Check Disk Space¶
Open a Terminal window.
Use SSH to connect to your Alation Analytics host.
Run the command below to check the disk space available in the partition where Alation Analytics is installed.
df -h
At least
20%
space must be available. If you discover that less than20%
space is available, add more disk space before performing the update.
Note
To calculate available disk space, subtract the Use% value from 100%
: 100% - Use% = available space
. For example, if we had to calculate the available space for the example output below, the partition has 43%
available.
ssh-user@ip-10-13-14-108:~$ df -h /opt/alation-analytics/
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p1 388G 220G 169G 57% /
Update Extensions and Drop Custom Aggregate Functions¶
To ensure a successful database upgrade, you’ll need to update database extensions and drop any custom aggregate functions of types anyarray
and anyelement
before updating, in a similar way as this was done for the Postgres component of the Alation application.
Alation provides a script that helps prepare for the Alation Analytics PostgreSQL upgrade. By the time you update Alation Analytics, you should have used this script to prepare for the Postgres upgrade.
To prepare for the Alation Analytics PostreSQL update:
Open another Terminal window or tab.
Use SSH to connect to your Alation application host.
Enter the Alation shell.
sudo /etc/init.d/alation shell
Navigate to the directory with the script pgupgrade_helper.py. For example, if you have it in the /data1/tmp, then navigate to this directory.
Change the user to
alation
.sudo su alation
Run the script with the
--pre
and--aav2
parameters. The script will update extensions on Alation Analytics and check for the presence of custom aggregate functions of typesanyarray
andanyelement
. The script will not drop the functions yet, allowing for an analysis of what customizations your Alation Analytics database may have.python pgupgrade_helper.py --pre --aav2
Analyze the output. Look for the text
Aggregate functions Check [Failed]
, and a list of functions below it.If you don’t find this line and a list of functions, the script hasn’t identified any functions to be dropped and you can continue with the Alation Analytics update. Use
exit
to exit from thealation
user and continue to Step 3: Update Alation Analytics.If you see the line
Aggregate functions Check [Failed]
and a list of functions below it, those functions need to be dropped. Continue to the next step of this instruction.Example output where such a function was identified:
(env) PROD [alation@ip-10-13-57-97 ~]$ python pgupgrade_helper.py --pre --aav2 Pre flag: True Post flag: False CPU: None Drop Function: False AAv2 datastore: True Connected to database postgres [Success] Aggregate functions Check [Failed] - array_agg_array Updating Extensions [Success] Connected to database alation_analytics_v2 [Success] Updating Extensions [Success] Connected to database template1 [Success] Updating Extensions [Success] Pre Upgrade tasks completed [Success]
Run the script again with the
--pre
,--aav2
, and--drop-func
parameters. The script will retrieve the definitions of the identified aggregate functions, record them into a file, and drop the identified aggregate functions.python pgupgrade_helper.py --pre --aav2 --drop-func
Note
The
--pre
parameter is still required during the second run as it identifies the pre-upgrade stage of the process.Example output:
(env) PROD [alation@ip-10-13-57-97 ~]$ python pgupgrade_helper.py --pre --aav2 --drop-func Pre flag: True Post flag: False CPU: None Drop Function: True AAv2 datastore: True Connected to database postgres [Success] Aggregate functions Check [Failed] - array_agg_array Dropped function array_agg_array [Success] Updating Extensions [Success] Connected to database alation_analytics_v2 [Success] Updating Extensions [Success] Connected to database template1 [Success] Updating Extensions [Success] Saved all the dropped functions definition: /home/alation/recreate-agg-function.sql Pre Upgrade tasks completed [Success]
Check the contents of the /data1/tmp/recreate-agg-function.sql file. (The path to the file is located in the line
Saved all the dropped functions definition:
of the output and depends on where you have the script. The file is generated in your working directory). The file contains the information about the function definitions of the custom functions dropped from Alation Analytics.Copy the file to your local machine, renaming it to recreate-agg-function_AlationAnalytics.sql to differentiate from the Postgres file that you may have saved previously. It’s crucial to restoring the functions after the update.
Exit from the
alation
user.exit
Continue to Step 3: Update Alation Analytics.
Step 3: Update Alation Analytics¶
To update Alation Analytics:
Switch to the Terminal window where you’re SSH’ed to the Alation Analytics host.
Create a temporary directory, for example, tmp/update-analytics:
sudo mkdir /tmp/update-analytics
Note
Do not use the installation directory /opt/alation-analytics/ to create a temporary directory and extract the update package. Create a custom temporary directory that is located elsewhere and not at /opt/alation-analytics/.
In your browser, log in to the Alation Catalog and go to Admin Settings > Alation Analytics. There will be a warning message on top of the page when Alation Analytics requires an update.
In step 1 of the installation instructions on the Settings page, copy the Curl command for downloading the newer version and paste it into the Terminal window that is already SSH’ed into the Alation Analytics host.
Untar the downloaded Alation Analytics .tar file into the temporary directory that you created in the previous step:
sudo tar -C /tmp/update-analytics -xzf ./alation-analytics-<version>.tar
Determine whether you need to use one of the available installation flags:
Flag
Description
-w
Lets you specify which system user will own the installed files, configs, and logs.
Without this flag, ownership will be assigned to the root user.
-b
Traffic to the Postgres and RabbitMQ containers will be bound to a specific IP address. The address will be stored in /etc/default/alation-analytics.env.
Without this flag, the Postgres and RabbitMQ containers are exposed on all network interfaces (0.0.0.0) in the host.
Run the installer script with the update flag
-u
to update Alation Analytics. Run the installer as the root user (if you installed rootless Docker during the original installation) or using sudo. The installer stops the current Alation Analytics Docker containers and Alation Analytics Manager, removes the outdated images from Docker, moves the contents of the new images and Manager to the installation directory, and registers and starts the new images and Manager.To update with no additional flags:
sudo /tmp/update-analytics/alation-analytics-<version>/alation-analytics-installer-<version> -u
Example:
sudo /tmp/update-analytics/alation-analytics-1.1.0.139590/alation-analytics-installer-v-1.0.18 -u
To update and change the ownership to a different user, run the update command with the flag
-w
, as shown below, and follow the prompts in the console to provide a new username.sudo /tmp/update-analytics/alation-analytics-<version>/alation-analytics-installer-v-x.x.x -u -w <username>
Example:
sudo /tmp/update-analytics/alation-analytics-<version>/alation-analytics-installer-v-x.x.x -u -w aav2_admin
To update and bind incoming traffic to a specific IP address, run the update command with the flag
-b
:sudo /tmp/update-analytics/alation-analytics-<version>/alation-analytics-installer-v-x.x.x -u -b <ip_address>
Example:
sudo /tmp/update-analytics/alation-analytics-<version>/alation-analytics-installer-v-x.x.x -u -b 10.13.14.108
The RabbitMQ component requires basic authentication. If you haven’t set a RabbitMQ password yet, the Alation Analytics updater will automatically generate one for you and display it in the console when you run the update command. Copy and securely store this password. You’ll need to set it in alation_conf in a subsequent step of this instruction. For help with alation_conf, refer to Using alation_conf.
If the RabbitMQ password was generated for your instance during the update, then after the update, use alation_conf to set it in the parameter
alation_analytics-v2.rmq.config.password
:On the Alation server, enter the Alation shell.
sudo /etc/init.d/alation shell
Change to the
alation
user.sudo su alation
Set the value in alation_conf.
alation_conf alation_analytics-v2.rmq.config.password -s 'your_password'
Restart the Celery and Web components.
alation_supervisor restart celery:* web:*
Note
The
aamanager
service may not start automatically after the update of Alation Analytics.To check the status of
aamanager
, on the Alation Analytics host, use the commandsudo service aamanager status
.To start it manually, use the command:
sudo service aamanager start
.Update extensions and restore custom aggregate functions if you previously dropped any.
Step 4: Update Extensions and Restore Custom Aggregate Functions¶
After updating Alation Analytics, it is important to update the extensions and restore the functions if any were dropped to ensure compatibility with the new version.
Update Extensions¶
To update the extensions on Alation Analytics:
Switch to the Terminal window where you are SSH’ed to the Alation host.
We recommend running the next command using a terminal multiplexer, such as Screen. If Screen is available, start a screen session.
screen -S alation-postgres
Enter the Alation shell.
sudo /etc/init.d/alation shell
Change the user to
alation
.sudo su alation
Navigate to the directory where you placed the pgupgrade_helper.py file.
Run the script with the
--post
and--aav2
parameters. The script will update PostgreSQL extensions and run an analysis of the Alation Analytics PostgreSQL database.python pgupgrade_helper.py --post --aav2
Example output:
(env) PROD [alation@ip-10-13-57-97 ~]$ python pgupgrade_helper.py --post --aav2 Pre flag: False Post flag: True CPU: None Drop Function: False AAv2 datastore: True Connected to database template1 [Success] Updating Extensions [Success] Connected to database postgres [Success] Updating Extensions [Success] Connected to database alation_analytics_v2 [Success] Updating Extensions [Success] vacuumdb: vacuuming database "alation_analytics_v2" vacuumdb: vacuuming database "postgres" vacuumdb: vacuuming database "template1" Analyze AAv2 DB [Success] Post Upgrade tasks completed [Success]
Your next step depends on whether or not any functions were dropped before the update:
If you previously didn’t identify and drop any custom aggregate functions, do the following:
Exit from the
alation
user:exit
Exit from the Alation shell:
exit
Continue to Step 5: Initiate the Alation Analytics Database.
If you previously dropped any custom aggregate functions, then restore them manually.
Restore Custom Aggregate Functions¶
Review the function definitions from the recreate-agg-function_AlationAnalytics.sql file generated by the script in Step 2: Prepare for the PostgreSQL Upgrade.
Update the SQL code as necessary to be compatible with the newer PostgreSQL version. You may need to change
anyarray
toanycompatiblearray
andanyelement
toanycompatible
or update the SQL code in other ways, depending on your specific customization.After preparing the SQL code, switch to the Terminal window where you’re SSH’ed to the Alation Analytics host.
Use the
docker exec
command to enter the bash shell for the Postgres container.sudo docker exec -it postgres bash
Enter the Postgres shell.
bash-5.0# psql -U postgres
Run the following command for each function to be recreated, replacing the placeholder
func-definition
with the SQL code of the specific definition you are restoring.CREATE AGGREGATE func-definition;
Example:
CREATE AGGREGATE public.array_agg_array(anyarray) (sfunc = array_cat, stype = anyarray);
Exit the Postgres shell.
\q
Continue to Step 5: Initiate the Alation Analytics Database.
Step 5: Initiate the Alation Analytics Database¶
Log in to Alation and navigate to Admin Settings > Alation Analytics page. In the Alation Analytics installation instructions section, click the Initiate Analytics Database button to run the migration and finalize the update.
Test your Alation Analytics instances post-update. If you identify any issues, contact Alation Support.