Update Alation Analytics Database to 16.2 or 16.4 and a Compatible Release

Customer Managed Applies to customer-managed instances of Alation

Use the steps on this page to update Alation Analytics to version 2024.3.1, 2024.3, 2024.1.5, or 2024.1.4 from versions requiring the Alation Analytics PostgreSQL database upgrade to a newer major version. This includes the following updates:

• From 2023.3.x to 2024.3.1 or 2024.3.2 (upgrade to version 16.4)

• From 2023.3.x to 2024.3 (upgrade to version 16.2)

• From 2023.3.x to 2024.1.5 (upgrade to version 16.2)

• From 2023.3.x to 2024.1.4 (upgrade to version 16.2)

• From 2024.1.0, 2024.1.1, 2024.1.2.x, or 2024.1.3.x to 2024.3.1 or 2024.3.2 (upgrade to version 16.4)

• From 2024.1.0, 2024.1.1, 2024.1.2.x, or 2024.1.3.x to 2024.3 (upgrade to version 16.2)

• From 2024.1.0, 2024.1.1, 2024.1.2.x, or 2024.1.3.x to 2024.1.5 (upgrade to version 16.2)

• From 2024.1.0, 2024.1.1, 2024.1.2.x, or 2024.1.3.x to 2024.1.4 (upgrade to version 16.2)

These steps require the role of the Server Admin.

To update Alation Analytics:

• Step 1: Verify Backup Availability

• Step 2: Prepare for the PostgreSQL Upgrade

• Step 3: Update Alation Analytics

• Step 4: Update Extensions and Restore Custom Aggregate Functions

Step 1: Verify Backup Availability

Ensure you have a valid latest Alation Analytics backup. Refer to Alation Analytics V2 Backup and Restore to take a backup of Alation Analytics.

Step 2: Prepare for the PostgreSQL Upgrade

As part of the update, the Alation Analytics PostgreSQL database will be upgraded to a newer version. To ensure a successful update:

• Check Disk Space

• Update Extensions and Drop Custom Aggregate Functions

Check Disk Space

1. Open a Terminal window.

2. Use SSH to connect to your Alation Analytics host.

3. Run the command below to check the disk space available in the partition where Alation Analytics is installed.

   df -h
   

4. At least 20% space must be available. If you discover that less than 20% space is available, add more disk space before performing the update.

Note

To calculate available disk space, subtract the Use% value from 100%: 100% - Use% = available space. For example, if we had to calculate the available space for the example output below, the partition has 43% available.

ssh-user@ip-10-13-14-108:~$ df -h /opt/alation-analytics/
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p1  388G  220G  169G  57% /

Update Extensions and Drop Custom Aggregate Functions

To ensure a successful database upgrade, you’ll need to update database extensions and drop any custom aggregate functions of types anyarray and anyelement before updating, in a similar way as this was done for the Postgres component of the Alation application.

Alation provides a script that helps prepare for the Alation Analytics PostgreSQL upgrade. By the time you update Alation Analytics, you should have used this script to prepare for the Postgres upgrade.

To prepare for the Alation Analytics PostreSQL update:

1. Open another Terminal window or tab.

2. Use SSH to connect to your Alation application host.

3. Enter the Alation shell.

   sudo /etc/init.d/alation shell
   

4. Navigate to the directory with the script pgupgrade_helper.py. For example, if you have it in the /data1/tmp, then navigate to this directory.

5. Change the user to alation.

   sudo su alation
   

6. Run the script with the --pre and --aav2 parameters. The script will update extensions on Alation Analytics and check for the presence of custom aggregate functions of types anyarray and anyelement. The script will not drop the functions yet, allowing for an analysis of what customizations your Alation Analytics database may have.

   python pgupgrade_helper.py --pre --aav2
   

7. Analyze the output. Look for the text Aggregate functions Check [Failed], and a list of functions below it.

   • If you don’t find this line and a list of functions, the script hasn’t identified any functions to be dropped and you can continue with the Alation Analytics update. Use exit to exit from the alation user and continue to Step 3: Update Alation Analytics.

   • If you see the line Aggregate functions Check [Failed] and a list of functions below it, those functions need to be dropped. Continue to the next step of this instruction.

     Example output where such a function was identified:

     (env) PROD [alation@ip-10-13-57-97 ~]$ python pgupgrade_helper.py --pre --aav2
     Pre flag: True
     Post flag: False
     CPU: None
     Drop Function: False
     AAv2 datastore: True
     Connected to database postgres [Success]
     Aggregate functions Check [Failed]
     - array_agg_array
     Updating Extensions [Success]
     Connected to database alation_analytics_v2 [Success]
     Updating Extensions [Success]
     Connected to database template1 [Success]
     Updating Extensions [Success]
     Pre Upgrade tasks completed [Success]
     

8. Run the script again with the --pre, --aav2, and --drop-func parameters. The script will retrieve the definitions of the identified aggregate functions, record them into a file, and drop the identified aggregate functions.

   python pgupgrade_helper.py --pre --aav2 --drop-func
   

   Note

   The --pre parameter is still required during the second run as it identifies the pre-upgrade stage of the process.

   Example output:

   (env) PROD [alation@ip-10-13-57-97 ~]$ python pgupgrade_helper.py --pre --aav2 --drop-func
   Pre flag: True
   Post flag: False
   CPU: None
   Drop Function: True
   AAv2 datastore: True
   Connected to database postgres [Success]
   Aggregate functions Check [Failed]
   - array_agg_array
   Dropped function array_agg_array [Success]
   Updating Extensions [Success]
   Connected to database alation_analytics_v2 [Success]
   Updating Extensions [Success]
   Connected to database template1 [Success]
   Updating Extensions [Success]
   Saved all the dropped functions definition: /home/alation/recreate-agg-function.sql
   Pre Upgrade tasks completed [Success]
   

9. Check the contents of the /data1/tmp/recreate-agg-function.sql file. (The path to the file is located in the line Saved all the dropped functions definition: of the output and depends on where you have the script. The file is generated in your working directory). The file contains the information about the function definitions of the custom functions dropped from Alation Analytics.

10. Copy the file to your local machine, renaming it to recreate-agg-function_AlationAnalytics.sql to differentiate from the Postgres file that you may have saved previously. It’s crucial to restoring the functions after the update.

11. Exit from the alation user.

    exit
    

12. Continue to Step 3: Update Alation Analytics.

Step 3: Update Alation Analytics

To update Alation Analytics:

1. Switch to the Terminal window where you’re SSH’ed to the Alation Analytics host.

2. Create a temporary directory, for example, tmp/update-analytics:

   sudo mkdir /tmp/update-analytics
   

   Note

   Do not use the installation directory /opt/alation-analytics/ to create a temporary directory and extract the update package. Create a custom temporary directory that is located elsewhere and not at /opt/alation-analytics/.

3. In your browser, log in to the Alation Catalog and go to Admin Settings > Alation Analytics. There will be a warning message on top of the page when Alation Analytics requires an update.

4. In step 1 of the installation instructions on the Settings page, copy the Curl command for downloading the newer version and paste it into the Terminal window that is already SSH’ed into the Alation Analytics host.

5. Untar the downloaded Alation Analytics .tar file into the temporary directory that you created in the previous step:

   sudo tar -C /tmp/update-analytics -xzf ./alation-analytics-<version>.tar
   

6. Determine whether you need to use one of the available installation flags:

   Flag	Description
   -w	Lets you specify which system user will own the installed files, configs, and logs. Without this flag, ownership will be assigned to the root user.
   -b	Traffic to the Postgres and RabbitMQ containers will be bound to a specific IP address. The address will be stored in /etc/default/alation-analytics.env. Without this flag, the Postgres and RabbitMQ containers are exposed on all network interfaces (0.0.0.0) in the host.

7. Run the installer script with the update flag -u to update Alation Analytics. Run the installer as the root user (if you installed rootless Docker during the original installation) or using sudo. The installer stops the current Alation Analytics Docker containers and Alation Analytics Manager, removes the outdated images from Docker, moves the contents of the new images and Manager to the installation directory, and registers and starts the new images and Manager.

   • To update with no additional flags:

     sudo /tmp/update-analytics/alation-analytics-<version>/alation-analytics-installer-<version> -u
     

     Example:

     sudo /tmp/update-analytics/alation-analytics-1.1.0.139590/alation-analytics-installer-v-1.0.18 -u
     

   • To update and change the ownership to a different user, run the update command with the flag -w, as shown below, and follow the prompts in the console to provide a new username.

     sudo /tmp/update-analytics/alation-analytics-<version>/alation-analytics-installer-v-x.x.x -u -w <username>
     

     Example:

     sudo /tmp/update-analytics/alation-analytics-<version>/alation-analytics-installer-v-x.x.x -u -w aav2_admin
     

   • To update and bind incoming traffic to a specific IP address, run the update command with the flag -b:

     sudo /tmp/update-analytics/alation-analytics-<version>/alation-analytics-installer-v-x.x.x -u -b <ip_address>
     

     Example:

     sudo /tmp/update-analytics/alation-analytics-<version>/alation-analytics-installer-v-x.x.x -u -b 10.13.14.108
     

8. The RabbitMQ component requires basic authentication. If you haven’t set a RabbitMQ password yet, the Alation Analytics updater will automatically generate one for you and display it in the console when you run the update command. Copy and securely store this password. You’ll need to set it in alation_conf in a subsequent step of this instruction. For help with alation_conf, refer to Using alation_conf.

   

9. If the RabbitMQ password was generated for your instance during the update, then after the update, use alation_conf to set it in the parameter alation_analytics-v2.rmq.config.password:

   1. On the Alation server, enter the Alation shell.

      sudo /etc/init.d/alation shell
      

   2. Change to the alation user.

      sudo su alation
      

   3. Set the value in alation_conf.

      alation_conf alation_analytics-v2.rmq.config.password -s 'your_password'
      

   4. Restart the Celery and Web components.

      alation_supervisor restart celery:* web:*
      

   Note

   The aamanager service may not start automatically after the update of Alation Analytics.

   To check the status of aamanager, on the Alation Analytics host, use the command sudo service aamanager status.

   To start it manually, use the command: sudo service aamanager start.

10. Update extensions and restore custom aggregate functions if you previously dropped any.

Step 4: Update Extensions and Restore Custom Aggregate Functions

After updating Alation Analytics, it is important to update the extensions and restore the functions if any were dropped to ensure compatibility with the new version.

Update Extensions

To update the extensions on Alation Analytics:

1. Switch to the Terminal window where you are SSH’ed to the Alation host.

2. We recommend running the next command using a terminal multiplexer, such as Screen. If Screen is available, start a screen session.

   screen -S alation-postgres
   

3. Enter the Alation shell.

   sudo /etc/init.d/alation shell
   

4. Change the user to alation.

   sudo su alation
   

5. Navigate to the directory where you placed the pgupgrade_helper.py file.

6. Run the script with the --post and --aav2 parameters. The script will update PostgreSQL extensions and run an analysis of the Alation Analytics PostgreSQL database.

   python pgupgrade_helper.py --post --aav2
   

   Example output:

   (env) PROD [alation@ip-10-13-57-97 ~]$ python pgupgrade_helper.py --post --aav2
   Pre flag: False
   Post flag: True
   CPU: None
   Drop Function: False
   AAv2 datastore: True
   Connected to database template1 [Success]
   Updating Extensions [Success]
   Connected to database postgres [Success]
   Updating Extensions [Success]
   Connected to database alation_analytics_v2 [Success]
   Updating Extensions [Success]
   vacuumdb: vacuuming database "alation_analytics_v2"
   vacuumdb: vacuuming database "postgres"
   vacuumdb: vacuuming database "template1"
   Analyze AAv2 DB [Success]
   Post Upgrade tasks completed [Success]
   

7. Your next step depends on whether or not any functions were dropped before the update:

   • If you previously didn’t identify and drop any custom aggregate functions, do the following:

     1. Exit from the alation user: exit

     2. Exit from the Alation shell: exit

     3. Continue to Step 5: Initiate the Alation Analytics Database.

   • If you previously dropped any custom aggregate functions, then restore them manually.

Restore Custom Aggregate Functions

1. Review the function definitions from the recreate-agg-function_AlationAnalytics.sql file generated by the script in Step 2: Prepare for the PostgreSQL Upgrade.

2. Update the SQL code as necessary to be compatible with the newer PostgreSQL version. You may need to change anyarray to anycompatiblearray and anyelement to anycompatible or update the SQL code in other ways, depending on your specific customization.

3. After preparing the SQL code, switch to the Terminal window where you’re SSH’ed to the Alation Analytics host.

4. Use the docker exec command to enter the bash shell for the Postgres container.

   sudo docker exec -it postgres bash
   

5. Enter the Postgres shell.

   bash-5.0# psql -U postgres
   

6. Run the following command for each function to be recreated, replacing the placeholder func-definition with the SQL code of the specific definition you are restoring.

   CREATE AGGREGATE func-definition;
   

   Example:

   CREATE AGGREGATE public.array_agg_array(anyarray) (sfunc = array_cat, stype = anyarray);
   

7. Exit the Postgres shell.

   \q
   

8. Continue to Step 5: Initiate the Alation Analytics Database.

Step 5: Initiate the Alation Analytics Database

Log in to Alation and navigate to Admin Settings > Alation Analytics page. In the Alation Analytics installation instructions section, click the Initiate Analytics Database button to run the migration and finalize the update.

Test your Alation Analytics instances post-update. If you identify any issues, contact Alation Support.