Project Archival Tool

Last updated 4 months ago

Overview

The Archival Tool Addon is a premium addon designed to help you clean up your storage by archiving or deleting published product versions files.

For example, if you have 100 versions of a render product for shot01, you might want to delete all but the last two, and possibly back up the remaining files to your archive directory.

Currently, you must run the tool manually as needed; it does not yet support periodic scheduling for cleanup.

Preparation

To begin, you must install the addon and ensure it's included in your production bundle. The next essential step is to adjust the Archival Tool Addon Settings according to your project or studio preferences.

Also, ensure you have the AYON launcher installed on your machine. You can find the steps for this in the Getting Started with AYON Pipeline article.

Using Archival Tool

The Archival Tool workflow involves first analysing the folder you want to clean up, generating a report and configuration, and then providing actions to apply those settings.

Follow these steps to use the tool:

  1. Analyse Folders:

    • Select the project folder from which you want the analysis to start. This can be a root folder in your project or a sub-folder.

    • In the folder web actions menu, select the “Analyse in Archival Tool” action.

  2. View Report:

    Once the analysis is complete, a report with a unique ID will be created. You can view this report in the “Archival Tool” tab within your project.

  3. Access Report Details:

    Click the Details button associated with the report entry to view the configuration and available actions.

  4. Update and Run:

    You'll be redirected to the report's detail page, where you can update the configuration via the Update config button or execute the delete action via the Run button. Alternatively, click Back to return to the main report view.

  5. Check Report Status:

    You can check the status of your report entry in the main report view tab within the Archival Tool interface. This view provides two key statuses:

    • analyse_finished: Indicates that the initial analysis of the selected folders and products is complete.

    • delete_finished: Indicates that the cleanup action (deletion or archiving) has been fully executed based on the report's configuration.

Archival Tool Addon Settings

Publish Area Configs

Setting Location: ayon+settings://archival_tool/publish

This setting allows you to define multiple profiles. You can add a new profile using the + button.

Each profile consists of:

  • Profile name: The name of the profile (e.g., default).

  • Product types: A list of product types to be deleted (e.g., pointcache).

  • Product names: A regex pattern for product names to be deleted (e.g., render.*).

  • Keep versions: The number of the last versions that should be kept and not deleted.

  • Delete in DB*: When enabled, version records will be deleted from the database.

  • Delete files: When enabled, version files will be deleted from storage.

  • Archive in ftrack: When enabled, the versions will be archived in ftrack.

  • Copy kept files*: When enabled, files that are to be kept (based on keep versions and filtering Product types or Product names) will be copied.

  • Copy to directory*: The destination folder where kept files should be copied, providing simple backup functionality.

Items marked with an asterisk (*) are options that specifically control the delete action.

Work Area Configs

Setting Location: ayon+settings://archival_tool/work

This setting consists of profiles you can add with the + button.

Each profile consists of:

  • Profile name: The name of the profile (e.g., default).

  • Directory regex: A regex pattern for sub-folders to include.

  • Delete known files: When enabled, files matching AYON's naming convention will be deleted.

  • Delete unknown files: When enabled, files not created by AYON (e.g., artist-uploaded references) will be deleted.

  • Delete folders: When enabled, empty folders will be deleted.

  • Copy kept files*: When enabled, files associated with versions that should remain will be copied.

  • Copy kept files path*: The destination folder where kept files should be copied, providing simple backup functionality.

Items marked with an asterisk (*) are options that specifically control the delete action.

Archival Tool CLI

This section, intended for TDs, outlines how to access the Archival Tool from the command-line interface (CLI) using the AYON executable. For additional information, refer to Launcher - Admin Notes.

The Archival Tool Addon provides the following CLI commands: analyse and delete.

Example
cd <ayon-launcher-installation-location>
./ayon_console.exe addon archival_tool <command> [OPTIONS]

analyse

The analyse command is used to traverse your project hierarchy below the specified input folder IDs and save the results as a report in the AYON server database.

Example
cd <ayon-launcher-installation-location>
./ayon_console.exe addon archival_tool analyse --project <project-name> --input <folder-id(s)>

Option

Description

--project <TEXT>

The project name

--input <TEXT>

A selection of folder IDs separated by a comma. e.g., 698caf20ac0b11efbf2a95506def575f, 698caf20ac0b11efbf2a95506def575f

Output of analyse:

  • A timed log is added to the addon directory (typically, {AYON_LAUNCHER_STORAGE_DIR}/addons/archival_tool_x.x.x/ayon_archival_tool/logs).

  • A new report is saved in AYON's database under the project.archival_tool_reports table.

  • The new report becomes visible in the Archive Tool tab on the AYON server, where you can find the report ID needed for the delete command.

delete

The delete command is used to execute the deletion or archiving of entries based on the configuration associated with the specified report ID.

Example
cd <ayon-launcher-installation-location>
./ayon_console.exe addon archival_tool delete --project <project-name> --report_id <report-id>

Option

Description

--project <TEXT>

The project name.

--report_id <TEXT>

The ID of the report, found in the Archive Tool tab.