Automations
An Automation is an automated process of controlling workflows on your Files.com site.
Automations are different from Behaviors because Behaviors are associated with a current folder, while Automations apply across your entire site.
Automations are never removed when folders are removed, while Behaviors are removed when the associated folder is removed.
Path Matching
The path attribute specifies which folders this automation applies to.
It gets combined with the source attribute to determine which files are actually affected by the automation.
Note that the path attribute supports globs, and only refers to folders.
It's the source attribute, which also supports globs, combined with the path attribute that determines which files are affected, and automations only operate on the files themselves.
Additionally, paths in Automations can refer to folders which don't yet exist.
Path Globs
Although Automations may have a path specified, it can be a glob (which includes wildcards), which affects multiple folders.
* matches any folder at that level of the path, but not subfolders. For example, path/to/* matches path/to/folder1 and path/to/folder2, but not path/to/folder1/subfolder.
** matches subfolders recursively. For example, path/to/** matches path/to/folder1, path/to/folder1/subfolder, path/to/folder2, path/to/folder2/subfolder, etc.
? matches any one character.
Use square brackets [] to match any character from a set. This works like a regular expression, including negation using ^.
Curly brackets {} can be used to denote parts of a pattern which will accept a number of alternatives, separated by commas ,.
These alternatives can either be literal text or include special characters including nested curly brackets.
For example {Mon,Tue,Wed,Thu,Fri} would match abbreviated weekdays, and 202{3-{0[7-9],1?},4-0[1-6]}-* would match dates from 2023-07-01 through 2024-06-30.
To match any of the special characters literally, precede it with a backslash and enclose that pair with square brackets. For example to match a literal ?, use [\?].
Globs are supported on path, source, and exclude_pattern fields. Globs are not supported on remote paths of any kind or for any field.
By default, Copy and Move automations that use globs will implicitly replicate matched folder structures at the destination. If you want to flatten the folder structure, set flatten_destination_structure to true.
Automation Triggers
Automations can be triggered in the following ways:
custom_schedule: The automation will run according to either the reusable Site-level Schedule selected byschedule_idor its own custom schedule fields fordays_of_week(0-based) andtimes_of_day. A time zone may be specified viatime_zonein Rails TimeZone name format.daily: The automation will run in a pickedinterval. You can specifyrecurring_dayorrecurring_daysto select one or more day numbers inside the interval.webhook: the automation will run when a request is sent to the corresponding webhook URL.action: The automation will run when a specific action happens, e.g. a file is created or downloaded.
Future enhancements will allow Automations to be triggered by an incoming email, or by other services.
Currently, all Automation types support all triggers, with the following exceptions: Create Folder and Run Remote Server Sync are not supported by the action trigger.
Automations can be triggered manually if trigger is not set to action.
Destinations
The destinations parameter is a list of paths where files will be copied, moved, or created. It may include formatting parameters to dynamically determine the destination at runtime.
Relative vs. Absolute Paths
In order to specify a relative path, it must start with either ./ or ../. All other paths are considered absolute. In general, leading slashes should never be used on Files.com paths, including here. Paths are interpreted as absolute in all contexts, even without a leading slash.
Files vs. Folders
If the destination path ends with a /, the filename from the source path will be preserved and put into the folder of this name. If the destination path does not end with a /, it will be interpreted as a filename and will override the source file's filename entirely.
Formatting Parameters
Action-Triggered Automations
%tf: The name of the file that triggered the automation.%tp: The path of the file that triggered the automation.%td: The directory of the file that triggered the automation.%tb: The name of the file (without extension) that triggered the automation.%te: The extension of the file that triggered the automation.
For example, if the triggering file is at path/to/file.txt, then the automation destination path/to/dest/incoming-%tf will result in the actual destination being path/to/dest/incoming-file.txt.
Parent Folders
To reference the parent folder of a source file, use %p1, %p2, %p3, etc. for the first, second, third, etc. parent folder, respectively.
To reference path components from the root downward, use %P1, %P2, %P3, etc. for the first, second, third, etc. path component, respectively.
For example, if the source file is at accounts/file.txt, then the automation destination path/to/dest/%p1/some_file_name.txt will result in the actual destination being path/to/dest/accounts/some_file_name.txt.
If the source file is at partner/app/team/inbound/file.txt, then the automation destination path/to/dest/%P1/%P3/%P4/file.txt will result in the actual destination being path/to/dest/partner/team/inbound/file.txt.
Source File Name
To reference the name of the source file being processed, use the following tokens:
%Ff: The name of the source file, with extension.%Fb: The name of the source file, without extension.%Fe: The extension of the source file.%Fl: The name of the source file, with extension, converted to lowercase.%Fn: The name of the source file, without non-alphanumeric characters, with extension.%Fp: The name of the source file, with extension, spaces removed, lowercase, non-ASCII normalized.
For example, if the source file is Daily Report.xlsx and the destination is archive/%Y-%m-%d/%Fb.xlsx, the resolved destination will be archive/2024-01-15/Daily Report.xlsx.
Dates and Times
%Y: The current year (4 digits)%m: The current month (2 digits)%B: The current month (full name)%d: The current day (2 digits)%H: The current hour (2 digits, 24-hour clock)%M: The current minute (2 digits)%S: The current second (2 digits)%z: UTC Time Zone (e.g. -0900)
For example, if the current date is June 23, 2023 and the source file is named daily_sales.csv, then the following automation destination path/to/dest/%Y/%m/%d/ will result in the actual destination being path/to/dest/2023/06/23/daily_sales.csv.
Replacing Text
To replace text in the source filename, use the destination_replace_from and destination_replace_to parameters. This will perform a simple text replacement on the source filename before inserting it into the destination path.
For example, if the destination_replace_from is incoming and the destination_replace_to is outgoing, then path/to/incoming.txt will translate to path/to/outgoing.txt.
Automation Types
There are several types of automations: Create Folder, Copy File, Move File, Delete File and, Run Remote Server Sync.
Create Folder
Creates the folder with named by destinations in the path named by path.
Destination may include formatting parameters to insert the date/time into the destination name.
Example Use case: Our business files sales tax for each division in 11 states every quarter. I want to create the folders where those sales tax forms and data will be collected.
I could create a Create Folder automation as follows:
- Trigger:
daily - Interval:
quarter_end - Path:
AccountingAndTax/SalesTax/State/*/ - Destinations:
%Y/Quarter-ending-%m-%d
Note this assumes you have folders in AccountingAndTax/SalesTax/State/ already created for each state, e.g. AccountingAndTax/SalesTax/State/CA/.
Delete File
Deletes the file with path matching source (wildcards allowed) in the path named by path.
Copy File
Copies files in the folder named by path to the path specified in destinations.
The automation will only fire on files matching the source (wildcards allowed). In the case of an action-triggered automation, it will only operate on the actual file that triggered the automation.
If the parameter limit exists, the automation will only copy the newest limit files in each matching folder.
Move File
Moves files in the folder named by path to the path specified in destinations.
The automation will only fire on files matching the source (wildcards allowed). In the case of an action-triggered automation, it will only operate on the actual file that triggered the automation.
If the parameter limit exists, the automation will only move the newest limit files in each matching folder.
Note that for a move with multiple destinations, all but one destination is treated as a copy.
Run Remote Server Sync
The Run Remote Server Sync automation runs the remote server syncs specified by the sync_ids.
Typically when this automation is used, the remote server syncs in question are set to the manual
scheduling mode (manual to true via the API) to disable the built in sync scheduler.
Import File
Retrieves files from one or more URLs and saves the results under the path specified in destinations.
The URLs to retrieve are specified as a JSON array in the import_urls property.
The recognized keys are:
name: The file name which will be used to save the returned content. Required.%tokens will be replaced as described under Formatting Parameters.url: The URL which will be requested. Required.method: The HTTP method to be used for the request. May be eithergetorpost(case insensitive). Defaults toget.headers: Optional headers to be included in the request.%tokens in the values will be replaced as described under Formatting Parameters.content: Optional body to send for POST request. If supplied as a string,%tokens will be expanded. If supplied as a JSON Object,%tokens will be expanded for top-level values. Other JSON types will be sent as-is.
Help us build the future of Automations
Do you have an idea for something that would work well as a Files.com Automation? Let us know! We are actively improving the types of automations offered on our platform.
Retrying Failures
Automations will automatically retry individual action steps up to 3 times, with pauses between retries that increase from 15 seconds to 1 minute. If individual action steps fail after our 3rd attempt, that action will fail. If every action step in an Automation Run fails, that automation run will move to a failure status. If at least one step succeeds and one step fails, that automation run will move to a partial_failure status.
Automation Runs can be retried automatically when they enter a failure or partial_failure status as described above. A retry will re-run the automation from scratch, including the "planning" phase, which expands globs (wildcards) and identifies which files to transfer or skip.
Retrying of entire Automation Runs must be explicitly enabled by setting the retry_on_failure_interval_in_minutes and retry_on_failure_number_of_attempts values on the Automation.
Resource Schema
Required
| Property | Description |
|---|---|
| automation String | Automation type Possible values: create_folder, delete_file, copy_file, move_file, as2_send, run_sync, import_file, v2 |
Optional
| Property | Description |
|---|---|
| workspace_id Int64 | Workspace ID |
| always_serialize_jobs Bool | Ordinarily, we will allow automation runs to run in parallel for non-scheduled automations. If this flag is true we will force automation runs to be serialized (run one at a time, one after another). This can resolve some issues with race conditions on remote systems at the cost of some performance. |
| always_overwrite_size_matching_files Bool | Ordinarily, files with identical size in the source and destination will be skipped from copy operations to prevent wasted transfer. If this flag is true we will overwrite the destination file always. Note that this may cause large amounts of wasted transfer usage. This setting has no effect unless overwrite_files is also set to true. |
| description String | Description for the this Automation. |
| definition Object | Automation v2 graph definition. |
| destination_replace_from String | If set, this string in the destination path will be replaced with the value in destination_replace_to. |
| destination_replace_to String | If set, this string will replace the value destination_replace_from in the destination filename. You can use special patterns here. |
| destinations List | Destination Paths |
| disabled Bool | If true, this automation will not run. |
| exclude_pattern String | If set, this glob pattern will exclude files from the automation. Supports globs, except on remote mounts. |
| import_urls Dynamic | List of URLs to be imported and names to be used. |
| flatten_destination_structure Bool | Normally copy and move automations that use globs will implicitly preserve the source folder structure in the destination. If this flag is true, the source folder structure will be flattened in the destination. This is useful for copying or moving files from multiple folders into a single destination folder. |
| group_ids List | IDs of Groups for the Automation (i.e. who to Request File from) |
| ignore_locked_folders Bool | If true, the Lock Folders behavior will be disregarded for automated actions. |
| interval String | If trigger is daily, this specifies how often to run this automation. One of: day, week, week_end, month, month_end, quarter, quarter_end, year, year_end |
| legacy_folder_matching Bool | If true, use the legacy behavior for this automation, where it can operate on folders in addition to just files. This behavior no longer works and should not be used. |
| name String | Name for this automation. |
| overwrite_files Bool | If true, existing files will be overwritten with new files on Move/Copy automations. Note: by default files will not be overwritten on Copy automations if they appear to be the same file size as the newly incoming file. Use the always_overwrite_size_matching_files option in conjunction with overwrite_files to override this behavior and overwrite files no matter what. |
| path String | Path on which this Automation runs. Supports globs, except on remote mounts. This must be slash-delimited, but it must neither start nor end with a slash. Maximum of 5000 characters. |
| path_time_zone String | Timezone to use when rendering timestamps in paths. |
| recurring_day Int64 | If trigger type is daily, this specifies a day number to run in one of the supported intervals: week, month, quarter, year. |
| recurring_days List | If trigger type is daily, this specifies one or more day numbers to run in one of the supported intervals: week, month, quarter, year. |
| schedule_id Int64 | If trigger is custom_schedule, the reusable Schedule used instead of the automation's schedule fields. |
| retry_on_failure_interval_in_minutes Int64 | If the Automation fails, retry at this interval (in minutes). Acceptable values are 5 through 1440 (one day). Set to null to disable. |
| retry_on_failure_number_of_attempts Int64 | If the Automation fails, retry at most this many times. Maximum allowed value: 10. Set to null to disable. |
| schedule_days_of_week List | If trigger is custom_schedule, Custom schedule description for when the automation should be run. 0 is Sunday, 1 is Monday, etc. |
| schedule_times_of_day List | Times of day to run in HH:MM format (24-hour). For custom_schedule, run at these times on specified days of week. For daily, run at these times on the scheduled interval date. |
| schedule_time_zone String | Time zone for the schedule. If not set, times are interpreted as UTC. |
| source String | Source path/glob. See Automation docs for exact description, but this is used to filter for files in the path to find files to operate on. Supports globs, except on remote mounts. |
| sync_ids List | IDs of Syncs to run by this Automation. |
| trigger_actions List | If trigger is action, this is the list of action types on which to trigger the automation. Valid actions are create, copy, move, archived_delete, update, read, destroy |
| trigger String | How this automation is triggered to run. Possible values: manual, daily, custom_schedule, webhook, email, action |
| user_ids List | IDs of Users for the Automation (i.e. who to Request File from) |
| value Dynamic | A Hash of attributes specific to the automation type. |
| holiday_region String | Skip the automation if there is a formal, observed holiday for this region. |
Read Only
| Property | Description |
|---|---|
| id Int64 | Automation ID |
| converted_automation_id Int64 | ID of this v1 Automation's converted v2 copy, when present. |
| deleted Bool | Indicates if the automation has been deleted. |
| inbound_email_address String | If trigger is email, this is the address that triggers the Automation. |
| last_modified_at String | Time when automation was last modified. Does not change for name or description updates. |
| original_automation_id Int64 | ID of the retained v1 Automation for a converted Automation. Re-enable that v1 Automation to pause this copy and revert the conversion. |
| version Int64 | Current Automation v2 definition version. |
| schedule Dynamic | If trigger is custom_schedule, Custom schedule description for when the automation should be run in json format. |
| human_readable_schedule String | If trigger is custom_schedule or daily with times, Human readable schedule description for when the automation should be run. |
| user_id Int64 | User ID of the Automation's creator. |
| webhook_url String | If trigger is webhook, this is the URL of the webhook to trigger the Automation. |
Resource Import
This Resource supports importing using the following syntax:
Data Source Schema
Required
| Property | Description |
|---|---|
| id Int64 | Automation ID |
Read Only
| Property | Description |
|---|---|
| workspace_id Int64 | Workspace ID |
| always_serialize_jobs Bool | Ordinarily, we will allow automation runs to run in parallel for non-scheduled automations. If this flag is true we will force automation runs to be serialized (run one at a time, one after another). This can resolve some issues with race conditions on remote systems at the cost of some performance. |
| always_overwrite_size_matching_files Bool | Ordinarily, files with identical size in the source and destination will be skipped from copy operations to prevent wasted transfer. If this flag is true we will overwrite the destination file always. Note that this may cause large amounts of wasted transfer usage. This setting has no effect unless overwrite_files is also set to true. |
| automation String | Automation type Possible values: create_folder, delete_file, copy_file, move_file, as2_send, run_sync, import_file, v2 |
| converted_automation_id Int64 | ID of this v1 Automation's converted v2 copy, when present. |
| deleted Bool | Indicates if the automation has been deleted. |
| description String | Description for the this Automation. |
| definition Object | Automation v2 graph definition. |
| destination_replace_from String | If set, this string in the destination path will be replaced with the value in destination_replace_to. |
| destination_replace_to String | If set, this string will replace the value destination_replace_from in the destination filename. You can use special patterns here. |
| destinations List | Destination Paths |
| disabled Bool | If true, this automation will not run. |
| exclude_pattern String | If set, this glob pattern will exclude files from the automation. Supports globs, except on remote mounts. |
| import_urls Dynamic | List of URLs to be imported and names to be used. |
| inbound_email_address String | If trigger is email, this is the address that triggers the Automation. |
| flatten_destination_structure Bool | Normally copy and move automations that use globs will implicitly preserve the source folder structure in the destination. If this flag is true, the source folder structure will be flattened in the destination. This is useful for copying or moving files from multiple folders into a single destination folder. |
| group_ids List | IDs of Groups for the Automation (i.e. who to Request File from) |
| ignore_locked_folders Bool | If true, the Lock Folders behavior will be disregarded for automated actions. |
| interval String | If trigger is daily, this specifies how often to run this automation. One of: day, week, week_end, month, month_end, quarter, quarter_end, year, year_end |
| last_modified_at String | Time when automation was last modified. Does not change for name or description updates. |
| legacy_folder_matching Bool | If true, use the legacy behavior for this automation, where it can operate on folders in addition to just files. This behavior no longer works and should not be used. |
| name String | Name for this automation. |
| original_automation_id Int64 | ID of the retained v1 Automation for a converted Automation. Re-enable that v1 Automation to pause this copy and revert the conversion. |
| overwrite_files Bool | If true, existing files will be overwritten with new files on Move/Copy automations. Note: by default files will not be overwritten on Copy automations if they appear to be the same file size as the newly incoming file. Use the always_overwrite_size_matching_files option in conjunction with overwrite_files to override this behavior and overwrite files no matter what. |
| path String | Path on which this Automation runs. Supports globs, except on remote mounts. This must be slash-delimited, but it must neither start nor end with a slash. Maximum of 5000 characters. |
| path_time_zone String | Timezone to use when rendering timestamps in paths. |
| version Int64 | Current Automation v2 definition version. |
| recurring_day Int64 | If trigger type is daily, this specifies a day number to run in one of the supported intervals: week, month, quarter, year. |
| recurring_days List | If trigger type is daily, this specifies one or more day numbers to run in one of the supported intervals: week, month, quarter, year. |
| schedule_id Int64 | If trigger is custom_schedule, the reusable Schedule used instead of the automation's schedule fields. |
| retry_on_failure_interval_in_minutes Int64 | If the Automation fails, retry at this interval (in minutes). Acceptable values are 5 through 1440 (one day). Set to null to disable. |
| retry_on_failure_number_of_attempts Int64 | If the Automation fails, retry at most this many times. Maximum allowed value: 10. Set to null to disable. |
| schedule Dynamic | If trigger is custom_schedule, Custom schedule description for when the automation should be run in json format. |
| human_readable_schedule String | If trigger is custom_schedule or daily with times, Human readable schedule description for when the automation should be run. |
| schedule_days_of_week List | If trigger is custom_schedule, Custom schedule description for when the automation should be run. 0 is Sunday, 1 is Monday, etc. |
| schedule_times_of_day List | Times of day to run in HH:MM format (24-hour). For custom_schedule, run at these times on specified days of week. For daily, run at these times on the scheduled interval date. |
| schedule_time_zone String | Time zone for the schedule. If not set, times are interpreted as UTC. |
| source String | Source path/glob. See Automation docs for exact description, but this is used to filter for files in the path to find files to operate on. Supports globs, except on remote mounts. |
| sync_ids List | IDs of Syncs to run by this Automation. |
| trigger_actions List | If trigger is action, this is the list of action types on which to trigger the automation. Valid actions are create, copy, move, archived_delete, update, read, destroy |
| trigger String | How this automation is triggered to run. Possible values: manual, daily, custom_schedule, webhook, email, action |
| user_id Int64 | User ID of the Automation's creator. |
| user_ids List | IDs of Users for the Automation (i.e. who to Request File from) |
| value Dynamic | A Hash of attributes specific to the automation type. |
| webhook_url String | If trigger is webhook, this is the URL of the webhook to trigger the Automation. |
| holiday_region String | Skip the automation if there is a formal, observed holiday for this region. |