Transfers configuration#

Introduction#

This page is a reference documentation for the configuration options available when defining a client-side file transfer in SFTPPlus.

A transfer configuration defines the rules based on which files or folders are transferred between two locations or inside a location.

For more information about how to implement secure file transfers, see the separate documentation page for transfers operations.

For local scripts, destination commands, and the execute_* configuration options, see external transfer commands.

For archive locations, file naming, and automatic cleanup, see file archiving.

enabled#

Default value:

'Yes'

Optional:

No

From version:

2.6.0

Values:

  • Yes

  • No

Description:

Determines whether the transfer should be automatically started at server startup.

name#

Default value:

''

Optional:

No

From version:

2.6.0

Values:

  • Any text.

Description:

Human-readable short text used to identify this transfer.

description#

Default value:

''

Optional:

Yes

From version:

2.6.0

Values:

  • Any text.

Description:

Human-readable text that describes the purpose of this transfer.

cluster_activation#

Default value:

'all'

Optional:

No

From version:

5.13.0

Values:

  • all

  • controller

  • node

  • Pool UUID

Description:

When clustering is enabled, it determines on which SFTPPlus instance this component is enabled.

When configured with all, it will be active on any SFTPPlus instance, including the controller and all nodes.

When configured with controller, it will only be active on the SFTPPlus controller instance.

When configured with node, it will be active on all SFTPPlus node instances.

Configure it with the UUID of a pool to make it active on all nodes in that pool.

Note

The enabled configuration is not ignored. When configured with enabled = No this will not be active, regardless of the cluster_activation configuration.

source_path#

Default value:

Empty

Optional:

Yes

Values:

  • Absolute path on the local file system.

  • Relative path to the server installation folder.

From version:

2.10.0

Description:

Path to the monitored source folder.

recursive#

Default value:

No

Optional:

Yes

From version:

2.10.0

Values:

  • Yes

  • No

Description:

Determines whether the monitor should look for source files and folders only in the configured path, or recurse in all its descendant folders.

changes_poll_interval#

Default value:

10

Optional:

Yes

From version:

3.0.0

Values:

  • Number of seconds.

Description:

This is the time interval that defines the delay used to observe the changes in the monitored path and compare snapshots, a record of any changes, in a monitored folder.

When set to zero or a negative number, the default value is used.

stable_interval#

Default value:

10

Optional:

Yes

From version:

2.10.0

Values:

  • Number of seconds.

Description:

This is the interval after which a file is considered stable if no changes are made to it.

These values should be greater than the value of changes_poll_interval.

When the transfer is executed on a cluster node this value is increased by the failover_interval configured for that node's pool. On the cluster controller, there is no additional delay.

source_filter#

Default value:

Empty

Optional:

Yes

From version:

2.10.0

Values:

  • Globbing expression containing wildcard characters.

  • Regular expression

  • Empty

Description:

It can be configured using a Globbing expression or regular expression.

It defines the pattern used to select source files to be transferred. For more details about the syntax used by this configuration option see the matching expression documentation.

Only files matching the expression will be transferred.

A globbing expression can contain multiple file mask filtering rules, separated by the pipe character |.

If a globbing expression doesn't include path separators, it only matches the file name. The parent path is ignored.

When using regular expressions, only file names are matched.

Leave it empty to transfer all files.

Since 4.16.0, the globbing expression can be used to filter based on the full file path, not only on the file name part.

delete_source_on_success#

Default value:

Yes

Optional:

yes

Values:

  • Yes

  • No

From version:

4.0.0

Description:

Whether to delete the source file after a successful transfer.

You can use the archive_success_path configuration option to have the file removed from the source directory and moved into another directory.

If the transfer fails, the source is not removed, even when this is set to Yes.

delete_source_parent_delay#

Default value:

0

Optional:

yes

Values:

  • Number of seconds

From version:

4.10.0 This defines whether the parent directory of the source file is also removed on successful transfer.

Set it to 0 to not delete the source directory.

It is configured with a number of seconds after which, if no other file is transferred inside the targeted directory, the directory is removed.

The parent directory is not removed right away to allow other external application to write all the required files for that directory.

The base source directory is never removed. This configuration only removes the direct parent directory of the file that was transferred. The directory is not removed if it contains other files or directories.

The configuration is valid for both local and remote source locations.

Note

This configuration is ignored for non-recursive transfers. This configuration has no effect if the source file is not configured to also be removed via delete_source_on_success.

ignore_missing_source_files#

Default value:

No

Optional:

yes

Values:

  • Yes

  • No

From version:

5.13.0

Description:

Whether to ignore the fact that a file was first detected by the transfer, but by the time the transfer needs to actually act on the file content the file is no longer present on the source.

If this is set to No the transfer will consider this case a failure and will emit a failure event.

If this is set to Yes the transfer is considered successful. A separate event is emitted to inform that the file was ignored.

minimum_transfer_count#

Default value:

0

Optional:

Yes

Values:

  • Number of files

From version:

5.1.0

Description:

Minimum number of files expected to be successfully transferred in a time interval.

The time interval is defined by the minimum_transfer_interval option.

The event with ID 60019 is emitted if fewer files are successfully transferred in the configured time interval.

You can configure SFTPPlus to trigger an email notification for this event. You can also configure your log watch to trigger an action for it.

Use the default value of 0 to not emit an event if no files were transferred.

minimum_transfer_interval#

Default value:

86400

Optional:

Yes

Values:

  • Number of seconds

From version:

5.1.0

Description:

Time interval, in seconds, used to count successfully transferred files.

This configuration is ignored if minimum_transfer_count = 0`

The number of files transferred in the configured interval is checked every 10 minutes. This means that the configured value should be larger than 600.

However, this value can't be larger than the number of seconds in 370 days: 31968000.

source_filter_age#

Default value:

Empty

Optional:

Yes

From version:

3.51.0

Values:

  • Number of seconds

Description:

This option can be used to instruct SFTPPlus to ignore older files.

It is configured as a number of seconds. Files with a modified time older than the configured number of seconds are not transferred.

Leave it empty or set it to 0 to transfer all files, regardless of the modified time.

transfer_memory_duration#

Default value:

0

Optional:

Yes

Values:

  • Number of days

From version:

4.0.0

Description:

The number of days to keep track of files transferred in the past.

Already transferred files are not re-transferred as long as they have the same name, size, and modified timestamp. This is only true for transfers newer than the configured number of days.

Use the default value of 0 to not keep track of transferred files and to transfer any file found in the source directory.

ignore_duplicate_paths#

Default value:

No

Optional:

Yes

Values:

  • Yes

  • No

From version:

4.0.0

Description:

When set to Yes this relaxes the rules based on which duplicated files are not re-transferred, by ignoring a file as long as it has the same path and name. The file is not re-transferred, even when it has a new size or modified date.

For duplicated files to be recognized, the transfer_memory_duration needs to be enabled. This configuration is ignored when transfer_memory_duration = 0.

transfer_chunk_size#

Default value:

65536

Optional:

Yes

Values:

  • Number of bytes greater or equal to 1024

From version:

5.22.0

Description:

A file is transferred as a series of fragments / chunks.

This defines the size in bytes of each fragment.

This can be used to adjust the size based on the network performance.

It should be configured with a value of at least 1024 bytes.

Greater values can improve the transfer performance, but can increase memory usage.

destination_uuid#

Default value:

Local file system

Optional:

Yes

Values:

  • UUID of a defined location.

  • Empty - local file system location.

From version:

2.10.0

Description:

UUID of the location used as the destination for this transfer or empty to use the local file system location.

destination_path#

Default value:

Empty

Optional:

Yes

Values:

  • Absolute path on the local file system.

  • Relative path to the server installation folder.

From version:

2.10.0

Description:

Path to the destination folder of this transfer.

destination_temporary_suffix#

Default value:

Empty

Optional:

Yes

From version:

3.45.0

Values:

  • Empty.

  • Short text.

Description:

This option allows uploading a file using a temporary name, renaming to its initial name once all the file's content was transferred.

Leave it empty to always transfer a file to its destination using the original name.

Note

This configuration option is ignored when destination_path_actions is set using the temporary-suffix or fixed-path actions.

batch_interval#

Default value:

0

Optional:

Yes

Values:

  • 0 to disable batch transfer.

  • Number of seconds to wait for new files to be part of a batch.

From version:

3.0.0

Description:

You can configure the transfer to send each file as an independent transfer, or group multiple files into a single transfer. This is called a time-based batch transfer.

Note

This option is ignored when batch_marker_path is defined. A transfer can only run as time-based batching or file marker batching.

If the transfer of one or more files from the batch fails in batch mode, the whole transfer is considered to have failed. The transfer is considered to have been successful only when all files from a batch have been successfully transferred.

A batch transfer is started if no new changes are recorded in the configured time interval. Each new change will reset the time interval.

If a transfer is stopped or fails while a batch transfer is waiting for the batch interval, the whole batch will be canceled, and no files will be transferred.

If a transfer is suspended while a batch transfer is waiting for the batch interval, the files accumulated until then will be transferred.

To disable batch mode and transfer each file as an individual transfer, set this to 0.

When batch mode is disabled, execute_before, execute_after_success and execute_after_failure commands are executed for each individual file.

When the batch mode is enabled, execute_before is called before transferring the first file from the batch, while the execute_after_success and execute_after_failure commands are called after the last file from the batch was transferred.

batch_marker_path#

Default value:

''

Optional:

Yes

Values:

  • Name of a file.

  • Globbing expression.

  • Regular expression.

  • Multiple names of files or expressions. One per line. (Since 4.22.0)

From version:

3.36.0

Description:

You can configure the transfer to wait for a file with a specific file path or file path pattern before transferring the files.

This is called a file marker-based batch transfer.

The marker file needs to be located inside the source path.

The matching expression should be defined in such a way as to match the whole path of the targeted file, and not just the file name.

Multiple expressions can be defined to configure the transfer to wait for multiple files. Each expression should match a single file.

When one expression is matching multiple files, the transfer will fail. This indicates a likely misconfiguration, which could cause non-deterministic transfers.

For transfer with a recursive source, you can configure the marker at any place inside the source sub-directories. The batch transfer will include any files located in the same directory as the batch marker as well as any files in the sub-directories.

The transfer will start 15 seconds after the marker file was detected.

Leave it empty to not delay the transfer based on a marker/flag file path.

For more details about defining a transfer with a file marker, see source monitoring.

retry_count#

Default value:

2

Optional:

Yes

From version:

3.0.0

Values:

  • 0

  • Positive integer

Description:

This is the number of times a failed file transfer is retried. The retries for failed file transfer are done after retry_wait seconds. If the file transfer still fails after all retries, the file is ignored. Any other file continues to be transferred.

It is also the number of retries in the case the source directory is no longer available. The retries for a failed source directory, are done after changes_poll_interval seconds. If the source directory is still not available after all retries, the whole transfer fails and no other file is processed.

When set to 0, failed file transfers are never retried.

For batch mode transfers, each failed transfer of a file inside the batch is retried.

retry_wait#

Default value:

60

Optional:

Yes

From version:

3.0.0

Values:

  • 0

  • Positive integer

Description:

Number of seconds to wait before retrying a failed transfer.

When set to 0, there will be no waiting time. As soon as a file transfer fails, it will be retried.

schedule#

Default value:

Empty

Optional:

Yes

From version:

3.0.0

Values:

  • Empty to have the transfer always active.

  • Comma-separated list of 24HOUR:MINUTE-ACTION_NAME actions.

  • Comma-separated list of WEEK_DAY-24HOUR:MINUTE-ACTION_NAME actions (Since 3.44.0).

  • Comma-separated list of MONTH_DAY-24HOUR:MINUTE-ACTION_NAME actions (Since 5.11.0).

  • Multiple lines of comma-separated values of actions.

Description:

Comma-separated list with scheduled actions for this transfer.

Times are defined using a 24-hour based on the local time zone.

Supported actions are:

  • start

  • stop

Here are the valid week day values, in ISO 8601 order:

  • Monday

  • Tuesday

  • Wednesday

  • Thursday

  • Friday

  • Saturday

  • Sunday

The numbered month days run from month_day_1 through month_day_31. An action is skipped if the configured day does not occur in that month.

Use month_day_last for the last day of the month, or month_day_second_last for the day before it. Neither relative month-end day can be combined with numbered days 26 through 31 in the same schedule.

For longer schedule definition you can define the actions across multiple lines.

You can't mix week based scheduling with month based scheduling.

Time resolution is one minute. Please contact us if you need a higher resolution.

For more details about defining a transfer with a scheduler, see the transfer operation guide.

overwrite_rule#

Default value:

fail

Optional:

Yes

From version:

3.0.0

Values:

  • fail - abort transfer if the destination file already exists.

  • overwrite - always overwrite existing files with the content of the new source files.

  • disable - don't check for existing files and always try to transfer the file.

  • skip - don't transfer the source file when the destination exists.

  • timestamp-always - always transfer the source file with an amended timestamp and prevent accidental overwriting on the destination.

  • timestamp-to-new-file - transfer the source file with a timestamp only if a file with the same original name exists at the destination.

  • timestamp-to-existing-file - transfer the source file with the original name and, when a file with the same original name exists at the destination, rename the existing file at the destination.

Description:

The rule used to decide how a transfer handles the overwriting of an existing file at the destination.

When the file is transferred with temporary-suffix, it will check for the existence of the final file name on the destination, not the file name with the temporary suffix.

When set to overwrite it will emit an event when the destination file is overwritten.

When set to skip it will not transfer the source file and the source file is not removed from the source directory. An event is emitted to inform that the file was skipped.

Note

The transfer does nothing to prevent external applications from tampering with the existing file during a transfer. It assumes that no other application is managing the local or remote files during a transfer.

destination_path_actions#

Default value:

Empty

Optional:

Yes

From version:

3.36.0

Values:

  • path-match-expression, transformation, parameter-1, parameter-2

  • path-match-expression, transformation

  • List of rules, separated by newlines.

Description:

Rules defining the action taken to the destination path and file name, as transferred to the destination location.

This is a comma-separated configuration value.

The first value is the full source path-matching expression for the source file with a path relative to the configured source_path. Globbing expression or regular expression can be used. For more details, see the matching expression documentation.

The second value is the name of the action performed on the source file name and source path in order to define the destination file name and destination path.

You can specify multiple rules, one per line. Leading and trailing spaces are ignored. Empty lines are ignored.

When a path is matching multiple rules, it will have the effect of multiple actions applied for the same path. The actions are applied one by one, from top to bottom as defined in the configuration. The result from the first action is used as the input for the second action.

Leave it empty to not define any actions and have the same file path and file name as from the source.

When action rules are defined, and the current transferred file does not match any rule, the transfer will fail. You can use a file *, no-action rule to explicitly instruct SFTPPlus that you don't want an action for the file names not matching any of the previous rules.

The supported actions are:

  • transform - This is the generic operation based on which the destination path can be defined as a different name in comparison with the source path. The whole path can be transformed, not only the filename. It can be used for a rename operation, as well as for inserting a timestamp.

  • only-filename - Use only the source file name and discard the source path. This results in a flattened destination. This action has no parameters.

  • replace-separator - Takes a single parameter, which is the character(s) used to replace the source path separator.

  • lowercase - Will use the same file path as source, but in all lower case. This action has no parameters.

  • temporary-suffix - Will use a temporary file name suffix to transfer the file, renaming it back after all the content was transferred.

  • temporary-prefix - Will use a temporary file name prefix to transfer the file, renaming it back after all the content was transferred.

  • fixed-path - Will use the same path for all the transferred files.

  • no-action - Will keep the file path and file name as in the source. This action has no parameters.

For more details and examples on the available actions, see destination path conversion.

destination_content_actions#

Default value:

Empty

Optional:

Yes

Values:

  • path-match-expression, transformation, parameter

  • List of rules, separated by newlines.

From version:

4.31.0

Description:

Configures content conversion during a file transfer. The source content is unchanged. Leave empty to transfer files without conversion.

For supported conversions, rule syntax, and examples, see source content conversion.

destination_fallback_path#

Default value:

Empty

Optional:

Yes

From version:

4.24.0

Values:

  • Empty.

  • Path on destination.

Description:

This option allows defining a destination path that is used when failing to copy the file to the normal destination path.

This option is designed to be used with non-recursive transfers. When used with recursive transfers the functionality of this option is limited. SFTPPlus will not try to create a missing destination sub-directory. It will also not try to create the missing sub-directories inside the fallback path. Instead, files for missing sub-directories are attempted to be delivered to the fallback path, without creating the source directory structure in the fallback directory. Get in touch with our support team, if you want SFTPPlus to create the source directory structure inside the fallback directory.

This is designed to be used together with destination_path_actions, when the destination path is dynamic.

See also destination path conversion.