4.18. Transfers

A transfer configuration defines the rules based on which files or folders are transferred between two locations or inside a location.

Please consult the type configuration option to see the list of supported transfer types.

4.18.1. Adding a new transfer

Adding a new transfer configuration is done by creating a new section inside the configuration file. The name of the section should be prefixed with transfers/ and followed by the transfer’s UUID.

The transfer’s UUID can be any unique string used to identify the transfer. Once defined, the UUID should not be changed.

For more information about UUIDs, please see the dedicated UUID documentation.

For example, to add a new transfer configuration of type monitor called Exported orders

[transfers/b904e6a6-c29b-4ccf-8abd-edcae4d3324f]
name = Exported orders
description = Nightly orders exported by the accounting application.
type = copy
source_path = path/to/exported_orders
recursive = y
destination_path = path/to/exported_orders
recursive = y

4.18.2. Transfer options

Each transfer configuration section has the following options:

4.18.2.1. enabled

Default value:

‘Yes’

Optional:

No

From version:

2.6.0

Values:
  • Yes
  • No
Description:

Determines whether the transfer should be automatically started at server startup.

4.18.2.2. name

Default value:

‘’

Optional:

No

From version:

2.6.0

Values:
  • Any text.
Description:

Human-readable short text used to identify this transfer.

4.18.2.3. description

Default value:

‘’

Optional:

Yes

From version:

2.6.0

Values:
  • Any text.
Description:

Human-readable text that describes the purpose of this transfer.

4.18.2.4. type

Default value:

‘’

Optional:

No

From version:

2.6.0

Values:
  • copy
  • move
Description:

This option specifies the type of the transfer.

When the copy type is configured, the transfers will copy files from source to destination and process them using the configured actions: executing external commands before and after transferring the files, archiving the files, etc.

The move type is similar to copy, but, once the transfer is successful, source files are removed from the source location. Source files will still be archived, if configured so. If transfers fail, no matter the cause, the files are not removed from the source location.

4.18.2.5. source_path

Optional:

Yes

Default value:

Disabled

Values:
  • Absolute path on the local file system.
  • Relative path to the server installation folder.
From version:

2.10.0

To version:

None

Description:

Path to the monitored source folder.

4.18.2.6. recursive

Optional:

Yes

Default value:

No

From version:

2.10.0

Values:
  • Yes
  • No
Description:

Determines whether the monitor should look for source files and folders only in the configured path, or recurse in all its descendant folders.

4.18.2.7. changes_poll_interval

Default value:

10

Optional:

Yes

From version:

3.0.0

Values:
  • Number of seconds.
Description:

Frequency of the checks for changes in the monitored path, in seconds.

Lower values help detect changes quicker, but increase the load, CPU, and network usage for both the local and the remote servers.

4.18.2.8. stable_interval

Default value:

10

Optional:

Yes

From version:

2.10.0

Values:
  • Number of seconds.
Description:

Number of seconds after which a file is considered stable if no changes are made to it.

When a new file is created or a file starts to be changed, it is not processed right away. It might be that an external program is still changing its content after the initial file creation or file modification. This will allow the external program to finish handling the file. After the last modification is observed, a configured time interval is allowed to pass. The file is processed only if no other changes to the file are observed after the configured interval. Each change will reset the interval.

This needs to be a multiple of changes_poll_interval.

4.18.2.9. source_filter

Default value:

Disabled

Optional:

Yes

From version:

2.10.0

Values:
  • Globbing expression containing wildcard characters.
  • Regular expression
  • Empty
  • Disabled
Description:

Globbing expression or regular expression used to select source files to be transferred. For more details see the matching expression documentation

Only files matching the expression will be transferred.

Only file names are filtered, all folder names will be transferred.

Leave it empty or set it to Disabled to transfer all files.

4.18.2.10. destination_uuid

Optional:

Yes

Default value:

Local file system

Values:
  • UUID of a defined location.
  • empty - local file system location.
From version:

2.10.0

To version:

None

Description:

UUID of the location used as destination for this transfer or empty to use the local file system location.

4.18.2.11. destination_path

Optional:

Yes

Default value:

Disabled

Values:
  • Absolute path on local file system.
  • Relative path to server installation folder.
From version:

2.10.0

To version:

None

Description:

Path to the destination folder of this transfer.

4.18.2.12. batch_interval

Optional:

Yes

Default value:

0

Values:
  • 0 to disable batch transfer.
  • Number of seconds to wait for new files to be part of a batch.
From version:

3.0.0

To version:

None

Description:

You can configure the transfer to send each file as an independent transfer, or group multiple files into a single transfer; what is called a ‘batch mode’.

If the transfer of one or more files from the batch fails in batch mode, the whole transfer is considered to have failed. The transfer is considered to have been successful only when all files from a batch have been successfully transferred.

A batch transfer is started if no new changes are recorded in the configured time interval. Each new change will reset the time interval.

If a transfer is stopped or fails while a batch transfer is waiting for the batch interval, the whole batch will be cancelled, and no files will be transferred.

If a transfer is suspended while a batch transfer is waiting for the batch interval, the files accumulated until then will be transferred.

To disable batch mode and transfer each file as an individual transfer, set this to 0.

When batch mode is disabled, execute_before, execute_after_success and execute_after_failure commands are executed for each individual file.

When batch mode is enabled, execute_before is called before transferring the first file from the batch, while the execute_after_success and execute_after_failure commands are called after the last file from the batch was transferred.

4.18.2.13. execute_before

Default value:

Disabled

Optional:

Yes

From version:

2.7.0

Values:
  • Path to local script or executable to call before a file is transferred.
Description:

The executable is called with the full path of the file to be transferred. The SOURCE_PATH environment variable is also set to the path of the processed file.

For batch transfers, only the first file from the batch is sent as an argument.

The transfer continues only if the command’s exit code is 0. This means that, when the exit code is not 0, the file is not transferred (copied, moved, etc.), and no other actions are done for this transfer.

Warning

On Windows, executables located in Unicode paths and monitored Unicode paths are not yet supported.

This option is not yet supported on Windows XP and Windows Server 2003.

4.18.2.14. execute_after_success

Default value:

Disabled

Optional:

Yes

From version:

2.9.0

Values:
  • Path to local script or executable to call after a file is successfully transfered.
Description:

Please see the description of the execute_before configuration option.

For batch transfers, the command is called with the path of the last file from the batch.

The command is not called when execute_before fails.

4.18.2.15. execute_after_failure

Default value:

Disabled

Optional:

Yes

From version:

2.9.0

Values:
  • Path to local script or executable to call after a file fails to be transferred.
Description:

Please see the description of the execute_before configuration option.

For batch transfers, the command is called with the path of the last file from the batch.

This is called once, after all retries have been executed. The command is not called if execute_before fails.

4.18.2.16. execute_on_destination_before

Default value:

Disabled

Optional:

Yes

From version:

3.0.0

Values:
  • List of commands to be executed in the destination location.
Description:

Before starting the transfer of a file or a batch, execute in the destination location the list of configured commands.

Each command should be defined on a separate line or delimited with a semicolon (;).

The commands are executed using an embedded client shell. The shell is already connected to the destination location. There is no need for an explicit open command. For the list of supported commands, please check the client shell documentation.

Transfer continues only if commands are successful. When one of the commands fails, the file is not transferred (copied / moved / etc.), and no other actions are completed for this transfer.

4.18.2.17. execute_on_destination_after_success

Default value:

Disabled

Optional:

Yes

From version:

3.0.0

Values:
  • List of commands to be executed in the destination location.
Description:

See description of the execute_on_destination_before configuration option.

The commands are executed if the file or the batch have been successfully transferred.

4.18.2.18. execute_on_destination_after_failure

Default value:

Disabled

Optional:

Yes

From version:

3.0.0

Values:
  • List of commands to call after a file fails to be transferred.
Description:

See description of the execute_on_destination_before configuration option.

The commands are executed if the file or the batch transfer have failed.

4.18.2.19. archive_success_path

Default value:

Disabled

Optional:

Yes

From version:

3.0.0

Values:
  • Path on local filesystem.
  • Disabled
Description:

Path to a folder in the local file system used to keep a copy of successfully transferred files.

To disable archiving, leave this option empty, or set it to Disabled.

To prevent overwriting previous files, new files are copied to the archive folder with timestamps inserted in their names.

Timestamps are inserted before file extensions. When a file has no extension, the timestamp is appended to the file name as an extension.

Timestamps have the following format:

.YEAR-MONTH-DAY-HOUR-MINUTES-SECONDS-MILLISECONDS-RANDOM

A file named README.rst will be archived as README.2014-12-03-13-00-57-967636-036.rst, while a file named README as README.2014-12-03-13-00-57-967636-036.

Note

Archiving is disabled when a transfer source or destination location is not a local folder.

4.18.2.20. archive_failure_path

Default value:

Disabled

Optional:

Yes

From version:

3.0.0

Values:
  • Path in the local file system.
  • Disabled
Description:

Path to local folder in the local file system used to keep a copy of unsuccessful file transfers.

To disable archiving, leave this option empty, or set it to Disabled.

To prevent overwriting previous files, new files are copied to the archive folder with timestamps inserted in their names. See archive_success_path for more details about the timestamp’s format.

When the remote file is partially transferred, the partial file is archived.

Warning

When the source is a remote location and the file has not yet been copied to the local file system, an empty file is copied in the archive.

Note

Archiving is disabled when a transfer has both source and destination as remote locations.

4.18.2.21. retry_count

Default value:

2

Optional:

Yes

From version:

3.0.0

Values:
  • 0
  • Positive integer
Description:

This is the number of times a failed file transfer is retried.

When set to 0, failed file transfers are never retried.

For batch mode transfers, each failed transfer of a file inside the batch is retried.

4.18.2.22. retry_wait

Default value:

60

Optional:

Yes

From version:

3.0.0

Values:
  • 0
  • Positive integer
Description:

Number of seconds to wait before retrying a failed transfer.

When set to 0, there will be no waiting time. As soon as a file transfer fails, it will be retried.

4.18.2.23. schedule

Default value:

Empty

Optional:

Yes

From version:

3.0.0

Values:
  • empty to have the transfer always active.
  • Comma-separated list of 24HOUR:MINUTE-ACTION_NAME values.
Description:

Comma-separated list with scheduled actions for this transfer. Times are defined using a 24-hour clock. Supported actions are: * start * stop

Time resolution is one minute. Please contact us if you need a higher resolution.

For example, to have the transfer started daily at 10:00, and stopped at 14:00, use:

schedule = 10:00-start, 14:00-stop

To have the transfer started daily at 10:00, stopped at 14:00, restarted at 17:00, and stopped at 18:00, use:

schedule = 10:00-start, 14:00-stop, 17:00-start, 18:00-stop

You can also have the scheduler extended over two consecutive days. To have the transfer started daily at 23:00 and then stopped the next day at 01:00, use:

schedule = 23:00-start, 01:00-stop

4.18.2.24. overwrite_rule

Default value:

fail

Optional:

Yes

From version:

3.0.0

Values:
  • fail - abort transfer if destination file already exists.

  • overwrite - always overwrite existing files with the content

    of the new source files. Emits an event when a destination file is overwritten.

  • timestamp-always - always transfer the source file with an amended timestamp.

  • timestamp-to-new-file - transfer the source file with a timestamp only if a file with the same original name exists at the destination.

  • timestamp-to-existing-file - transfer the source file with the original name and, when a file with the same original name exists at the destination, rename the existing file at the destination.

Description:

Rule used to decide how a transfer handles the overwriting of an existing file at the destination.

Warning

The transfer does nothing to prevent external applications from tampering with the existing file during a transfer. It assumes that no other application is managing the local or remote files during a transfer.