6.1. Client-Side Transfers Operations¶
- Source Location
- File Content Transfer
- Recursive Source Transfer
- Delete Source Directories
- Destination Location
- Transferring files with a temporary name
- Configuring the Destination File Name and Path
- Transfer States
- Location States
- Fault-tolerant transfers
- Fault-tolerant source monitoring
- Resumed transfers
- Hard Links and Symbolic Links
- Resolving duplicate / successive events
- Executing commands using pre/post transfer executables or scripts
- Executing pre-/post- transfer destination commands
- Working with the source monitor
- Transfer scheduling
- Batch transfer operation
Transfers are the pro-active component of SFTPPlus. While a file transfer service only acts in response to requests from remote clients, transfers actively check local or remote paths and initiate transfers based on changes in those folders.
Transfers are executed by transferring files between a source location and a destination location.
Source location changes are detected by regularly checking the source folder for changes.
When a change is observed on the monitored source location, the monitor emits one of the following event types:
- File modified
- File/folder created
- File/folder deleted
- File/folder moved
Here is a list of file system changes which are ignored:
- Folder modified (since they are emitted whenever a member is created / removed / moved inside that folder)
- Permissions / Security / Attribute changes
For transfers of type Local filesystem only, created, moved, or removed folders will only emit an informational log event, without triggering the prior transfer command. The same behaviour is present when a file is removed.
For transfers of type other than Local filesystem only, folder events and file removed event are ignored, and no informational event is emitted.
Source events are observed only inside the configured path, and the server will ignore events outside the configured path. While in most cases this is the desired result, it affects ‘moved’ events. Moved events are only observed inside the location path. When you move a file from a path outside the monitored path, a create event is emitted. When you move a file to a path outside the monitored path, a delete event is emitted.
A transfer can monitor only direct ancestors of a folder, for non-recursive monitors, or all ancestors for recursive ones.
When the source location is not available (for example, its connection failed after all retries) the transfer will become stalled and no files will be transfered.
Once changes are observed in the source, SFTPPlus will start doing the actual transfer of the file’s content.
Multiple files might be created at the source in the same time. All of them are observed as soon as they are stable.
To limit the load to the remote destination, SFTPPlus will only transfer one file at a time. Multiple files are not transferred in parallel in the same time.
All the files detected in the source folder of a transfer are placed into a queue. As soon as the current file is transferred, a new file is picked from the queue.
To thwart erroneous behaviour, SFTPPlus will only keep files into the queue up to 100 000 files. If more files are created and not transferred, the transfer will fail and stop.
A recursive transfer is one in which you configure the base path for a transfer and SFTPPlus will traverse the whole path and sub-path and transfer files from there.
You can configure a transfer to check the source path for any file which is found in the source path itself or in any of the sub-paths.
The source path can be on a local filesystem or remote location such as a SFTP or FTPS server.
See below for an example:
The files will be transferred to the destination using a folder structure relative to the source path. On destination, the parent folder of each file is automatically created. The full source path hierarchy is not preserved.
For a recursive source transfer, there is a performance issue to consider. For each sub-folder, SFTPPlus will need to make a separate request (local or remote) to get the content and attributes of each members of that folder. When you have 1000 sub-folders, SFTPPlus will make at least 1000 requests.
In contrast, if you set up a non-recursive file transfer scenario, the files will be transferred only from a single folder and will make a single request.
See below for a simplified example:
In the above scenario, only two sub-folders have transfer rules associated with them. This means that SFTPPlus will only monitor filesystem changes within those two sub-folders, and not with the main home folder. Transfers are triggered based on the rules relevant to the two folders. Other benefits to this method beyond performance include; greater control for the administrator for their managed file transfer setup, more detailed logging of events associated with the transfer and monitored folders, and better troubleshooting capacity if the issues are traced back to the transfer rules themselves.
In addition, specific transfer rules can be configured based on custom conditions to trigger the transfer. The conditions could depend on factors such as the schedule, the file type (if not all files), the directory itself, directory actions that trigger the transfer rule and more.
For recursive transfers you can configure SFTPPlus to automatically delete the parent source directory, if becomes empty.
Below is an example in which, after successfully transferring a file from path
E:\\Reports\\2020-12-28\\sales.pdf, the file is removed right away
while the parent is removed after 1 hour, if there are no other files
in the same source path:
[transfers/15d46e5a-25f6-11e9-87b3-0b5ae5eda581] source_path = E:\Reports\ delete_source_on_success = True delete_souorce_parent_delay = 3600
Any other file transferred from the same parent source path will reset the delay after which the delete operation is executed.
The destination’s locations are accessed only when a file which needs to be transferred is detected on the source location.
When the destination location is not available, the changes detected on the source location are still observed but no transfer is attempted. As soon as the destination location is available again, new changes observed on the destination will be processed.
The transfer will automatically start or resume the destination location when needed. It will not try to start the destination location, when the location has failed.
A transfer can be configured to send / upload / push the file’s content using a temporary name, renaming to its initial name once all the data was received by the server.
This is done using the destination_temporary_suffix configuration option.
In the example below, the content of the file E:ReportsINV-2019-08-23.pdf is send to the remote server as a file /inbox/INV-2019-08-23.pdf.tmp. Once the file was received on the server, it is renamed from /inbox/INV-2019-08-23.pdf.tmp to /inbox/INV-2019-08-23.pdf:
[transfers/15d46e5a-25f6-11e9-87b3-0b5ae5eda581] source_path = E:\Reports\ destination_path = /inbox destination_temporary_suffix = .tmp
When destination_temporary_suffix is configured, any file transfer to the destination will be done using a temporary name.
In the case in which you only want a selected set of files transferred using temporary names or want to transfer files using a temporary prefix, you can leave destination_temporary_suffix empty and set destination_path_actions with the temporary-suffix or temporary-prefix actions. See the dedicated documentation for the temporary-suffix action.
The destination_path_actions option can be defined for a transfer and will allow using a destination path and file name different than the source path and file name.
This is configured using a set of rules which are applied based on expression matching the source path.
To highlight misconfiguration or omission in configuration, when a transfer has a destination_path_actions configured and it tries to transfer files not matching any of the configured rules, SFTPPlus will stop the whole transfer with a failure and will not process any further files.
The base destination path, as defined in destination_path is not affected by the file path and file name transformation.
A transfer can have multiple rules defined for the same path and the actions will be applied in series. The result from one action will be passed to the next one. Having the following configuration:
[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1] enabled: yes name: copy-to-remote recursive: yes source_path: c:\out\sales source_filter: * delete_source_on_success: Yes destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c destination_path: /Outbox/ destination_path_actions: *.pdf, only-filename *.pdf, lowercase
the files will be transferred as follows:
c:\out\sales\Report.PDF -> /Outbox/report.pdf c:\out\sales\18\Jan\Outcome.PDF -> /Outbox/outcome.pdf
In the following sections you will find detailed information for each of the available actions.
The no-action rule does not change the transferred path or name.
It is used to explicitly inform SFTPPus that you don’t want a transformation for certain file type and to transfer the file with the same path and name as is found on the source.
The only-filename transformation is used to flatten a path defined across multiple directories by using only the file name.
The parent path as found on the source is ignored.
This can be used to transfer a recursive transfer on source to a non-recursive transfer on destination.
Having the following configuration:
[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1] enabled: yes name: copy-to-remote recursive: yes source_path: c:\out\sales source_filter: * delete_source_on_success: Yes destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c destination_path: /Outbox/ destination_path_actions: *.pdf, only-filename *.txt, no-action
the files will be transferred as follows:
c:\out\sales\Report.PDF -> /Outbox/Report.PDF c:\out\sales\Report.txt -> /Outbox/Report.txt c:\out\sales\18\Jan\Outcome.PDF -> /Outbox/Outcome.PDF c:\out\sales\18\Jan\Outcome.txt -> /Outbox/18/Jan/Outcome.txt
The temporary-suffix action allows uploading a file using a temporary name, renaming to the initial name once all the file content was transferred.
The temporary-suffix action is required only when a selected set of files need to be transferred with temporary names. When you want all files transferred with temporary names, use the destination_temporary_suffix configuration option.
It takes a single option consisting of the characters used as the temporary suffix.
With the following configuration, the file
be first transferred to
/Outbox/report.pdf.tmp, then finally renamed to
[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1] enabled: yes name: push-to-remote source_path: c:\out\sales source_filter: * delete_source_on_success: Yes destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c destination_path: /Outbox/ destination_path_actions: *.pdf, temporary-suffix, .tmp
When a transfer fails before completing the transfer of the file’s content, the incompletely transferred file on destination keeps the temporary suffix.
The temporary-prefix action allows uploading a file using a temporary prefixed name, renaming to the initial name once all the file content was transferred.
It takes a single option consisting of the characters used as the temporary prefix.
With the following configuration, the file
be first transferred to
/Outbox/TEMP_report.pdf, then finally renamed to
[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1] enabled: yes name: push-to-remote source_path: c:\out\sales source_filter: * delete_source_on_success: Yes destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c destination_path: /Outbox/ destination_path_actions: *.pdf, temporary-prefix, TEMP_
When a transfer fails before completing the transfer of the file’s content, the incompletely transferred file on destination keeps the temporary prefix.
The replace-separator transformation is used to flatten a path defined across multiple directories and to convert it into a simple file name.
It takes a single option which is the character or characters used to substitute the path separators. Depending on the source filesystem, it will replace the / or \ character.
For recursive transfers, the destination path in this transformation is relative to the base source_path.
Having the following configuration:
[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1] enabled: yes name: copy-to-remote recursive: yes source_path: c:\out\sales source_filter: * delete_source_on_success: Yes destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c destination_path: /Outbox/ destination_path_actions: *.pdf, replace-separator, -- *.txt, replace-separator, _
the files will be transferred as follows:
c:\out\sales\Report.PDF -> /Outbox/Report.PDF c:\out\sales\Report.txt -> /Outbox/Report.txt c:\out\sales\report.csv -> Fail as it does not match any rule. c:\out\sales\18\Jan\Report.PDF -> /Outbox/18--Jan--Report.PDF c:\out\sales\18\Jan\Report.txt -> /Outbox/18_Jan_Report.txt c:\out\sales\18\Jan\report.csv -> Fail as it does not match any rule.
The lowercase is a simple transformation which will transfer the file using destination file path and file name all in lower case. It takes no options.
If the remote filesystem is case-insensitive and a file with the same name (but different cases) exists, the remote name is kept.
Below is an example for the lowercase transformation:
[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1] enabled: yes name: copy-to-remote recursive: yes source_path: /out/sales source_filter: * delete_source_on_success: Yes destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c destination_path: /Outbox/ destination_path_actions: *.pdf, lowercase *, no-action
With the above configuration the following files will be transferred
as exemplified below.
Note that the base destination path as defined in the transfer configuration,
is not affected by the file path and file name transformation.
report.txt are transferred since they match the
/out/sales/Report.PDF -> /Outbox/report.pdf /out/sales/Report.txt -> /Outbox/Report.txt /out/sales/Jan/Report.PDF -> /Outbox/jan/report.pdf /out/sales/Jan/Report.txt -> /Outbox/Jan/Report.txt
The fixed-path action is a simple transformation which will transfer the file using a fixed destination. It takes a single option, the full destination path. The destination_path configuration is ignored for the fixed-path action.
If this transformation is enabled, the temporary suffix behaviour is disabled.
Below is an example for the fixed-path transformation:
[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1] enabled: yes name: copy-to-remote recursive: yes source_path: /out/sales source_filter: * delete_source_on_success: Yes destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c destination_path: /Outbox/ destination_path_actions: *, fixed-path, //'PDEB.OU11.DRSAFG(+1)'
With the above configuration, the following files will be transferred as exemplified below. Note that the base destination path as defined in the transfer configuration, is ignored:
/out/sales/Report.PDF -> //'PDEB.OU11.DRSAFG(+1)' /out/sales/Report.txt -> //'PDEB.OU11.DRSAFG(+1)' /out/sales/Jan/Report.PDF -> //'PDEB.OU11.DRSAFG(+1)' /out/sales/Jan/Report.txt -> //'PDEB.OU11.DRSAFG(+1)'
A transfer can be in one of the following states:
- Stopped - the transfer is not active and is not configured to be active at any time.
- Started - the transfer is active and processing files from the source.
- suspended - the transfer was started, but it is outside of its activity schedule.
- Source failed - the source location is not available and files at source are not processed. Processing will be started as soon as the source location is available, either by fixing the errors and manually starting it or by an automated location reconnection process.
- Destination failed - The destination location is not available. Processing will resume as soon as the destination location is available, either by fixing the errors or manually starting it.
- Failed - a critical error occurred while executing the transfer and the transfer was stopped. Manual intervention is required to restart the transfer.
A location can be in one of the following states:
- Stopped - no transfer is using this location and it was not manually started.
- Started - the location is connected to the remote location.
- Disconnected - the location is not connected to the remote location, but it will auto-reconnect when required by a transfer.
- Failed - the location failed to connect to the remote location. Manual intervention is required to restart the location.
SFTPPlus will do its best to perform the requested transfers and under certain circumstance will retry on transfer failures.
From the configuration, you can define the number of times to retry and the time interval to wait after retries.
Once all retries are exhausted, the transfer will fail and SFTPPlus will no longer try to process/transfer that file.
Files which were previously failed to be transferred, can be processed/transfered again by restarting the transfer.
When a transfer is started, SFTPPlus will check for the required permissions to monitor the source path for changes, the configured path exists and that this path is valid.
If the source path is no longer available after the transfer was started and the initial check operations were successful, SFTPPlus will not consider the transfer failed. It will wait for the next configured changes_poll_interval to pass and will try again. This is done for an unlimited number of times and an event is emitted on each failure. Once the source path is available again, the transfer is resumed and new or changed files will be transferred.
The type of source path errors which are considered not critical for the purpose of source monitoring varies based on the file transfer protocol.
The most common errors found on all supported protocols are:
- Source path not found.
- Source path without read or listing permissions.
- Connection closed while source was checked.
For the HTTP and WebDAV protocol, below is a non-exhaustive list of HTTP codes for which source monitoring will continue:
- 401 UNAUTHORIZED
- 403 FORBIDDEN
- 404 NOT FOUND
- 429 TOO MANY REQUESTS
- 500 INTERNAL SERVER ERROR
- 503 SERVICE UNAVAILABLE
- 504 GATEWAY TIMEOUT
Resuming unfinished transfers, either upload or download, is supported across available file transfer protocols as follows:
Our roadmap includes adding support for resuming transfers across all available protocols. In the case that resume support is not available for your preferred transfer protocol, please contact us.
When an event occurs at the source location, it is not signaled right away. A buffer time interval is used to remove duplicate events and emit the event only once the file or folder is stable.
For example, when a large file is copied, the operating system will emit a stream of events after each intermediary change. The monitor will queue these changes and merge them into a single event.
The time frame in which duplicate events are resolved is 1 second. After an event occurs, it waits 1 second before emitting the event. If no event occurs for the same source, the event is emitted. Otherwise, the event is merged and the time interval is extended by another second in which it can be further merged with another event, or emitted.
Here is the list of events which are emitted. Path is used as a generic reference to a file or a folder:
- Existing - for paths which are already present when the process is started.
- Modify - for a path which has a changed size or modify time.
- Delete - for a path which is no longer in the monitored path. Either removed from there or moved to a path which is not monitored.
- Create - for a path which has appeared in the monitored path. Either created there or moved from a path which is not monitored.
- Moved - for a path which was renamed / moved inside the monitored path.
Here are the rules based on which events are resolved:
- Existing + Modify = Existing
- Existing + Delete = Ignore
- Modify + Modify = Modify (and wait for next event)
- Modify + Delete = Delete (without delay)
- Modify + Move = Move + Modify (modify is last)
- Create + Modify = Create (and wait for next event)
- Create + Delete = Discard (file was created for a very short time)
- Create + Move = Create (at the moved path)
- Delete.File + Create.File = Modify (the file was completely replaced)
- Delete.Folder + Create.File = Create.File
- Moved + Created = Moved + Created (with delay)
- Moved + Modified = Moved + Modified (with delay)
- Moved + Deleted = Deleted for the source path
SFTPPlus allows configuring a transfer to execute at certain transfer steps:
- Execute before processing a source file
- Execute after success of a file transfer
- Execute after failure of a file transfer
The configuration from this example,
will copy (indicated by delete_source_on_success: No) files from a source
(indicated by source_path and source_uuid) to a destination (indicated
by destination_path and destination_uuid).
A timestamp-always is set in the overwrite_rule in order to always
copy the source file with an amended timestamp and prevent accidental
overwriting on the destination.
It will poll (via changes_poll_interval) for changes to the folder every
10 seconds of no changes to a file (indicated by stable_interval),
it will consider the file stable and ready to be processed.
A script (saved as a local file at
C:\acme\sftpplus\transfer-jobs\after-success.bat) is executed once a file
that has been successfully transferred is considered stable.
The script is configured as:
[transfers/7db823d8-05f8-4481-be98-b87a826ded28] execute_after_success = C:\acme\sftpplus\transfer-jobs\after-success.bat
There are additional processing actions within the custom script. Processing actions include:
- Creating a zipped archive from a target source.
- Affixing a date to the archive.
- Moving the archive to another destination.
An example of the above configuration:
[transfers/7db823d8-05f8-4481-be98-b87a826ded28] enabled = Yes name = inbox-transfer description = Transfer Inbox files. source_uuid = local-file-system-source-uuid source_path = C:\acme\sftpplus\accounting\inbox delete_source_on_success: No destination_uuid = local-file-system-destination-uuid destination_path = C:\acme\sftpplus\accounting\inbox-final execute_after_success = C:\acme\sftpplus\transfer-jobs\after-success.bat stable_interval = 10 changes_poll_interval = 5 overwrite_rule = timestamp-always batch_interval = 0 retry_count = 0 retry_wait = 0
SFTPPlus allows configuring a transfer to execute a list of commands via the following options:
- Execute after success on destination
- Execute before on destination
- Execute before success on destination.
These commands need to be available in the SFTPPlus client-shell implementation.
To check what commands are available, please start the client-shell and run:
To find out more about a particular command, run:
> help COMMAND_NAME
These commands are configured as part of the Transfer section in the Local Manager GUI and it can be added in the configuration text file like below:
[transfers/99feb83f-a99d-42f1-a86c-1562c3281bb6] name = transfer_sample execute_on_destination_after_success = rename /path1 /path2; delete /path3 execute_on_destination_before = client-shell-COMMAND /file/path execute_on_destination_before = rename /path-to/ /path2 delete /path3
Transfers can executed either a single command or multiple commands.
Multiple commands are separated by a semicolon (;) or on a separate line.
When the changes_poll_interval passes, the application will read all files, folders, and attributes (size, last modification time, etc) found in the monitored folder. Each snapshot is then compared with the previous.
If there are changes between the snapshots, they are notified to the application.
Before doing the actual notification, the application will wait for changes to be considered stable as defined by another option, stable_interval.
Lower values help detect changes quicker, but increase the load, CPU, and network usage for local and remote servers.
The poll interval might drift.
For example, with a value of
72000 and a transfer started at
the product might be busy at
In this case, the check is done only at
15:36 and the next check will be
When a new file is created or a file starts to be modified, it is not processed by SFTPPlus right away. This allows an external program to finish handling the file.
If no other changes to the file are observed after this interval, it is then processed.
Criteria for a file to be considered stable:
- Size is not changed.
- Last modified time attribute is not changed.
- The configured stable interval has elapsed.
After the last modification is observed, this interval is then allowed to pass.
Each change will reset the stable interval for the file itself since each file has its own stable counter.
Since changes can only be observed with a resolution defined by changes_poll_interval, the stable_interval needs to be a multiple.
The following will check new files every hour and transfer stable files after two hours, even if the stable_interval is lower:
[transfers/b904e6a6-c29b-4ccf-8abd-edcae4d3324f] changes_poll_interval = 3600 stable_interval = 60
The file age is determined based on the file’s modified attribute.
Some file copy tools provided by the operating system or 3rd party might preserver the file attribute during the copy operation. In this case, even if you make a new copy of a file, it will be observed as an old file by the SFTPPlus system.
Windows Explorer or command prompt tools will preserve the original modified date of the copied file.
Linux and macOS tools will update the modified date of the copied files. The modified data is preserved only when the copy operation is done in archive mode.
A transfer can be configured to be active all the time or be active based on a simple scheduler.
The scheduler available in SFTPPlus allows you to define transfer start and transfer stop events in a 24 hour interval or weekly interval. The current scheduler has no knowledge of the exact day of month or year. It only knows about current time inside the day and current day inside the week.
To create a scheduled transfer, you will to define start and stop action on different hours across a day or week.
For example, to have the transfer started daily at 10:00, and stopped at 14:00, you can use the short notation:
[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028] schedule = 10:00-start, 14:00-stop
or the weekly longer notation:
[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028] schedule = Monday-10:00-start, Monday-14:00-stop Tuesday-10:00-start, Tuesday-14:00-stop Wednesday-10:00-start, Wednesday-14:00-stop Thursday-10:00-start, Thursday-14:00-stop Friday-10:00-start, Friday-14:00-stop Saturday-10:00-start, Saturday-14:00-stop Sunday-10:00-start, Sunday-14:00-stop
To have the transfer started daily at 10:00, stopped at 14:00, restarted at 17:00, and stopped at 18:00, use:
[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028] schedule = 10:00-start, 14:00-stop, 17:00-start, 18:00-stop
You can also have the scheduler extended over two consecutive days. To have the transfer started daily at 23:00 and then stopped the next day at 01:00, use:
[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028] schedule = 23:00-start, 01:00-stop
For defining actions at different days throughout the week, you can add the name of the day before the time value. To have a transfer started at 10:00, stopped at 14:00 every Monday, Wednesday, and Friday, use:
[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028] schedule = Monday-10:00-start, Monday-14:00-stop Wednesday-10:00-start, Wednesday-14:00-stop Friday-10:00-start, Friday-14:00-stop
You can also have start and stop events spanning different days. To have a transfer which is active from Monday to Wednesday and then again active from Friday to Saturday, use:
[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028] schedule = Monday-10:00-start, Wednesday-18:00-stop Friday-10:00-start, Saturday-18:00-stop
At this stage, the transfer’s changes_poll_interval is not aligned with the schedule. We are currently working on improving this to align the interval with the schedule.
For example, the configuration below will schedule to check files every 2 hours (or 7200 seconds) between 8AM and 6PM:
[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028] schedule = 08:00-start, 18:00-stop changes_poll_interval = 7200
If the transfer was started at
13:35, it will do a check at
instead of what you might expect to be done at
The next check will be scheduled for
Please get in touch if you want a transfer with a complex scheduling rules.
The default transfer operation for SFTPPlus is to transfer a file as soon as it is detected and considered stable. Each file will be processed as an independent transfer. The pre/post command hooks and events are executed as a separate process for each file.
By using the batch_interval configuration option, you can instruct SFTPPlus to only start transferring files once no new files are detected in the specified time interval.
All the files detected during this interval will be transferred together as a single transfer. The pre/post command hooks and events are executed once for the batch of files.
You can also configure SFTPPlus to wait and batch/group the files based on a file with a specific path. This is done by using the batch_marker_path configuration option. You can configure an exact file path or a pattern of file paths which can trigger the transfer.
Once a file with a path matching the batch_marker_path is detected, SFTPPlus will wait for 15 seconds before it begins transferring all the previous files. Should there be new files created within the 15 seconds interval after the marker, it will be included in the batch transfer.
The matching expression is for the file path and not for the file name. If you want to match a file name complete.txt located in any directory, you should configure it as *complete.txt.
The marker file is also transferred.
For more details about defining a file marker, see the matching expression documentation.
When filtering source files which are to be transferred,
the filtering rules should also include the file marker.
The following example will filter the source for both .pdf and .desc
The transfer will start 15 seconds after a report.desc files is
generated in any of the subdirectories for
For example, in
[transfers/a4747959-25de-470a-b76e-afe08071a70d] enabled: yes name: Move report set. description: Move all report files once the report is finalized. A report.desc file is generated by the external reporting process to signal that the process is complete. recursive: Yes source_uuid: c4b74ed0-b952-4edc-be24-7d8e1e6d8068 source_path: c:\jobs\reports source_filter: m/*.(pdf|desc)/ delete_source_on_success: Yes destination_uuid: 92eb50d4-bfc2-4b15-a369-f0d74dc68dac destination_path: /dropbox/ batch_interval: 0 batch_marker_path: *done.desc
With the configuration from above, assume that you have the following files:
c:\jobs\reports\ACC-123\in.pdf c:\jobs\reports\ACC-123\out.pdf c:\jobs\reports\ACC-123\response\report.pdf c:\jobs\reports\MON-542\office.pdf c:\jobs\reports\MON-542\storage.pdf c:\jobs\reports\MON-542\response\report.pdf c:\jobs\reports\MON-542\done.desc
then the following actions are taken on these files
c:jobsreportsACC-123in.pdf - Not transferred c:jobsreportsACC-123out.pdf - Not transferred c:jobsreportsACC-123responsereport.pdf - Not transferred c:jobsreportsMON-542office.pdf - Transferred c:jobsreportsMON-542storage.pdf - Transferred c:jobsreportsMON-542responsereport.pdf - Transferred c:jobsreportsMON-542done.desc - Transferred
The batch marker should match a single file for a certain directory and its descendants. The following file structure is valid and the files will be transferred since the marker files are in different folders which are not a descendant of another:
c:\jobs\reports\ACC-123\in.pdf c:\jobs\reports\ACC-123\out.pdf c:\jobs\reports\ACC-123\done.desc - Marker file c:\jobs\reports\MON-542\storage.pd c:\jobs\reports\MON-542\done.desc - Marker file
The following file structure will fail since
is triggered by both
c:\jobs\reports\MON-542\done.desc - Marker file c:\jobs\reports\MON-542\storage.pdf - Triggered by MON-542\done.desc c:\jobs\reports\MON-542\resp\done.desc - Marker file c:\jobs\reports\MON-542\resp\report.pdf - Triggered by MON-542\done.desc and MON-542\resp\done.desc
When a transfer is stopped and there are files queued for a batch transfer, SFTPPlus will not transfer the files but will instead emit a warning to inform the user of this action.
When a transfer is suspended, and there are files in a batch which are not yet triggered, the files are not transferred and no warning info is emitted. The files will be transferred once the transfer is resumed and the batch is triggered by one of the batch triggering conditions.
When a batch transfer is in progress and the transfer is suspended, the active batch will continue to be transferred.