SFTPPlus Documentation

Start Page 6. Client-side Usage Instructions 6.1. Client-Side Transfers Operations

6.1. Client-Side Transfers Operations

6.1.1. Introduction

Transfers are the pro-active component of SFTPPlus. While a file transfer service only acts in response to requests from remote clients, transfers actively check local or remote paths and initiate transfers based on changes in those folders.

Transfers are executed by transferring files between a source location and a destination location.

6.1.2. Source Location

Source location changes are detected by regularly checking the source folder for changes.

When a change is observed on the monitored source location, the monitor emits one of the following event types:

  • File modified
  • File/folder created
  • File/folder deleted
  • File/folder moved

Here is a list of file system changes which are ignored:

  • Folder modified (since they are emitted whenever a member is created / removed / moved inside that folder)
  • Permissions / Security / Attribute changes

For transfers of type Local filesystem only, created, moved, or removed folders will only emit an informational log event, without triggering the prior transfer command. The same behaviour is present when a file is removed.

For transfers of type other than Local filesystem only, folder events and file removed event are ignored, and no informational event is emitted.

Source events are observed only inside the configured path, and the server will ignore events outside the configured path. While in most cases this is the desired result, it affects ‘moved’ events. Moved events are only observed inside the location path. When you move a file from a path outside the monitored path, a create event is emitted. When you move a file to a path outside the monitored path, a delete event is emitted.

A transfer can monitor only direct ancestors of a folder, for non-recursive monitors, or all ancestors for recursive ones.

When the source location is not available (for example, its connection failed after all retries) the transfer will become stalled and no files will be transfered.

6.1.3. File Content Transfer

Once changes are observed in the source, SFTPPlus will start doing the actual transfer of the file’s content.

Multiple files might be created at the source in the same time. All of them are observed as soon as they are stable.

To limit the load to the remote destination, SFTPPlus will only transfer one file at a time. Multiple files are not transferred in parallel in the same time.

All the files detected in the source folder of a transfer are placed into a queue. As soon as the current file is transferred, a new file is picked from the queue.

To thwart erroneous behaviour, SFTPPlus will only keep files into the queue up to 100 000 files. If more files are created and not transferred, the transfer will fail and stop.

6.1.4. Recursive Source Transfer

A recursive transfer is one in which you configure the base path for a transfer and SFTPPlus will traverse the whole path and sub-path and transfer files from there.

You can configure a transfer to check the source path for any file which is found in the source path itself or in any of the sub-paths.

The source path can be on a local filesystem or remote location such as a SFTP or FTPS server.

See below for an example:

Example of a recursive file transfer via the changes poll interval

The files will be transferred to the destination in the same destination path, using a flat structure. The source path hierarchy is not preserved.

Please get in touch if you want to have a transfer in which the destination will mirror the source path structure.

For a recursive source transfer, there is a performance issue to consider. For each sub-folder, SFTPPlus will need to make a separate request (local or remote) to get the content and attributes of each members of that folder. When you have 1000 sub-folders, SFTPPlus will make at least 1000 requests.

In contrast, if you set up a non-recursive file transfer scenario, the files will be transferred only from a single folder and will make a single request.

See below for a simplified example:

A non-recursive file transfer using specific transfer rules

In the above scenario, only two sub-folders have transfer rules associated with them. This means that SFTPPlus will only monitor filesystem changes within those two sub-folders, and not with the main home folder. Transfers are triggered based on the rules relevant to the two folders. Other benefits to this method beyond performance include; greater control for the administrator for their managed file transfer setup, more detailed logging of events associated with the transfer and monitored folders, and better troubleshooting capacity if the issues are traced back to the transfer rules themselves.

In addition, specific transfer rules can be configured based on custom conditions to trigger the transfer. The conditions could depend on factors such as the schedule, the file type (if not all files), the directory itself, directory actions that trigger the transfer rule and more.

6.1.5. Destination Location

The destination’s locations are accessed only when a file which needs to be transferred is detected on the source location.

When the destination location is not available, the changes detected on the source location are still observed but no transfer is attempted. As soon as the destination location is available again, new changes observed on the destination will be processed.

The transfer will automatically start or resume the destination location when needed. It will not try to start the destination location, when the location has failed.

6.1.6. Transferring files with a temporary name

A transfer can be configured to send / upload / push the file’s content using a temporary name, renaming to its initial name once all the data was received by the server.

This is done using the destination_temporary_suffix configuration option.

In the example below, the content of the file E:ReportsINV-2019-08-23.pdf is send to the remote server as a file /inbox/INV-2019-08-23.pdf.tmp. Once the file was received on the server, it is renamed from /inbox/INV-2019-08-23.pdf.tmp to /inbox/INV-2019-08-23.pdf:

[transfers/15d46e5a-25f6-11e9-87b3-0b5ae5eda581]
source_path = E:\Reports\
destination_path = /inbox

destination_temporary_suffix = .tmp

When destination_temporary_suffix is configured, any file transfer to the destination will be done using a temporary name.

In the case in which you only want a selected set of files transferred using temporary names, you can leave destination_temporary_suffix empty and set destination_path_actions with the temporary-suffix action. See the dedicated documentation for the temporary-suffix action.

6.1.7. Configuring the Destination File Name and Path

The destination_path_actions option can be defined for a transfer and will allow using a destination path and file name different than the source path and file name.

This is configured using a set of rules which are applied based on expression matching the source path.

To highlight misconfiguration or omission in configuration, when a transfer has a destination_path_actions configured and it tries to transfer files not matching any of the configured rules, SFTPlus will stop the whole transfer with a failure and will not process any further files.

The base destination path, as defined in destination_path is not affected by the file path and file name transformation.

A transfer can have multiple rules defined for the same path and the actions will be applied in series. The result from one action will be passed to the next one. Having the following configuration:

[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1]
enabled: yes
name: copy-to-remote
type: move

recursive: yes
source_path: c:\out\sales
source_filter: *

destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c
destination_path: /Outbox/

destination_path_actions:
  *.pdf, only-filename
  *.pdf, lowercase

the files will be transferred as follows:

c:\out\sales\Report.PDF          -> /Outbox/report.pdf
c:\out\sales\18\Jan\Outcome.PDF  -> /Outbox/outcome.pdf

In the following sections you will find detailed information for each of the available actions.

6.1.7.1. no-action

The no-action rule does not change the transferred path or name.

It is used to explicitly inform SFTPPus that you don’t want a transformation for certain file type and to transfer the file with the same path and name as is found on the source.

6.1.7.2. only-filename

The only-filename transformation is used to flatten a path defined across multiple directories by using only the file name.

The parent path as found on the source is ignored.

This can be used to transfer a recursive transfer on source to a non-recursive transfer on destination.

Having the following configuration:

[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1]
enabled: yes
name: copy-to-remote
type: move

recursive: yes
source_path: c:\out\sales
source_filter: *

destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c
destination_path: /Outbox/

destination_path_actions:
  *.pdf, only-filename
  *.txt, no-action

the files will be transferred as follows:

c:\out\sales\Report.PDF          -> /Outbox/Report.PDF
c:\out\sales\Report.txt          -> /Outbox/Report.txt
c:\out\sales\18\Jan\Outcome.PDF  -> /Outbox/Outcome.PDF
c:\out\sales\18\Jan\Outcome.txt  -> /Outbox/18/Jan/Outcome.txt

6.1.7.3. temporary-suffix

The temporary-suffix action allows uploading a file using a temporary name, renaming to the initial name once all the file content was transferred.

Note

The temporary-suffix action is required only when a selected set of files need to be transferred with temporary names. When you want all files transferred with temporary names, use the destination_temporary_suffix configuration option.

It takes a single option consisting of the characters used as the temporary suffix.

With the following configuration, the file c:outsalesreport.pdf will be first transferred to /Outbox/report.pdf.tmp, then finally renamed to /Outbox/report.pdf:

[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1]
enabled: yes
name: push-to-remote
type: move

source_path: c:\out\sales
source_filter: *

destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c
destination_path: /Outbox/

destination_path_actions:
  *.pdf, temporary-suffix, .tmp

When a transfer fails before completing the transfer of the file’s content, the incompletely transferred file on destination keeps the temporary suffix.

6.1.7.4. replace-separator

The replace-separator transformation is used to flatten a path defined across multiple directories and to convert it into a simple file name.

It takes a single option which is the character or characters used to substitute the path separators. Depending on the source filesystem, it will replace the / or \ character.

For recursive transfers, the destination path in this transformation is relative to the base source_path.

Having the following configuration:

[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1]
enabled: yes
name: copy-to-remote
type: move

recursive: yes
source_path: c:\out\sales
source_filter: *

destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c
destination_path: /Outbox/

destination_path_actions:
  *.pdf, replace-separator, --
  *.txt, replace-separator, _

the files will be transferred as follows:

c:\out\sales\Report.PDF          -> /Outbox/Report.PDF
c:\out\sales\Report.txt          -> /Outbox/Report.txt
c:\out\sales\report.csv          -> Fail as it does not match any rule.

c:\out\sales\18\Jan\Report.PDF   -> /Outbox/18--Jan--Report.PDF
c:\out\sales\18\Jan\Report.txt   -> /Outbox/18_Jan_Report.txt
c:\out\sales\18\Jan\report.csv   -> Fail as it does not match any rule.

6.1.7.5. lowercase

The lowercase is a simple transformation which will transfer the file using destination file path and file name all in lower case. It takes no options.

Note

If the remote filesystem is case insensitive and a file with the same name (but different cases) exists, the remote name is kept.

Below is an example for the lowercase transformation:

[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1]
enabled: yes
name: copy-to-remote
type: move

recursive: yes
source_path: /out/sales
source_filter: *

destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c
destination_path: /Outbox/

destination_path_actions:
  *.pdf, lowercase
  *, no-action

With the above configuration the following files will be transferred as exemplified below. Note that the base destination path as defined in the transfer configuration, is not affected by the file path and file name transformation. Files like report.txt are transferred since they match the no-action rule:

/out/sales/Report.PDF          -> /Outbox/report.pdf
/out/sales/Report.txt          -> /Outbox/Report.txt

/out/sales/Jan/Report.PDF      -> /Outbox/jan/report.pdf
/out/sales/Jan/Report.txt      -> /Outbox/Jan/Report.txt

6.1.7.6. fixed-path

The fixed-path action is a simple transformation which will transfer the file using a fixed destination. It takes a single option, the full destination path. The destination_path configuration is ignored for the fixed-path action.

Note

If this transformation is enabled, the temporary suffix behaviour is disabled.

Below is an example for the fixed-path transformation:

[transfers/313a9679-bc15-42e9-80ef-e2305959a3d1]
enabled: yes
name: copy-to-remote
type: move

recursive: yes
source_path: /out/sales
source_filter: *

destination_uuid: 5f304286-8d56-41f9-ba57-420a2303e90c
destination_path: /Outbox/

destination_path_actions:
  *, fixed-path, //'PDEB.OU11.DRSAFG(+1)'

With the above configuration, the following files will be transferred as exemplified below. Note that the base destination path as defined in the transfer configuration, is ignored:

/out/sales/Report.PDF          -> //'PDEB.OU11.DRSAFG(+1)'
/out/sales/Report.txt          -> //'PDEB.OU11.DRSAFG(+1)'

/out/sales/Jan/Report.PDF      -> //'PDEB.OU11.DRSAFG(+1)'
/out/sales/Jan/Report.txt      -> //'PDEB.OU11.DRSAFG(+1)'

6.1.8. Transfer States

A transfer can be in one of the following states:

  • Stopped - the transfer is not active and is not configured to be active at any time.
  • Started - the transfer is active and processing files from the source.
  • suspended - the transfer was started, but it is outside of its activity schedule.
  • Source failed - the source location is not available and files at source are not processed. Processing will be started as soon as the source location is available, either by fixing the errors and manually starting it or by an automated location reconnection process.
  • Destination failed - The destination location is not available. Processing will resume as soon as the destination location is available, either by fixing the errors or manually starting it.
  • Failed - a critical error occurred while executing the transfer and the transfer was stopped. Manual intervention is required to restart the transfer.

6.1.9. Location States

A location can be in one of the following states:

  • Stopped - no transfer is using this location and it was not manually started.
  • Started - the location is connected to the remote location.
  • Disconnected - the location is not connected to the remote location, but it will auto-reconnect when required by a transfer.
  • Failed - the location failed to connect to the remote location. Manual intervention is required to restart the location.

6.1.10. Fault-tolerant transfers

SFTPPlus will do its best to perform the requested transfers and under certain circumstance will retry on transfer failures.

From the configuration, you can define the number of times to retry and the time interval to wait after retries.

Once all retries are exhausted, the transfer will fail and SFTPPlus will no longer try to process/transfer that file.

Files which were previously failed to be transferred, can be processed/transfered again by restarting the transfer.

6.1.11. Fault-tolerant source monitoring

When a transfer is started, SFTPPlus will check for the required permissions to monitor the source path for changes, the configured path exists and that this path is valid.

If the source path is no longer available after the transfer was started and the initial check operations were successful, SFTPlus will not consider the transfer failed. It will wait for the next configured changes_poll_interval to pass and will try again. This is done for an unlimited number of times and an event is emitted on each failure. Once the source path is available again, the transfer is resumed and new or changed files will be transferred.

The type of source path errors which are considered not critical for the purpose of source monitoring varies based on the file transfer protocol.

The most common errors found on all supported protocols are:

  • Source path not found.
  • Source path without read or listing permissions.
  • Connection closed while source was checked.

For the HTTP and WebDAV protocol, below is a non-exhaustive list of HTTP codes for which source monitoring will continue:

  • 401 UNAUTHORIZED
  • 403 FORBIDDEN
  • 404 NOT FOUND
  • 429 TOO MANY REQUESTS
  • 500 INTERNAL SERVER ERROR
  • 503 SERVICE UNAVAILABLE
  • 504 GATEWAY TIMEOUT

6.1.12. Resumed transfers

Resuming unfinished transfers, either upload or download, is supported across available file transfer protocols as follows:

Protocol Download Upload
FTP/FTPS Yes Yes
SFTP Yes Yes
SCP No No
HTTP Yes No

Note

Our roadmap includes adding support for resuming transfers across all available protocols. In the case that resume support is not available for your preferred transfer protocol, please contact us.

6.1.14. Resolving duplicate / successive events

When an event occurs at the source location, it is not signaled right away. A buffer time interval is used to remove duplicate events and emit the event only once the file or folder is stable.

For example, when a large file is copied, the operating system will emit a stream of events after each intermediary change. The monitor will queue these changes and merge them into a single event.

The time frame in which duplicate events are resolved is 1 second. After an event occurs, it waits 1 second before emitting the event. If no event occurs for the same source, the event is emitted. Otherwise, the event is merged and the time interval is extended by another second in which it can be further merged with another event, or emitted.

Here is the list of events which are emitted. Path is used as a generic reference to a file or a folder:

  • Existing - for paths which are already present when the process is started.
  • Modify - for a path which has a changed size or modify time.
  • Delete - for a path which is no longer in the monitored path. Either removed from there or moved to a path which is not monitored.
  • Create - for a path which has appeared in the monitored path. Either created there or moved from a path which is not monitored.
  • Moved - for a path which was renamed / moved inside the monitored path.

Here are the rules based on which events are resolved:

  • Existing + Modify = Existing
  • Existing + Delete = Ignore
  • Modify + Modify = Modify (and wait for next event)
  • Modify + Delete = Delete (without delay)
  • Modify + Move = Move + Modify (modify is last)
  • Create + Modify = Create (and wait for next event)
  • Create + Delete = Discard (file was created for a very short time)
  • Create + Move = Create (at the moved path)
  • Delete.File + Create.File = Modify (the file was completely replaced)
  • Delete.Folder + Create.File = Create.File
  • Moved + Created = Moved + Created (with delay)
  • Moved + Modified = Moved + Modified (with delay)
  • Moved + Deleted = Deleted for the source path

6.1.15. Executing commands using pre/post transfer executables or scripts

SFTPPlus allows configuring a transfer to execute at certain transfer steps:

  • Execute before processing a source file
  • Execute after success of a file transfer
  • Execute after failure of a file transfer

The configuration from this example, will copy (indicated by type) files from a source (indicated by source_path and source_uuid) to a destination (indicated by destination_path and destination_uuid). A timestamp-always is set in the overwrite_rule in order to always copy the source file with an amended timestamp and prevent accidental overwriting on the destination. It will poll (via changes_poll_interval) for changes to the folder every 5 seconds. After 10 seconds of no changes to a file (indicated by stable_interval), it will consider the file stable and ready to be processed.

A script (saved as a local file at C:\acme\sftpplus\transfer-jobs\after-success.bat) is executed once a file that has been successfully transferred is considered stable.

The script is configured as:

[transfers/7db823d8-05f8-4481-be98-b87a826ded28]
execute_after_success = C:\acme\sftpplus\transfer-jobs\after-success.bat

There are additional processing actions within the custom script. Processing actions include:

  • Creating a zipped archive from a target source.
  • Affixing a date to the archive.
  • Moving the archive to another destination.

An example of the above configuration:

[transfers/7db823d8-05f8-4481-be98-b87a826ded28]
enabled = Yes
name = inbox-transfer
description = Transfer Inbox files.
type = copy
source_uuid = local-file-system-source-uuid
source_path = C:\acme\sftpplus\accounting\inbox
destination_uuid = local-file-system-destination-uuid
destination_path = C:\acme\sftpplus\accounting\inbox-final
execute_after_success = C:\acme\sftpplus\transfer-jobs\after-success.bat
stable_interval = 10
changes_poll_interval = 5
overwrite_rule = timestamp-always
batch_interval = 0
retry_count = 0
retry_wait = 0

6.1.16. Executing pre-/post- transfer destination commands

SFTPPlus allows configuring a transfer to execute a list of commands via the following options:

  • Execute after success on destination
  • Execute before on destination
  • Execute before success on destination.

These commands need to be available in the SFTPPlus client-shell implementation.

To check what commands are available, please start the client-shell and run:

> help

To find out more about a particular command, run:

> help COMMAND_NAME

These commands are configured as part of the Transfer section in the Local Manager GUI and it can be added in the configuration text file like below:

[transfers/99feb83f-a99d-42f1-a86c-1562c3281bb6]
name = transfer_sample
execute_on_destination_after_success = rename /path1 /path2; delete /path3
execute_on_destination_before = client-shell-COMMAND /file/path
execute_on_destination_before =
rename /path-to/  /path2
delete /path3

Transfers can executed either a single command or multiple commands.

Multiple commands are separated by a semicolon (;) or on a separate line.

6.1.17. Working with the source monitor

6.1.17.1. Understanding the changes poll interval

When the changes_poll_interval passes, the application will read all files, folders, and attributes (size, last modification time, etc) found in the monitored folder. Each snapshot is then compared with the previous.

If there are changes between the snapshots, they are notified to the application.

Before doing the actual notification, the application will wait for changes to be considered stable as defined by another option, stable_interval.

Note

Lower values help detect changes quicker, but increase the load, CPU, and network usage for local and remote servers.

The poll interval might drift.

For example, with a value of 72000 and a transfer started at 15:35, the product might be busy at 15:35. In this case, the check is done only at 15:36 and the next check will be scheduled for 17:36.

6.1.17.2. Changing the file stable period settings

When a new file is created or a file starts to be modified, it is not processed by SFTPPlus right away. This allows an external program to finish handling the file.

If no other changes to the file are observed after this interval, it is then processed.

Criteria for a file to be considered stable:

  • Size is not changed.
  • Last modified time attribute is not changed.
  • The configured stable interval has elapsed.

After the last modification is observed, this interval is then allowed to pass.

Each change will reset the stable interval for the file itself since each file has its own stable counter.

Since changes can only be observed with a resolution defined by changes_poll_interval, the stable_interval needs to be a multiple.

The following will check new files every hour and transfer stable files after two hours, even if the stable_interval is lower:

[transfers/b904e6a6-c29b-4ccf-8abd-edcae4d3324f]
changes_poll_interval = 3600
stable_interval = 60

6.1.18. Transfer scheduling

A transfer can be configured to be active all the time or be active based on a simple scheduler.

The scheduler available in SFTPPlus allows you to define transfer start and transfer stop events in a 24 hour interval or weekly interval. The current scheduler has no knowledge of the exact day of month or year. It only knows about current time inside the day and current day inside the week.

To create a scheduled transfer, you will to define start and stop action on different hours across a day or week.

For example, to have the transfer started daily at 10:00, and stopped at 14:00, you can use the short notation:

[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028]
schedule = 10:00-start, 14:00-stop

or the weekly longer notation:

[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028]
schedule = Monday-10:00-start, Monday-14:00-stop
           Tuesday-10:00-start, Tuesday-14:00-stop
           Wednesday-10:00-start, Wednesday-14:00-stop
           Thursday-10:00-start, Thursday-14:00-stop
           Friday-10:00-start, Friday-14:00-stop
           Saturday-10:00-start, Saturday-14:00-stop
           Sunday-10:00-start, Sunday-14:00-stop

To have the transfer started daily at 10:00, stopped at 14:00, restarted at 17:00, and stopped at 18:00, use:

[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028]
schedule = 10:00-start, 14:00-stop, 17:00-start, 18:00-stop

You can also have the scheduler extended over two consecutive days. To have the transfer started daily at 23:00 and then stopped the next day at 01:00, use:

[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028]
schedule = 23:00-start, 01:00-stop

For defining actions at different days throughout the week, you can add the name of the day before the time value. To have a transfer started at 10:00, stopped at 14:00 every Monday, Wednesday, and Friday, use:

[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028]
schedule = Monday-10:00-start, Monday-14:00-stop
           Wednesday-10:00-start, Wednesday-14:00-stop
           Friday-10:00-start, Friday-14:00-stop

You can also have start and stop events spanning different days. To have a transfer which is active from Monday to Wednesday and then again active from Friday to Saturday, use:

[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028]
schedule = Monday-10:00-start, Wednesday-18:00-stop
           Friday-10:00-start, Saturday-18:00-stop

At this stage, the transfer’s changes_poll_interval is not aligned with the schedule. We are currently working on improving this to align the interval with the schedule.

For example, the configuration below will schedule to check files every 2 hours (or 7200 seconds) between 8AM and 6PM:

[transfers/6b775eec-f98b-4527-811f-fd8c5c71a028]
schedule = 08:00-start, 18:00-stop
changes_poll_interval = 7200

If the transfer was started at 13:35, it will do a check at 13:35 instead of what you might expect to be done at 14:00 The next check will be scheduled for 15:35.

Please get in touch if you want a transfer with a complex scheduling rules.

6.1.19. Batch transfer operation

The default transfer operation for SFTPPlus is to transfer a file as soon as it is detected and considered stable. Each file will be processed as an independent transfer. The pre/post command hooks and events are executed as a separate process for each file.

By using the batch_interval configuration option, you can instruct SFTPPlus to only start transferring files once no new files are detected in the specified time interval.

All the files detected during this interval will be transferred together as a single transfer. The pre/post command hooks and events are executed once for the batch of files.

You can also configure SFTPPlus to wait and batch/group the files based on a file with a specific path. This is done by using the batch_marker_path configuration option. You can configure an exact file path or a pattern of file paths which can trigger the transfer.

Once a file with a path matching the batch_marker_path is detected, SFTPPlus will wait for 15 seconds before it begins transferring all the previous files. Should there be new files created within the 15 seconds interval after the marker, it will be included in the batch transfer.

Note

The matching expression is for the file path and not for the file name. If you want to match a file name complete.txt located in any directory, you should configure it as *complete.txt.

The marker file is also transferred.

For more details about defining a file marker, see the matching expression documentation.

When filtering source files which are to be transferred, the filtering rules should also include the file marker. The following example will filter the source for both .pdf and .desc files. The transfer will start 15 seconds after a report.desc files is generated in any of the subdirectories for c:\jobs\reports. For example, in c:\jobs\reports\ACC-2018-09-23\report.desc or c:\jobs\reports\ACC-2018-09-24\report.desc

[transfers/a4747959-25de-470a-b76e-afe08071a70d]
enabled: yes
name: Move report set.
description: Move all report files once the report is finalized.
   A report.desc file is generated by the external reporting process
   to signal that the process is complete.
type: move

recursive: Yes
source_uuid: c4b74ed0-b952-4edc-be24-7d8e1e6d8068
source_path: c:\jobs\reports
source_filter: m/*.(pdf|desc)/

destination_uuid: 92eb50d4-bfc2-4b15-a369-f0d74dc68dac
destination_path: /dropbox/

batch_interval: 0
batch_marker_path: *done.desc

With the configuration from above, assume that you have the following files:

c:\jobs\reports\ACC-123\in.pdf
c:\jobs\reports\ACC-123\out.pdf
c:\jobs\reports\ACC-123\response\report.pdf
c:\jobs\reports\MON-542\office.pdf
c:\jobs\reports\MON-542\storage.pdf
c:\jobs\reports\MON-542\response\report.pdf
c:\jobs\reports\MON-542\done.desc

then the following actions are taken on these files

c:jobsreportsACC-123in.pdf - Not transferred c:jobsreportsACC-123out.pdf - Not transferred c:jobsreportsACC-123responsereport.pdf - Not transferred c:jobsreportsMON-542office.pdf - Transferred c:jobsreportsMON-542storage.pdf - Transferred c:jobsreportsMON-542responsereport.pdf - Transferred c:jobsreportsMON-542done.desc - Transferred

The batch marker should match a single file for a certain directory and its descendants. The following file structure is valid and the files will be transferred since the marker files are in different folders which are not a descendant of another:

c:\jobs\reports\ACC-123\in.pdf
c:\jobs\reports\ACC-123\out.pdf
c:\jobs\reports\ACC-123\done.desc            - Marker file
c:\jobs\reports\MON-542\storage.pd
c:\jobs\reports\MON-542\done.desc            - Marker file

The following file structure will fail since MON-542\response\report.pdf is triggered by both MON-542\response\done.desc and MON-542\done.desc

c:\jobs\reports\MON-542\done.desc        - Marker file
c:\jobs\reports\MON-542\storage.pdf      - Triggered by MON-542\done.desc
c:\jobs\reports\MON-542\resp\done.desc   - Marker file
c:\jobs\reports\MON-542\resp\report.pdf  - Triggered by MON-542\done.desc
                                           and MON-542\resp\done.desc

6.1.19.1. Batch transfer and scheduling

When a transfer is stopped and there are files queued for a batch transfer, SFTPPlus will not transfer the files but will instead emit a warning to inform the user of this action.

When a transfer is suspended, and there are files in a batch which are not yet triggered, the files are not transferred and no warning info is emitted. The files will be transferred once the transfer is resumed and the batch is triggered by one of the batch triggering conditions.

When a batch transfer is in progress and the transfer is suspended, the active batch will continue to be transferred.