5.8. Client-Side Transfers Operations

5.8.1. Introduction

Transfers are the pro-active component of SFTPPlus. While a file transfer service only acts in response to requests from remote clients, transfers actively check local or remote paths and initiate transfer based on changes in those folders.

Transfers are executed by transferring files between a source location and a destination location.

5.8.2. Source Location

Source location changes are detected by regularly checking the source folder for changes.

When a change is observed on the monitored source location, the monitor emits one of the following event types:

• file modified
• file/folder created
• file/folder deleted
• file/folder moved

Here is a list of file system changes which are ignored:

• Folder modified (since they are emitted whenever a member is created / removed / moved inside that folder)
• Permissions / Security / Attribute changes

For transfers of type Local filesystem only, created, moved, or removed folders will only emit an informational log event, without triggering the prior transfer command. The same behaviour is present when a file is removed.

For transfers of type other than Local filesystem only, folder events and file removed event are ignored, and no informational event is emitted.

Source events are observed only inside the configured path, and the server will ignore events outside the configured path. While in most cases this is the desired result, it affects ‘moved’ events. Moved events are only observed inside the location path. When you move a file from a path outside the monitored path, a create event is emitted. When you move a file to a path outside the monitored path, a delete event is emitted.

A transfer can monitor only direct ancestors of a folder, for non-recursive monitors, or all ancestors for recursive ones.

When the source location is not available (for example its connection failed after all retries) the transfer will become stalled and no files will be transfered.

5.8.3. Destination Location

The destination’s locations are accessed only when a file which needs to be transferred is detected on the source location.

When the destination location is not available the changes detected on the source location are still observed but no transfer is attempted. As soon as the destination location is available again, new changes observed on the destination will be processed.

5.8.4. Transfer States

A transfer can be in one of the following states:

• stopped - the transfer is not active and is not configured to be active at any time
• started - the transfer is active and processing files from the source
• suspended - the transfer was started, but it is outside of its activity schedule
• stalled - the source location is not available and files at source are not processed. Processing will be started as soon as the source location is available, either by fixing the errors and manually starting it or by an automated location reconnection process.
• stalled - The destination location is not available. Processing will resume as soon as the destination location is available, either by fixing the errors or manually starting it.
• failed - a critical error occurred while executing the transfer and the transfer was stopped. Manual intervention is required to restart the transfer.

5.8.5. Location States

A location can be in one of the following states:

• stopped - no transfer is using this location and it was not manually started
• started - the location is connected to the remote location
• stalled - the location is not connected to the remote location but it can auto-connect when required by a transfer.
• failed - the location failed to connect to the remote location. Manual intervention is required to restart the location.

5.8.6. Hard Links and Symbolic Links

Hard links and symbolic links are observed as normal files.

When both the link and original are monitored, changes to the link will trigger notifications for the original file as well as any other hard links pointing to it.

When only the link is monitored, changes to the original source will also trigger notifications.

5.8.7. Resolving duplicate / successive events

When an event occurs at the source location, it is not signalled right away. A buffer time interval is used to remove duplicate events and emit the event only once the file or folder is stable.

For example, when a large file is copied, the operating system will emit a stream of events after each intermediary change. The monitor will queue these changes and merge them into a single event.

The time frame in which duplicate events are resolved is 1 second. After an event occurs, it waits 1 second before emitting the event. If no event occurs for the same source, the event is emitted. Otherwise, the event is merged and the time interval is extended by another second in which it can be further merged with another event, or emitted.

Here are the rules based on which events are resolved:

• modify + modify = modify (and wait for next event)
• modify + delete = delete (without delay)
• create + modify = create (and wait for next event)
• create + delete = discard (file was created for a very short time)
• delete + create = modify (the file was completely replaced)
• moved + created = moved + created (with delay)
• moved + modified = moved (and wait for next event)