5.11. Events, Event Handlers and the audit trail

5.11.1. Introduction

While operating, SFTPPlus will emit a set of events. Each event has a unique ID and defines a specific operation carried out by the server.

A common action for an event is to send it to one of the supported logging systems. In addition, arbitrary actions can be attached to an event by configuring event handlers.

Event handlers are asynchronous / independent components and server operations will continue regardless of the side effects produced by the event handler.

All data used by the event handlers is Unicode, encoded as UTF-8.

The integrated logging system, keeps an audit trail for each action/event produced while performing management tasks or file transfer operations. The server can also send events to a SysLog server or to the Windows EventLog, but after an entry is sent, the server cannot retrieve or manage those logs.

5.11.2. ‘Past Activity’ page in Local Manager

Past activities (or the audit trail) can also be accessed in the Local Manager through the ‘Past Activity’ page.

The web browser adds an interactive way of viewing past activities with the SFTPPlus server:

• Filter by database
• Filter by date and time
• View x events per page
• Filter by event ID, peer IP, username, and summary
• Further view the details of each event.

Please see below for an example page filtered by database (SQLite).

5.11.3. ‘Account Activity’ page in Local Manager

Activities of accounts set up in SFTPPlus are reported in Local Manager.

5.11.4. Failure events

All failure events are part of the failure group.

Internal server errors and errors which should not occur during normal operation are part of the failure and failure-critical groups. All critical failures have a details data field which contains information about the cause of the error.

Configuration errors and other errors which cause one of the components to stop, or prevent it from starting, are part of the failure and failure-high group.

5.11.5. The Audit Trail

This section describes how the audit trail works in order to help you use it and integrate it as per your requirements. For configuring the audit system, please see event handlers configuration.

Each audit entry contains the following information:

• Unique event ID
• Date and time
• Component ID
• Account name
• IP and port of remote client
• Human readable message
• Structured log data as JSON.

While the plain text log format does not list the structured log data, this is available in the format used for storing audit entries in a database.

Here is an example of text file log format:

20076 2014-06-07 19:44:05 ftp-only-1 Process 0.0.0.0:0 Service “ftp” started on “0.0.0.0:10023” using “ftp” protocol.

10033 2014-06-07 19:44:10 ftp-only-1 Unknown 127.0.0.1:51290 New FTP/FTPS client connection made.

20009 2014-06-07 19:44:10 ftp-only-1 test_user 127.0.0.1:51290 User authenticated as application account using password credentials.

10059 2014-06-07 19:44:10 ftp-only-1 test_user 127.0.0.1:51290 User successfully logged on “/srv/iop_files/test_user” as “/”.

10061 2014-06-07 19:44:17 ftp-only-1 test_user 127.0.0.1:51292 Passive transfer requested.

10022 2014-06-07 19:44:17 ftp-only-1 test_user 127.0.0.1:51292 Listening on port 9000 for the next passive request.

10077 2014-06-07 19:44:17 ftp-only-1 test_user 127.0.0.1:51292 Processing STOR command for file “/remote_put”.

For Syslog the log format conforms to RFC 3164. Here is an example:

<30>Apr 19 18:08:11 host test-server-name: 20156 admin 127.0.0.1:33674 Successfully started event handler “SQLite Event Handler” of type database.

<30>Apr 19 18:09:53 host test-server-name: 20156 admin 127.0.0.1:33674 Successfully started event handler “sample-digital-signature” of type digital-signature-validation.

<30>Apr 19 18:19:28 host test-server-name: 50002 admin 127.0.0.1:33674 Configuration read from local manager.

<30>Apr 19 21:38:34 host test-server-name: 20156 Process 0.0.0.0:0 Successfully started database connector “SQLite” of type sqlite.

Here is an example of structured data attached to an event with ID 10077:

account:
    name: test_user
    peer:
      protocol: TCP
      port: 51490
      address 127.0.0.1
path: /remote_put

Plain text log files can automatically rotate based on size, date or time of the day. Older files can be automatically removed, based on configurable rules. This helps limit the size of a single file and control the total disk space used by the plain text logging system.

Audit entries stored in a database can be viewed and filtered using the Local Manager service.

Note

Currently, there is no option to remove/clean log entries stored in a database from within SFTPPlus.

You can, however, use external tools to remove logs stored in a database.

5.11.6. Filtering the handled events

When an event handler is defined there are a set of rules based on which you can include or exclude the events handled by the configured event handler.

Multiple rules can be defined and an event is handled if all rules are matching the targeted event. That is, the combined rules are defining a logical conjunction (AND).

Note that inside a single rule the configured values can form a disjunction (OR), this is based on the characteristics of each rule.

For example, to only send the events from users mcr_adm or acct_aut with event IDs 20156 to Syslog, you can define the following configuration:

[event-handlers/73a23-23ead2]
enabled: Yes
name: Syslog Notification
description: Sends logs to backup Syslog server.
type: syslog
path: /dev/log

usernames: mcr_adm, acct_aut
target: 20156, 20127, 50002

In this case, only events by the accounts (mcr_adm OR acct_aut) AND event IDs (20156 OR 20127 OR 50002) are sent to the specified Syslog path. Any other event IDs outside of this will not be logged.

For more details check the configuration documentation available for each of the filtering rules.

5.11.6.1. Event ID

A simple and straight forward rule is to include only event IDs.

For example, to only send event IDs 20156 or 20127 or 50002 to Syslog, you can define the following configuration. Subsequently, event IDs 5421 or 2341 are not matched since they are not listed as a target:

[event-handlers/73a23-23ead2]
enabled: Yes
name: Syslog Backup
description: Sends logs to backup Syslog server.
type: syslog
path: /dev/log

target: 20156, 20127, 50002

The target configuration option also support event exclusion by prefixing the event ID with with the exclamation mark (!).

If you want to send all the events to Syslog, with the exception of the event IDs 20156 or 20127, you can define the following configuration:

[event-handlers/73a23-23ead2]
enabled: Yes
name: Syslog Main Source
description: Sends filtered logs to Syslog server.
type: syslog
path: /dev/log

target: !20156, !20127

5.11.6.2. Username / Account Name

Most of the events emitted by the server-side components will have associated accounts or usernames.

These are part of the authenticated group and are emitted for a session for which the authentication process was successful.

You can filter these type of events based on the associated user or account name.

Note that the filtering is done based on the name, and not the UUID. This is done to accommodate various authentication methods which do not have the concept of UUID.

For example, to only send the events from users mcr_adm or acct_aut to Syslog you can define the following configuration:

[event-handlers/73a23-23ead2]
enabled: Yes
name: Syslog Notification
description: Sends notification to Syslog server.
type: syslog
path: /dev/log

usernames: mcr_adm, acct_aut

5.11.6.3. Components

SFTPPlus is built around a modular design in which multiple components are interacting in order to perform the manged file transfer flow.

Components are file transfer services, authentication methods, transfers, resources and event handlers.

Each component has an unique ID (UUID) and the handler can filter events based on these IDs.

By using the UUID, you can rename a component without having to update the configuration of the associated event handlers.

For example, to send all the events emitted by certain authentication methods to Syslog, you can use the following configuration:

[authentications/98cd32-54acde2]
enabled: yes
type: deny-username
name: deny-admin-accounts
description: Filter all admin accoutns
usernames: root, adm, administrator

[authentications/753acd23e-edc2317c]
enabled: Yes
type: ip-time-ban
name: ip-time-ban-auth
description: Ban for 30 seconds after 3 failures
ban_interval = 30
ban_after_count = 3

[event-handlers/73a23-23ead2]
enabled: Yes
name: Syslog Notification
description: Sends rogue auth requests to Syslog.
type: syslog
path: /dev/log

components: 753acd23e-edc2317c, 98cd32-54acde2

5.11.6.4. Attached structured data

Each event emitted by SFTPPlus has a data attribute with an arbitrary structure.

The members of the data attribute are specific to each event. This filter is usually used together with the target option, to also filter based on event IDs.

If an event does not have the required data member, the event will not be handled.

The following example will only send the PDF files uploaded via SFTP to Syslog. Non PDF files are not sent to Syslog. The PDF files are defined using the regular expression .*.pdf:

[event-handlers/73a23-23ead2]
enabled: Yes
name: Syslog Notification
description: Sends PDF file operations to Syslog.
type: syslog
path: /dev/log

target: 30068
data_filter: path, m/.*\.pdf/

5.11.7. HTTP / HTTPS POST Handlers

To read more about HTTP POST event handlers in the audit trail, please go to the Developer Documentation.

5.11.8. Database handlers

All fields that are displayed in the Past Activity page have an associated, single column, index. The sole exception is the Summary/message field, in order to limit database/table size.