Documentation
11.12. Events, Event Handlers and the audit trail¶
11.12.1. Introduction¶
While operating, SFTPPlus will emit a set of events. Each event has a unique ID and defines a specific operation carried out by the server.
A common action for an event is to send it to one of the supported logging systems. This covers the Accounting requirement of the AAA (Authentication, Authorization, and Accounting) security design framework.
For information on covering the Authentication process, check the authentication section and for the Authorization process, see the authorization section.
In addition, arbitrary actions can be attached to an event by configuring event handlers.
Event handlers are asynchronous / independent components and server operations will continue regardless of the side effects produced by the event handler.
All data used by the event handlers is Unicode, encoded as UTF-8.
The integrated logging system, keeps an audit trail for each action/event produced while performing management tasks or file transfer operations. The server can also send events to a SysLog server or to the Windows EventLog, but after an entry is sent, the server cannot retrieve or manage those logs.
11.12.2. Event groups as an alternative to logging levels¶
SFTPPlus does not use any form of logging level or severity level conventions to classify and categorize the generated log entries. Instead, the event groups are provided as a way to classify the entries. A single event can belong to multiple groups.
A special group is the failure group, which contains all events which may be considered an error.
A failure event is emitted when a client-side or server-side request produces an outcome that is either an error (when request is processed and returns an error as defined in the protocol) or a failure (when the request was not processed).
For example, a client which requested to create a folder that already exists may be ignored since the client may also ignore the error. In other cases, you may want to investigate the failure and update the client to not try to create a folder which already exists.
Internal server errors and errors which should not occur during normal operations are part of the failure and failure-critical groups. All critical failures have a details data field which contains information about the cause of the error.
Configuration errors and other errors which cause one of the components to stop, or prevent it from starting, are part of the failure and failure-high group.
11.12.3. Activity Log page in Web Manager¶
For low volume installations, the "Activity Log" (or the audit trail) can be used within the Web Manager to view and filter logs.
Using a web browser you can perform interactive searches within the events generated in the past.
Here are a few actions available:
Filter by event handler
Filter by date and time
View x events per page
Filter by event ID, peer IP, username, and summary
Further view the details of each event.
Please see below for an example page.
Note that the embedded database, used by default by the Web Manager, is not suitable for high volume installations.
In the case in which your installation generates thousand of entries each day, we recommend using specialized log management tools external to SFTPPlus. For more information about managing logs in high volume websites, see
11.12.4. Account Activity page in Web Manager¶
Activities of accounts set up in SFTPPlus are reported in Web Manager.
11.12.5. The Audit Trail¶
This section describes how the audit trail works in order to help you use it and integrate it as per your requirements. For configuring the audit system, please see event handlers configuration.
Each audit entry contains the following information:
Unique event ID
Date and time
Component ID
Account name
IP and port of remote client
Human readable message
Structured log data as JSON.
While the plain text log format does not list the structured log data, this is available in the format used for storing audit entries in a database.
Here is an example of text file log format:
20156 2014-06-07 19:44:05 ftp-only-1 Process 0.0.0.0:0 Service "ftp" started on "0.0.0.0:10023" using "ftp" protocol.10033 2014-06-07 19:44:10 ftp-only-1 Unknown 127.0.0.1:51290 New FTP/FTPS client connection made.20009 2014-06-07 19:44:10 ftp-only-1 test_user 127.0.0.1:51290 User authenticated as application account using password credentials.10059 2014-06-07 19:44:10 ftp-only-1 test_user 127.0.0.1:51290 User successfully logged on "/srv/iop_files/test_user" as "/".10022 2014-06-07 19:44:17 ftp-only-1 test_user 127.0.0.1:51292 Expecting client connection on 32.12.123.12:9054 for the next EPSV passive request.10077 2014-06-07 19:44:17 ftp-only-1 test_user 127.0.0.1:51292 Processing STOR command for file "/remote_put".
For Syslog the log format conforms to RFC 3164. Here is an example:
<30>Apr 19 18:08:11 host test-server-name: 20156 admin 127.0.0.1:33674 Successfully started event handler "SQLite Event Handler" of type database.<30>Apr 19 18:09:53 host test-server-name: 20156 admin 127.0.0.1:33674 Successfully started event handler "sample-digital-signature" of type digital-signature-validation.<30>Apr 19 18:19:28 host test-server-name: 50002 admin 127.0.0.1:33674 Configuration read from local manager.<30>Apr 19 21:38:34 host test-server-name: 20156 Process 0.0.0.0:0 Successfully started database connector "SQLite" of type sqlite.
Here is an example of structured data attached to an event with ID 10077:
account:
name: test_user
peer:
protocol: TCP
port: 51490
address 127.0.0.1
path: /remote_put
Plain text log files can automatically rotate based on size, date or time of the day. Older files can be automatically removed, based on configurable rules. This helps limit the size of a single file and control the total disk space used by the plain text logging system.
Audit entries stored in a database can be viewed and filtered using the Web Manager service.
11.12.6. Filtering the handled events¶
When an event handler is defined there are a set of rules based on which you can include or exclude the events handled by the configured event handler.
Multiple rules can be defined and an event is handled if all rules are matching the targeted event. That is, the combined rules are defining a logical conjunction (AND).
Note that inside a single rule the configured values can form a disjunction (OR), this is based on the characteristics of each rule.
For example, to only send the events from users mcr_adm or acct_aut
with event IDs 20156 to Syslog, you can define the following
configuration:
[event-handlers/00feb81f-a99d-42f1-a86c-1562c3281bd9]
enabled: Yes
name: Syslog Notification
description: Sends logs to backup Syslog server.
type: syslog
url: file:///dev/log
usernames: mcr_adm, acct_aut
target: 20156, 20127, 50002
In this case, only events by the accounts (mcr_adm OR acct_aut) AND
event IDs (20156 OR 20127 OR 50002) are sent to the specified Syslog
path.
Any other event IDs outside of this will not be logged.
For more details check the configuration documentation available for each of the filtering rules.
11.12.6.1. Event ID¶
A simple and straight forward rule is to include only event IDs.
For example, to only send event IDs 20156 or 20127 or 50002 to Syslog, you can define the following configuration. Subsequently, event IDs 5421 or 2341 are not matched since they are not listed as a target:
[event-handlers/1ee4337a-22f7-4a67-9a77-5c3a508a8158]
enabled: Yes
name: Syslog Backup
description: Sends logs to backup Syslog server.
type: syslog
url: file:///dev/log
target: 20156, 20127, 50002
The target configuration option also support event exclusion by prefixing the event ID with with the exclamation mark (!).
If you want to send all the events to Syslog, with the exception of the event IDs 20156 or 20127, you can define the following configuration:
[event-handlers/27a31405-a963-4fb9-b4ee-09d415b1a30a]
enabled: Yes
name: Syslog Main Source
description: Sends filtered logs to Syslog server.
type: syslog
url: file:///dev/log
target: !20156, !20127
11.12.6.2. Username / Account Name¶
Most of the events emitted by the server-side components will have associated accounts or usernames.
These are part of the authenticated group and are emitted for a session for which the authentication process was successful.
You can filter these type of events based on the associated user or account name.
Note that the filtering is done based on the name, and not the UUID. This is done to accommodate various authentication methods which do not have the concept of UUID.
For example, to only send the events from users mcr_adm or acct_aut
to Syslog you can define the following configuration:
[event-handlers/7db823d8-05f8-4481-be98-b87a826ded28]
enabled: Yes
name: Syslog Notification
description: Sends notification to Syslog server.
type: syslog
url: file:///dev/log
usernames: mcr_adm, acct_aut
If you want to send all the events to Syslog, except the users mcr_adm or acct_aut,
the configuration will look like this.
If you prefer, you can add a space between the exclamation mark and the username:
[event-handlers/7db823d8-05f8-4481-be98-b87a826ded28]
enabled: Yes
name: Syslog Notification
description: Sends a notification to the Syslog server.
type: syslog
url: file:///dev/log
usernames: !mcr_adm, ! acct_aut
11.12.6.3. Components¶
SFTPPlus is built around a modular design in which multiple components are interacting in order to perform the manged file transfer flow.
Components are file transfer services, authentication methods, transfers, resources and event handlers.
Each component has an unique ID (UUID) and the handler can filter events based on these IDs.
By using the UUID, you can rename a component without having to update the configuration of the associated event handlers.
For example, to send all the events emitted by certain authentication methods to Syslog, you can use the following configuration:
[authentications/ac547e16-a3ff-4fc3-a6ab-142af2744f50]
enabled: yes
type: deny-username
name: deny-admin-accounts
description: Filter all admin accoutns
usernames: root, adm, administrator
[authentications/22a9d8fb-068d-4a63-8d5d-0ce94ef22a25]
enabled: Yes
type: ip-time-ban
name: ip-time-ban-auth
description: Ban for 30 seconds after 3 failures
ban_interval = 30
ban_after_count = 3
[event-handlers/27b8e2b1-7971-416d-af14-6a8aae2ac46e]
enabled: Yes
name: Syslog Notification
description: Sends rogue auth requests to Syslog.
type: syslog
url: file:///dev/log
components: 27b8e2b1-7971-416d-af14-6a8aae2ac46e,
ac547e16-a3ff-4fc3-a6ab-142af2744f50
To send all the events to Syslog, with the exception of a few components, you can configure the handler as the following configuration:
[event-handlers/27b8e2b1-7971-416d-af14-6a8aae2ac46e]
enabled: Yes
name: Syslog Notification
description: Sends rogue auth requests to Syslog.
type: syslog
url: file:///dev/log
components: !27b8e2b1-7971-416d-af14-6a8aae2ac46e,
!ac547e16-a3ff-4fc3-a6ab-142af2744f50
11.12.6.4. Source address of the remote peer¶
You can include or exclude events based on the source IP address of the remote peer.
If for example, your load balancers are sending probes usiing the
172.26.2.45 and 196.168.9.34 IP addresses you might want to exclude
those connections from the main logs.
The example from below will send all the events to Syslog, with the exception of the connection made from certain IP addresses:
[event-handlers/27b8e2b1-7971-416d-af14-6a8aae2ac46e]
enabled: Yes
name: Syslog Notification
description: Sends all connections to Syslog excepting the probes.
type: syslog
url: file:///dev/log
source_addresses: !172.26.2.45, !196.168.9.34
11.12.6.5. Attached structured data¶
Each event emitted by SFTPPlus has a data attribute with an arbitrary structure.
The members of the data attribute are specific to each event. This filter is usually used together with the target option, to also filter based on event IDs.
If an event does not have the required data member, the event will not be handled.
The following example will only send the PDF files uploaded via SFTP
to Syslog.
Non PDF files are not sent to Syslog.
The PDF files are defined using the regular expression .*\.pdf:
[event-handlers/1ee4337a-22f7-4a67-9a77-5c3a508a8158]
enabled: Yes
name: Syslog Notification
description: Sends PDF file operations to Syslog.
type: syslog
url: file:///dev/log
target: 30068
data_filter: path, m/.*\.pdf/
11.12.7. HTTP / HTTPS POST Handlers¶
To read more about HTTP POST event handlers in the audit trail, please go to the Developer Documentation.
11.12.8. Database handlers¶
All fields that are displayed in the Activity Log page have an associated, single column, index. The sole exception is the Summary / message field, in order to limit database/table size.
When using the database event handler, SFTPPlus will store / update the newly generated events in an SQLite3 database file.
As each installation will have its own data and log retention policy, SFTPPlus will not delete any event.
This can result in a database which will continuously grow in size.
We assume that each site administrator will set the auto-delete feature to manage the size of the database.
The name of the database used to store the logs is chevah_log_entries. In SQLite, to check all the fields from the table, you can use:
| sqlite> SELECT * FROM sqlite_master WHERE type='table' and
name='chevah_log_entries';
To delete all log entries older than a certain timestamp, you can use:
sqlite> DELETE FROM chevah_log_entries WHERE timestamp < '1526469347' ;