Documentation

12.3. HTTP POST Event Handler¶

Introduction
Request body
Response codes
Response body
Redundant HTTP URL / Endpoints
Fault-tolerant / Resilient HTTP requests
Sending custom JSON data in HTTP POST request

12.3.1. Introduction ¶

The HTTP POST Event Handler is where you can integrate SFTPPlus with your web resource. Simply create an event handler that will send an HTTP POST request to your remote HTTP resource based on specified server event IDs.

In the Local Manager GUI, create a new Event Handler of type Send as HTTP Post request or add the url in the configuration file for the event-handlers UUID:

[event-handlers/6d32ee50-b277-49e5-b2f4-c70eeea289a7]
url = http://www.acme.io/http-post-hook-url

For an example configuration that targets the server event ID 40007 and specifying that the JSON format be used to send the event in the HTTP body, see below:

[event-handlers/6d51ed1e-35ec-41d7-8b51-53e56c716212]
enabled = True
type = http
name = Example file transfer hook for ACME
description = Send an HTTP POST hook when the event is raised
target = 40007
url = http://www.acme.io/http-post-hook-url
http_content_type = json

12.3.2. Request body ¶

When an HTTP or HTTPS handler is used, SFTPPlus will initiate an HTTP POST client request to the configured URL with a body containing one or more events, together with the identity of the server making the request.

The request body can be formatted as JSON, XML, or using a custom template.

Warning

The server member found in the root container is deprecated and will be removed in SFTPPlus v5. You should use the server attribute of each event.

name: uuid
type: string
optional: No
description: UUID of the server emitting this event.

Each event from the events array contains the following attributes.

The overall structure of the event object is presented below.

The following variables (case-insensitive) are provided as context data containing information about the event being triggered:

id
uuid
message
account.name
account.email
account.peer.address
account.peer.port
account.peer.protocol
account.peer.family
account.uuid
component.name
component.type
component.uuid
timestamp.cwa_14051
timestamp.iso_8601
timestamp.iso_8601_fractional
timestamp.iso_8601_local
timestamp.iso_8601_basic
timestamp.iso_8601_compact
timestamp.timestamp
server.name
server.uuid
data.DATA_MEMBER_NAME
data_json

The members of data are specific to each event. See Events page for more details regarding the data available for each event.

Many events have data.path and data.real_path, together with the associated data.file_name, data.directory_name, data.real_file_name, and data.real_directory_name.

Below is the description for the main members of the event object.

name: id
type: string
optional: No
description: ID of this event. See Events page for the list of all available events.

name: message
type: string
optional: No
description: A human readable description of this event.

The timestmap contains the following attributes:

name: timestamp
type: string
optional: No
description: Date and time at which this event was created, as Unix timestamp with milliseconds.

name: cwa_14051
type: string
optional: No
description: Date and time in CWA 14051 at which this event was emitted.

The account contains the following attributes:

name: uuid
type: string
optional: No
description: UUID of the account emitting this event. In case no account is associated with the event, this will be the special process account. In case the associated account is not yet authenticated this will be the special peer account.

name: name
type: string
optional: No
description: Name of the account emitting this event.

name: token
type: string
optional: Yes
description: For Windows local or domain accounts a token that can be use to impersonate the account. For Azure AD accounts, when extra api_scopes are configured, this is the latest OAuth2 token that can be use to obtain access to an extra API or refresh a token.

name: peer
type: JSON Object
optional: No
description: Address of the peer attached to this account. This might be a local or remote address, depending on whether the account is used for client side or server side interaction.

The peer contains the following attributes:

name: address
type: string
optional: No
description: IP address of this connection.

name: port
type: integer
optional: No
description: Port number of this connection.

name: protocol
type: string
optional: No
description: OSI Layer 4 transport layer protocol used for this connection in the form of either TCP or UDP.

The component contains the following attributes:

name: uuid
type: string
optional: No
description: UUID of the component (service or transfer) emitting this event.

name: type
type: string
optional: No
description: Type of the component emitting this event.

The server contains the following attributes:

name: uuid
type: string
optional: No
description: UUID of the server emitting this event.

name: type
type: string
optional: No
description: Type of the server emitting this event.

Below is an example for a POST request containing two JSON events:

POST /remote/url
Content-Type: application/json

{
  "events": [
    {
      "id": "10025",
      "timestamp": {
        "timestamp":  "1510742418.2341",
        "cwa_14051": "2017-11-15 10:40:18"
        },
      "data": {
        "path": "/path/of/file/as/seen/by/client",
        "details": "Some details about failure."
        },
      "account": {
        "uuid": "0c12a7f9-484a-45de-b622-8a5d96061328",
        "name": "mike",
        "peer": {
          "address": "12.442.23.34",
          "port": 2345,
          "family": "IPv4",
          "protocol": "TCP"
          }
        },
      "component": {
        "uuid": "dff314a6-c594-48dc-8e34-5270fd6cb635",
        "type": "ssh",
        "name": "SFTP-Internal"
        }

      },
    {
      "id": "20040",
      "timestamp": {
        "timestamp":  "1510742419.1245",
        "cwa_14051": "2017-11-15 10:40:19"
        },
      "data": {
        "subject": "event specific data."
        },
      "account": {
        "uuid": "0c12a7f9-484a-45de-b622-8a5d96061328",
        "name": "mike",
        "peer": {
          "address": "2006:4820:4060::8844",
          "port": 2355,
          "family": "IPv6",
          "protocol": "TCP"
          }
        },
      "component": {
        "uuid": "dff314a6-c594-48dc-8e34-5270fd6cb635",
        "type": "ssh",
        "name": "SFTP-Internal"
        }
      },
      "server": {
        "uuid": "cc5c804d-0a3c-4c4c-b651-eba6fc3b5902",
        "name": "AZWMPreProd01"
        }
    ]
}

Below is an example for a POST request containing one XML SOAP event:

POST /remote/url
Content-Type: application/soap+xml; charset=utf-8

<?xml version="1.0" ?>
<soap:Envelope
  soap:encodingStyle="http://www.w3.org/2003/05/soap-encoding"
  xmlns:soap="http://www.w3.org/2003/05/soap-envelope/">
  <soap:Body xmlns:m="http://www.sftpplus.com/event">
    <m:Events>
      <m:Event>
        <account>
          <peer>
            <protocol>TCP</protocol>
            <port>55216</port>
            <family>IPv6</family>
            <address>::ffff:127.0.0.1</address>
          </peer>
          <uuid>user_uuid</uuid>
          <name>user</name>
        </account>
        <created>1591402889.06</created>
        <timestamp>
          <cwa_14051>2020-06-06 01:21:29</cwa_14051>
          <timestamp>1591402889.06</timestamp>
          <iso_8601>2020-06-06T00:21:29Z</iso_8601>
          <iso_8601_fractional>
            2020-06-06T00:21:29.056312084198Z</iso_8601_fractional>
          <iso_8601_local>2020-06-06T01:21:29+01:00</iso_8601_local>
        </timestamp>
        <component>
          <type>http</type>
          <name>http</name>
          <uuid>http-1</uuid>
        </component>
        <server>
          <uuid>test-server-uuid</uuid>
          <name>test-server-name</name>
        </server>
        <message>
          Successfully uploaded file at &quot;/path/file.txt&quot;.
          Wrote 168691 bytes at 48227205.7845 bytes/second
          in 0.003497838974 seconds.
        </message>
        <data>
          <total_write>168691</total_write>
          <real_path>
            /srv/ftp/path/file.txt</real_path>
          <write_speed>48227205.7845</write_speed>
          <avatar>
            <peer>
              <protocol>TCP</protocol>
              <port>55216</port>
              <family>IPv6</family>
              <address>::ffff:127.0.0.1</address>
            </peer>
            <uuid>user_uuid</uuid>
            <name>user</name>
          </avatar>
          <duration>0.003497838974</duration>
          <path>/path/file.txt</path>
        </data>
        <id>40017</id>
      </m:Event>
    </m:Events>
  </soap:Body>
</soap:Envelope>

12.3.3. Response codes ¶

Once the server has successfully received and processed the events, it should respond with HTTP code 200:

Status: 200 OK

HTTP code 204 is also a valid response code:

Status: 204 No Content

When the remote HTTP server is busy and can't process more requests, the response should display HTTP code 503 together with a Retry-After header with a value expressed in seconds. The current events are discarded.:

Status: 503 Service Unavailable
Retry-After: 3600

For any response code, other than 200 and 204, SFTPPlus will consider that the requests failed to be successfully processed by the remote HTTP endpoint.

Failed requests are not retried and the event handler will stop sending events after the configured number of consecutive failures.

An event is emitted for every event which failed to be processed by the remote HTTP server. For example, when the event ID 40007 is triggered (opening an existing folder) and its handling failed, the server-side SFTPPlus log will specify the server return code. In this case, 203 is the specified server return code:

20174 2017-05-12 14:12:12 other-event-http-uuid Process 0.0.0.0:0 Failed to handle event 40007 by "file-transfer-hooks". Server returned 203:Non-Authoritative Information

12.3.4. Response body ¶

For remote servers which are sending the error inside an HTTP response with code 200, you can use the expected_response configuration to define a matching expression to detect a successful response. In the following example, SFTPPlus will treat the HTTP response as an error if it doesn't contain the <Status>OK</Status> value anywhere in the body:

[event-handlers/69fb02f4-a896-11ea-b3ee-134d4ef1f480]
enabled: yes
type: http
name: file-transfer-hooks
url: https://web-hooks.example.com/received-files.api

expected_response=*<Status>OK</Status>*

You can use the negated regular expression to consider a success any response which doesn't contain the <Status>Error</Status> value, like in the following example:

[event-handlers/69fb02f4-a896-11ea-b3ee-134d4ef1f480]
enabled: yes
type: http
name: file-transfer-hooks
url: https://web-hooks.example.com/received-files.api

expected_response=e/.*<Status>Error</Status>.*/i

When using regular expressions to match a value that spans multiple lines, you will need to explicitly include the newline characters. This is not required for globbing expression, as the newline characters are included in the * (match anything) glob. Below is an example of defining multi lines expression:

[event-handlers/69fb02f4-a896-11ea-b3ee-134d4ef1f480]
enabled: yes
type: http
name: file-transfer-hooks
url: https://web-hooks.example.com/received-files.api

expected_response=e/.*<Status>.*\n.*Error.*\n.*</Status>.*/i

The above example which will consider the response a failure if the body contains either of the values:

<Status>Error</Status>

or

<Status>
  Error
</Status>

12.3.5. Redundant HTTP URL / Endpoints ¶

An HTTP POST event handler can be configured with more than one URL to provide redundant event handling.

If handling of an event failed for an URL, SFTPPlus will re-try to handle the event using the next URL from the list.

If handling of an event succeeded, SFTPPlus will not send the request to the following URLs from the list.

Below is an example of an event handler configured with multiple URLs:

[event-handlers/6d32ee50-b277-49e5-b2f4-c70eeea289a7]
url = http://www.acme.io/events, https://fallback.acme.io/events

SFTPPlus will send the HTTP requests to URL http://www.acme.io/events. URL https://fallback.acme.io/events is used only when the request failed to be handled by the HTTP endpoint at URL http://www.acme.io/events

When the request fails for an URL, the usage of that URL will be suspended and resumed after 5 minutes.

If all URLs are not available, SFTPPlus will emit an error informing that processing the event via HTTP POST failed, not retrying the delivery.

12.3.6. Fault-tolerant / Resilient HTTP requests ¶

It is possible for a request to the HTTP endpoint to return a failure response. SFTPPlus can be configure to retry the same URL on failure.

For example, the following configuration will make a first request to http://www.acme.io/events and if it fails, it will wait 0.5 second and then try again. If the request still fails, it will retry for 1 seconds before retrying for a second try. If the second request will fail, the request is considered as a permanent failure and the request will not be sent again to the URL.

Below is the configuration for the retry scenario described above:

[event-handlers/6d32ee50-b277-49e5-b2f4-c70eeea289a7]
url = http://www.acme.io/events
retry_count = 2
retry_increase = 0.5

It is possible to comply the URL retry configuration with a fallback URL. In this case, it will retry the first url twice and then will attempt the same request on the next URL:

[event-handlers/6d32ee50-b277-49e5-b2f4-c70eeea289a7]
url = http://www.acme.io/events, https://fallback.acme.io/events
retry_count = 2
retry_increase = 0.5

12.3.7. Sending custom JSON data in HTTP POST request ¶

Each payload sent by the HTTP POST event handler contains the following members, as documented in previous sections:

{
  "events": [ EVENT_DATA ],
  "server": { SERVER_DATA }
}

You can configure the event handler to add custom values to the payload. The extra values are configured in JSON format and can be nested structures.

The keys and values can contain variables which are replaced with values based on the event's data.

For example to send the event as a Slack Incoming WebHook message, you can use this configuration:

[event-handlers/b904ed23-v254-4ccf-8abd-edcae4d3324f]
url = https://hooks.slack.com/services/n2unjSpQQ4L6JIOrHoO9CKXl
extra_data = {
    "text": "{message}",
    "attachments": [
      {
      "pretext": "New event {id} SFTPPlus *MFT-023-QA*",
      "author_name": "{account.name}",
      "author_icon": "https://www.sftpplus.com/static/images/logo-80.png",
      "text": "{message}",
      "footer": "Sent by SFTPPlus HTTP Post Event",
      "mrkdwn_in": ["pretext"],
      "fields": [
        {"title": "IP", "value": "{account.peer.address}"},
        {"title": "Port", "value": "{account.peer.port}"},
        {"title": "Server ID", "value": 132},
        ]
      }
    ]}

Note the usage of {id} or {message} variables inside the key names or string values. In this case, they payload sent by the HTTP POST event handler contains the following members:

{
  "events": [ EVENT_DATA_AS_BEFORE ],
  "server": { SERVER_DATA_AS_BEFORE },
  "text": "Stopped authentication "AD DAP" of type ldap. Failed at start",
  "attachments": [
    {
    "pretext": "New event 20157 SFTPPlus *MFT-023-QA*",
    "author_name": "Process",
    "author_icon": "https://www.sftpplus.com/static/images/logo-80.png",
    "text": "Stopped authentication "AD DAP" of type ldap.",
    "footer": "Sent by SFTPPlus HTTP Post Event",
    "mrkdwn_in": ["pretext"],
    "fields": [
      {"title": "IP", "value": "182.12.31.21", "short": true},
      {"title": "Port", "value": "346", "short": true},
      {"title": "Server ID", "value": 132, "short: true"},
      ]
    }
}

On Slack, the message will look like

12.2. HTTP API User Authentication and Configuration

12.4. Python API Event Handler

Documentation

12.3. HTTP POST Event Handler¶

12.3.1. Introduction¶

12.3.2. Request body¶

12.3.3. Response codes¶

12.3.4. Response body¶

12.3.5. Redundant HTTP URL / Endpoints¶

12.3.6. Fault-tolerant / Resilient HTTP requests¶

12.3.7. Sending custom JSON data in HTTP POST request¶

12.3.1. Introduction ¶

12.3.2. Request body ¶

12.3.3. Response codes ¶

12.3.4. Response body ¶

12.3.5. Redundant HTTP URL / Endpoints ¶

12.3.6. Fault-tolerant / Resilient HTTP requests ¶

12.3.7. Sending custom JSON data in HTTP POST request ¶