Articles from resource category

Understanding the exchange between SFTP Client and SFTP Server

Thu 08 March 2018 | article Written by Hannah Suarez

Why read this?

As part of meeting the Accounting component of the AAA (Authorization, Authentication and Accounting) framework, each event and action on the server and/or the client-side are recorded by SFTPPlus. These events have an associated Event ID which is also publicly searchable both on our website and on the internal documentation included in the software package that you have downloaded.

System and network administrators touching on logs - be it in the most verbose format or not - may find this article describing the breakdown of such logs helpful.

For this example, we will be touching on SFTPPlus SFTP transfers from both the client-side and server-side only. Please do not hesitate to get in touch with us if you are interested in learning more about other file transfer protocols.

SFTPPlus SFTP Server-side Perspective

Initial configuration notes

If you are currently evaluating SFTPPlus, please follow our documentation to learn more about how you can configure your database and event handlers to suit your specifications.

Read more about configuring databases with SFTPPlus.

Read more about configuring event handlers. These provide further ways to configure SFTPPlus to create logging actions based on the events recorded.

Even if you are an existing customer, you can follow our documentation links above in order to refresh your knowledge on configuring SFTPPlus version 3. For those on legacy versions, please consult the documentation relevant to your version.

Example logs from SFTPPlus

The following are snippets when logging in for the first time from a GUI client to an SFTPPlus 3.30.0 SFTP server.

A new connection has been made to the service sftp-1. Knowing the service name is useful in case there are multiple other SFTP services running:

| 30014 2018-02-27 17:28:53 sftp-1 Unknown 127.0.0.1:58032
  New SSH connection made.
| 2018-02-27 17:28:53 30014 New SSH connection made.

The following are authentication methods associated with the server and confirmation of which methods are not active. There may be more methods, depending on how many of these are set up and enabled. To simplify the login process, please make sure to disable all unused authentication methods.:

| 20138 2018-02-27 17:28:55 some-http-auth-uuid Unknown 127.0.0.1:58032
  Ignoring http authentication "auth-over-remote-http" for "user" since it
  is not active.
| 2018-02-27 17:28:55 20138 Ignoring http authentication "auth-over-remote-http"
  for "user" since it is not active.
| 20138 2018-02-27 17:28:55 ldap-uuid Unknown 127.0.0.1:58032 Ignoring
  ldap authentication "LDAP against local test server" for "user" since it
  is not active.

The following logs list out a successful authentication of user using the ssh-key:

| 20137 2018-02-27 17:28:55 test-server-uuid Unknown 127.0.0.1:58032
  Account "user" of type "application" authenticated as "user" by
  application authentication "Application Accounts" using ssh-key.
| 2018-02-27 17:28:55 20137 Account "user" of type "application"
  authenticated as "user" by application authentication "Application
  Accounts" using ssh-key.

The following log message confirms the type of permissions allowed for the account and an active transfer that is already running:

| 20182 2018-02-27 17:28:55 Process user 127.0.0.1:58032 Account "user"
  logged in with permissions [[u'allow-full-control'], [u'/main_folder/*', u'allow-full-control'],
  [u'*.PDF', u'allow-read']]. Files uploaded as: test.txt

The following confirms that the user has logged into and now has access to the folder as the root ("/") folder:

| 30011 2018-02-27 17:28:55 Process user 127.0.0.1:58032 Subsystem SFTP
  successfully started in "/root/home/node/user/" as "/".
| 2018-02-27 17:28:55 30011 Subsystem SFTP successfully started in
  "/root/home/node/user/" as "/".
| 30060 2018-02-27 17:28:55 Process user 127.0.0.1:58032 Canonical file
  name requested for ".".
| 2018-02-27 17:28:55 30060 Canonical file name requested for ".".
| 30060 2018-02-27 17:28:55 Process user 127.0.0.1:58032 Canonical file
  name requested for "/.".
| 2018-02-27 17:28:55 30060 Canonical file name requested for "/.".
| 30019 2018-02-27 17:28:55 Process user 127.0.0.1:58032 Listing folder "/".
| 2018-02-27 17:28:55 30019 Listing folder "/".
| 30020 2018-02-27 17:28:55 Process user 127.0.0.1:58032 Successfully
  listed folder "/".
| 2018-02-27 17:28:55 30020 Successfully listed folder "/".

SFTPPlus SFTP Client-side Perspective

Initial configuration notes

If you are currently evaluating SFTPPlus, please follow our client side documentation.

The SFTPPlus Client software utilizes the command-line client-shell to access remote file servers using the interactive shell.

Even if you are an existing customer, you can follow our documentation links above in order to refresh your knowledge on configuring SFTPPlus version 3. For those on legacy versions, please consult the documentation relevant to your version.

Example logs from SFTPPlus

Let's connect with SFTPPlus Client using the SFTP protocol on port 10022. The following log details the UUID of the sftp service and confirms the connections:

| $ ./bin/client-shell.sh sftp://user@localhost:10022 -p pass
  --ssh-server-fingerprint 06:cb:46:2b:9a:9a:c4:10:54:f0:ea:2f:b6:05:cb:a0
| SFTPPlus (3.31.0) file transfer client shell
| > connect
| 20140 2018-03-05 16:40:59 51e1db00-8214-4b68-96fe-58470b8b2fc5 Process
  0.0.0.0:0 Connecting resource "sftp".
| 30072 2018-03-05 16:40:59 Process user localhost:10022 Location sftp
  connected to the SSH server.
| 30076 2018-03-05 16:40:59 Process user localhost:10022 Client SFTP
  subsystem initialized for location sftp.
| 20141 2018-03-05 16:40:59 51e1db00-8214-4b68-96fe-58470b8b2fc5 Process
  0.0.0.0:0 Resource "sftp" successfully connected.
| 20156 2018-03-05 16:40:59 51e1db00-8214-4b68-96fe-58470b8b2fc5 Process
  0.0.0.0:0 Successfully started location "sftp" of type sftp.

On the event that the SFTP connections fails, the log will state a number of details. The event ID is 30073. The event will communicat the host key algorithm that is in use to identify the server-side, the cipher used to receive data, the HMAC for both sent and received data, key exchange algorithm, cipher used for sent data and the name of the location associated for this event. Below is an example of the event that has been emitted has part of this new SFTP connection.:

| 30073 2018-03-05 16:36:16 Process user localhost:10022 Connection to
  SSH server was lost for location sftp. Protected using host-key:ssh-rsa key-exchange:
  diffie-hellman-group-exchange-sha256 in-hmac:hmac-sha2-256
  in-cipher:aes256-ctr out-hmac:hmac-sha2-256 out-cipher:aes256-ctr

Providing that the SFTP connection succeeds, supported actions are logged as either a success like below:

| > gattrs remote_get
| 60071 2018-03-05 16:41:22 Process Process 0.0.0.0:0 Successfully got
  attributes for "Reports_2018" on "sftp".
| name: Reports_2018
| path: Reports_2018
| size: 128
| modified: 2018-02-16 16:15:21
| is_file: False
| is_folder: True

Or error details are caught with an explanation message as to why:

| > get unknown_file
| 20145 2018-03-05 16:42:08 Process Process 0.0.0.0:0 Failed to resolve
  text for event id "60054" with data "{'path': 'unknown_file\xc8\x9bu',
  'location': u'sftp', 'avatar':
  <chevah.server.identity.avatar.ProcessAvatar object at 0x10efc3110>,
  'details': "'ascii' codec can't decode byte 0xc8 in position 9: ordinal
  not in range(128)"}". 'ascii' codec can't decode byte 0xc8 in position
  9: ordinal not in range(128)

SFTPPlus SFTP Exchange - Detailed Verbose OpenSSH Logs

Initial configuration notes

Following from that, you can use the built-in the client-side or server-side software that you are utilizing. SFTPPlus offers logging functionalities both for the client-side and server-side. Network administrators using other software, such as sftp -vvv, for client or server may wish to use additional logging functionalities.

Example with sftp -vvv output

These lines mean that SSH protocol 2.0 is being utilized with the version of OpenSSH:

debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.6

This line indicates which protocol version is in use service-side and which version:

debug1: Remote protocol version 2.0, remote software version SFTPPlus_3.30.0

This indicates which algorithms are preferred. You may opt to only select the strongest availability supported in your system first. In this case the ordering is logical as it moves from the more secure algorithm down to a less secure algorithm.:

| debug3: order_hostkeyalgs: prefer hostkeyalgs:
  ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa

These are the key exchange algorithms that are available.:

| debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,
  ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,
  diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,
  diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,
  diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c

These are the host key algorithms.:

| debug2: host key algorithms: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,
  rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,
  ecdsa-sha2-nistp384-cert-v01@openssh.com,
  ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,
  ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519

These are the ciphers used from client to server (ctos) and from server to client (stoc):

| debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,
  aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com

| debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,
  aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com

These are the ciphers used from client to server (ctos) and from server to client (stoc):

| debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,
  hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,
  hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,
  hmac-sha2-256,hmac-sha2-512,hmac-sha1

| debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,
  hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,
  hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,
  hmac-sha2-256,hmac-sha2-512,hmac-sha1

These are the compression algorithms used from client to server (ctos) and from server to client (stoc):

debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib

This is the key exchange initialized proposal from the host server:

| debug2: peer server KEXINIT proposal
| debug2: KEX algorithms: diffie-hellman-group-exchange-sha256,
  diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1,
  diffie-hellman-group14-sha1
| debug2: host key algorithms: ssh-dss,ssh-rsa
| debug2: ciphers ctos:
  aes256-ctr,aes256-cbc,aes192-ctr,aes192-cbc,aes128-ctr,aes128-cbc,
  3des-ctr,3des-cbc
| debug2: ciphers stoc:
  aes256-ctr,aes256-cbc,aes192-ctr,aes192-cbc,aes128-ctr,aes128-cbc,
  3des-ctr,3des-cbc
| debug2: MACs ctos: hmac-sha2-256,hmac-sha1
| debug2: MACs stoc: hmac-sha2-256,hmac-sha1
| debug2: compression ctos: none,zlib
| debug2: compression stoc: none,zlib

These are the key exchange algorithms used from server to client and client to server:

| debug1: kex: algorithm: diffie-hellman-group-exchange-sha256
| debug1: kex: host key algorithm: ssh-rsa
| debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256
  compression: none
| debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256
  compression: none

This is the SSH version 2 key exchange Diffie-Hellman Group Exchange request. This specifies the size of the SSH prime moduli being calculated by the SFTP server as indicated in the SFTPPlus /configuration/ file. When you first initialize SFTPPlus version 3, the Time Type Tests Tries Size Generator Modulus is generated and saved in ssh-service.moduli. This file contains primes ranging in size from 1023 to 8191 bits. An example of the contents for the .moduli file is below:

| 20060827134212 2 6 100 3071 2
  D3230D237572ECE9F92358715EBAC3A4D89F2D6B4DC39F056450263BEF1665FBD
  7B93916ABC867B7064802159D273C7EB01C5F9281A3D6DCCB7CF997D385998EC0E1FA3319AFE771A90ADBACEB414A02
  0630D7C7F161FAFEC6C9FC06D3205C712AAE8848A1B2C21DFF301C7FFC0B75D13F060A313C32AFEEAF1493F641760EB
  EF38829B3371699D2A3264D0ECEB4E5C19581ED8C57699F559B9828BBFE147952E289F0E171C9C60335DD2F492CB409
  A4DB97BDF86E2DBA605064DB040A3DF5678E24F66718CA115C95C892FF7AEDFAABC2E6414716298CEC1A604270FEADF
  191B7C8A59C238C395A65442C0B963BF83025BED3951A271B7440EC7687C31DE63355DA7FEAC15DC962C7BF7614EB59
  B077B9889AD8703DFE98AC99615B722A0ABE89956D1058E025C7733420CB51D7E1608EFF2C0A30C9A5EB77CCA02C6B0
  0CE781B172001C6C458630890062E27CE307D513A7686A69D1D548DE8334B13136D9E842A5E17FD67522C93823E03F0
  8AEE8024AF5D88B2EE01D4D9980084EFD5D943

In the following example below, a SSH moduli prime from 2048 to 8192 bits are used. Specifically, a moduli with a range from 4092 to 8192 are sent for the SSH message key exchange Diffie-Hellman group exchange request as indicated on debug1 line below (SSH2_MSG_KEX_DH_GEX_REQUEST(2048<8192<8192)) Once sent, the server uses the moduli file, the same file that was initialized as part of the SFTPPlus installation steps, in order to crack the shared secret. The server provides its host key back to the client along with the algorithm used as indicated by the final line as Server host key: ssh-rsa SHA256:hdSfa7gb2O984malHerkwerj3m20dHb6Yuwl0&hbxFj.

See the rest of the output below:

| debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(2048<8192<8192) sent
| debug3: receive packet: type 31
| debug1: got SSH2_MSG_KEX_DH_GEX_GROUP
| debug2: bits set: 4092/8192
| debug3: send packet: type 32
| debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
| debug3: receive packet: type 33
| debug1: got SSH2_MSG_KEX_DH_GEX_REPLY
| debug1: Server host key: ssh-rsa
  SHA256:hfSfa0gb2O884malLerkwerj3m20dBb6Yuwl0&hbxGj

The client then checks to see if the host key is located within the known_hosts file:

| debug3: hostkeys_foreach: reading file "/root/home/node/.ssh/known_hosts"
| debug3: record_hostkey: found key type RSA in file
  /root/home/node/.ssh/known_hosts:8
| debug3: load_hostkeys: loaded 1 keys from [12.345.678.90]:10022

A few more steps occur to verify this server host name and port:

ddebug1: Host '12.345.678.90]:10022' is known and matches the RSA host key.
ddebug1: Found key in /root/home/node/.ssh/known_hosts:8

This is the server rekey interval:

debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 4294967296 blocks

The following are SSH keys found:

debug2: key: imported-openssh-key (0x7e403ff95550), agent
debug2: key: /root/home/node/.ssh/id_rsa (0x0)
debug2: key: /root/home/node/.ssh/id_dsa (0x0)
debug2: key: /root/home/node/.ssh/id_ecdsa (0x0)
debug2: key: /root/home/node/.ssh/id_ed25519 (0x0)

The following are authentication methods that can continue, the preferred authentication order, remaining preferred:

| debug3: send packet: type 5
| debug3: receive packet: type 6
| debug2: service_accept: ssh-userauth
| debug1: SSH2_MSG_SERVICE_ACCEPT received
| debug3: send packet: type 50
| debug3: receive packet: type 51
| debug1: Authentications that can continue: password,publickey
| debug3: start over, passed a different list password,publickey
| debug3: preferred publickey,keyboard-interactive,password
| debug3: authmethod_lookup publickey
| debug3: remaining preferred: keyboard-interactive,password
| debug3: authmethod_is_enabled publickey
| debug1: Next authentication method: publickey
| debug1: Offering public key: RSA
  SHA256:F8zPRFytcYU8PERggkPDs+D32TRgvVm4H3BBJduo+de
  /root/home/node/.ssh/id_rsa
| debug3: send_pubkey_test
| debug3: send packet: type 50
| debug2: we sent a publickey packet, wait for reply

The server will go through the exchange to authenticate until the final preferred method is reached - the password method. Upon success, the client enters an interactive session with the server.

There will also be additional verbose logs after entering an interactive session, such as a brief snippet below:

debug2: fd 6 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x08
debug2: client_session2_setup: id 0
debug1: Sending environment.
debug3: Ignored env _system_type
debug1: Sending env LANG = en_CA.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env _system_arch
debug3: Ignored env XPC_FLAGS
debug3: Ignored env _system_version
debug3: Ignored env XPC_SERVICE_NAME
debug3: Ignored env rvm_version
debug3: Ignored env _system_name
debug1: Sending subsystem: sftp

Evaluating SFTPPlus MFT

This article was written as of SFTPPlus version 3.31.0.

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

Install SFTPPlus MFT today either as an on-premise solution supported on Windows, Linux, AIX, OS X, Solaris, HP-UX, FreeBSD or on the cloud as Docker containers or AWS instances.

Email us at sales@proatria.com or fill in the form below to start your evaluation version today.

• • •

SFTPPlus is not affected by the Meltdown and Spectre Vulnerabilities

Wed 21 February 2018 | article security Written by Adi Roiban

Meltdown and Spectre are vulnerabilities based on CPU design flaws which require the attacker to be able to execute application code which is created to exploit these vulnerabilities.

SFTPPlus secure file transfers does not allow any arbitrary application code execution. It will only read and write data without executing it. This is standard behaviour for doing file transfers over FTPS or HTTPS.

The SSH implementation of SFTPPlus is only allowed for the SFTP and SCP protocols. Shell access or any other SSH execution is denied. The SCP protocol is implemented using an embedded SCP protocol and no external scp application is called.

For the purpose of managed file transfers, SFTPPlus allows the execution of pre-configured application code with the pre and post transfer hooks. As long as the SFTPPlus is configured with trusted applications, this does not constitute an attack vector.

If you are running SFTPPlus Itanium architectures, for example with HPUX, you are not affected by these vulnerabilities, no mater what other software is in used on those systems.

SPARC architecture (example with Solaris 10) and POWER (example with AIX 7.1) are affected by the Spectre, while not being affected by Meltdown.

The embedded devices based on ARM64 CPUs are also affected by Spectre.

Administrators using the SFTPPlus MFT Client with pre and post transfer hooks should review the configuration and make sure that the hooks will trigger calls to trusted applications.

This article was written as of SFTPPlus version 3.31.0.

• • •

Introducing SFTPPlus to high availability and resiliency

Tue 23 January 2018 | article infrastructure Written by Hannah Suarez

Where does SFTPPlus sit in your IT infrastructure

The SFTPPlus software stands at the OSI Layer 7 or the TCP Layer 4. In order to have a fully fault tolerant system, you need to implement resilience at all the other layers including the OS. SFTPPlus can be integrated with external tools in order to meet the requirements for a fault tolerant infrastructure.

For those not familiar with OSI and TCP please read on.

SFTPPlus on the OSI

The OSI model is a model that characterizes and standardizes communication functions. The layers range from layer 1 right through to layer 7. In the OSI, or Open Systems Interconnection model, SFTPPlus sits in the OSI Layer 7 or on the application layer.

The application layer sits at the top of the OSI model and is the software, hence the name application, layer between the end-user and the networking layers underneath.

In order to have a fault tolerant system, SFTPPlus on the upper layer 7 will need to be integrated with the bottom layers.

SFTPPlus on the TCP

In addition to the OSI model, another way of understanding where SFTPPlus plays a role in your infrastructure is via the TCP layer. SFTPPlus sits in the TCP Layer 4 or the application layer. This is the topmost layer which defines the TCP/IP application protocols and how SFTPPlus interfaces with the Transport layer, the layer below the application layer, and other services that use the network.

Installing SFTPPlus in high availability and resilient environments

The following are introductory information for this topic.

About high availability

High availability means creating a system that is always available for use. It could be a percentage of 99.99% uptime guaranteed. In this case, you will be looking at a downtime of merely five minutes of time over the course of the year.

There are extra items that one can add to ensure that this system is available at the guaranteed uptime rate. In this case, one can look into active-active or active-passive scenarios. To build a system that is highly available means that there may be an additional cost associated with ensuring this.

About resilience

The following can be deduced as a definition of a resilient control system:

"A resilient control system is one that maintains state awareness and an accepted level of operational normalcy in response to disturbances, including threats of an unexpected and malicious nature"

High availability and resilience tend to be used interchangeably. However, having a highly available system does not necessarily mean that all required functions are still in use and available. This is where having a resilient system come into action. Even if a system has high availability, can it still function to a required level of standard, operational normalcy? You will still wish to utilize a system with the same users, storage and database as found in the usual system.

About fault tolerance

On the event of failure, the system remains available in order to maintain the high uptime. There may be a performance break or slow down but the services are still available.

You may add additional devices or protocols for a fault tolerant system - RAID set up, multiple network paths for fault tolerance (on the event of a failed network path) and load balancers are such examples.

About clustering

Clustering involves creating a cluster of two or more nodes or members that work together in order to perform an action. They can be grouped in the following major types; storage, high availability, load balancing and high performance clusters.

The main clusters that relates to SFTPPlus in a given system are high availability and load balancing types of clusters.

High availability clusters involve the provision of highly available services by ensuring that any single points of failure are eliminated. This is done by failing over services from one cluster node to another should that node be no longer in operation. This ensures the ability to maintain data integrity.

Load balancing clusters sends off network requests to a number of cluster nodes in order to balance the request load among the cluster nodes. This ensures scalability of a network since administrators can match the number of nodes according to load requirements through load balancing algorithms.

How can SFTPPlus be integrated in these environments

Diagram example: Integration for load balancing

Integration for load balancing

Diagram example: Integration for high availability

Integration for high availability

Active-Active and Active-Passive Scenarios

Active-active and Active-passive are two types of cluster configurations in a high availability scenario.

The details between these two scenarios are laid out below from Sybase.

Active-Passive configurations

Setup: A single Adaptive Server runs either on the primary node or on the secondary node. The Adaptive Server runs on the primary node before a fail over and the secondary node after fail over.

Failover: When a system fails over, the Adaptive Server and its associated resources are relocated to, and restarted on, the secondary node.

Failback: Failback is a planned fail over or relocation of the Adaptive Server and its resources to the primary node. Failback is not required, but can be done for administrative purposes.

Client Connection failover: During failover and failback, clients connect to the same Adaptive Server to resubmit uncommitted transactions. Clients with the failover property reestablish their connections automatically.

How to set up SFTPPlus in active-passive scenarios

In this infrastructure scenario, the second system is offline and only commences when the main SFTPPlus system is down.

Since the server.ini configuration is stored in a single file, you can create a file copy task to keep the system configurations in sync. Make sure to also transfer additional files that are required - such as SSH keys, and SSL keys and certificates - to ensure a smooth transition. When it is time to use the secondary system, the SFTPPlus instance will then read the latest server.ini configuration file.

Active-Active configurations

Setup: Two Adaptive Servers are configured as companion servers, each with independent workloads. These companions run on the primary and secondary nodes, respectively, as individual servers until one fails over.

Failover: When fail over occurs, the secondary companion takes over the devices, client connections, and so on from the primary companion. The secondary companion services the failed-over clients, as well as any new clients, until the primary companion fails back and resumes its activities.

Failback: Failback is a planned event during which the primary companion takes back its devices and client connections from the secondary companion to resume its services.

Client Connection failover: During failover, clients connect to the secondary companion to resubmit their uncommitted transactions. During failback, clients connect to the primary companion to resubmit their transactions. Clients with the failover property reestablish their connections automatically.

How to set up SFTPPlus in active-active scenarios

In this infrastructure scenario, both SFTPPlus systems are receiving and processing requests. If one system goes down, the other will handle all the requests.

To implement SFTPPlus in this scenario, a simple file copy will not work. This is because running SFTPPlus instances will not check changes in the local file configuration (server.ini) in order to reconfigure. In addition, there are other files that are also required - such as all SSH keys in use and other related files, all SSL certificates required, any logs that need to be kept for auditing purposes, any externally referenced scripts used in pre- and post- transfer processing, and so on.

One method of achieving an active/active implementation is to manually set up the 2 nodes to rely on a single external authentication method (HTTP or LDAP). In this way, accounts are managed in the single external system, and those accounts will be automatically available for both SFTPPlus instances.

Installing SFTPPlus for disaster recovery

Disaster recovery is part of business continuity plans (or business continuity and resiliency plans) which is the process of creating systems of prevention and recovery to deal with potential threats to a company. The use of the term “recovery” has also been used when talking about resiliency.

Providing that the server configuration and related configuration files are properly maintained and backed-up, you can integrate SFTPPlus as part of your disaster recovery plans.

Conclusion and next steps

The application of these does not immediately guarantee results in achieving high availability or resiliency. Please consider these guides merely as a layer within multiple others when implementing a high available, resilient and secure managed file transfer solution.

Since features are constantly changed, we did not touch on any specifics within SFTPPlus. Please consult our documentation for the configuration and operations information, as well as practical users guides.

Evaluating SFTPPlus MFT

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

Install SFTPPlus MFT today either as an on-premise solution supported on Windows, Linux, AIX, OS X, Solaris, FreeBSD, HP-UX or on the cloud as Docker containers or AWS instances.

Email us at sales@proatria.com to start your evaluation version today.

For licensing queries please contact sales@proatria.com.

Addendum

This resource is written as of SFTPPlus version 3.29.0.

• • •

Choosing the best protocols for securing data and file transfers

Mon 22 January 2018 | article security Written by Hannah Suarez

Why read this guide

In order to implement a secure managed file transfer system, you will need a good understanding of the supported services and protocols involved.

This article provides an overview of the supported protocols, including the advantages and disadvantages of these protocols as well as situations for the use of these services.

The first part focuses on protocols that we recommend you reconsider in using and the rest of the article is followed by services that we do recommend.

Protocols to reconsider when securing data and file transfers

The following, FTP and HTTP, are covered below as they both pose two services that offer the least advantage in terms of securing data and file transfers.

File Transfer Protocol (FTP)

Shortform for File Transfer Protocol, the objectives of FTP are 1) to promote sharing of files (computer programs and/or data), 2) to encourage indirect or implicit (via programs) use of remote computers, 3) to shield a user from variations in file storage systems among hosts, and 4) to transfer data reliably and efficiently.

FTP has had a long evolution over many years starting with its beginnings published as RFC 114 on 16 April 1971. Over time there has been other forms of file transfer protocols made available as there had been vulnerabilities and weaknesses with FTP such as:

  • Brute force attacks which is attacking via computing credential combinations.
  • FTP bounce attacks which is an exploit enabling an attacker to use the PORT command to request access to ports indirectly through the use of the target machine as a man in the middle request.
  • Packet capture through the use of packet capture tools.
  • Port stealing where traffic directed at a port is stolen or intercepted by an attacker.
  • Spoofing attack where the attacker may use a tool to try multiple instances of an IP address in order to assume the correct, and therefore spoofing, the host address of the target machine.
  • Username enumeration is part of the discovery, or enumeration, process prior to an attack of a network or service by obtaining usernames associated with the service.

There are also limitations to the protocol. For example, there is no ability to encrypt data on transit. Data in transit can be sniffed using freely available tools since the transmissions of usernames, passwords, commands and other data is not encrypted. An attacker can run a packet sniffer over the network can sniff out FTP credentials. In addition, there is no integrity checking of files to ensure that data integrity remains since this is not included as a feature of the protocol.

Situations to use the FTP service:

There is a chance that your initial file transfer system may even be in FTP, depending on the age of the system. However, FTP has many security weaknesses and vulnerabilities as mentioned previously.

Those wishing to continue to use FTP and to do so in a secure manner may find themselves integrating other software to ensure security, creating additional scripts or taking on board additional maintenance. For example, there is no built-in integrity check however scripting work can be done to create a checksum integrity checking process and added at the end of a file transfer. There is also further additional overhead in ensuring that FTP remains secure such as integration with other applications for additional layers of security.

The FTP service is accessible by enabling the service and then configuring the address, port, passive port range, passive address, idle data connection timeout and more.

Hypertext Transfer Protocol (HTTP)

HTTP, shortform for Hypertext Transfer Protocol, is an application-level protocol for distributed, collaborative, hypermedia information systems. It is a generic, stateless, protocol which can be used for many tasks.

HTTP, in use by the World-Wide Web global initiative since 1990, is a protocol used in many tasks related to common usage of the web such as browsing websites. This is relevant to SFTPPlus since we offer a browser-based file management utility.

Without SSL/TLS, there is no way to encrypt data in transit. Other downsides to HTTP, in the context of SFTPPlus, include:

  • Packet capture through the use of packet capture tools.
  • Man-in-the-middle attack where the attacker intercepts and relays false malicious content between two parties.
  • Credentials are sent in a plain text encoding when using the SFTPPlus HTTP Basic Auth API.

Situations to use the HTTP service:

HTTP may be used internally within a highly secured network where there are already mechanisms in place to protect the environment.

We offer HTTP based micro-services and endpoints as part of our public API. In this case, the API is used in conjunction with other security mechanisms in place for the environment.

The HTTP service is accessible by enabling the service and then configuring the address, port, idle connection timeout and maximum concurrent connections.

Protocols to consider when securing data and file transfers

The following are protocols and services that we do recommend for securing data and file transfers. This is not an exhaustive list.

Implicit FTPS or FTPS Implicit SSL (FTPIS)

FTPIS, or implicit FTPS, is the use of the FTP protocol where secure data transfer is invoked via SSL as soon as the connection starts or after the OK reply is sent by the server. In implicit mode, an FTPS client is expected to “immediately expected to challenge the FTPS server with a TLS ClientHello message. If such a message is not received by the FTPS server, the server should drop the connection.” This means that the use of SSL is implied. This is illustrated in the diagram above.

The advantage is that this service is safer than the use of the FTP protocol due to implementing SSL meaning that data transmission is encrypted.

Implicit FTPS or FTPS Implicit SSL (FTPIS)

Situations to use the FTPIS service:

Use FTPIS when you wish to use a more secure FTP for file transfer and where SSL does not need to be invoked prior to login. However, if possible, use FTPES as described further below.

Explicit FTPS or FTPS with Explicit SSL (FTPES)

In explicit mode, an FTPS client must “explicitly request” security from an FTPS server and then step up to a mutually agreed encryption method. If a client does not request security, the FTPS server can either allow the client to continue in insecure mode or refuse the connection.

The advantage is that this service is safer than the use of the FTP protocol due to implementing SSL. Prior to user connection, both the server and client must negotiate the level of security used.

Explicit FTPS or FTPS Explicit SSL (FTPES)

Situations to use the FTPES service:

Use FTPES when you wish to use a more secure FTP for file transfer and where SSL needs to be invoked prior to login. However it should be noted that this not ensure that each and every session and data transfer is secure. FTPES is only a tool allowing the client/server to negotiate the accepted level of security with each session.

Notes for both FTPES and FTPIS

Since both are FTPS (FTP over TLS/SSL), they share some common advantages as listed, non-exhaustively, below:

  • The advantages afforded by SSL is used - certificate authorities, certification revocation lists, transmission encryption and more.
  • Certain regulations and compliance obligations may require data transmissions to be encrypted but it should be noted the difference between FTPES and FTPIS when it comes to which stage the encryption occurs.
  • The protocols make use of TLS (Transport Layer Security) encryption. It should be noted that in SFTPPlus, the TLS version can be used

SSH File Transfer Protocol or Secure File Transfer Protocol (SFTP)

SFTP is a network protocol that allows for file access, transfer and management capabilities over the SSH (Secure Shell) protocol channel.

The advantages for SFTP include:

  • Designed to be used to implement a secure remote file system service and also a secure file transfer service.
  • Runs over a secure channel, SSH, so that the server has already authenticated the client. The identity of the client user should also be available to the protocol.
  • Data is encrypted based on a configured cipher list agreed upon by the server and client.
  • There is the option to implement user access via SSH keys only or via a combination of password and SSH keys. If authenticating via SSH keys, the client does not need to go through password recollection so long as the SSH key is correctly configured on the server.
  • Certain regulations and compliance obligations may require data transmissions to be encrypted.

The SFTP protocol follows a simple request-response model where each request and response contains a sequence number and multiple requests may be pending simultaneously.

Situations to use the SFTP service:

The protocol assumes that both ends of the connection have been authenticated and that the connection has privacy and integrity features already in place and that security issues are left to the underlying transport protocol.

Since the protocol provides file system management feature, the server must have the correct access controls in place and implement correct authorization and enforce access controls.

In this case, when you implement SFTP ensure that you are doing so within an AAA (authorization, authentication, auditing/accounting) security design framework on SFTPPlus.

HTTP over SSL/TLS (HTTPS)

HTTPS, shortform for HTTP over TLS, provides security measures in using HTTP via SSL and its successor, TLS.

The HTTP protocol is further secured via SSL and its successor, TLS (Transmission Layer Security), thus this is referred to as HTTPS. HTTPS provides end-to-end security for browser-based applications.

Other advantages to using HTTPS:

  • TLS can harden TCP against Man-in-the-middle attacks where clients and servers exchange certificates which are issued and verified by a trusted third party called a certificate authority (CA).
  • HTTP Public Key Pinning (HPKP) allows HTTPS website to overcome impersonation via the use of fraudulent certificates.
  • Certain regulations and compliance obligations may require data transmissions to be encrypted

Situations to use the HTTPS service:

Since the SFTPPlus file management utility is accessible via the web browser, the HTTPS service is a more secure alternative compared to HTTP.

HTTPS is a must especially when the resource is going to be public (Internet) facing.

The HTTPS service is accessible by enabling the service and then configuring the SSL/TLS options such as the SSL cipher list, allowed SSL/TLS methods, SSL certificate, SSL key, certificate authority, certification revocation list and more.

Conclusion and next steps

The application of one protocol over the other does not immediately guarantee security. Please consider these services merely as a layer within multiple others when implementing a secure managed file transfer solution.

Since features are constantly changed, we did not touch on any specifics within SFTPPlus. Please consult our documentation for the configuration and operations information, as well as practical users guides.

Evaluating SFTPPlus MFT

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

Install SFTPPlus MFT today either as an on-premise solution supported on Windows, Linux, AIX, OS X, Solaris, FreeBSD, HP-UX or on the cloud as Docker containers or AWS instances.

Email us at sales@proatria.com to start your evaluation version today.

For licensing queries, please contact sales@proatria.com.

Addendum

This resource is written as of SFTPPlus version 3.29.0.

• • •

Securing data and file transfers between SFTPPlus and third parties

Sun 21 January 2018 | article security Written by Hannah Suarez

Why read this article

In order to have a fully established file transfer and sharing system, you need to implement integration at all the other layers including the OS. SFTPPlus can be integrated with external tools and third parties in order to help establish these integration requirements.

This article is written for those new to SFTPPlus and those involved in the business function of securing file transfer software. Topics are covered for various levels of knowledge to reach a wider audience.

Where does SFTPPlus sit in your IT infrastructure

The SFTPPlus software stands at the OSI Layer 7 or the TCP Layer 4. SFTPPlus can be integrated with external tools in order to secure data and file transfer with third parties.

For those not familiar with OSI and TCP please read on.

SFTPPlus on the OSI

The OSI model is a model that characterizes and standardizes communication functions. The layers range from layer 1 right through to layer 7. In the OSI, or Open Systems Interconnection model, SFTPPlus sits in the OSI Layer 7 or on the application layer.

The application layer sits at the top of the OSI model and is the software, hence the name application, layer between the end-user and the networking layers underneath.

SFTPPlus on the TCP

In addition to the OSI model, another way of understand where SFTPPlus plays a role in your infrastructure is via the TCP layer. SFTPPlus sits in the TCP Layer 4 or the application layer. This is the topmost layer which defines the TCP/IP application protocols and how SFTPPlus interfaces with the Transport layer, the layer below the application layer, services to use the network.

While the above information only provides a brief introduction, this should help you understand how SFTPPlus integrates with the networking components (the file transfer protocols) as well as the components on the application level (other systems).

Introduction to file transfer protocols supported by SFTPPlus

For those making the leap to a managed file transfer service for the first time, they may also be thinking about networking components such as which file transfer protocols to support. If your company is currently using FTP, then you may want to switch to FTPES or FTPIS for better security. Or perhaps SFTP for features such as the security of key exchanges.

File transfers are secured via the support of file transfer protocols such as SFTP, FTPS, FTPS, and HTPS. For those that are not familiar, please read the overview of the supported protocols from our Quick Start section in our documentation.

Logging and monitoring network operations with SFTPPlus

Introduction

When you create file transfer systems that interact with services, protocols, databases, users and data, it is important to ensure that the system is being protected and monitored from unauthorized modifications, use, access or destruction.

Ensuring that activities are under proper monitoring and logging is also an important aspect of secure file transfer infrastructures. Should there be access attempts from that particular IP range? Should this file transfer service port be in use at all? Are there subsets of data that are critical enough that an alert mechanism be put in place for failed transfers in that source? The answers to questions like these will help provide the basis for meeting logging and auditing requirements for your infrastructure through our audit and logging mechanisms.

Databases

SFTPPlus can be integrated with external database and logging tools such as SQLite, MySQL, Syslog, Windows Event Log.

These integrations will help in any logging, reporting and auditing obligations as is the case for organizations seeking to meet compliance with bodies such as GDPR, HIPAA, GPG13 and more.

SFTPPlus will keep a detailed log of any file which is transferred and can include details about the initial transfer request and the status of the request finalization, logs status changes to device and configurations, login attempts, connection attempts to and from the server or client, session activities and more.

Email resources for email alerts

Users can set up vital email alerts to monitor for any specific server events.

By creating a new email resource, SFTPPlus formulates outgoing emails as though they were coming from an email client.

Access to an SMTP server is necessary. You may use any email service available on your network (public or private). You may use anonymous SMTP, or a preexisting account on that server/service. A local or private SMTP server may also be used.

Once the email resource and Event Handler are configured, the body of any email sent will consist of a JSON object for the event that triggered the email. The recipient(s) and subject line of the email are configurable items.

Integrating SFTPPlus with third parties via the API

SFTPPlus provides public APIs which extend the identity management, file access, audit functionality and more of the SFTPPlus MFT Server using HTTP based micro-services / endpoints. The HTTP APIs can be used to integrate file transfer processes with disparate systems, such as web applications, that need to interface with SFTPPlus.

While it is targeted to HTTP, the HTTP API is used by integrator only as a layer of operations underneath secured networks such as having the services only available under corporate VPNs or proxies.

For more details about the API, please consult our developer section in the documentation.

Potential use case for HTTP Authorization

A potential use case for the API is utilizing the HTTP Authorization for SFTPPlus. This is deployed in a DMZ where certificate-based authentication cannot be used, only a username/password authentication with a time limit expiration.

Case study with load balancers and HTTPS push from third party via SFTPPlus API

Case study with load balancers and HTTPS push from third party via SFTPPlus API

This case study involves integrating with a third party. In this case, the third party is a web application functioning as the client.

Through the HTTPS service that is available with SFTPPlus, the third party developer works with the SFTPPlus HTTP API to authenticate the users and to allow them to upload content (push) to the SFTP servers.

Subsequently the corporate, or internal network, are authenticating via the SSH key exchange (as one of the possible methods) and pulling content from the servers.

While the topic is covered in another article, the servers are also set up for high availability and resilient environment via a load balancer. In this case, the load balancer is utilizing the weighted round-robin algorithm depending on the server ‘weight’.

Ensuring high availability is also a pathway to secure file transfer operations by making sure that critical data is constantly available. Load balancers can be used with SFTPPlus to set up high availability.

Through a combination of choosing secure file transfer protocols, smart use of the SFTPPlus API and structuring file transfer operations to function in a high availability environment, you can further secure your data and file transfers with your company.

Integrating SFTPPlus with DMZ and buffer zones

A DMZ (or demilitarized zone) is implemented in order to separate servers and other resources from the external or public-facing facing Internet and their internal, trusted networks will run through a number of different configuration options. The standard example used is two firewalls, one firewall for the external or public facing resources and the other for the internal resources, serving a subnetwork.

Case study with DMZ and buffer zones

Case study with DMZ and buffer zones

For a case study on how SFTPPlus is integrated with a DMZ, see above for an example of an internal company user transferring files towards external, public-facing servers.

In this case, the FTP/SFTP ‘Inbox’ folder which resides within the DMZ will utilize the SFTPPlus file-dispatcher Event Handler to dispatch files to the ‘Outbox’ folder within the DMZ.

The file-dispatcher event handler has been configured to move files from the SFTPPlus Inbox folder to the SFTPPlus Outbox folder based on a matching expression - either global or regular expression. In this instance, we can say that the matching expression is for all PDF files. All of this happens with the DMZ which acts as a buffer zone between the media PC in an internal network to external servers.

From the SFTPPlus Outbox folder, which serves as a cache of matched files, users can initiate a client transfer to the external public facing servers outsize the DMZ.

Implementing AAA (Authentication, Authorization and Accounting) frameworks

SFTPPlus allows for an AAA system to be implemented. AAA refers to Authentication, Authorization and Accounting. It is a system to mediate and manage network access based on the process of identifying a user (authentication), granting or denying access to the user (authorization) and keeping track of the user’s activities on the network resource (accounting).

Accounting part of the AAA framework

Accounting part of the AAA framework

As SFTPPlus operates, it will emit a set of events which contains a unique ID and defines a specific operation carried out by the server. A common action for an event is to send it to one of the supported logging systems. This covers the Accounting requirement of the AAA (Authentication, Authorization, and Accounting) security design framework.

Utilizing an accounting, also seen as auditing, framework is a way to ensure that any compliance or logging obligations and requirements are met.

Authorization part of the AAA framework

Authorization part of the AAA framework

The use of authorization is one of the fundamental aspects of network and resource management security. By building an authorization framework, you can ensure that users have correct access to network resources.

In the above diagram, we have two users in the same department or user group but both of these users have different access requirements. After authentication via the authentication server, how does an administrator ensure that the correct authentication framework is applied? One user can only have read-only rights to shared folder and full-control for a common home folder. Whereas another user has full-control-allowed access to both the common folder and all other folders underneath, including a shared folder with the first user.

The permissions framework can be set up on a global or on a per-path basis

In the above diagram, the permissions framework can be set up on a global or on a per-path basis, including fine-grain details such as permissions for matching expressions.

Even after a user group is authenticated and the correct users are in their respective accounts, a solid authorization framework will ensure that any additional user access rights policies are applied.

Authentication part of the AAA framework

The server-side security of SFTPPlus is designed based on the Authentication, Authorization and Accounting (AAA) components. Authentication can be integrated with external third parties - Windows Domain Accounts - or with external resources such as via the domain controller, via the SSH RSA/DSA keys or SSL certificates.

When compiling how you will secure your system, it is important to take stock of how you are mediating and managing network access based on meeting authentication, authorization and accounting requirements.

Integrating SFTPPlus with post-processing actions

Integrating SFTPPlus with post-processing actions

SFTPPlus can function suitably when anti-virus applications are installed to protect the environment on the machine. This integration is done as part of the transfer configuration for post-processing actions.

Most anti-virus applications have a real-time protection component that will scan files on creation, when accessed, and on execution. These operations will not affect the overall performance of the system.

Case Study - Virtual machines

Case Study - Virtual machines

To further secure data and file transfers, users can create two installations running in active / passive mode behind a load balancer. These two instances will share the same users, database and storage.

Running in AWS, a new instance is created when one dies to maintain high availability.

SFTPPlus can be integrated with a third party virtual private cloud, as well as load balancers to ensure high availability and resiliency.

Conclusion and next steps

The application of these does not immediately guarantee security. Please consider these guides merely as a layer within multiple others when implementing a secure managed file transfer solution.

Since features are constantly changed, we did not touch on any specifics within SFTPPlus. Please consult our documentation for the configuration and operations information, as well as practical users guides.

Evaluating SFTPPlus MFT

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

Install SFTPPlus MFT today either as an on-premise solution supported on Windows, Linux, AIX, OS X, Solaris, HP-UX, FreeBSD or on the cloud as Docker containers or AWS instances.

Email us at sales@proatria.com to start your evaluation version today.

For licensing queries, please contact sales@proatria.com.

Addendum

This resource is written as of SFTPPlus version 3.29.0.

• • •