Articles and news

SFTPPlus Release 3.48.0

Mon 27 May 2019 | general release

We are announcing the latest release of SFTPPlus version 3.48.0.

New Features

  • HTTP POST event handler can now be configured to automatically retry on network and HTTP errors. [server-side][http-api] [#2619]
  • It is now possible to configure a file transfer service to emit debugging events for the low-level protocol used. [http][ftp][ftps][sftp][scp][server-side] [#2697]
  • The Python Extension event handler now handles events on multiple CPUs. In previous versions all events were handled by a single CPU. [#5262]
  • A new destination path action named single-file was added to transfer multiple source files as a single destination file. [client-side] [#4054]
  • You can now disable the overwriting rule for a transfer destination. In this way, the file is uploaded right away, without doing any extra requests on the server. [client-side] [#4054]
  • Debian 9 is now a supported platform. [#3353]

Defect Fixes

  • When changing the current folder in FTP, the SFTPPlus server now only checks that the path is a folder and that path traversal is allowed. It no longer tries to see if the operating system allows listing content. Asking the operating system to list content for every target directory could have caused performance issues. [server-side][ftp][ftps] [#2111]
  • You can now use a local directory with a large number of files (more than 10.000), as the source for a transfer. [client-side] [#1319]
  • The local filesystem source location no longer stops to monitor the source on I/O errors. It will log an error and retry to get the content again after changes_poll_interval seconds. [client-side] [#3350]
  • The SysV and OpenRC init scripts now work when executed as root user. This was a defect introduced in 3.42.0. [#3353]

Deprecations and Removals

  • The Python Extension event handler no longer takes a parent argument. The events are no longer handled in separate threads. Instead, they are added to a queue to be executed on a dedicate CPU. [#5262]
  • Support for Ubuntu 16.04 on ARM64 was removed. [#3353]
  • Support for Debian 8 was removed. [#3353]

You can check the full release notes here.

• • •

SFTPPlus Release 3.47.0

Thu 11 April 2019 | general release

We are announcing the latest release of SFTPPlus version 3.47.0.

New Features

  • You can now configure multiple domains for a free Let's Encrypt certificate using the subjectAlternativeName field. [server-side][ftps][https] [#5108]
  • A new event handler of type external-executable was added to execute external scripts or programs. [#5234]
  • Windows Server 2019 is now a supported platform. [#5241-1]
  • The bundled OpenSSL libraries in Windows, SLES 11, and OS X were updated to versions 1.1.1b, adding support for TLS 1.3. [#5241]

Defect Fixes

  • The WebDAV location now ignores HTTP proxy errors when they occur while monitoring a remote SharePoint Online site. [client-side][https] [#5114-1]
  • The WebDAV location now works with multiple parallel transfers from the same SharePoint Online source. [client-side][https] [#5114]
  • The SFTP and SCP file transfer services will no longer block the whole SFTPPlus process during the SSH handshake. [server-side][sftp][scp] [#5202]

Deprecations and Removals

  • Event with ID 20057, emitted when execute_at_startup times out, was removed and replaced by event with ID 20056. [#5234]

You can check the full release notes here.

• • •

Endpoint FTPS and SFTP server for DWP GFTS

Tue 02 April 2019 | compliance

A red floppy disk.

The electronic data interchange (EDI) of the Department for Work and Pensions (DWP) in the United Kingdom can be done via the Generic File Transfer Service (GFTS) gateway.

This article is aimed at companies which need to exchange files and data with the DWP.

For example, as an housing association you will exchange documents with the DWP Housing to manage the Universal Credit payments and deductions.

These entities are referred by DWP as creditor server or endpoint FTPS server.

The GFTS options is not available to Local Authorities or Local Councils. E-Transfer system should be used instead.

In practice, this means that as a partner to DWP you will have to set up and host an Explicit FTPS server. DWP is operating an FTPS client and actively pushes data to you.

Electronic data interchange (EDI) is the concept of electronically communicating information that was traditionally communicated on paper, such as purchase orders and invoices.

Connection Security

The connection between your company and DWP is secured using certificate-based mutual TLS authentication (mTLS) (also referred to as two-way authentication). DWP will provide the SSL certificate used by their client, while your company will have to provide the SSL certificate used by your FTPS server.

With SFTPPlus you can use a certificate generated by any certificate authority (public or your private CA).

Integration with the Let's Encrypt Certificate Authority is provided via the HTTP-01 challenge. SFTPPlus can seamlessly obtain and use a certificate from the Let's Encrypt CA. The certificate is automatically renewed.

On top of the security provided by the TLS/SSL layer, username/password credentials are used to identify the requests from DWP.

SFTPPlus can support a multi-channel architecture, allowing you to use the same SFTPPlus server for exchanging files with multiple partners, not only with DWP.

Read more about securing FTPS server with SFTPPlus in our dedicated documentation page.

Client / Server Data Exchange

FTPS is an open standard file transfer protocol built on a client-server model architecture.

The client is the active component which controls when and what type of file transfer operation to perform. The client generates an authenticated connection to the server and asks the server to push or pull files. DWP will act as a client.

The server is the reactive component which controls who can perform file transfer operations and what kind of file operations are allowed. The server stays idle and only becomes active once it receives a connection from the client. Your system will act as a server.

Once the data is pushed by DWP, it will reside as files on your system. From there it will be further processed and consumed by your business system.

ProAtria DWP Expertise

ProAtria, the developer of SFTPPlus, is a long-term partner for the projects deployed at DWP. We have helped with the migration from insecure FTP to Explicit and Implicit FTPS systems and with the migration from legacy Solaris-based systems to a modern Linux-based cloud infrastructure.

We can help you understand the Code of Connection (CoCo) document and make sure the people from your organization will understand the requirements and security measurements.

We are involved in the delivery and maintenance of the Digital Children’s Platform (DOS 012) and the data exchange between DWP and the Scottish Government.

We offer broad expertise into the data exchange with DWP and DVLA. Our customers benefit of help and consultancy for their DWP and DVLA related projects without any additional cost.

A server rack.

Evaluating SFTPPlus MFT

The features listed in this article are just a selected few out of many integration and configuration options that are available today. Feel free to talk to the Support team about your requirements with file transfer software.

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

SFTPPlus MFT is available as an on-premise solution supported on Windows, Linux, and macOS.

It is also available on the cloud as Docker containers, AWS or Azure instances and many other cloud providers.

Request a trial using the form below.

• • •

SFTPPlus Release 3.46.0

Mon 11 March 2019 | general release

We are announcing the latest release of SFTPPlus version 3.46.0.

New Features

  • The HTTP/HTTPS file transfer service now supports downloading multiple files at once as a Zip file. [server-side][web-api][http][https] [#5093]
  • It is now possible to set up password expiration for accounts and groups. [server-side][security] [#5146]
  • It is now possible to configure the preferred size of the group in the SSH Diffie-Hellman group key exchange method. [server-side][sftp][scp] [#5205]
  • The file dispatcher event handler now supports the copy action. This will copy the source file to one or more destinations, without removing the source file. [server-side][client-side] [#5210]
  • The file dispatcher event handler now supports the rename action. This will rename the source file (with an atomic move operation) without overwriting an existing file. [server-side][client-side] [#5220]

Defect Fixes

  • An event is now emitted when a file is closed after it was open for reading through the HTTP file transfer service. [server-side][http][https] [#5093]
  • The HTTP/HTTPS file transfer service now responds with 401 Unauthorized for requests made with 100 Continue when no credentials are provided in the request. [server-side][http][https] [#5223]

You can check the full release notes here.

• • •

FTP client uploads with temporary names

Thu 21 February 2019 | ftp client-side blog

A drawer with file tabs.

When closely investigating managed file transfers, pushing a file to a remote FTP server turns out to be just a phase in a series of interlinked processes.

Once the file arrives on the FTPS server, it is read and further processed. The next step might involve downloading the file or copying it to another processing area.

When uploading a large file, copying or pulling it before completing the transfer can result in corrupted file data. For example, a pull operation might start before the file is fully uploaded, with only a fragment of the original file available for download.

Another common case in which data corruption may happen is when a partial upload occurs because of connection failures during transfer. A client starts sending a file to the server, but at some point connection is lost. Maybe the client VM was powered off unexpectedly or the network became temporarily unavailable for too long. This will result in a partial file being left on the server, which can be accidentally processed by the next stage in our process.

This is a serious issue with FTP and FTPS connections. FTP protocols do not mandate sending the total file size before an upload. Furthermore, they do not make use of an explicit end-of-file marker. An FTP client signals the completion of an upload by simply closing the data connection.

To mitigate this problem, a file locking mechanism can be implemented by uploading files using temporary names and then renaming them back to their initial names once all the data was pushed by the client.

Clients like WinSCP will use temporary names formed by appending a non-configurable .filepart extension to the initial file names.

In SFTPPlus you can configure a file transfer to use any suffix / extension during the upload, you are not restricted to the .filepart one. . For example, you can use the .tmp or .incomplete extensions.

Screenshot with transfer destination in SFTPPlus.

By using temporary names you can implement a process in which transferred files are locked while their contents are being uploaded. The chained process will ignore files with temporary names, only handling transferred files after the final rename operation.

On most file systems the rename operation is atomic and very fast.

The same technique can be used to lock a file while uploading through SFTP transfers.

The SCP protocol does not provide a rename operation, but the total file size is advertised in the SCP upload request, which happens before the client starts pushing the content of the file.

Read more about transferring files with temporary names in our documentation page.

Evaluating SFTPPlus MFT

The features listed in this article are just a selected few out of many integration and configuration options that are available today. Feel free to talk to the Support team about your requirements with file transfer software.

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

SFTPPlus MFT is available as an on-premise solution supported on Windows, Linux, and macOS.

It is also available on the cloud as Docker containers, AWS or Azure instances and many other cloud providers.

Request a trial using the form below.

• • •