Articles from blog category

Endpoint FTPS and SFTP server for DWP GFTS

Tue 02 April 2019 | blog

A red floppy disk.

The electronic data interchange (EDI) of the Department for Work and Pensions (DWP) in the United Kingdom is done via the Generic File Transfer Service (GFTS) gateway.

This article is aimed at companies which need to exchange files and data with the DWP. These entities are referred by DWP as creditor server or endpoint FTPS server.

The information is also valid for the E-Transfer systems used by local councils.

In practice, this means that as a partner to DWP you will have to set up and host an Explicit FTPS server. DWP is operating an FTPS client and actively pushes data to you.

Electronic data interchange (EDI) is the concept of electronically communicating information that was traditionally communicated on paper, such as purchase orders and invoices.

Connection Security

The connection between your company and DWP is secured using certificate-based mutual TLS authentication (mTLS) (also referred to as two-way authentication). DWP will provide the SSL certificate used by their client, while your company will have to provide the SSL certificate used by your FTPS server.

With SFTPPlus you can use a certificate generated by any certificate authority (public or your private CA).

Integration with the Let's Encrypt Certificate Authority is provided via the HTTP-01 challenge. SFTPPlus can seamlessly obtain and use a certificate from the Let's Encrypt CA. The certificate is automatically renewed.

On top of the security provided by the TLS/SSL layer, username/password credentials are used to identify the requests from DWP.

SFTPPlus can support a multi-channel architecture, allowing you to use the same SFTPPlus server for exchanging files with multiple partners, not only with DWP.

Read more about securing FTPS server with SFTPPlus in our dedicated documentation page.

Client / Server Data Exchange

FTPS is an open standard file transfer protocol built on a client-server model architecture.

The client is the active component which controls when and what type of file transfer operation to perform. The client generates an authenticated connection to the server and ask the server to push or pull files. DWP will act as a client.

The server is the reactive component which controls who can perform file transfer operations and what kind of file operations are allowed. The server stays idle and only becomes active once it receives a connection from the client. Your system will act as a server.

Once the data is pushed by DWP, it will reside as files on your system. From there it will be further processed and consumed by your business system.

ProAtria DWP Expertise

ProAtria, the developer of SFTPPlus, is a long-term partner for the project deployed at DWP. We have helped with the migration from insecure FTP to Explicit and Implicit FTPS systems and with the migration from legacy Solaris-based systems to a modern Linux-based cloud infrastructure.

We are involved in the delivery and maintenance of the Digital Children’s Platform (DOS 012) and the data exchange between DWP and the Scottish Government.

We offer broad expertise into the data exchange with DWP and DVLA. Our customers benefit of help and consultancy for their DWP and DVLA related projects without any additional cost.

A server rack.

Evaluating SFTPPlus MFT

The features listed in this article are just a selected few out of many integration and configuration options that are available today. Feel free to talk to the Support team about your requirements with file transfer software.

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

SFTPPlus MFT is available as an on-premise solution supported on Windows, Linux, and macOS.

It is also available on the cloud as Docker containers, AWS or Azure instances and many other cloud providers.

Request a trial using the form below.

• • •

FTP client uploads with temporary names

Thu 21 February 2019 | ftp client-side blog

A drawer with file tabs.

When closely investigating managed file transfers, pushing a file to a remote FTP server turns out to be just a phase in a series of interlinked processes.

Once the file arrives on the FTPS server, it is read and further processed. The next step might involve downloading the file or copying it to another processing area.

When uploading a large file, copying or pulling it before completing the transfer can result in corrupted file data. For example, a pull operation might start before the file is fully uploaded, with only a fragment of the original file available for download.

Another common case in which data corruption may happen is when a partial upload occurs because of connection failures during transfer. A client starts sending a file to the server, but at some point connection is lost. Maybe the client VM was powered off unexpectedly or the network became temporarily unavailable for too long. This will result in a partial file being left on the server, which can be accidentally processed by the next stage in our process.

This is a serious issue with FTP and FTPS connections. FTP protocols do not mandate sending the total file size before an upload. Furthermore, they do not make use of an explicit end-of-file marker. An FTP client signals the completion of an upload by simply closing the data connection.

To mitigate this problem, a file locking mechanism can be implemented by uploading files using temporary names and then renaming them back to their initial names once all the data was pushed by the client.

Clients like WinSCP will use temporary names formed by appending a non-configurable .filepart extension to the initial file names.

In SFTPPlus you can configure a file transfer to use any suffix / extension during the upload, you are not restricted to the .filepart one. . For example, you can use the .tmp or .incomplete extensions.

Screenshot with transfer destination in SFTPPlus.

By using temporary names you can implement a process in which transferred files are locked while their contents are being uploaded. The chained process will ignore files with temporary names, only handling transferred files after the final rename operation.

On most file systems the rename operation is atomic and very fast.

The same technique can be used to lock a file while uploading through SFTP transfers.

The SCP protocol does not provide a rename operation, but the total file size is advertised in the SCP upload request, which happens before the client starts pushing the content of the file.

Read more about transferring files with temporary names in our documentation page.

Evaluating SFTPPlus MFT

The features listed in this article are just a selected few out of many integration and configuration options that are available today. Feel free to talk to the Support team about your requirements with file transfer software.

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

SFTPPlus MFT is available as an on-premise solution supported on Windows, Linux, and macOS.

It is also available on the cloud as Docker containers, AWS or Azure instances and many other cloud providers.

Request a trial using the form below.

• • •

Restrict user with trusted IPs for SFTP and FTPS

Fri 15 February 2019 | security blog

Computer security illustration.

It is common practice to secure a file transfer server using firewall rules which only allow incoming connections from trusted partners.

Let's assume you have a US partner named "ACME Inc", connecting to your server from IP 1.1.1.1 using the user acme-inc, and another German partner called "AlleWerkzeuge AG", connecting to your server from IP 5.5.5.5 using the user alle-werkzeuge-ag.

You can configure your firewall to only allow connections from a list of trusted IPs like 1.1.1.1 and 5.5.5.5, but the firewall doesn't know about usernames. So it will allow the account acme-inc to connect even if the connection is initiated from 5.5.5.5, which is an IP outside of the ACME Inc network.

To complement firewall restrictions, SFTPPlus allows defining a fixed list of trusted IP rules from which it will allow connections for a specific user.

Such a configuration can be defined per user, but also per group, with multiple users inheriting their configuration from the group.

To restrict a specific user to connect through SFTP or FTPS to the file transfer server only from a certain IP (or IPs), you can use the source_ip_filter configuration option in SFTPPlus.

The remote access is denied when the user connects from a source address which is not whitelisted.

Below is a screenshot from our web-based management console demonstrating such a configuration.

Screenshot of SFTPlus account configuration.

Read more about securing your SFTP/FTPS and HTTPS services with SFTPPlus in our documentation page.

Evaluating SFTPPlus MFT

The features listed in this article are just a selected few out of many integration and configuration options that are available today. Feel free to talk to the Support team about your requirements with file transfer software.

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

SFTPPlus MFT is available as an on-premise solution supported on Windows, Linux, and macOS.

It is also available on the cloud as Docker containers, AWS or Azure instances and many other cloud providers.

Request a trial using the form below.

• • •

HTTPS, FTPS, and SFTP with Docker and OpenShift

Wed 30 January 2019 | blog

For some time, we have been maintaining an MIT-licensed GitHub repository to ease the creation and running of SFTPPlus Docker instances.

Whether you are already a customer of ours, or currently evaluating SFTPPlus, simply head over to our GitHub repository, clone, and follow the instructions to run an SFTPPlus instance in Docker.

To further aid in quickly trying out a version of SFTPPlus in Docker, we are pleased to announce the creation of a SFTPPlus Docker Hub repo.

Docker Hub banner

You can now simply pull from Docker Hub our latest 3.44.0 trial image for Red Hat Enterprise Linux 7.0 / CentOS 7.0 (or other compatible OS'es) with a single command:

docker pull proatria/sftpplus-trial:3.44.0-centos7

In this way, you can evaluate a dockerized FTPS and SFTP server with minimum effort.

The Docker Hub repository only contains the evaluation version. For production use you will most probably want to change the configuration to meet your requirements.

To build your own Docker image, check the scripts and instructions used to build the evaluation image in our aforementioned GitHub repo.

Similar commands can be used to deploy the Debian Linux 8 image pushed to Docker Hub as:

proatria/sftpplus-trial:3.44.0-debian8

To offer a Docker image with minimal disk size, our Docker Hub repository also covers Alpine Linux, a distribution for "power users who appreciate security, simplicity and resource efficiency". To get the Alpine Linux 3.7 image, use:

proatria/sftpplus-trial:3.44.0-alpine37
OpenShift banner

Our Dockerfile and the images derived from it do not require running the process as root inside the container. Therefore, you can deploy them in OpenShift with a single command as well:

oc new-app proatria/sftpplus-trial:3.44.0-centos7

A users guide for deploying SFTPPlus with Docker containers is available in our Docker documentation page.

Evaluating SFTPPlus MFT

The features listed in this article are just a selected few out of many integration and configuration options that are available today. Feel free to talk to the Support team about your requirements with file transfer software.

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

SFTPPlus MFT is available as an on-premise solution supported on Windows, Linux, and macOS.

It is also available on the cloud as Docker containers, AWS or Azure instances and many other cloud providers.

Request a trial using the form below.

• • •

Announcing the SFTPPlus and Docker repository

Wed 31 January 2018 | blog

Docker containers have been a constant presence in the worlds of DevOps and cloud computing. We have recognized this only through a passing mention in our product page that SFTPPlus can run in a Docker container.

However, we have not gone beyond that, until now.

Now announcing SFTPPlus and Docker

We are pleased to announce the creation and release of a dedicated, public, MIT-licensed repository to make the creation and running of Dockerfiles more accessible. Whether you are already a customer of ours, currently evaluating SFTPPlus or you are interested in seeing a managed file transfer service run in Docker. Simply head over to our repo, clone, and follow the instructions to run an SFTPPlus instance in Docker.

SFTPPlus running in a Docker container does not lose functionality and makes full use of the infrastructure provided by a Docker container. You can audit and archive SFTPPlus server events (also knows as logs) using the default Docker log driver.

Once you have set up the SFTPPlus Dockerfile, what's next? Why not use Docker Compose to run multi-container Docker applications.

You can use Compose to create the following specialized instances like:

  • SFTPPlus instance - Handle file transfers over SFTP / FTPS / WebDAV. Data storage is backed by a volume.
  • Authentication and Authorization instance - Respond to authentication and authorization requests over HTTP. You can use this instance to authenticate other services inside your deployment.
  • Audit instance - Receive, over HTTP, events and logs generated by SFTPPlus. Use this instance to process logs and events from other services.
  • File Processor instance - Receives events over HTTP in order to further process them based on the rules specified by your business logic.

See our Github and documentation

You can view the scripts and instructions to get started quickly in our GitHub repository.

A users guide is available in our Docker documentation page.

Evaluating SFTPPlus MFT

The features listed in this article are just a selected few out of many integration and configuration options that are available today. Feel free to talk to the Support team about your requirements with file transfer software.

SFTPPlus MFT Server supports FTP, Explicit FTPS, Implicit FTPS, SFTP, SCP, HTTP and HTTPS.

SFTPPlus MFT is available as an on-premise solution supported on Windows, Linux, and macOS.

It is also available on the cloud as Docker containers, AWS or Azure instances and many other cloud providers.

Request a trial using the form below.

• • •