Quality Assurance

Automated tests

Our testing process involves planning, designing, implementing and executing functional tests.

Automated testing is utilized and targeted to functional tests.

The test suite, comprising of more than 4500 automated tests, are executed prior to each release. The test suite is also extended to accommodate any new functional tests and test scenarios that have been introduced for that release.

Releases will need to pass all automated tests. Even if there is one failed test, this will block the release until all tests have passed.

Manual tests

Manual smoke test scenarios include checking that the release can be installed, upgraded over an existing installation and can perform basic file transfer operations.

Others - Soak, Systems Integration and Unit Integration Tests

Soak testing coverage is currently low and it only slightly increases the test coverage already in place.

We do not cover benchmark and performance testing. Due to the number of unique environments in which the product is used, it is not possible to design and implement all possible production scenarios.

We recommend that system integration testing / black box testing and unit integration testing be performed. This is to evaluate the product and that it meets the actual systems, environments, processes and interactions that are specific to your requirements.

Tested platforms

The following is a list of systems in our automated test suite used to test our products:

  • AIX 5.3 - IBM Power System P710 with a POWER7+ CPU.
  • AIX 7.1 - IBM Power System P710 with a POWER7+ CPU.
  • HPUX 11.31 v3 - HP Integrity RX1620 with Itanium2 CPU.
  • OS X 10.8 with OpenSSL 0.9.8 provided by Apple on Intel Xeon E3-1265 v2
  • MacOS Sierra 10.12 with OpenSSL 1.0.2 provided by the Homebrew community on Intel Xeon E3-1265 v2
  • Red Hat Enterprise Linux on x86 and x86_64 - Intel Xeon E3-1265 v2 and AMD Phenom II X6 1090T
  • Red Hat Enterprise Linux on POWER8 - IBM Power System S822L with PowerKVM
  • Red Hat Enterprise Linux on IBM z Systems mainframes - IBM System z provided by the Linux for System z Test Drive program and on the Hercules mainframe emulator
  • SUSE Enterprise Linux on x86 and x86_64 - Intel Xeon E3-1265 v2 and AMD Phenom II X6 1090T
  • Ubuntu Linux on x86 and x86_64 - Intel Xeon E3-1265 v2 and AMD Phenom II X6 1090T
  • Ubuntu Linux on ARM64 - HP Moonshot m400
  • Ubuntu Linux on POWER8 - IBM Power System S822L with PowerKVM
  • Solaris on x86_64 - Intel Xeon E3-1265 v2 and AMD Phenom II X6 1090T
  • Solaris 10 on SPARC Sparc Enterprise T2000
  • Solaris 11 on SPARC Sparc Enterprise T5240
  • Windows 2003, 2008 and 2012 Server on x86 and x86_64 - Intel Xeon E3-1265 v2
  • FreeBSD 10 on X86_64

Testing your configuration

You may want to use different deployment environments in order to test not only the configuration but also the entire system supporting the file transfer operations.

Our customers can deploy SFTPPlus on "development", "testing" and "staging" environments as part of the process of supporting a final production environment.

The following are examples of different environments:

Local / Development

Each team member can install SFTPPlus on her/his machine for testing. The sysadmin may have some servers on which they test with SFTPlus and consider various configuration variants. In this environment, you may run partial transfers.

Testing / Integration

This environment could include a server where a specific configuration is applied and on which end to end transfers can be executed. They target the functional requirements. In this stage, security / monitoring / performance testing are not implemented and firewall rules might be relaxed. Syadmins, developers and QA team members have access to this type of environment for testing.

Staging / Pre-Production

This stage is a very similar environment to the production environment. The same hardware is used to help with performance testing. The same firewall rules are also used to test security. Only the sysadmin might have access to this environment in order to to mirror production access restrictions.

Please see the Installation Validation and Troubleshooting page for additional notes to test your SFTPPlus configuration and installation.

We allow the use of existing SFTPPlus licenses for the use of setting up testing environment/s prior to the production stage.

For any further questions about how testing is done or for a trial, please do not hesitate to get in touch with us. For further questions about licensing during the testing stages, please contact our Sales team.