Using TestStand with Jenkins in CI/CD Workflows

Jenkins is an open-source automation server commonly used to perform Continuous Integration (CI) tasks such as building, testing, and delivering applications. TestStand can be integrated with Jenkins or other CI systems for several use cases, including:

• Continuously executing test sequences to detect integration issues early (for example, identifying missing dependencies such as uncommitted subVIs in shared repositories).
  
  
• Validating test code across multiple developers to ensure consistency between local development environments and shared builds.
  
  
• Integrating hardware validation into a CI pipeline, where TestStand executes tests for hardware requirements and software/hardware interaction.
  
  
• Publishing TestStand results alongside software unit test results in the same CI system, enabling a unified view of system quality.

By continuously building and testing code, CI practices help identify problems early in the development process. For example, in a team where multiple engineers contribute to test development, a sequence may run correctly on one machine but fail in the shared environment due to missing components. Running automated tests in a CI system allows these issues to be detected and resolved immediately, avoiding late-stage troubleshooting.

In more advanced workflows, TestStand can be part of a larger CI system involving both software and hardware teams. Software unit tests validate code functionality, while TestStand sequences verify hardware behavior and system integration. By publishing all results back to the CI server, teams can adopt a unified, test-driven approach and quickly address any failures.

Command Line Tools

The primary communication method between a CI server and a TestStand system is through the command line. The following TestStand utilities have command-line interfaces that can be used for this purpose:

• TestStand Sequence Editor and Operator Interfaces
• TestStand Sequence Analyzer Application
• TestStand File Diff and Merge Utility
• TestStand Deployment Utility
• Sequence File Converter Utility
• XML Packaging Utility

Hardware and Software Requirements

You will need a Windows computer or virtual machine on your network that can be used exclusively for building. It is not recommended to use this machine for software development purposes - doing so can cause copies of dependencies to exist in search directories that can be inadvertently pulled into a build. You will also need a licensed copy of all software and drivers required to build your test system. 

Please, refer to the Activating and Licensing TestStand article from the TestStand help to learn more about licensing a TestStand installation on a CI/CD server machine or container.

Setting up a Build Machine for use with TestStand

1. Set up a local administrator account for the build server and log on.
2. Install all NI and 3rd party software required to build your test system.
3. Since Jenkins runs as a service, no user dialogs can be shown during execution. If you plan on running any executions as part of your build or test process, modify the following TestStand properties to prevent dialogs from showing:
   1. Modify the following Station Options:
      1. Configure » Station Options » User Manager » Check User Privileges - uncheck this box to prevent the login dialog, alternatively - configure the system to automatically log on the Windows user.
      2. Configure » Station Options » Execution » On Run-Time Error - set this property to 'Run Cleanup.'
      3. Configure » Station Options » Time Limits - make sure none of these properties are set to "Prompt for Action."
      4. Configure » Station Options » Preferences - Uncheck the option to "Prompt to Find Files."
   2. Ensure that the process model does not display any dialogs that require user interaction.
   3. Ensure that the client sequence file does not display any dialogs that require user interaction.
4. Change the DCOM settings of the TestStand auto Manager (TSAutoMgr.exe) to Launching User.
   1. Open the Run Window (Win + R).
   2. Type dcomcnfg and press enter.
   3. Navigate to Component Services»Computers»My Computer»DCOM Config»NI TestStand AutoMgr.
   4. Right-click NI TestStand AutoMgr and select Properties.
   5. In the Identity tab, change the user account to The launching user.
      
      

5. Note: The TestStand Version Selector can reset DCOM settings. If you intend to manage multiple TestStand versions on a single build machine, consider automatically setting this property via the RunAs registry key.

Setting up Jenkins

1. Download and install the latest Jenkins Long Term Support (LTS) msi for Windows.
2. When the installer is done running, it should automatically open up a web browser and navigate to 'http://localhost:8080/'.
   
   

Note: You will see HTTP Error 404 if the Jenkins service did not start properly. Port 8080 is the default port for LabVIEW web services and several other applications - if this port is already in use, Jenkins will not be able to start. To change the default port, open Jenkins.xml from the installation directory and modify the --httpPort attribute to an open port. If you still have problems starting the service, check the jenkins.err.log file in the installation directory for detailed error information.
 

8. Before configuring Jenkins, configure the service to run as a local administrator account. Services run as the SYSTEM user by default, which can cause problems with the TestStand deployment utility.
   
   1. Press Win+R and type 'services.msc' to open services.
   2. Right-click on the Jenkins Service and choose 'Properties.'
   3. On the 'Log On' tab, choose 'This Account' and enter the local admin credentials.

9. Press OK and restart the service.
10. Navigate to the Jenkins URL: http://localhost:<port> 
11. You will need to log in with the initial admin password, which should be given to you by Jenkins.
12. Once logged in, you can install any plugins that you'd like. The Pipeline Plugin Suite is a good place to start.
13. Create an initial admin user when prompted.
14. Refer to the Jenkins User Documentation for more getting started information and best practices.

Integrating TestStand with Jenkins Pipelines

ou can integrate the full TestStand suite—including the Sequence Editor or Operator Interfaces, Sequence Analyzer, and Deployment Utility—into Jenkins by using Jenkins Pipelines to orchestrate command-line execution of these tools (see this article from Jenkins docs to learn more about Pipelines).

In this approach, a Jenkins pipeline defines a series of stages that represent your workflow (for example, validation, execution, and packaging). Within each stage, Jenkins invokes the appropriate TestStand application through its command-line interface, allowing all operations to run in a fully automated and non-interactive manner.
Conceptually, this means:

• Using the Sequence Analyzer in an early stage to validate sequence files and enforce development standards.
• Using the Sequence Editor or Operator Interfaces (via command line) to execute test sequences and generate results.
• Using the TestStand Deployment Utility to build and package deployments as part of release or distribution stages.

By chaining these command-line operations inside pipeline stages, Jenkins can manage the flow of actions, evaluate results (such as exit codes), and determine whether the pipeline should continue or fail. This provides a consistent mental model where each TestStand tool plays a specific role in the CI/CD lifecycle, and Jenkins acts as the orchestrator that automates and coordinates them.