Introduction
Continuous Integration (CI) allows for the automation of testing, building, and releasing software. A CI setup includes a shared repository which developers can push changes to, and a CI server to automate test, build, or other processes whenever those changes are made.
In this tutorial, we’ll walk through setting up Jenkins as our CI server and using it to automate graphical diffing and testing of VIs that we push to GitHub, our shared repository. Jenkins will monitor GitHub Pull Requests for changes and will comment and check if the LabVIEW project can build successfully. Jenkins and GitHub are not the only options, and they may not be the best options for your needs.
We are using them today because they are commonly used in industry and are free and simple to set up. The steps in this tutorial can be modified for your specific Continuous Integration or Source Code Management systems.
Software Required:
Note: This tutorial uses Jenkins on a Windows machine. This means that file paths and commands may be Windows-specific. Please modify any file paths and/or commands to your specific Operating System.
LabVIEWCLI Setup
The
LabVIEW Command Line Interface (LabVIEWCLI) is installed alongside LabVIEW 2018+ and is used to run LabVIEW commands, such as unit testing, through the Command Line. To use the LabVIEWCLI, you must enable VI Server:
Tools >> Options >> VI Server. Make sure the TCP/IP checkbox is enabled and leave the Port as 3363.
Python Setup
The Python
requests library must be installed in order for Jenkins to post results to GitHub. One way to do this is by opening a command prompt and running the following command:
pip install requests
GitHub Setup
Repository Setup
We’ll create three GitHub repositories. Each of these repositories will be used for one of the following:
- Host the script files, library files, and infrastructure items needed for the build server.
- Host the actual example application.
- Host images used during the VI diffing step.
To create these repositories:
1. Create or log in to your GitHub account. Make sure your email is verified.
2. Click on the New button next to Repositories:
3. On the
Create a new repository screen, fill out the Repository name and Description:
4. Click Create repository to create the repository.
5. Repeat steps 2-4 using the below information. This will be the repository for the actual example application.
6. Repeat steps 2-4 with the following information. This will be used to host images for diffing.
7. Open a terminal with Git. If you installed Git Bash on Windows, open Git Bash:
8. Create a local directory to maintain your Jenkins files. For this example, we’ll be using the C:/Users/<User>/Documents/GitHub directory. To create and navigate to this directory, run the following commands:
cd “C:/Users/<User>/Documents”
mkdir GitHub
cd GitHub
9. Run the following command (using the URL copied from Step 5) to copy the Git repository and its contents onto your machine:
git clone https://github.com/<Organization Name>/<Repository Name>.git
You can also obtain the HTTPS URL of the repository by clicking on the Copy button on the repository page:
10. Repeat Step 9 for the myApplication repository.
11. In Windows Explorer, place the contents of ../myBuildsystem from the example files located here in the ../myBuildSystem directory located on your machine:
12. Navigate to the directory in the Git terminal, and run the following command to add all files within the directory to the Git repository:
cd “C:/Users/<User>/Documents/GitHub/<Repository Name>”
git add .
13. Run this command to commit all changes and add a comment:
git commit -m “Added files”
If you can’t run the command because you’re missing credentials, you can set them:
14. Run this command to push all committed changes to the remote repository (i.e. GitHub):
git push origin master
Now, the repository contains all necessary components we will need for this example.
15. Next, we’ll perform similar steps to set up the example application repository. In Windows Explorer, place the contents of ../myApplication from the example files located here in the ../myApplication directory located on your machine:
16. Open up ‘Jenkinsfile’ in the ../myApplication directory. Change the lvVersion and lvBitness variables according to which version of LabVIEW you are using. For instance, with LabVIEW 2018 32-bit, your Jenkinsfile will look like this:
17. Run the commands from steps 12-14, this time for the myApplication repository:
Now, your files are set up and in place for Jenkins to use.
Personal Access Token
For Jenkins to access your GitHub information, it will need a Personal Access Token. We will generate Personal Access token through GitHub:
1. On the GitHub website, click on your Avatar >> Settings
2. On the left-hand side of the Settings page, click on Developer Settings:
3. Click on Personal access tokens >> Generate new token.
4. Enter a descriptive name for the token, such as ‘Jenkins LabVIEW Token’, and select the repo scope.
5. At the bottom of the page, click Generate token.
6. Copy down your Personal access token. This is your only chance to copy the token. Keep your access token safe, so others can’t access your GitHub.
Congrats! You’ve set up the GitHub repositories you’ll need for this example, and you have the personal access token so Jenkins can access your GitHub information. Next, we’ll configure Jenkins to use these items.
Jenkins Setup
Jenkins Initial Setup
Note: This tutorial uses the classic Jenkins view, with minimal plugins to get the example running. You can use additional plugins such as Blue Ocean for an enhanced UI experience.
1. Install Jenkins based on your Operating System using the official Jenkins help, and navigate to the address of your Jenkins server in your browser. A few additional notes (all of which are already mentioned in the aforementioned Jenkins help):
a. The default Jenkins address is http://localhost:8080
b. The first time you launch Jenkins, you will need to unlock it with the admin password from the Console log output
2. When prompted to Customize Jenkins, select the Select plugins to install option. Here, make sure the following Plugins are selected:
a. GitHub Branch Source
b. Folders Plugin
c. Pipeline: GitHub Groovy Libraries
You can leave the other Plugin options as-is.
3. Click Install at the bottom of the page to install the Plugins.
4. Once plugins are complete, you’ll be prompted to create a user. Enter your credentials and click Save and Continue.
5. On the following Instance Configuration page, leave the Jenkins URL as-is.
6. Click Start using Jenkins to begin your Jenkins configuration.
Setting up environment variables and credentials
Environment variables can be accessed across Jenkins jobs. We will want certain values accessible when Jenkins tries to build, test and diff your application. Since the GitHub organization name, access token, and picture repository will likely be the same across Jenkins jobs, we will set them in our Jenkins configuration.
1. From the Jenkins dashboard, navigate to the system configuration page: Manage Jenkins >> Configure System
2. Navigate to the ‘Global properties’ section.
3. Check the Environment variables box to display the List of variables.
4. Click Add to add a new environment variable. Fill out the Name as shown below (BUILD_SYSTEM_REPO), and enter 'myBuildSystem' in the Value field:
5. Click Add another time and add the LV_BUILD_OUTPUT_DIR environment variable. This should be an empty local directory anywhere on your computer, that LabVIEW will build and output files to. For instance, you may use something like C:\Windows\Temp\jenkins.
6. Click Add two more times to add the ORG_NAME variable and the PIC_REPO variable. The value fields for these variables should be your organization’s name (instead of ‘branchNI’), and myPicRepo, respectively. These are the two repositories we created in the GitHub Setup section. Your Environment variables section should look like this:
7. Click Save at the bottom of the screen to save your changes.
8. Now, navigate to the Credentials page: Jenkins Dashboard >> Credentials >> System >> Global credentials >> Add credentials
9. Use the following settings, entering your GitHub access token as the 'secret' field:
Setting up the Global Library
The Global Library contains the script files and other components that will be used each time Jenkins tries to build. In this example, we are hosting them in the myBuildSystem repository. We will link Jenkins to that repository so it can use those files for each job.
1. On the main Jenkins dashboard: Manage Jenkins >> Configure System
2. Under the ‘Global Pipeline Libraries section’, click Add and fill out the credentials for the myBuildSystem repository. Make sure to check the Load implicitly checkbox. Your options should look like this:
Here, ‘master’ refers to the master branch of the repository. The steps we performed in the GitHub Setup section pushed all files to the ‘master’ branch. More info on GitHub flows here.
3. Now, we’ll want to select the Retrieval method. For this example, we’ll use Modern SCM. Select the Modern SCM radio button, followed by the GitHub radio button under ‘Source Code Management’:
4. Start by adding Credentials. Click the Add dropdown next to the credentials field >> Jenkins.
5. On the Jenkins Credentials Provider screen, leave the default options, and enter your GitHub Username. The Password will be your GitHub Access Token. Click Add once the credentials are filled out.
6. Once the credentials are created, select them from the Credentials dropdown.
7. Fill out the ‘Owner’ field and select ‘myBuildSystem’ from the ‘Repository’ dropdown. Leave the other options as-is. Your fields should look like this:
8. Click Save at the bottom of the screen to save your changes.
We’ve successfully linked Jenkins to the global library – this means that the files hosted on the myBuildSystem repository can be accessed and used by Jenkins!
Setting up the Jenkins pipeline
Finally, we’ll set up Jenkins to scan and automate testing and diffing of your myApplication repository.
1. From the Jenkins dashboard, select New Item. Enter a name for the item and select GitHub Organization.
2. Click OK to create the item. You will be redirected to the configuration page.
3. Under the ‘Projects’ section:
a. Select your ‘Credentials’ from the dropdown menu and change the value of the ‘Owner’ field to your GitHub organization name.
b. In ‘Behaviors’, change the Discover branches option to Only branches that are also filed as PRs.
4. Make sure the ‘Script Path’ value is Jenkinsfile. Your ‘Projects’ section should look like this:
5. Under the ‘Scan Organization Triggers’ section, select the Interval that you want Jenkins to scan your repository for changes. If Jenkins detects a change, it will initiate the testing and diffing process. For this example, we will use 10 minutes.
Note: While the method used in this tutorial sets up Jenkins to scan GitHub for changes, there are methods for GitHub to trigger Jenkins builds whenever a change happens. This involves exposing your Jenkins server so that GitHub can communicate with it through WebHooks.
6. Click Save to save your changes. Through this pipeline, Jenkins will now scan your repository based on the interval you just configured.
Confirmation
Let’s confirm that everything works. We’ll do this by configuring our LabVIEW project, making changes to the VIs in our myApplication directory, and creating a Pull Request on GitHub.
1. We will create a branch to the ‘myApplication’ GitHub repository. Start by opening a Git terminal and navigating to the local ../’myApplication’ directory.
2. To create and checkout or use a branch, enter the following commands:
git branch myBranch
git checkout myBranch
3. Navigate to the ../myApplication/source directory.
4. Open the Jenkins.lvproj file.
5. Expand the Build Specifications, and double click myBuildSpec to open its properties:
6. Modify the ‘Destination directory’ to the specific directory you set for the LV_BUILD_OUTPUT_DIR environment variable value, in the Jenkins Setup section. This will be the output directory of the build specification.
7. Next, we will modify the VIs. First, open Add.vi. This is a simple VI that adds two numbers together and returns a result.
8. Make some cosmetic changes to Add.vi (don’t change anything that would alter the functionality of the VI), like so:
9. Click File >> Save to save the VI changes, and close Add.vi .
10. Now, open Subtract.vi. This is a simple VI that subtracts two numbers and returns a result.
11. Make cosmetic changes to Subtract.vi; for example:
12. Click File >> Save to save changes, and close Subtract.vi .
13. Close Jenkins.lvproj. Save files if prompted.
14. With the Git terminal open, make sure we are still in the ../myApplication directory. Run the following commands to push our changes to the ‘myBranch’ branch on the ‘myApplication’ repository:
git add .
git commit -m “Cosmetic VI changes”
git push origin myBranch
15. With our changes pushed to the Branch, we’ll go on GitHub to create a Pull Request. First, navigate to the ‘myApplication’ repository on https://www.github.com.
16. Navigate to the ‘myBranch’ Branch using the dropdown:
17. On the ‘myBranch’ page, click on New pull request.
18. Fill out the pull request details as shown. Click Create pull request to create the request.
19. Wait for Jenkins to perform its check based on the interval you set, or navigate to your Jenkins dashboard >> GitHub Builder (or whatever you named your Jenkins pipeline) >> Scan Organization Now. Then, navigate to myApplication>> Scan Repository Now.
20. To view the progress of your job, you can navigate to the Pull Requests tab.
21. Click on the pull request name (‘PR-1’ in this case). The progress of each step is displayed on the main page. The ‘Build History’ section on the bottom left displays the jobs run.
22. If the job was successful, each stage should be Green, and the latest Build in the ‘Build History’ section should have a blue dot next to it. Additionally, you’ll see the latest artifacts (files you can upload to Jenkins), and the unit test results:
23. If you navigate to your Pull Request on GitHub (Repository Page >> Pull requests tab >> Click on the Pull Request), you should also see the comment that Jenkins posted.
24. If you’re satisfied with the results, you can scroll to the bottom of the Pull Request and click Merge pull request to merge the changes with the master branch.
25. If the job was not successful, on the Pull Request page in Jenkins, click on the build number that failed, and click on Console Output on the left sidebar to view error information.
