Modifying the Failure Chain of the HTML Report to Include All Failed Steps

Updated Dec 3, 2024

Environment

Software

  • TestStand

TestStand contains a failure chain section in the header of the HTML and ASCII text reports. The first item in the failure chain shows the step whose failure caused the UUT to fail, the remaining items show the sequence call steps through which the execution reaches the failing step. For a majority of users this information will be sufficient. However, some users may need to have all failed steps displayed in the failure chain. This document outlines one of the possible ways that can be implemented to expand the failure chain to include all failed steps in the HTML report. 

Part A: Adding a Local Variable

  1. Copy the file %teststand%\Components\Models\TestStandModels\reportgen_html.seq to %teststandpublic%\Components\Models\TestStandModels\. 
  2. Open the sequence file reportgen_html.seq under %teststandpublic%\Components\Models\TestStandModels\.
  3. Navigate to PutOneResultInReport_lmpl at the sequence pane.
    teststand sequence pane.png  
  4. Add a new local variable of type Number to this sequence and name it NumElements. This variable will be used to track the total number of elements within the Critical Failure Stack.


NOTE:

  1. If you are using TestStand 64 bit, you may replace %teststand% to %teststand64% to easily search for the sequence file.. 
  2. Remember not to modify contents of the %teststand%\Components\Models directory. Always copy the components you want to customize to the %teststandpublic%\Components\Models\ directory. TestStand will automatically use the files from the User directory over those in the NI directory.

 

Part B: Creating the Add All Failed Step and Modifying its Preconditions

The Update Failure Stack step is used to add elements to the Critical Failure Stack array.

  1. Click the Update Failure Stack step and select Properties from the step settings. Examine the Preconditions. Notice that this step is set to run if the following expression is true.
    Parameters.FailureChainActive && PropertyExists("Parameters.Result.TS.StepCausedSequenceFailure") && Parameters.Result.TS.StepCausedSequenceFailure

    There are actually three conditions in this expression that must be true in order for the step to execute. The condition shown on the third line above is true only if the current step being documented is the one that caused the sequence to fail. This portion of the expression, Parameters.Result.TS.StepCausedSequenceFailure, is a boolean that is only set to True for a failed step that caused the sequence to fail.
    teststand update failure stack.png 
  2. Add another step that will add all failed steps to the Critical Failure Stack array by following the steps below:
    1. Make a copy of the Update Failure Stack step and paste it directly below the original.
    2. Rename this step to Add All Failed Steps.
    3. Modify the preconditions of Add All Failed Steps to run only when the preceding Update Failure Stack step is not executed and the status of the step being documented is set to Failed.
    4. Click the Add All Failed Steps step and select Properties from the step settings.
    5. Navigate to the Preconditions properties.
    6. Click the Precondition Builder button and modify the preconditions expression.
      teststand add all failed steps.png
      1. Click Insert AllOf from the Precondition Builder window.
      2. Click the Update Failure Stack on the Insert Step Status box and make sure to enable the Negate box. Click Insert Step Executed. You will now see that the condition has been updated to NOT EXECUTED Update Failure Stack.
      3. Click on Insert New Expression to insert a new line on the Condition box.
      4. At the Edit/View Expression box, key in the below expression:
        Parameters.Result.Status == "Failed"
      5. Click OK to close the Precondition Builder window.
        teststand precondition builder.png 
    7. Add "&& (Locals.NumElements < 1)", without the quotes, to the end of the preconditions of the step Update Failure Stack.  This ensures all failing steps will be shown if there are subsequences being called from your MainSequence. 

 
Part C: Modifying the Expression of the Add All Failed Steps Step

To modify the expression of the Add All Failed Steps step follow the steps below:

  1. Click the Add All Failed Steps step and select Expression tab on the step settings pane.
  2. Modify the expression control value of this step to contain the following expressions separated by a comma.
    Locals.NumElements = GetNumElements(Parameters.UUT.CriticalFailureStack),

SetNumElements(Parameters.UUT.CriticalFailureStack, Locals.NumElements + 1),

Parameters.UUT.CriticalFailureStack[Locals.NumElements].StepName = Parameters.Result.TS.StepName,

Parameters.UUT.CriticalFailureStack[Locals.NumElements].SequenceName = Parameters.SequenceName,

Parameters.UUT.CriticalFailureStack[Locals.NumElements].SequenceFileName = Parameters.SequenceFileName,

Parameters.UUT.CriticalFailureStack[Locals.NumElements].ResultId = Parameters.Result.TS.ID，

Parameters.Result.TS.StepCausedSequenceFailure = True

    NOTE    : For cosmetics, you can add returns within a single expression by pressing the <Ctrl> and <Enter> keys together.
With these expressions, you keep track of the current size of the Critical Failure Stack array and add new elements each time this step executes, which is when the previous step has not executed and the status of the step from your test sequence is set to Failed.


Part D: Modifying the Loop Options

Modify the loop options of the step Add Failure Stack Items located in the sequence AddReportHeader of the reportgen_html.seq sequence file following the steps below.

TestStand loops on this step as many times as there are items in the Critical Failure Stack array. Currently, the loop index of this step is set to decrement down to zero as the step loops. You need to modify this so that the loop index starts at 0 and increments as the step loops, otherwise all of the steps that failed will be displayed in reverse order, last step to first step, in the Failure Chain.

  1. Click the step Add Failure Stack Items and select Properties from the step settings.
    teststand add failure stack utems_Looping.png 
  2. Select the Looping tab and modify it as below.
    1. Make sure to select the Loop type as Custom and check the box for Record Result of Each Iteration.
    2. Set the Loop Initialization Expression as
      RunState.LoopIndex = 0
    3. Set the Loop Increment Expression as
      RunState.LoopIndex += 1
    4. Set the Loop While Expression as
      RunState.LoopIndex <= GetNumElements(Parameters.UUT.CriticalFailureStack) - 1
    5. Set the Loop Status Expression as
      "Done"

 

Part E: Configure the Result Processing

 

Follow the below steps to configure the result processing correctly to make sure that this reportgen_html.seq sequence file can work correctly with the test sequence.

  1. Select Configure >> Result Processing to launch the dialog box.
  2. Click the Options icon on the Result Processing dialog box to launch another Report Options dialog box.
    teststand result processing.png 
  3. Choose the report format as HTML_Document and select Sequence (Easier to modify, and calls ModifyReportEntry callbacks) on the Select a Report Generator for Producing the Report Body option.
    teststand report options.png 

 

Save the sequence file and run your test sequence by selecting Execute»Test UUTs or Execute»Single Pass. The failure chain in the report header should now contain links to all steps and subsequences that have failed.

 

Additional Notes:

This is not the only possible solution to add all the failure chains in TestStand report. There are several ways to do so and the user needs to be familiarized with the process model and report generation process in TestStand.

Check out this article: TestStand Report Generation and Customization to understand what are the options and methods available to customize the report generation.

Related Links

Other Support Options

Ask the NI Community

Collaborate with other users in our discussion forums



Request Support from an Engineer

A valid service agreement or active software subscription may be required, and support options vary by country.