Understanding Numeric Formatting in Teststand Reports and Database Logs

Issue Details

While working with TestStand, I noticed that numeric values were displayed differently depending on whether they were included in an ATML report or logged to a database. In ATML reports, the numbers appeared with noticeably less precision, while the same values in the database retained full detail. 

Solution

The inconsistency in numeric formatting between ATML reports and database logs in TestStand is caused by how the NumericFormat property is applied to numeric PropertyObjects. This property, if set, overrides the default formatting defined in the Report Options dialog and directly affects how numeric values are rendered in ATML reports. In contrast, the database plugin does not apply this property when storing values in the DATA column, though it may log the format string in the DISPLAY_FORMAT column.

To ensure consistent numeric formatting across both ATML and database outputs, users can follow these best practices:

1. Avoid setting NumericFormat on individual variables unless specific formatting is required. These overrides can be difficult to trace and may lead to unexpected results.
2. Use the Report Options callback to define a global numeric format:
   • Add a statement step in the ReportOptions callback to set:

     Parameters.ReportOptions.NumericFormat = "%g"
     

3. For ATML reports, configure the plugin to use a consistent high-precision format:
   • Set ModelPlugin.PluginSpecific.Options.NumericFormat = "%$.15g"
4. Remove conflicting format settings before report generation:
   • Traverse the MainSequenceResults tree and clear the NumericFormat property from all numeric PropertyObjects by setting it to an empty string.
5. Understand plugin behavior:
   • The ATML plugin uses NumericFormat if present; otherwise, it defaults to the global setting.
   • The database plugin stores numeric values with high precision regardless of NumericFormat, but may log the format string separately.

By applying these steps, users can maintain consistent numeric formatting across different report types and avoid precision loss or formatting discrepancies.

Additional Information

• The NumericFormat property is accessible via the Sequence Editor by right clicking a variable and selecting Numeric Format.

• In the database schema, PROP_RESULT.DATA stores numeric values as strings with high precision, independent of NumericFormat.

• The DISPLAY_FORMAT column may contain the format string from NumericFormat, but it does not affect the stored value.