ChnLinGen Generates Data Points That Are Slightly off From Expected Value in DIAdem

Updated Jun 15, 2023

Reported In

Software

  • DIAdem

Issue Details

I am using the ChnLinGen function in DIAdem to create a channel with equidistant values but instead of exact values like -1, -0.8, -0.6, -0.4, I am receiving a channel with the following values: -1, -0.79999..., -0.6, -0.39999... Is this expected behavior? How can I create a channel with the values I want? 

Solution

This is a result of digital computing. At different circumstances (step width, number of values to be generated, etc.) this effect – jumping of the last bit - can occur. In DIAdem you can use a rounding function after you have generated the channel like below to prevent this effect. 
Set ChnResult = ChnRound("[1]/Time", "Standard", "Automatic", 6, False)

Additional Information

Digital computing and this effect can be seen in other applications as well. For instance, if you go to Excel and generate such an array, you will get the same effect.

Other Support Options

Ask the NI Community

Collaborate with other users in our discussion forums



Request Support from an Engineer

A valid service agreement or active software subscription may be required, and support options vary by country.