A customer is using the PXIe-4082, a 6 1/2 digit DMM. They expect the measurement data to be in the format of 6.5 digits, such as 'X.XXXXXX5' or 'X.XXXXXX0'. However, they are receiving more than 6.5 digits.
The reason you see 15 digits is because the DBL (Double Precision Floating Point) format has a precision of 15 digits. However, as mentioned, the absolute resolution is 5 µV for a 6 1/2 digit resolution.
Collaborate with other users in our discussion forums
A valid service agreement or active software subscription may be required, and support options vary by country.
Helpful
Not Helpful
