Overview of the TDM Data Model
The TDM file format specifies that descriptive information such as comments, units, and channel names are saved in a header file with the extension TDM, while the measurement, simulation or analysis results are saved in one or more binary file(s). The TDM and binary files are linked through references that are created automatically when using the LabWindows/CVI DIAdem Connectivity Library, LabVIEW Data Storage VIs, or programmatically when using the TDM Header Writer C API.
TDM files use XML to manage and structure all the descriptive information contained in the TDM header file. Each TDM file contains descriptive information on three different levels – the root, group, and channel, as illustrated in Figure 1.
Figure 1. Hierarchical Structure of a TDM File
One TDM Root object, also referred to as file, can contain several Groups, and each group can contain several Channels. The TDM data model has a defined set of properties at each of the three levels while also providing functionality for you to insert your own custom properties.
The descriptive information located in the TDM header file is a key benefit of this format because it provides an easy way to document your data without having to design a custom header structure. As your measurement requirements grow, you do not have to redesign your application; you simply extend the TDM data model to meet your expanding needs.
The division between header information and measurement data makes the TDM files very flexible by giving you the opportunity to acquire data using multiple methods capable of storing binary data and later document the information with a TDM header file. The TDM Header Writer C API provides an easy-to-use interface to create the TDM header file.
It is common for standard binary files to contain measurement information from different data channels acquired at different rates and stored with different data types. TDM files can store data in signed or unsigned integers ranging from one to eight bytes in length, and single and double-precision floating-point numbers (Note: Although the TDM header file can describe the previously mentioned data types, LabView versions prior to 8 and DIAdem versions prior to 10 will not read 64-bit integers and 8-bit signed integers.). The data in the binary file can be stored in both big and little endian byte order. The ability of the TDM header file to accommodate mixed data rates and data types will be shown in the following examples.
Creating a TDM Header File for a Single-Channel Binary File
One of the characteristics of TDM files is that they are capable of storing both very simple and complex data sets. One of the simplest binary files used to create a TDM header file is a single-channel binary file. This file consists of a single block of binary data. A block is a set of contiguous data points describing the value of one or more channels. The easiest way of creating such a file is to use the ArrayFile example that is shipped with LabWindows/CVI. Running this example will generate a binary file with 100 double-precision floating-point numbers.
The code for creating a TDM header for the single-channel binary file can be found in Figure 2.
#include <tdmwriter.h>
int main(int argc, char* argv[])
{
// Declare the TDM File's State, Error and Binary Channel Index
tdmwState* state = NULL;
tdmwError error = TDMW_NOERROR;
uInt32 channelIndex = NULL;
// Create a TDM File
state = tdmwOpen(L"TDM_File.TDM",&error);
// Link Binary File to Newly Created TDM File
tdmwSetupFile(state, L"BinaryFile1.dat", TDMW_LITTLE_ENDIAN, &error);
// Define the Format of the Binary File
channelIndex = tdmwSetupChannel(state, 0, 100, TDMW_T_FLOAT64, &error);
// Create Root Element
tdmwCreateRoot(state, L"Root", 1, &error);
// Create Channel Group
tdmwCreateChannelGroup(state, L"Group Name", 1, &error);
// Create Channel
tdmwCreateChannel (state, L"Channel 1", channelIndex, &error);
// Close TDM File
tdmwClose(state,&error);
return 0;
}
Figure 2. Create a TDM Header File for a Single-Channel Binary File
The first step in creating a TDM header file is to create the file itself. The tdmwOpen function will create a TDM file in the same directory where the executable is located. The file name is set by the tdmwOpen function’s first argument. As an output, this function returns a reference pointing to the newly created file, stored in the state variable, that will be used by future functions when creating the TDM file. The function also outputs its error code through the third argument. This error value will be used by all the functions that create the TDM header file. If the value of the error variable is not equal to TDMW_NOERROR none of the functions of the TDM Header Writer C API will execute. The only exception is tdmwClose that will execute even if there was an error in order to close the reference to the file.
Using the reference to the TDM file, the tdmwSetupFile functions links the binary file containing the data we want to describe with the TDM header file. This function has four parameters, the TDM file reference, the name of the binary file, byte order of binary data and the error output. TDM files support binary files stored both in big and little endian byte order. In the case of the binary file created by the ArrayFile example, the file is stored in little endian byte order. Therefore, we will use the TDMW_LITTLE_ENDIAN constant as an argument for the byte order parameter.
Now that we have linked the binary data to the TDM header file we must specify the format of the binary data. The tdmwSetupChannel function defines the format of the binary file by accepting the following arguments, the TDM file reference, byte offset, number of samples, data type and error. The byte offset specifies the number of bytes at the beginning of the file that must be ignored before reaching the measurement data. This functionality facilitates the use of existing binary files because you can ignore headers written at the beginning of the file. The Number of Samples parameter specifies how many samples are stored in the file. The Data Type parameter determines in which format the data was stored for this particular channel. In the case of the ArrayFile example, the data was stored as a double-precision floating-point number. The tdmwSetupChannel function returns a binary channel index for the channel whose format we specified. This index will be used later when we define the properties for the channel.
Using the reference for the TDM header file, the tdmwCreateRoot function creates the file Root object, defines its name and sets the number of groups which will eventually be created. The next level of the TDM file structure is the Group. Because we have only one Group in the above example, we will call the tdmwCreateChannelGroup function once. If we had more than one Group, the function would have to be called once for each group. The four parameters for this function specify its TDM file reference, name, number of channels and error output.
The last step in creating the TDM header file is to specify the properties for channels. This operation is performed by the tdmwCreateChannel function. This function parameters define the TDM file reference, channel name, binary channel index and error. The binary channel index was created previously when we defined the format of the binary file using the tdmwSetupChannel function. Finally, to complete the creation of the TDM file, we must close the file using the tdmwClose function.
Creating a TDM Header File for a Multichannel Binary File
One of the great advantages of TDM header files is that they can describe many different types of data sets. The most common use-case for TDM header files is documenting binary data acquired from multiple channels.
In order to link data from multiple channels in a single binary file to a TDM header, the channel data must be stored either end to end or interleaved. The most common case is data from multiple channels stored in a single, interleaved binary block. This binary block will not only be described by its length, the number of samples per channel, but also by its width, the number of bytes containing one sample for all channels. Figure 3 illustrates the structure of a binary file containing four channels and n samples.
Figure 3. Structure of an Interleaved Binary Block
In this case the binary block length is n. If the data type of all of the channels was an 8-byte double-precision floating-point, the block width would be 32 bytes. Note that the channels in the same binary block may be stored with differing data types. Note also that the channels must be consistently interleaved in consecutive order.
One of the most common ways of acquiring data from multiple channels using C is to use the DAQmxReadAnalogF64 function of the NI-DAQmx C API. The DAQmxReadAnalogF64 function can output data both end-to-end and interleaved. The output formatting is set to interleaved by the inputing a value of DAQmx_Val_GroupByScanNumber to the Fill_Mode parameter of the function. In the following example we will acquire 25 samples from each of four channels using the NI-DAQmx C API in LabWindows/CVI and save them to a binary file in interleaved format. (Note: TDM header files describing binary files in interleaved format can be read only in LabVIEW versions 8 or later and DIAdem version 9.1 SP2 or later.).
The process of creating a TDM header file for a binary file containing multiple channels is very similar to creating one for a binary file with a single channel. The first of two differences are that instead of calling the tdmwSetupChannel once to define the format of the binary file, we will call the tdmwSetupBlockChannel once for each of the channels in the file. The second difference is that instead of calling the tdmwCreateChannel once to define the properties of a single channel, we will call it one time for each of the channels in the file.
The first two steps of creating the TDM header file for a binary file with multiple channels will be the same as those used to create a TDM header file for a binary file with a single channel, as can be seen in Figure 4.
#include <tdmwriter.h>
int main(int argc, char* argv[])
{
// Declare the TDM File's State and Error
tdmwState* state = NULL;
tdmwError error = TDMW_NOERROR;
// Declare Channel Indeces and Channel Names
uInt32 channelIndex[4];
wchar_t channelName[10];
// Declare Number of Channels and Size of Each Sample in Bytes
int channelCount = 4;
int sampleSize = 8;
// Create a TDM File
state = tdmwOpen(L"TDM_File.TDM",&error);
// Link Binary File to Newly Created TDM File
tdmwSetupFile(state, L"Multi_Channel_Binary_File.dat",
TDMW_LITTLE_ENDIAN, &error);
// Define the Format of the Binary File
for (int i = 0 ; i < channelCount ; i++)
{
channelIndex[i] = tdmwSetupBlockChannel(state, 0,
channelCount * sampleSize,
i * sampleSize,
25, TDMW_T_FLOAT64,
&error);
}
// Create Root Element
tdmwCreateRoot(state, L"Root", 1, &error);
// Create Channel Group
tdmwCreateChannelGroup(state, L"Group Name", 4, &error);
// Create Channels
for (i = 0 ; i < channelCount ; i++)
{
swprintf(channelName, L"Channel %d", i + 1);
tdmwCreateChannel (state, channelName, channelIndex[i], &error);
}
// Close TDM File
tdmwClose(state,&error);
return 0;
}
Figure 4. Create a TDM Header File for a Multichannel Binary File
The third step of the process involves using the tdmwSetupBlockChannel function instead of tdmwSetupChannel. The tdmwSetupBlockChannel is used because with it we can specify the size of a block in the binary file as well as the relative position of the channels in the block. The function is called once for each of channels in the file, therefore it is in a For loop that iterates once for each channel. The first two parameters of the tdmwSetupBlockChannel are the TDM file reference and the block offset. The block offset determines the number of bytes from the beginning of the file where the first block of data is located. The next parameter is the block width in bytes. The fifth parameter determines the file block length.
All of the previous parameters will have the same values for all the channels in our file. The fourth parameter, Channel Byte Offset, will be different for each channel. This parameter specifies the number of bytes from the beginning of the file to the first sample of a certain Channel. The last two parameters determine the data type that the samples were stored in and return the error for the function. The tdmwSetupBlockChannel function outputs a block channel index for each of the channels whose format we specified. These four Block Channel Indices are appended to an array that will be used later by the tdmwCreateChannel function to set the channels’ properties.
There is very little difference between the last four steps of creating a TDM header file for a multichannel and a single-channel binary file. The TDM file only has one root object so this is created in the same way, and because we created only one group, this operation is identical except that the Number of Channels parameter is now 4. The main difference lies in that the tdmwCreateChannel function is called once for each channel in the binary file. The block channel indices coming from the tdmwSetupBlockChannel function are accepted as a parameter in the tdmwCreateChannel function.
Creating a TDM Header File for Multiple Binary Files
One of the biggest drawbacks of using binary files is that any documentation or header information is limited to the file where it resides. TDM header files are much more flexible and enable you to document and organize different binary files into a cohesive structure. These binary files can have different samples counts, byte order, formatting, and data types. Figure 5 demonstrates how to create a TDM header file for two different binary files. One file contains a block of binary data for a single channel of data in double-precision floating-point format and the second a block containing a single channel of data in 32-bit integer format.
#include <tdmwriter.h>
int main(int argc, char* argv[])
{
// Declare the TDM File's State, Error and Channel Indeces
tdmwState* state = NULL;
tdmwError error = TDMW_NOERROR;
uInt32 channelIndex[2];
// Create a TDM File
state = tdmwOpen(L"TDM_File.TDM",&error);
// Link Binary File to Newly Created TDM File
tdmwSetupFile(state, L"Single_Channel_DBL.bin", TDMW_BIG_ENDIAN, &error);
// Define the Format of the Bianary File
channelIndex[0] = tdmwSetupChannel(state, 0, 10, TDMW_T_FLOAT64, &error);
// Link Binary File to Newly Created TDM File
tdmwSetupFile(state, L"Single_Channel_I32.bin", TDMW_BIG_ENDIAN, &error);
// Define the Format of the Bianary File
channelIndex[1] = tdmwSetupChannel(state, 0, 10, TDMW_T_INT32, &error);
// Create Root Element
tdmwCreateRoot(state, L"Root", 2, &error);
// Create Channel Group
tdmwCreateChannelGroup(state, L"Group 1 Name", 1, &error);
// Create Channel
tdmwCreateChannel (state, L"Channel 1", channelIndex[0], &error);
// Create Channel Group
tdmwCreateChannelGroup(state, L"Group 2 Name", 1, &error);
// Create Channel
tdmwCreateChannel (state, L"Channel 2", channelIndex[1], &error);
// Close TDM File
tdmwClose(state,&error);
return 0;
}
Figure 5. Create a TDM Header File for Multiple Binary Files
The process of building a TDM header file for multiple binary files follows the same structure as creating one for a single binary file. First, the TDM header is created using the tdmwOpen function. Then the tdmwSetupFile function is used to link the first binary file to the TDM header. After linking the binary file we define its format using the tdmwSetupChannel function which will return a Binary Channel Index that is used later by the tdmwCreateChannel function. The big difference when building a TDM header file for two binary files is that the tdmwSetupFile and tdmwSetupChannel functions must be called twice, once for each file. It is necessary that the tdmwSetupChannel function be called immediately after the tdmwSetupFile function for each file. The two previously mentioned functions must also be called consecutively for each of the files.
To assign properties to the binary file in the TDM header file, we will use the same three functions used in the previous two examples. The process is the same as assigning properties for a single file, except that in this case we will be creating two groups, one for each binary file, instead of one. We will call the tdmwCreateGroup function followed by the tdmwCreateChannel function consecutively for each binary file. It is required that these two functions be called in that order, otherwise the TDM header file is not created correctly. The binary channel indices created by the two tdmwSetupChannel functions are used by their respective tdmwCreateChannel function. Finally, the TDM header is closed by the tdmwClose function.
Creating a TDM Header File and Setting Properties
Using TDM files, you can specify predefined and custom properties for your data at different hierarchical levels. This functionality makes TDM files excellent for documenting large data sets with expanding documentation needs. Using the TDM header writer C API, you can assign and create properties programmatically as Figure 6 illustrates.
#include <tdmwriter.h>
int main(int argc, char* argv[])
{
// Declare the TDM File's State and Error
tdmwState* state = NULL;
tdmwError error = TDMW_NOERROR;
// Create a TDM File
state = tdmwOpen(L"TDM_File.TDM",&error);
// Create Root Element
tdmwCreateRoot(state, L"Root", 1, &error);
// Assign Value to a Root Property
tdmwAddPropertyString(state, TDMW_P_REGISTERTXT1,
L"1:23:45.678 PM 1/2/2003", &error);
// Create Channel Group
tdmwCreateChannelGroup(state, L"Group Name", 1, &error);
// Create and Assign a Custom Property
tdmwAddCustomPropertyDouble(state, L"Applied_Voltage", 1.23, &error);
// Close TDM File
tdmwClose(state,&error);
return 0;
}
Figure 6. Create a TDM Header File for Multiple Binary Files
The first two steps of assigning properties to a TDM file are to create a TDM file and a Root element. To set an existing property, use the tdmwAddProperty functions. The TDM header writer C API has a function for each of the four data types available for properties. In this case, we will use tdmwAddPropertyString function to store property as a string. This function accepts four arguments, the TDM file reference, the name of the property, its value and an error output variable. This function can be used to set properties at either the Root, Group or Channel level. The tdmwAddProperty function must follow the function creating the hierarchical level that the property will be describing. The next step in the sequence involves creating a Group. If you want to create a custom property you can use one of the tdmwAddCustomProperty functions. Each of the four functions in this group is used to create a custom property with a different data type. In Figure 6 we created a custom property as a double so we used the tdmwAddCustomPropertyDouble function. The four arguments used by the function are the TDM file reference, the name of the property to be created, its value and an error output variable. Like the tdmwAddProperty* function, the tdmwAddCustomProperty function must be called following the creation of the hierarchical level to which the property will be assigned. The last step of the process closes the TDM file.