Removing Invalid Data Spikes in DIAdem

Updated Sep 5, 2018

Reported In

Software

  • DIAdem

Issue Details

I have a data file I want to process using DIAdem, but I have spikes of invalid data I would like to remove before processing the rest of the data. Is there a way to filter this data?
 

Solution

You can use the Even Search functions functions to set the spikes to a specific value and if required, you can combine them with the Flag Functions to remove them from the channel instead.

Note: if the spikes are unipolar, skip the Event Search Functions section. It helps making simpler to delete all the wrong data points when the spikes are bipolar (positive and negative value spikes).

Even Search Functions
You can use this functions to give different values to the spikes, following the next steps.
1. Open the Event Search or Event Search (Free Formula) dialog box in DIAdem in ANALYSYS > Channels Functions.
2. Chose the channel you want to filter the spikes from.
3. Click on the ... button and enter the desired limits.
4. Select the Result Storage tab.
5. Under Condition fulfilled, select the Use value from channel option and select the same Channel as in step 2.
6. Under Condition not fulfilled, you can chose to replace the values with data from a different channel or to convert all data outside the limits to a single value. For this example, chose Use single value and set the data to an acceptable value (i.e. 0). 7. Once you click Calculate, a new channel will be created with all the spikes will be changed to the value you selected in step 6. If you want to completely remove the data points instead of just changing them, set them to an intentional value outside the data range to make them easy to identify and refer to the Flag Functions section to remove them. The example below sets the values to 80 to then remove them as shown in the Flag Functions section.
Flag Functions
Use the Flag Functions to remove the unwanted data following the steps below:
1. In DIAdem's VIEW panel, add the data to a 2D Axis System.
2. Select the Frame Cursor in the toolbar and create a frame around the unwanted data.
3. Click the Set flags button in the 2D Axis System specific toolbar.
4. Select Flags: Remove data points to convert the data points to No Value.
5. Optionally, you can select Flags: Interpolate NoValues to remove the No Value points and replace them with an interpolation of the adjacent values.

At the end of the process, you should see the clean signal as shown below.

Additional Information

As of DIAdem 2018, it is not possible to use the Event Search functions in a macro. If you want to automate this process on a Script, refer to Commands and Variables for Working with Data in the DIAdem Help and find the Commands for an Event Search.