Increasing the Size of the CPU Buffer for Capturing Traces on NI Linux Real-Time Controllers

This document explains how to increase the size of the CPU buffers when acquiring traces on a NI Linux Real-Time (RT) controller through ftrace, while performing the procedure presented in the following tutorial: Tracing on NI Linux Real-Time to allow more information to be captured for each CPU, if further analysis is required for troubleshooting purposes.

To execute this procedure, it is necessary to have access to the RT controller's Secure Shell Server (SSH). For more information, please refer to the following article: Accessing the Shell on NI Linux Real-Time Devices

1. Access your RT controller's SSH. 
2. Stop any ongoing tracing and clear the CPUs' buffers by executing the following commands

echo 0 > /sys/kernel/debug/tracing/tracing_on
echo 0 > /sys/kernel/debug/tracing/trace

3.  Adjust the size of the CPUs buffers for capturing traces

echo <new size in kB> | tee /sys/kernel/debug/tracing/buffer_size_kb 

Where <new size in kB> represents the new size in kilobytes (kB) that is going to be used for all CPU cores. When setting this value, please consider the following:

• • • The size must be a multiple of 4 kB.
    • The maximum size depends on the available memory on your RT controller. Be cautious when configuring large buffer sizes, as RAM is limited and shared with the rest of the system. 
    • The size configured through the command above is per CPU (core). For example, if your controller has 4 cores and you set 32,768 kB, it actually reserves 4 x 32 MB ≈ 128 MB in total.
    • This configuration does not persist across reboots. 

4. Verify that the new size has been set correctly by running the command shown below. The console output should display a value close to the one you configured. 

cat /sys/kernel/debug/tracing/buffer_size_kb

5. Capture the required traces from your system, as explained in the Tracing on NI Linux Real-Time guide. 

After capturing the traces, if you run the trace-cmd extract command to generate the binary trace file, you should see that the number of bytes captured for each CPU core is equal to or very close to the buffer size that was configured in the previous steps. This will also be reflected in the total size of the generated binary file.

For example, if you set the size as 32,768 kB you can expect an output similar to image shown below: