SystemLink Jupyter Notebook Not Outputting Data

Updated Mar 8, 2024

Reported In

Software

  • SystemLink
  • SystemLink Jupyter Notebook Add-On

Issue Details

  • I have created a Jupyter Notebook to output a DataFrame into a SystemLink report, but when I execute the notebook, SystemLink does not show any data.
  • I have a Grafana dashboard that is linked to a Jupyter Notebook. My dashboard panel is always empty, and when I inspect the DataFrame JSON (from Query) source, it's an empty array. How can I troubleshoot this?
  • I want to use a Jupyter Notebook as a data source in Grafana, but when I select my notebook I see the below error. What could be causing this?
The output for the notebook does not match the expected SystemLink format.

Grafana output error.PNG

Solution

This error occurs when the structure of the Jupyter Notebook output data is incorrect. Common causes for incorrect data are:
  • Mismatched parameters in the Jupyter Notebook.
  • Incorrect output data type.
  • Incorrect output data structure.
  • Errors occurring in the Jupyter Notebook.

To resolve this issue, follow the relevant sections below.
 

Cross-Checking the Notebook Parameters With the Inputs and Outputs

  1. In the Code Cell that defines your input parameters, open the Property Inspector.
  2. Cross-check the properties and syntax of your parameters with the inputs and outputs defined within the notebook.
    • For example, the image below indicates that the notebook expects to receive a numeric input called num_elements, and output a single [External] DataFrame with an ID of randomdata. In this scenario it's important to check that:
      • The num_elements input parameter is defined and used within the notebook.
      • A [External] Dictionary with type, id and data Keys is defined, where id is set to randomdata and data is a Dictionary of a DataFrame.

Property Inspector Showing Cell Metadata:
Cell Metadata.PNG
 

Checking the Output Data Type

  1. In the Code Cell that defines your input parameters, open the Property Inspector.
  2. Scroll through the Cell Metadata an identify the type associated with your output(s).
  3. Check that the type matches your output data structure. Valid values are data_frame or scalar.
    • In the example below, the output data is expected to be a DataFrame.
    • Check the notebook to ensure that the data is contained within a DataFrame, and that the DataFrame is inside a Dictionary that is glued to the notebook.

Cell MetaData Defining the Output Type:

Output cell metadata.PNG

Code Cell Defining the Dictionary that Includes the DataFrame and Sets the Type Accordingly:

Code Cell Defining Output.PNG
 

Checking the Output Data Structure

  • The output data structure should resemble the following:
    1. Initial output data is a Dictionary.
    2. The Dictionary is converted to a Pandas DataFrame.
    3. The DataFrame is included in a new Dictionary with Keys columns and values, where values is the DataFrame values as a list.
    4. The Dictionary from step 3 is included in a new Dictionary with Keys type, id and data, where data is the Dictionary from steps 3.
    5. The Dictionary from step 4 is glued to the notebook as a list
  • The images below show how each of these steps is implemented.

Code Cell Defining the Initial Dictionary and Converting it to a DataFrame (Steps 1 and 2 above):

Steps 1 and 2.PNG


DataFrame is Included in a Dictionary, Which is Then Included in Another Dictionary (Steps 3 and 4 above):

Steps 3 and 4.PNG


Final Dictionary is Glued to the Notebook as a List (Step 5 above):

step 5.PNG
 

Check the Jupyter Notebook for Errors

  • The output data may not be populated as expected if the notebook encounters an error in an earlier Code Cell. In Jupyter:
    1. Press the restart button.PNG button to restart the Kernel and run the entire notebook.
    2. If an error has occurred, it will be detailed below the problematic Code Cell.