Using a SystemLink Jupyter Notebook for a Grafana Query Variable

This article demonstrates how to use an output for a SystemLink Jupyter Notebook as a data source for a Grafana dashboard variable.
 

Prerequisites

• On the SystemLink Server PC, the NI SystemLink Server - Jupyter Hub Module must be installed.
• Grafana must be installed on the server PC. Refer to [External] GitHub: NI Grafana Plugins for instructions.

To use output data from a SystemLink Jupyter Notebook as a data source for a Grafana dashboard variable, the notebook must output string data and the Grafana dashboard variable must be a Query type.

Follow the steps below to use a SystemLink Jupyter Notebook as a data source for a Grafana dashboard variable:
 

1. Create a Jupyter Notebook that outputs string data.
   1. Define the output string data in a [External] Dictionary.
   2. Convert the Dictionary to a [External] Pandas DataFrame.
   3. Create a new Dictionary consisting of columns and values Keys to store the DataFrame.
   4. Store the Dictionary from step 3 in another Dictionary consisting of type, id and data Keys where type is data_frame and data is the Dictionary name.
      • The code snippets below demonstrate how to implement each step.

# import necessary modules for outputting SystemLink data
import pandas as pd
import scrapbook as sb

# Create dictionary of strings to output
dictionary = {
    "Variables" : ["string1", "string2", "string3", "string4", "string5"]
}

# Convert dictionary to dataframe
df = pd.DataFrame.from_dict(dictionary)
display(df)

# create a dictionary from the DataFrame
df_dict = {
    'columns': pd.io.json.build_table_schema(df, index=False)['fields'],
    'values': df.values.tolist(),
}

# Wrap df_dict in another dictionary that defines type, id and data
result = {
    'type': 'data_frame',
    'id': 'dataframe',
    'data': df_dict,
}

# Record output result with Scrapbook
sb.glue('result', [result])

2. Update the notebook parameters for the string output.
   1. Select the Code Cell that defines your input parameters. Note: if no input parameters are used, create an empty Code Cell at the beginning of the notebook with a comment to state that this cell is for parameters.
   2. Click the cog on the right-hand side of the notebook to open the Property Inspector.
   3. Expand the Advanced Tools section.
   4. Within the Cell Metadata block, ensure that the outputs section has:
      • A display_name for the output data. This is the name of the data as it appears in Grafana.
      • An id for the output data. This must match the id specified in step 1.d.
      • A type with value data_frame.
   5. The code snippet below is an example of Cell Metadata that expects a DataFrame output with id dataframe.

{
    "papermill": {
        "parameters": {}
    },
    "systemlink": {
        "namespaces": [
            "ni-testmanagement"
        ],
        "outputs": [
            {
                "display_name": "Variables",
                "id": "dataframe",
                "type": "data_frame"
            }
        ],
        "parameters": [],
        "version": 2
    },
    "tags": [
        "parameters"
    ]
}

3. Create a Query variable in the Grafana dashboard.
   • For instructions on how to create dashboard variables, refer to [External] Grafana Labs: Add and Manage Variables.
   • The example below shows a Query variable that uses a Jupyter Notebook as a data source. The Jupyter Notebook outputs Variables, which is a set of five different strings: string1, string2, string3, string4 and string5.
   • Click Apply ​​​​​​​to save the variable.

 

Download the attached Jupyter Notebook to test the example discussed throughout this article.