Difference between revisions of "Dashboard"

From SatNOGS Wiki
(dashboard screencap)
 
(9 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
__NOTOC__
  
== THIS IS A WORK IN PROGRESS AND SUBJECT TO CHANGE ==
+
== Introduction ==
 +
A SatNOGS dashboard website is available:
  
We are using InfluxDB and Grafana for our telemetry dashboards. InfluxDB is a time-series database designed to be efficient for time-series data like the telemetry we collect in SatNOGS.
+
* [https://dashboard.satnogs.org/ SatNOGS Dashboard]
  
I'm going to drop a few things to get the reader going on telemetry dashboard in SatNOGS, and expect that the reader also reference the [Grafana documentation](http://docs.grafana.org/) for more details on building dashboards.
+
[[File:Satnogs-dashboard-cap.png|800px|SatNOGS Dashboard Screen capture]]
  
This doc also assumes you have an account to edit dashboards.
+
===<span class="glyphicon glyphicon-dashboard"></span> https://dashboard.satnogs.org<nowiki/>===
 +
We are using [https://www.influxdata.com/ InfluxDB] and [https://grafana.com/ Grafana] for our telemetry dashboards. [https://www.influxdata.com/ InfluxDB] is a time-series database designed to be efficient for time-series data like the telemetry we collect in SatNOGS.
 +
 
 +
I'm going to drop a few things to get the reader going on telemetry dashboard in SatNOGS, and expect that the reader also reference the [https://docs.grafana.org Grafana documentation] for more details on building dashboards.
 +
 
 +
This doc also assumes you have an account to edit dashboards. You can generate an account by clicking the Sign In link, and then [[Get In Touch|contacting us]] to escalate your permissions. You will be put into a group with access to the "Scratchpad - New Users" folder at first.
  
 
Today there is a top level folder hierarchy designed as such:
 
Today there is a top level folder hierarchy designed as such:
  
* Telemetry (where telemetry dashboards should be stored)
+
* Telemetry (where telemetry dashboards should be stored, publicly accessible)
* Experiments (for dashboards displaying satellite experiment data)
+
* General (for stats about our data collection itself, meta-dashboards, publicly accessible)
* Sandbox (a playground for making grafana dashboards)
+
* Scratchpad (a playground for making grafana dashboards and testing functionality, not publicly accessible)
  
 +
=== Creating a dashboard ===
 
Clicking the + icon will allow you to create a new dashboard. When you do this a new panel is started for you. This panel can be resized at the lower-right corner, and moved around by dragging once more panels are added.
 
Clicking the + icon will allow you to create a new dashboard. When you do this a new panel is started for you. This panel can be resized at the lower-right corner, and moved around by dragging once more panels are added.
  
Now - before you get started please note that you have to save your changes!! They do not save automatically!
+
Now - before you get started please note that <u>'''you have to save your changes!! They do not save automatically!'''</u>
 +
 
 +
Click "Graph" to make this a graph panel. Click the graph title and then Edit to edit this specific graph.
 +
 
 +
[[File:Grafana panel edit.png|frameless|600x600px]]
  
Click "Graph" to make this a graph panel. Click the graph title and then Edit to edit this specific graph. A few things you will want to edit:
+
A few things you will want to edit:
  
 
==== General ====
 
==== General ====
Line 25: Line 37:
  
 
==== Metrics ====
 
==== Metrics ====
 +
Select the metric by clicking "select measurement". What you will see here are the names of all measurements available which are the NORAD ID of the satellite in the database. You can type part of the NORAD ID to filter.
  
First, select the data source. While the "default" data source should always be assumed to point to the SatNOGS data warehouse, it never hurts to be explicit about it.
+
[[File:Grafana select measurement.png|frameless|600x600px]]
  
[[File:grafanametric.png]]
+
In most cases you will want to leave the WHERE clause blank, but from here you can also filter data based on the tag that the data has (such as filtering on a specific observer).
  
Next, select the metric, by clicking "select measurement". What you will see here are the names of all measurements available. This is bound to get messy as we grow, so you will need to rely on schema documentation for the satellite you are working on. NOTE: underscores in variable names from kaitai may be truncated. You can type part of the variable name for suggestions.  
+
[[File:Grafana filter tag.png|frameless|600x600px]]
  
TODO: the kaitai schemas are not available to users today (they aren't exposed in the db ui yet), but are available in the admin console under "telemetries". In the meantime, if you are working on a dashboard and want a copy of the schema ping an admin.
+
Next, select the telemetry field you are looking to graph by clicking on field("value"). If there is data in the database that has been properly decoded you should see the relevant fields for that satellite displayed here:
  
After "WHERE" you can add conditions based on tags. The most important here will be "norad" to identify the satellite. NOTE: multiple satellites may share the same measurement variable names, so this will be important! "Select tag value" will give you options to pick from based on what is in the database today.
+
[[File:Grafana field value.png|frameless|600x600px]]
  
No data yet? Change your timeframe to something like "last 30 days" as the default of "last 6 hours" may not have data decoded (especially true for our dev instance, where we have no steady stream of recent data)
+
See any data in your graph yet? If not, change your timeframe in the upper-right corner to something like "last 30 days" as the default of "last 6 hours" may not have data decoded (especially true for our dev instance, where we have no steady stream of recent data)
  
[[File:grafanamath.png]]
+
SELECT: mean() (default), distinct(), and last() are likely to be the most useful for us. Keep in mind that there is the possibility for multiple data points from multiple stations and observations, some overlapping. InfluxDB handles this for us. The timeframe selected is also taken into account in these measurements. This line is where you may need to add math to make the number human-readable. For instance, for CubeBel-1, to calculate voltage we have to take (Measured value) / 4096×2.5×3.75, which looks like this:
  
SELECT: mean() (default) and last() are likely to be the most useful for us. Keep in mind that there is the possibility for multiple data points from multiple stations and observations, some overlapping. InfluxDB handles this for us. The timeframe selected is also taken into account in these measurements. This line is where you may need to add math to make the number human-readable. For instance, in the docs for CAS-4, to calculate voltage we have to take 6×(3.3/255)×(Measured value), which looks like this:
+
[[File:Grafana math.png|frameless|900x900px]]
  
 
GROUP BY:  you may wish to change the fill value here, there are a number of options. Be careful as this may be misleading to users if we are dealing with very sparse data sets, as this option will fill in any gaps, yet "none" or "linear" may be options to use to make the graphs look better.
 
GROUP BY:  you may wish to change the fill value here, there are a number of options. Be careful as this may be misleading to users if we are dealing with very sparse data sets, as this option will fill in any gaps, yet "none" or "linear" may be options to use to make the graphs look better.
Line 50: Line 63:
 
You may want to change the Unit here, as "Short" may be undesired for certain figures, and "None" would be better.
 
You may want to change the Unit here, as "Short" may be undesired for certain figures, and "None" would be better.
  
==== Display =====
+
==== Display ====
  
 
If you are doing a line graph and do not want the shading below the line, change the "Fill" value here to 0.
 
If you are doing a line graph and do not want the shading below the line, change the "Fill" value here to 0.
Line 63: Line 76:
  
 
"Save As" will allow you to make a copy of a graph and hack on it.
 
"Save As" will allow you to make a copy of a graph and hack on it.
 +
[[Category:Develop]]
 +
[[Category:Software]]
 +
[[Category:Infrastructure]]
 +
[[Category:Telemetry]]
 +
__NOEDITSECTION__

Latest revision as of 02:59, 20 January 2020


Introduction

A SatNOGS dashboard website is available:

SatNOGS Dashboard Screen capture

https://dashboard.satnogs.org

We are using InfluxDB and Grafana for our telemetry dashboards. InfluxDB is a time-series database designed to be efficient for time-series data like the telemetry we collect in SatNOGS.

I'm going to drop a few things to get the reader going on telemetry dashboard in SatNOGS, and expect that the reader also reference the Grafana documentation for more details on building dashboards.

This doc also assumes you have an account to edit dashboards. You can generate an account by clicking the Sign In link, and then contacting us to escalate your permissions. You will be put into a group with access to the "Scratchpad - New Users" folder at first.

Today there is a top level folder hierarchy designed as such:

  • Telemetry (where telemetry dashboards should be stored, publicly accessible)
  • General (for stats about our data collection itself, meta-dashboards, publicly accessible)
  • Scratchpad (a playground for making grafana dashboards and testing functionality, not publicly accessible)

Creating a dashboard

Clicking the + icon will allow you to create a new dashboard. When you do this a new panel is started for you. This panel can be resized at the lower-right corner, and moved around by dragging once more panels are added.

Now - before you get started please note that you have to save your changes!! They do not save automatically!

Click "Graph" to make this a graph panel. Click the graph title and then Edit to edit this specific graph.

Grafana panel edit.png

A few things you will want to edit:

General

Give it a meaningful title, of course. Description is not required, but offers a popup of more details to the user if you want to add more details.

Metrics

Select the metric by clicking "select measurement". What you will see here are the names of all measurements available which are the NORAD ID of the satellite in the database. You can type part of the NORAD ID to filter.

Grafana select measurement.png

In most cases you will want to leave the WHERE clause blank, but from here you can also filter data based on the tag that the data has (such as filtering on a specific observer).

Grafana filter tag.png

Next, select the telemetry field you are looking to graph by clicking on field("value"). If there is data in the database that has been properly decoded you should see the relevant fields for that satellite displayed here:

Grafana field value.png

See any data in your graph yet? If not, change your timeframe in the upper-right corner to something like "last 30 days" as the default of "last 6 hours" may not have data decoded (especially true for our dev instance, where we have no steady stream of recent data)

SELECT: mean() (default), distinct(), and last() are likely to be the most useful for us. Keep in mind that there is the possibility for multiple data points from multiple stations and observations, some overlapping. InfluxDB handles this for us. The timeframe selected is also taken into account in these measurements. This line is where you may need to add math to make the number human-readable. For instance, for CubeBel-1, to calculate voltage we have to take (Measured value) / 4096×2.5×3.75, which looks like this:

Grafana math.png

GROUP BY: you may wish to change the fill value here, there are a number of options. Be careful as this may be misleading to users if we are dealing with very sparse data sets, as this option will fill in any gaps, yet "none" or "linear" may be options to use to make the graphs look better.

You can also add multiple queries to the same graph with the Add Query button at the bottom

Axes

You may want to change the Unit here, as "Short" may be undesired for certain figures, and "None" would be better.

Display

If you are doing a line graph and do not want the shading below the line, change the "Fill" value here to 0.

Graph Settings

In the gear icon for a graph are the graph settings. Some things to change here:

  • Name
  • Folder (see the hierarchy above)
  • "Now delay now-" - I suggest 30d (the last 30d) as without it the default is 6 hours

"Save As" will allow you to make a copy of a graph and hack on it.