Difference between revisions of "Adding a new data decoder"

From SatNOGS Wiki
(Add step for automated field definition generation)
m (update link to satblog)
 
(6 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== How to create the parser for a new satellite downlink ==
+
== Introduction ==
 +
This is one of the last steps on your way to add support for a satellite in SatNOGS and visualize it in the SatNOGS [[Dashboard]]. For the previous steps have a look at [[Satellite Mission Guide]] and [[Satellite operator manual]] (those two pages will probably be merged in the future).
 +
 
 +
==How to create the parser for a new satellite downlink==
  
 
For satellite downlinks where SatNOGS provides demodulated data, but no decoder exists yet.
 
For satellite downlinks where SatNOGS provides demodulated data, but no decoder exists yet.
  
# Find and read documentation of the satellite downlink.
+
#Find and read documentation of the satellite downlink.
# Extract raw frames either from satnogs-network or satnogs-db.
+
#Extract raw frames either from satnogs-network or satnogs-db.
#* from network: Open good observations whith demodulated data,
+
#*from network: Open good observations whith demodulated data,
#*: then open the observation via the API,
+
#*:then open the observation via the API,
#*: copy the link to the payloads and download them (e.g. via wget)
+
#*:copy the link to the payloads and download them (e.g. via wget)
#* from db: Export frames of your satellite (how?), then convert them from hex to raw
+
#*from db: Export frames of your satellite (how?), then convert them from hex to raw
# Write the Kaitai Struct (.ksy-file)
+
#Write the Kaitai Struct (.ksy-file)
#* Open the [https://ide.kaitai.io/ Kaitai WebIDE]
+
#*Open the [https://ide.kaitai.io/ Kaitai WebIDE]
#* Create a new .ksy-format file (press the empty sheet button in the lower left)
+
#*Create a new .ksy-format file (press the empty sheet button in the lower left)
#* Upload the previously fetched raw frames as example data (press the cloud button in the lower left, select all files with raw frames)
+
#*Upload the previously fetched raw frames as example data (press the cloud button in the lower left, select all files with raw frames)
#* Write the kaitai struct following the guide [https://community.libre.space/t/starting-with-satnogs-decoders/3145 "Starting with satnogs-decoders"] (TODO: Move this guide into this wiki) Use existing parsers from the [https://gitlab.com/librespacefoundation/satnogs/satnogs-decoders satnogs-decoders] repository for reference.
+
#*Write the kaitai struct following the guide [https://community.libre.space/t/starting-with-satnogs-decoders/3145 "Starting with satnogs-decoders"] (TODO: Move this guide into this wiki) Use existing parsers from the [https://gitlab.com/librespacefoundation/satnogs/satnogs-decoders satnogs-decoders] repository for reference.
#* The "doc:" section which defines all fields in front of the satnogs decoder kaitai structs can be ignored for now.  They are ignored by kaitai, satnogs-specific and will be auto-generated in a later step.
+
#*The "doc:" section which defines all fields in front of the satnogs decoder kaitai structs can be ignored for now.  They are ignored by kaitai, satnogs-specific and will be auto-generated in a later step.
#* Add a link to all the documentation of the satellite downlink which was used to write the kaitai struct (this can be done with a doc key)
+
#*Add a link to all the documentation of the satellite downlink which was used to write the kaitai struct (this can be done with a doc key)
# Check the results of your parser
+
#Check the results of your parser
#* If another parser exists, cross-check your results with the output it provides.
+
#*If another parser exists, cross-check your results with the output it provides.
#* Sometimes there are examples frames or examples recordings provided. Use them for verification (if the data is well-known.)
+
#*Sometimes there are examples frames or examples recordings provided. Use them for verification (if the data is well-known.)
# Generate the fields definition using the [https://gitlab.com/librespacefoundation/satnogs/satnogs-decoders/blob/master/contrib/generate_field_docstring.py generate-field-docstring.py script] (TODO: Add detailed documentation here and in the README)
+
#Generate the fields definition using the [https://gitlab.com/librespacefoundation/satnogs/satnogs-decoders/blob/master/contrib/generate_field_docstring.py generate-field-docstring.py script] (TODO: Add detailed documentation here and in the README)
# Fork the [https://gitlab.com/librespacefoundation/satnogs/satnogs-decoders satnogs-decoders] repository and add your decoder as described in the [https://gitlab.com/librespacefoundation/satnogs/satnogs-decoders/tree/master#adding-a-new-decoder README]
+
#Fork the [https://gitlab.com/librespacefoundation/satnogs/satnogs-decoders satnogs-decoders] repository and add your decoder as described in the [https://gitlab.com/librespacefoundation/satnogs/satnogs-decoders/tree/master#adding-a-new-decoder README]
# Create a commit with your new kaitai struct, push it to your fork and submit a Merge Request to satnogs-decoders
+
#Create a commit with your new kaitai struct, push it to your fork and submit a Merge Request to satnogs-decoders
# Done.
+
#Done.
  
 
Next step: [https://wiki.satnogs.org/Telemetry_Dashboards Add a dashboard to visualize the new data]
 
Next step: [https://wiki.satnogs.org/Telemetry_Dashboards Add a dashboard to visualize the new data]
  
==== Resources ====
+
==See also==
* [http://doc.kaitai.io/user_guide.html Kaitai Struct User Guide]
+
 
* [https://ide.kaitai.io/ Kaitai WebIDE]
+
*[http://doc.kaitai.io/user_guide.html Kaitai Struct User Guide]
* [https://gitlab.com/librespacefoundation/satnogs/satnogs-decoders satnogs-decoders] repository
+
*[https://ide.kaitai.io/ Kaitai WebIDE]
[[Category:Telemetry]]
+
*[https://gitlab.com/librespacefoundation/satnogs/satnogs-decoders satnogs-decoders] repository
 +
*[[Decode Telemetry and Packets]]
 +
*[https://www.satblog.info/ Satellite specific telemetry decoders] (German)
 +
 
 +
[[Category:Develop]]
 +
[[Category:Software]]
 +
[[Category:RF Modes]]

Latest revision as of 17:59, 15 August 2022

Introduction

This is one of the last steps on your way to add support for a satellite in SatNOGS and visualize it in the SatNOGS Dashboard. For the previous steps have a look at Satellite Mission Guide and Satellite operator manual (those two pages will probably be merged in the future).

How to create the parser for a new satellite downlink

For satellite downlinks where SatNOGS provides demodulated data, but no decoder exists yet.

  1. Find and read documentation of the satellite downlink.
  2. Extract raw frames either from satnogs-network or satnogs-db.
    • from network: Open good observations whith demodulated data,
      then open the observation via the API,
      copy the link to the payloads and download them (e.g. via wget)
    • from db: Export frames of your satellite (how?), then convert them from hex to raw
  3. Write the Kaitai Struct (.ksy-file)
    • Open the Kaitai WebIDE
    • Create a new .ksy-format file (press the empty sheet button in the lower left)
    • Upload the previously fetched raw frames as example data (press the cloud button in the lower left, select all files with raw frames)
    • Write the kaitai struct following the guide "Starting with satnogs-decoders" (TODO: Move this guide into this wiki) Use existing parsers from the satnogs-decoders repository for reference.
    • The "doc:" section which defines all fields in front of the satnogs decoder kaitai structs can be ignored for now. They are ignored by kaitai, satnogs-specific and will be auto-generated in a later step.
    • Add a link to all the documentation of the satellite downlink which was used to write the kaitai struct (this can be done with a doc key)
  4. Check the results of your parser
    • If another parser exists, cross-check your results with the output it provides.
    • Sometimes there are examples frames or examples recordings provided. Use them for verification (if the data is well-known.)
  5. Generate the fields definition using the generate-field-docstring.py script (TODO: Add detailed documentation here and in the README)
  6. Fork the satnogs-decoders repository and add your decoder as described in the README
  7. Create a commit with your new kaitai struct, push it to your fork and submit a Merge Request to satnogs-decoders
  8. Done.

Next step: Add a dashboard to visualize the new data

See also