Difference between revisions of "Understanding satnogs-flowgraphs"
Matburnham (talk | contribs) (Created page with "I'm not entirely sure where to put this, but perhaps someone more experienced can move and link this from a sensible location. This started as a question continuing the [https...") |
Matburnham (talk | contribs) |
||
Line 1: | Line 1: | ||
I'm not entirely sure where to put this, but perhaps someone more experienced can move and link this from a sensible location. This started as a question continuing the [https://community.libre.space/t/how-to-open-satnogs-iq-dat-files/3793 How to open satnogs iq.dat files] thread but as I worked through after picking things up after initially having a problem I realised everything I'd described now magically worked! Typical. Worth documenting. | I'm not entirely sure where to put this, but perhaps someone more experienced can move and link this from a sensible location. This started as a question continuing the [https://community.libre.space/t/how-to-open-satnogs-iq-dat-files/3793 How to open satnogs iq.dat files] thread but as I worked through after picking things up after initially having a problem I realised everything I'd described now magically worked! Typical. Worth documenting. | ||
− | == Setting yourself up == | + | ==Setting yourself up== |
See elsewhere, but get GNU Radio up and running and a copy of satnogs-flowgraphs from GitLab. | See elsewhere, but get GNU Radio up and running and a copy of satnogs-flowgraphs from GitLab. | ||
Line 7: | Line 7: | ||
I'm using a virtual Ubuntu 19.10 machine grabbed from osboxes.org, but you can use whatever you like. I imagine you could even do it on the Raspberry Pi itself. I did end up setting this VM up as a 'development station' but you should only need to install GNU Radio, and any pre-requisites needed by satnogs-flowgraphs - so probably gr-satnogs, I can't quite remember. As you find out please edit this :-) | I'm using a virtual Ubuntu 19.10 machine grabbed from osboxes.org, but you can use whatever you like. I imagine you could even do it on the Raspberry Pi itself. I did end up setting this VM up as a 'development station' but you should only need to install GNU Radio, and any pre-requisites needed by satnogs-flowgraphs - so probably gr-satnogs, I can't quite remember. As you find out please edit this :-) | ||
− | == Storing an IQ recording == | + | ==Storing an IQ recording== |
There are options in satnogs-setup under Advanced > Radio to store IQ files: | There are options in satnogs-setup under Advanced > Radio to store IQ files: | ||
− | * ENABLE_IQ_DUMP: Enable IQ dump | + | *ENABLE_IQ_DUMP: Enable IQ dump |
− | * IQ_DUMP_FILENAME: Define IQ dump filename [/tmp/.satnogs/iq.raw] | + | *IQ_DUMP_FILENAME: Define IQ dump filename [/tmp/.satnogs/iq.raw] |
Beware, IQ files are huge (220MB for my APT pass example below). They're stored in /tmp so therefore in RAM and are overwritten with each pass. You can do [https://community.libre.space/t/uploading-compressed-iq-files-to-dropbox-or-any-other-cloud-storage-provider/5395 something clever] to upload them elsewhere, but for playing just grab a copy immediately after a pass. | Beware, IQ files are huge (220MB for my APT pass example below). They're stored in /tmp so therefore in RAM and are overwritten with each pass. You can do [https://community.libre.space/t/uploading-compressed-iq-files-to-dropbox-or-any-other-cloud-storage-provider/5395 something clever] to upload them elsewhere, but for playing just grab a copy immediately after a pass. | ||
Line 18: | Line 18: | ||
Don't worry about storing much else (which also vanishes quickly). You can easily grab waterfalls and OGG recording from the database via [https://github.com/deckbsd/glouton-satnogs-data-downloader Glouton]. | Don't worry about storing much else (which also vanishes quickly). You can easily grab waterfalls and OGG recording from the database via [https://github.com/deckbsd/glouton-satnogs-data-downloader Glouton]. | ||
− | == GNU Radio Companion == | + | ==GNU Radio Companion== |
Here I'm working with [https://network.satnogs.org/observations/2073316/ Observation #2073316] as an example. I've saved a copy of the IQ recording my station briefly stored at `/tmp/.satnogs/iq.raw`. This is about 220 MB. I even remembered to turn the IQ collection off afterwards or it would be storing stuff on top of each other forever. | Here I'm working with [https://network.satnogs.org/observations/2073316/ Observation #2073316] as an example. I've saved a copy of the IQ recording my station briefly stored at `/tmp/.satnogs/iq.raw`. This is about 220 MB. I even remembered to turn the IQ collection off afterwards or it would be storing stuff on top of each other forever. | ||
− | === So where did the IQ come from? === | + | ===So where did the IQ come from?=== |
In my case, I'm playing with an APT pass. It is therefore processed by [https://gitlab.com/librespacefoundation/satnogs/satnogs-flowgraphs/-/blob/master/satellites/noaa_apt_decoder.grc satnogs-flowgraphs/satellites/noaa_apt_decoder.grc]. Other signals use other flowgraphs. The default where nothing else is suitable appears to be [https://gitlab.com/librespacefoundation/satnogs/satnogs-flowgraphs/-/blob/master/generic/fsk.grc satnogs-flowgraphs/generic/fsk.grc]. | In my case, I'm playing with an APT pass. It is therefore processed by [https://gitlab.com/librespacefoundation/satnogs/satnogs-flowgraphs/-/blob/master/satellites/noaa_apt_decoder.grc satnogs-flowgraphs/satellites/noaa_apt_decoder.grc]. Other signals use other flowgraphs. The default where nothing else is suitable appears to be [https://gitlab.com/librespacefoundation/satnogs/satnogs-flowgraphs/-/blob/master/generic/fsk.grc satnogs-flowgraphs/generic/fsk.grc]. | ||
Line 30: | Line 30: | ||
That is, it's post Doppler correction, which includes an LO offset; and the values are scaled to shorts, although 16768 seems an weird number as I'd have expected a power of two - 16384? I don't imagine it makes a lot of difference as it'll still fit in the short just lose a little bit of dynamic range. | That is, it's post Doppler correction, which includes an LO offset; and the values are scaled to shorts, although 16768 seems an weird number as I'd have expected a power of two - 16384? I don't imagine it makes a lot of difference as it'll still fit in the short just lose a little bit of dynamic range. | ||
− | === Feeding your IQ in === | + | ===Feeding your IQ in=== |
Loading the flowgraph in requires some GNU Radio modules chaining together. Starting with [https://community.libre.space/t/how-to-open-satnogs-iq-dat-files/3793 How to open satnogs iq.dat files] you'll end up with something like this: | Loading the flowgraph in requires some GNU Radio modules chaining together. Starting with [https://community.libre.space/t/how-to-open-satnogs-iq-dat-files/3793 How to open satnogs iq.dat files] you'll end up with something like this: | ||
Line 40: | Line 40: | ||
The rest just reverses throttles the output to real-time, and converts the short values to complex real including the scaling division. | The rest just reverses throttles the output to real-time, and converts the short values to complex real including the scaling division. | ||
− | === Results === | + | ===Results=== |
Initially, the waterfall may look a bit busy compared to the one from the [https://network.satnogs.org/observations/2073316/ observation]. There's obviously some gain setting within the waterfall I've not yet mastered, or the Satnogs ones does something slightly cleverer. | Initially, the waterfall may look a bit busy compared to the one from the [https://network.satnogs.org/observations/2073316/ observation]. There's obviously some gain setting within the waterfall I've not yet mastered, or the Satnogs ones does something slightly cleverer. | ||
[[File:Waterfall.png|200px]] | [[File:Waterfall.png|200px]] | ||
+ | |||
+ | Yep. It turns out you can set the intesity min and max to better reflect the signal and get a cleaner display. Still not as good as the observation page one, but maybe that fiddles with the FFT size too. | ||
+ | |||
+ | [[File:Better.png|200px]] | ||
However, if you peer closer you can just about make out the feint telltales of an APT signal. | However, if you peer closer you can just about make out the feint telltales of an APT signal. | ||
Line 52: | Line 56: | ||
[[File:visual.png|300px]] | [[File:visual.png|300px]] | ||
− | === OGG === | + | ===OGG=== |
For completeness, you can do the same thing using the OGG file as a source. Again, you need to reverse any resampling or other changes that were done before saving to file before feeding back into the flowgraph at an appropriate point. The throttle is optional, and in the screenshot bypassed: | For completeness, you can do the same thing using the OGG file as a source. Again, you need to reverse any resampling or other changes that were done before saving to file before feeding back into the flowgraph at an appropriate point. The throttle is optional, and in the screenshot bypassed: | ||
[[File:flowgraph_all.png|800px]] | [[File:flowgraph_all.png|800px]] |
Revision as of 13:51, 30 April 2020
I'm not entirely sure where to put this, but perhaps someone more experienced can move and link this from a sensible location. This started as a question continuing the How to open satnogs iq.dat files thread but as I worked through after picking things up after initially having a problem I realised everything I'd described now magically worked! Typical. Worth documenting.
Contents
Setting yourself up
See elsewhere, but get GNU Radio up and running and a copy of satnogs-flowgraphs from GitLab.
I'm using a virtual Ubuntu 19.10 machine grabbed from osboxes.org, but you can use whatever you like. I imagine you could even do it on the Raspberry Pi itself. I did end up setting this VM up as a 'development station' but you should only need to install GNU Radio, and any pre-requisites needed by satnogs-flowgraphs - so probably gr-satnogs, I can't quite remember. As you find out please edit this :-)
Storing an IQ recording
There are options in satnogs-setup under Advanced > Radio to store IQ files:
- ENABLE_IQ_DUMP: Enable IQ dump
- IQ_DUMP_FILENAME: Define IQ dump filename [/tmp/.satnogs/iq.raw]
Beware, IQ files are huge (220MB for my APT pass example below). They're stored in /tmp so therefore in RAM and are overwritten with each pass. You can do something clever to upload them elsewhere, but for playing just grab a copy immediately after a pass.
Don't worry about storing much else (which also vanishes quickly). You can easily grab waterfalls and OGG recording from the database via Glouton.
GNU Radio Companion
Here I'm working with Observation #2073316 as an example. I've saved a copy of the IQ recording my station briefly stored at `/tmp/.satnogs/iq.raw`. This is about 220 MB. I even remembered to turn the IQ collection off afterwards or it would be storing stuff on top of each other forever.
So where did the IQ come from?
In my case, I'm playing with an APT pass. It is therefore processed by satnogs-flowgraphs/satellites/noaa_apt_decoder.grc. Other signals use other flowgraphs. The default where nothing else is suitable appears to be satnogs-flowgraphs/generic/fsk.grc.
That is, it's post Doppler correction, which includes an LO offset; and the values are scaled to shorts, although 16768 seems an weird number as I'd have expected a power of two - 16384? I don't imagine it makes a lot of difference as it'll still fit in the short just lose a little bit of dynamic range.
Feeding your IQ in
Loading the flowgraph in requires some GNU Radio modules chaining together. Starting with How to open satnogs iq.dat files you'll end up with something like this:
It took some time for me to work out the appropriate samples rates but I can't remember exactly how now. Scheduling a pass will generate them and then you can work it from there. Maybe I'll come back and update this later.
The rest just reverses throttles the output to real-time, and converts the short values to complex real including the scaling division.
Results
Initially, the waterfall may look a bit busy compared to the one from the observation. There's obviously some gain setting within the waterfall I've not yet mastered, or the Satnogs ones does something slightly cleverer.
Yep. It turns out you can set the intesity min and max to better reflect the signal and get a cleaner display. Still not as good as the observation page one, but maybe that fiddles with the FFT size too.
However, if you peer closer you can just about make out the feint telltales of an APT signal.
Following the flowgraph through to output shows an identical output to that on the observation. Result!
OGG
For completeness, you can do the same thing using the OGG file as a source. Again, you need to reverse any resampling or other changes that were done before saving to file before feeding back into the flowgraph at an appropriate point. The throttle is optional, and in the screenshot bypassed: