In this post I would like to introduce a new project that I have recently been working on.
RTKLIB can be a great tool for exploring the world of precision GNSS solutions. There are many configuration options (some say too many!) which allow the user to investigate the effects of different algorithm settings. It is also open source, so the user can see exactly what the code is doing behind the scene and even modify the code to explore options outside the available configuration set.
However, the code is quite complex and written in a dense style that can be quite intimidating to the casual explorer who wants to dig deeper. The compiled nature of C/C++ also makes the development platform more difficult to work in than a more interactive environment would be. These barriers mean that, despite its open source, only a small number of RTKLIB users actually ever dig into the core code. For those who want to take things a step further and experiment with or develop their own GNSS algorithms, RTKLIB may not be an optimal choice.
I have often thought that a Python version of RTKLIB would help minimize some of these barriers and make RTKLIB more useful as a learning and development tool. Recently I was made aware of a newly developed Python version of the code through a reference in the diary of Tomoji Takasu, the creator of RTKLIB. The code is called CSSRLib, and was written by Rui Hirokawa from Mitsubishi Electric. It is primarily intended to demonstrate the use of SSR correction data but includes a port into python of the pieces of RTKLIB needed to run PPK and PPP solutions. It also includes example data and wrapper scripts to run these solutions on the sample data. It was obviously a big effort and a significant accomplishment.
Although the CSSRLib package could be quite useful as-is for exploring precision GNSS solutions, the translation from C to Python is not strict enough to allow a user to jump directly back and forth between this code and RTKLIB.
I thought it would be an interesting exercise to rewrite the python code to be more closely aligned to RTKLIB, add all of the changes and enhancements in the demo5 version, and try to match the solutions of the demo5 solutions as closely as possible. This turned out to be a significantly more time-consuming proposition than I realized, but after several months of on-and-off work on the project, I finally was able to complete a code that, for the most part, meets these criteria.
It is not intended to be a substitute for the C version of RTKLIB since it only performs a small subset of the full library capability and runs noticeably slower. It currently runs only PPK solutions, although I would like to add PPP solutions later. The CSSRLib package supported the GPS and Galileo constellations. I have added support for GLONASS, but Beidou is still not supported. Since the resulting code is quite different in implementation and purpose from the original CSSRLib, I have created a new repository on Github for it, and named it rtklib-py. The code still shares many of its CSSRLib roots and so I have left the original copyright notices in the code files and have added acknowledgements to the original code.
The CSSRLib package includes an example dataset from a geodetic quality rover but since the demo5 code focuses on low-cost receivers, I have replaced the sample data with two other datasets, one from a u-blox F9P rover mounted on a vehicle roof, and a second dataset from the Google Smartphone Decimeter Challenge containing data from a smartphone mounted inside a vehicle.
A truly literal translation of the C code would run very slowly, primarily because Python is an interpreted language and C is a compiled language. To make the python code run at a reasonable speed, many of the for loops in RTKLIB have been replaced with Numpy array operations. This need to optimize, along with inherent differences between C and Python, means the codes are not identical but I have attempted to make the two codes as similar as possible in file names, function names, variable names, logic, and even comments. I also added a similar trace debug feature to the code that, when enabled, produces trace files that are very similar to the RTKLIB trace files. These will align quite closely when compared against each other with a file compare app and provides a way to confirm that the intermediate results match between the two code sets.
Here is an example of the trace file from RTKLIB on the left and rtklib-py on the right which demonstrates how similar they are. There are still small differences in the codes, which along with the iterative nature of the solution does cause the final positions to diverge by small amounts, but the intermediate results are for the most part very close.
The inputs (rinex files) and outputs are the same between the two codes, so RTKCONV and RTKPLOT in RTKLIB can be used to generate the rinex input files and plot the solution output files.
The config parameters in the new code have names and functionality very similar to the RTKLIB code although not all options are supported, particularly the ones used more frequently in PPP solutions than in PPK solutions. For the two sample data sets, the config parameters are defined in f9p_config.py and phone_config.py respectively. The top level script to run a solution is run_ppk.py. By default, it is setup to run the F9P sample data set but includes a commented out section to run a solution for the smartphone data. You can run a solution on your own data by modifying the files specified in run_ppk.py. You may also want to create a copy of one of the existing config files and adjust it for your data.
Below is a comparison of an RTKLIB forward solution to the F9P example data set on the left and an rtklib-py solution on the right, using the same configuration parameters for both.
Here is the difference between the two solutions. As you can see, they are quite close but not exactly the same.
Rtklib-py is not meant to be a replacement for RTKLIB and would not be a good choice for someone who is interested only in the final results. However, I am hoping it can be useful in several other possible ways:
- As a “map” to explore and learn how the inner details of RTKLIB work
- As a development environment to experiment with enhancements or adjustments to the RTKLIB algorithms. Due to the close alignment between the two packages, these can then be fairly easily ported back into the C/C++ version of RTKLIB when they are complete
- To debug issues found in RTKLIB within a more interactive environment
- To cut and paste pieces of the code into more custom solutions
One last goal is for this code is for it to be available as a tool for teams competing in this year’s Google Smartphone Decimeter Challenge competition.
The purpose of this post was just to introduce the new code so I won’t go into any more detail here. There are some brief instructions in the readme file in the Github repository for running the code but I do assume users are already reasonably comfortable with running Python applications.
I personally like to run Python in the Spyder IDE which provides an easy-to-work-in interactive environment and includes Numpy and other popular libraries in the installation. I’m sure, however, that there are other good development environments as well, if you prefer another option.
I’m always interested in other people’s thoughts on these topics so please leave any comments or suggestions you have in the comment section below.
13 thoughts on “A Python version of RTKLIB”
Hi, I have developed a program in Python that establishes a connection with a ublox EVK-M8T receiver, sends configuration messages to receive only RAWX and SFRBX type messages (I don’t know if others are needed), and then collects the received stream of bytes into a .ubx file (which I don’t know if it corresponds to what u-center normally generates when registering). My code removes the NMEA sentences and retains only the UBX messages (i.e., those beginning with bytes B5 65). I would like to be able to generate RINEX files from this file that I generate by collecting the bytes that arrive using this library. Is it possible to accomplish this? If so, I have some questions about it:
is the UBX file that I create by collecting the bytes coming from the receiver, without doing anything else, okay like this or should it be manipulated in other ways?
is it possible to generate the .XXo RINEX observations file (where XX is the year of observation) and get several RINEX files for the navigations, one per constellation (for example: .XXn for GPS, .XXg for GLONASS, .XXe for Galileo)?
Is it possible to choose the version of the RINEX file to generate?
If this library is not for me, how could I do it? I was reading about an RTKCONV tool and the possibility of launching it from the command line.
I apologize in advance for the triviality of my question, but I really have very little experience in this area. Thank you very much in advance.
The RTKCONV GUI app and the CONVBIN command line app in RTKLIB will both convert raw u-blox binary files to rinex. You can select the rinex version of the output files. The navigation information will default to separate files for the older rinex versions and their is an option to specify separate nav files for the newer rinex versions. There is no need to do anything to the raw receiver data before processing it with these apps. There is more info in the RTKLIB user manual.
By the way the STR2STR and STRSVR apps in RTKLIB will collect the raw data from the receiver and can also be used to configure the receiver.
Thank you very much! So basically I should use the RTKLIB library tools (in my case CONVBIN) and the Python library mentioned in the article is not useful. Or is there something at the Python script level to do all this (I think the issue of problems is decoding the ICDs of the various GNSS)?
In any case, CONVBIN and STR2STR are compatible for Linux/Windows and should not be installed, right?
Last question: currently in my Python program I open a serial connection and save all bytes until the connection (and then the file) is closed as is, except for the NMEA strings which I intercept and put in another file. Can this work or do I need to format the file in a particular way (as u-center would do when recording the stream)?
The python library is only a subset of RTKLIB and does not include the functionality that you are looking for. The executables for CONVBIN and STR2STR that I provide are compiled for Windows. If you want to run in linux you will need to clone the RTKLIB source code and build the executables yourself.
Capturing the raw stream from the receiver to a file with no modification or formatting will work fine. There is no need to remove the NMEA messages since RTKLIB will ignore them when parsing for observation and navigation data. It will parse the NMEA messages if you use RTKPLOT to plot a solution.
This is very useful code and learning material for students. Thanks for sharing!
This will be a great project, I hope everyone can participate in the development!
Tnx Tim for the great effort ! Will surely be appreciated by the community. Thank you.
Tim, This is really cool!! I run RTKNAVI with RTCMs that i stream from my C gps code. Are you thinking of creating an rtknavi.py as well by any chance?
Don Kelly Technologist | Engineer Agile Engineering, LLC
LinkedIn: http://www.linkedin.com/in/kellydak Cell: 281-221-2853 4403 Orange Leaf Court Houston, TX 77059
Hi Don. The python code is intended mostly as a development platform and in many cases would run too slowly for real-time use, so I probably won’t be creating an rtknavi.py. It would also require translation of the raw observations and navigation data to rinex form which is currently not supported.
It is great to see this development. I haven’t tried this python version yet but I am positive that it’s a good and worthy project. If possible and you would like to, I am planning to add some tools in the future, specifically for the PPK and RTK.
Thank you for your efforts and for sharing this. If you are interested we can discuss how to improve the project.
Hi Peshawa. Yes, I am always interested in hearing about improvements to RTKLIB. You can contact me at email@example.com.
Runs fine on a Mac. I did need to adjust the Windows raw pathnames to
datadir = '../data/phone'and
datadir = '../data/u-blox'
I cant wait to try this! unfortunately I wont be able to till late summer. Thank you for your efforts in this area. Maybe this will be a step towards a QGIS plugin that makes the post processing workflow more accessible to more folks.