Online RTKLIB post-processing demo service

[ Update 5/21/19:  Due to changes in the Google Gmail API, this service is no longer functional.  The easiest way to try out the demo5 RTKLIB code is to download the Windows executables from here.]

Recently, while experimenting with low cost dual frequency receivers, I discovered a few of the free online post-processing PPP services available from various academic and government organizations such as OPUS, CSRS, and AUSPOS.  These are a great way to easily compute a fast and accurate precise location for your base station for kinematc RTK/PPK work if you are using a dual frequency receiver.  I’ll leave the details to another post but bring them up here because they were the inspiration for a much more modest version I have put together for generating online PPK solutions using RTKLIB.

In this case, the intent is not so much to actually generate useful solution data, as it is to give new or potential RTKLIB users a chance to try out the software while avoiding some of the learning curve associated with setting it up and running it on their computer.

The PPP online services generally have a web page interface to upload the data and specify configuration options.  They then email the results to you a short time later.  To keep the implementation simple, the RtkExplorer online service works entirely through email.  You send the data and any custom config options in an email and it will email you a solution.  It runs entirely off of a single old laptop setup in the spare bedroom so is not capable of processing enormous amounts of data but hopefully will be able keep up with what I expect to be modest demand at the most.  My son has been back from college for the summer and is much better at coding things like Google APIs than I am, so he actually did most of the work with a little direction from me.

My hope is not only that it may be helpful to a few people getting started with RTKLIB, but also that it will help me improve the demo5 version of RTKLIB.  By seeing more data sets, especially those using low-cost receivers, I hope to better understand how people are using RTKLIB and what limitations they are running into.  So, even if you are already an RTKLIB expert, but run into a data set that you think it should have done a better job with, please submit it and get it added to the database.  I am sensitive to privacy issues, so while I may use relative position plots from the data to demonstrate issues in my posts, I will never share any absolute position information or anything else that would identify where the data came from.

Also, to help me evaluate how the demo5 version of code is working, and to provide a comparison for those who might be interested, two solutions are computed and plotted, one with the latest demo5 code and the other with the latest version of the official 2.4.3 code.

To help the user spot possible reasons for a poor solution, the observations for both base and rover are also plotted, along with some guidelines on common data quality issues to look for.  The solution files and configuration files for both solutions are also attached for more detailed analysis.

Before running the solutions, the data is briefly analyzed for receiver type and sample rate.  If it can be determined that both receivers were u-blox M8Ts, then the solution defaults to continuous ambiguity resolution and GLONASS AR enabled.  Otherwise, the solution defaults to both AR settings set to fix-and-hold with relatively low tracking gain.  Either way it defaults to kinematic mode and forward only solution.  Configuration inputs that are sample rate dependent are adjusted for the sample rate unless they are specified separately by the user.

If the user would like to run with any of the RTKLIB input configuration parameters set to something other than the selected defaults, that can be done by adding the appropriately set line from the config file to the body of the email.  This allows the user to specify a static or combined solution or any other variation that can be specified through the configuration file.  Any antenna types that are included in the ngs14.atx file can be specified.

For the time being, the only raw binary format that is supported is u-blox.  All other data must be in RINEX form.

Once I have a little more confidence that everything is working properly I will post a link and instructions to the rtkexplorer.com website but for now I’ll give some quick instructions here.

RTKLIB Demo Instructions

Send an email to rtklibexplorer@gmail.com that follows these rules:

1. The email subject must include the words “rtklib demo”
2. The base and rover data must be in attached files , either separate or zipped together and the total attachments must be less than 25 MB
3. If the files are zipped, zip the files directly, not the folder that the files are in. (This is a temporary restriction until I improve the code)
4. The rover files must have the letters “rov” in the file names
5. The base files must have the letters “base” in the file names
6. Valid file extensions are .ubx, .*nav, .obs, and .yy* where yy is the year (e.g.   .17o)
7. At least one navigation file from either the base or rover must be included as well as observation files from both rover and base.
8. If the base observations are in a Rinex file, the approximate base position in the header must be valid  and  in XYZ coordinates  (This is a temporary restriction until I improve the code)
9. Config values are optional and should be copied by line directly from the config file into the body of the email, one to a line (e.g.    pos1-posmode = static)

If you’ve done everything right (and everything on my end is working properly), about five minutes after you have sent the email, you should receive a response with observation and solution plots and attached solution and config files.  Here is an example response I received after sending in one of the M8T data sets from a recent post.

Hopefully everything will just work the first time, but please be patient if there are a few glitches getting started.  I will be monitoring the emails closely at first and will try to keep things running smoothly as much as possible.

So, go ahead and give it a try, and help me iron out any last few bugs and also develop a database of data sets for further demo5 code development.

PPK vs RTK: A look at RTKLIB for post-processing solutions

The “RTK” in RTKLIB is an abbreviation for “Real-time Kinematics”, but RTKLIB is probably used at least as often for “PPK” or “Post-Processed Kinematics” as it is for real-time work.  In applications like precision agriculture, where the solution is part of a real-time feedback loop, RTK is obviously a requirement, but in many other applications there is no need for a real-time solution.  For example, drones are often used for collecting photographic or other sensor data but only need precision positions after the fact to process the data.  PPK is simpler than RTK because there is no need for a real-time data link between GPS receivers and so is often preferable if there is a choice.  The downside of course is that if there is something wrong with the collected data, you may not find out until it’s too late.

For the most part, RTKLIB solutions are identical regardless if they are run on real-time data (RTK) or run on previously collected data (PPK).  The most significant exception to this rule is what RTKLIB calls the “Filter Type”.  This is selected in the configuration and can be set to forward, backward, or combined.  Forward is the default and this is the only mode that can be used in real-time solutions.  In forward mode, the observation data is processed through the kalman filter in the forward direction, starting with the beginning of the data and continuing through to the end.  Backward mode is the opposite,  data is run through the filter starting with the end of the data and continuing to the beginning.  In Combined mode, the filter is run both ways and the two results are combined into a single solution.   This mode is set using the “Filter Type” box in the Options menu if using one of the GUI apps, or with the “pos1-solytpe” input parameter in the configuration file if using a CUI app.

There are two advantages to a combined solution over a forward solution.  First of all, it gives two chances to find a fix for each data point.  Let’s say there is an anomaly in the middle of the data set that causes the solution to switch from fix to float and not come back to fix for some period of time.   It may cause both the forward and backward solutions to lose fix but they will lose fix on opposite sides of the anomaly.  By combining the two solutions we are likely to get a fix for everywhere except right at the anomaly.  Another case where it often helps is in recovering the beginning of a data set.  Let’s say the first fix didn’t occur until five minutes into the data set.  With a forward solution, you would need to guarantee that nothing important happened during that five minutes, but with a combined solution, the backward pass will normally provide a fix all the way to the very beginning of the data set so there is no lost data.

The second advantage of the combined solution is that it provides an extra level of validation of the results.  To understand how this happens, it’s important to understand how RTKLIB combines the forward and reverse solutions.  For each solution position point there are three possibilities; both passes are float, one is float and one is fix, or both are fixed.  If both passes generate a float position, then the combined result will be a float with a value equal to the average of the two positions.  If one is float, and the other is fix, the float is thrown away and the fix is used.  In the case where both are fixed, then RTKLIB will attempt to validate the result by comparing the two values.  If they differ by less than four sigma, then the result will be a fix, otherwise it will be downgraded to a float.  Either way, the value will be the average of the two positions.  This degrading the solution type when the answers from opposite directions differ provides an increased confidence in the solution, at least for points for which we got two fixed values.

I will show a couple examples of the differences between forward and combined modes.  The first example is a more typical case and demonstrates how combined mode will normally give you a higher fix percentage while at the same time increasing confidence in the solution.

The plots below were taken from an M8N receiver on a sailboat using a nearby CORS station as base.  With ambiguity resolution mode set to fix-and-hold, I was able to get a solution with nearly 100% fix except for the initial convergence, but I would prefer to use continuous ambiguity resolution because of the higher confidence of the solution.  In the position plots below, the top was run in forward mode, the middle in backwards mode, and the bottom in combined mode, all in continuous ambiguity resolution mode.

As you can see the forwards and backwards mode solutions are not bad but both have gaps of float in the middle as well as floats during the initial acquisition.  The combined solution though has almost 100% fix rate and in addition includes the additional confidence knowing that every point found the same solution when running the data in opposite directions.

This second example comes from a data set posted on the Emlid Reach forum with a question on why the combined solution was worse than the forward solution.  In the plots below, the top solution is forward, the middle is backward, and the bottom is combined.

This data was GPS and SBAS only, so had a fairly low number of satellites, also included a mix of poor observations and the solution was run with full tracking gain (i.e fix-and-hold with the default gain).  Both forward and backward runs found fixed (green) solutions and tracked them all the way through the data set.  However, at least one of them was most likely a false fix, causing the fix to be downgraded to float (yellow) for most of the combined solution as can be seen be seen in the bottom plot.

To confirm this, the plot below shows the difference between the forward and backward solutions.  As you can see, the two differ by a fairly substantial amount and it is not possible from this data to know which one is correct.

In this case, turning off fix-and-hold and running ambiguity resolution in continuous mode sheds some light on what may be going on.  The plots below are again forward, backward, and combined.  This time the forward solution loses fix early on and never recovers it, whereas the backwards solution maintains a fix through the whole data set and is probably correct since without fix-and-hold enabled, it is very unlikely to stay locked that long to an incorrect solution.  The backward solution is also consistent with the beginning of the forward solution, since the combined solution remains fixed in the early part of the data set where both forward and backward solutions are fixed.

Again, this can be confirmed by looking at the difference between the forward and backward solutions.  In this case they agree everywhere that both are fixed.

As this example demonstrates, if post-processing is an option, it often makes sense to run in combined mode with continuous ambiguity resolution instead of forward mode with fix-and-hold enabled.  The additional pass will increase the chances of getting a fixed solution without the risk of locking onto a false fix that fix-and-hold can cause.  Even if you find you can not disable fix-and-hold completely, it may allow you to reduce the tracking gain (pos2-varholdamb)

So one last question is why are there still some float values in the middle of the combined solution? We would expect that since the backwards solution is fixed and the forward solution is float, that the combined solution should just become the backwards solution and all but the very end should be fixed.

The answer to this question turns out to be the way the reverse pass of the kalman filter is initialized.  I have chosen in the demo5 code to not reset the filter between forward and reverse passes if continuous ambiguity resolution is selected.  If fix-and-hold is selected then the demo5 code does re-initialize the kalman filter between passes.  This is different from the release code which always resets the filter between passes.

In this case, the results would have been slightly better if the filter were re-initialized but most of the time I find that allowing the filter to stay converged avoids a large gap in the backwards solution during the active part of the data set where the filter is reconverging. With fix-and-hold enabled I have found the chance of staying locked to an incorrect fix is too high and so it is better to reset the filter.  This is a recent change and hasn’t yet made it into the released version of demo5 but I should get it out soon.  The current version of the demo5 code (b28a) does not reset the filter for either case.

Modifying the if statement in the existing code in postpos.c to match the line below will give you the newest behavior.  Removing the if statement altogether will cause the filter to always be reset and will match the release code.

The other factor to consider when deciding whether to run the filter type in forward or combined mode is that combined mode will take nearly twice as long to run since it is processing each data point twice.  Most of the time this shouldn’t be an issue since it is not being run in real-time.

So to summarize, my recommendation would be to use combined mode if you do not need a real-time solution as the only real cost is a small amount of additional computation time and it will give you both higher fix percentages and more confidence in those fixes.