RTKLIB Benchmarking: versions 2.4.2, 2.4.3, and demo5

It’s been about four years now since I created the demo5 branch of RTKLIB. During that time I have added a number of features and enhancements to the code with a focus on low-cost receivers (primarily u-blox) and moving rovers while at the same time keeping synced with the latest 2.4.3 code from the official RTKLIB code base. I thought it would be interesting to do a little benchmarking between the the two official versions of RTKLIB (2.4.2 and 2.4.3) and the demo5 code to see to what extent this evolution of the code has affected the results.

First of all, though, it’s probably worth a little discussion about versions 2.4.2 and 2.4.3. Source and executables for both versions are available on Github at https://github.com/tomojitakasu. The source is in the RTKLIB repository and the executables are in the RTKLIB_BIN repository. Both of these repositories default to the “master” branch which is the 2.4.2 code. This is what you will get unless you specifically request the “2.4.3” branch of the code. For several years, almost all the development activity was on the 2.4.3 branch and only very minimal changes were being made to the master branch. In Jan 2018, there was a merge of the 2.4.3 changes back to the master branch although it appears that not all of the changes in 2.4.3 were merged back into the master branch. Since then development has continued on 2.4.3 without another merge back to the master branch.

For most of my data analysis posts, I focus on a single data set, spending a fair bit of time to make sure I am only analyzing the usable parts of the data, possibly tweaking the configuration file for that specific data, and digging into any issues that crop up. In this case I didn’t do that. I picked nine raw data sets, all with u-blox M8T receivers and a moving rover, made no effort to filter out bad data, and used a single generic configuration file for all nine data sets. Eight of the data sets are those that I have previously uploaded to my website at http://rtkexplorer.com/downloads/gps-data/ and the ninth was from my most recent drive around the neighborhood with a u-blox M8T and an antenna on top of the car. I ran solutions for each of the three RTKLIB versions on all nine data sets. For each solution, I converted the data from u-blox binary to rinex using the same code version as I did for the solution, since the different codes will affect this conversion as well as the solution.

I ran the post-processed solutions in “combined” mode, meaning that the solution is run both forward and backward and the results are then combined. Not only does this tend to produce better results, but the results also have higher confidence since RTKLIB compares the forward and backwards solutions, sample by sample, and downgrades any sample where the solutions in both directions are fixed and the results differ by more than four standard deviations. This tends to do a good job of detecting and rejecting any false fixes in the results. However, it is not foolproof. If the solution is fixed in only one direction and float in the other, then there is no additional validation.

I used the same configuration settings for running each of the three RTKLIB versions on each of the data sets with a few exceptions. Versions 2.4.2 and 2.4.3 do not have the “arfilter” feature that automatically holds off new satellites until their phase bias estimates have converged enough to not break the ambiguity resolution so I increased the fixed hold off (arlockcnt) from 0 to 10 for the 2.4.2 and 2.4.3 codes. The outlier detection scheme is also different in the demo5 code from the other two versions, making it necessary to increase the outlier threshold (rejionno). In this case I increased the threshold from 1.0 to 30.0 for versions 2.4.2 and 2.4.3. Lastly, the 2.4.2 code runs very slowly if dynamics is enabled, so I turned dynamics off for this code.

The specific code versions for the experiment were: 2.4.2 p13, 2.4.3 b33 and demo5 b33b2.

I used fix percentage as a metric for the test. This isn’t a perfect metric because it doesn’t take into effect the accuracy of the non-fixed results and it can be affected by false fixes. Overall, though it is probably the best single metric for this kind of test, especially since running the solutions in combined mode should reject many of the false fixes.

The fix percentages for each test are listed below. The data set names correspond to the names of the sample data sets on my website.

Test	V 2.4.2	V 2.4.3	V2.4.3a	demo5	data set
1	37.7%	93.9%	96.3%	94.4%	union_0705
2	35.7%	9.8%	60.6%	85.5%	niwot2_car
3	52.4%	16.2%	89.0%	80.2%	drone_0414
4	85.8%	51.2%	93.8%	98.8%	m8t_niwot_0606
5	59.7%	21.4%	83.7%	97.1%	swift_m8t_road_0606
6	63.1%	10.0%	83.6%	99.0%	car_1114
7	54.6%	26.6%	73.3%	94.2%	car_0320
8	39.4%	11.2%	54.4%	98.1%	comnav_car
9	25.0%	9.2%	53.4%	98.1%	not uploaded
median	52.4%	16.2%	83.6%	97.1%

Experiment results in fix percentage

For the most part, the results match my expectations. Version 2.4.2 has the lowest fix percentage, 2.4.3 is in the middle, and demo5 has the highest. However, I ran into one very significant issue with the 2.4.3 code that I do not fully understand. In the table above, the “V2.4.3” column is the results using version 2.4.3 for both the conversion from raw binary to rinex as well as for the solution. As you can see, the fix percentages were very low for this test for all data sets except the first, significantly lower than even the 2.4.2 results. I did not fully debug this issue but the problem appears to be in the conversion from raw binary to rinex, not in the solution itself.

The V2.4.3a column is the results for running the 2.4.2 code for the raw binary to rinex conversion, and then the 2.4.3 code for the solution. This result is much more within my expectations for the 2.4.3 code. I suspect that the issue with the 2.4.3 rinex conversion is that when it is filtering out low quality observations it is not preserving the cycle-slips. RTKLIB can be very sensitive to unflagged cycle slips.

I am very curious if anyone who is a regular user of the 2.4.3 code can duplicate this result. As you can see from the first data set, it does not always occur, and is much less of a problem if the data does not have a large number of cycle slips, so it would need to be tested on a more challenging data set to see this issue.

Regarding the demo5 results, seven of the nine tests had over 94% fix rate and the median fix percentage was 97.1%. I consider this quite reasonable since all of these were fairly challenging data sets and most of them included at least a small amount of unusable data. The two data sets that did not perform as well (80-86% fix percentage) were both older. One did not include Galileo and the other was from a drone that had one particularly poor quality section of data. However both solutions had well over a 90% fix rate when run in the forward only direction which indicates the fixes in the combined solution were downgraded because of mismatches between the two directions. In one of these cases (test 3), the 2.4.3 solution obtained a higher fix rate than the demo5 code but it only got fixes in the forward direction, not in the backward direction so had no additional validation. Based on some discrete jumps in that solution, I suspect it would have also downgraded the fix percentage if it had achieved fix in the backwards direction.

Looking more closely at the cases where the demo5 solution points were downgraded to float for mismatches, it’s interesting because, at least at first glance, it appears that these were not false fixes, but discrepancies from using different combinations of satellites that were large enough to trigger the four standard deviation threshold. This is a little concerning and worthy of further investigation. Fortunately it only appears to occur when the data quality is fairly poor. However, this does emphasize the importance of insuring the best quality measurements possible, and not over-relying on RTKLIB to reject inaccurate solutions.

As always, I’d like to emphasize that these tests are intended only as one users snapshot of one fairly particular use case of RTKLIB and are not intended to be any kind of comprehensive analysis. Also, it’s important to understand that 99+% of the code in all versions of RTKLIB including the demo5 code are the result of many years of dedicated effort by Tomoji Takasu and his team at Tokyo University of Marine Science and Technology. My only contribution has been to add a few changes on top of this code to make it a little more focused on practical application for specific uses rather than a more generic academic tool.

Lastly, for reference, here’s a partial list of the most important configuration settings I used for this experiment for the demo5 code. The other two codes used the same settings with the exceptions I describe above.

pos1-posmode =kinematic # solution mode
pos1-soltype =combined # solution type (forward, backward, combined)
pos1-frequency =l1 # (l1, l1+l2, l1+l2+l5)
pos1-elmask =15 # min sat elevation to include in solution (deg)
pos1-snrmask_r =off # SNR mask rover (off, on)
pos1-snrmask_b =off # SNR mask base (off, on)
pos1-dynamics =on # add dynamic states to kalman filter (off, on)
pos1-navsys =15 # (1:gps+2:sbas+4:glo+8:gal+16:qzs+32:comp)
pos2-armode =fix-and-hold
pos2-gloarmode =on # Glonass AR mode (off, on, fix-and-hold, autocal)
pos2-bdsarmode =off # Bediou AR mode (off, on)
pos2-aroutcnt =50 # outage count to reset sat ambiguity (samples)
pos2-arminfix =50 # min # of fix samples to enable AR hold (samples)
pos2-rejionno =1.0 # phase bias outlier threshold (m)
pos2-maxage =30 # max age of differential (secs)
pos2-arthres =3.0 # minimum AR ratio for fix (m)
pos2-arthres1 =0.1 # max variance of position states to attempt AR (m)
pos2-varholdamb =0.1 # variance of fix-and-hold tracking feedback (cyc^2)
pos2-arfilter =on # automatic hold off for adding new sats to AR
pos2-arlockcnt =0 # fixed hold off for adding new sats to AR (samples)
pos2-minfixsats =4 # min sats required for fix
pos2-minholdsats =5 # min sats required for AR hold
pos2-mindropsats =10 # min sats required to drop sats from AR
stats-eratio1 =300 # ratio of input stdev of code to phase observations
stats-eratio2 =300 # ratio of input stdev of code to phase observations

A few simple debugging tips for RTKLIB

Unfortunately, error reporting is not one of RTKLIB’s strengths.  Sometimes it will halt with an error message that is less than helpful, other times it will just continue through the entire solution with Q=0 or Q=2 without ever getting a fix because of a minor error in the input file configuration or parameters. I suspect anyone who has used RTKLIB with any regularity has run into cases like this innumerable times. It still happens to me all the time, almost always because I have made a silly mistake somewhere. For a new RTKLIB user, it can be quite difficult to figure out why they are not getting a valid solution in cases like this.

In this post, I will demonstrate some examples of common errors and some techniques to help identify them. First let’s take a quick look at a couple of real-time solution examples with RTKNAVI, then I will jump to post-processing cases for the rest of the examples.

In my experience, the most common error when using RTKNAVI is to improperly specify the base location. To demonstrate this, I go to the “Positions” tab on the “Options” menu in RTKNAVI and switch the Base Station position type from “Average of Single Position” to “RTCM Position”. In this example, my base stream is u-blox binary and contains no RTCM messages so RTKLIB will not get any valid base location information and can not calculate a solution. However, rather than halting and letting the user know there is a problem, RTKNAVI just runs through the whole solution without ever reporting either an error or a valid position. To help understand what happened in a case like this, the first step is to click on the tiny unlabeled square above the “Start/Stop” button. This will bring up the RTK Monitor window. Choose “Error/Warning” from the Monitor options as shown in the image below to bring up additional error and warning information.

In this case, every sample reports an “initial base station position error”. This should be enough information to track down the problem fairly quickly. Often this will be the case. If the additional information in this window doesn’t help, then the next step is to enable the debug trace and look at the trace file. This is covered in more detail in the post-processing examples further below.

In my experience, the second most common reason to get no solution with RTKNAVI is that navigation messages haven’t been enabled for either base or rover receiver. If this is the case, then all the SNR bars will be gray. When navigation data is available, the bars will be colored as in the image above. Lack of SNR bar color is the easiest way to detect this problem but another symptom is that the “Error/Warning” window will report “No common satellites” for every sample even though satellites are appearing in the SNR window.

A more general troubleshooting technique that will work for either real-time or post-processing and for GUI or command line RTKLIB applications is the debug trace file. I will demonstrate a couple of examples of using this with RTKPOST but the same basic technique can also be used with RTKNAVI, RNX2RTKOP, RTKCONV, CONVBIN, or STRSVR.

To enable the debug trace, set “Output Debug Trace” in the “Options” menu to something other than “OFF”. For simple errors, level 2 is enough and will produce a less overwhelming amount of information. For more detailed information about the GNSS solution, especially the ambiguity resolution, level 3 is better. For more details about the communication streams, level 4 can be useful. Level 5 includes many of the solution matrices and is too detailed for most general debug. Level 1 only contains the error messages available in the error window and is not useful. The image below from the RTKPOST Options menu shows the trace level set to 3. I usually set it to level 3 since I am often looking at the details of the ambiguity resolution, but for simple debug I would recommend starting with level 2 and that is what I will use in the examples below.

Next, I will intentionally make some silly mistakes and show how they appear in the error window and the trace file.

Let’s start with an example where RTKPOST actually does a good job of reporting the error to demonstrate how it should work, and occasionally does. To keep the example simple I will use files in the root directory, with names: rover.obs, base.obs, and rover.nav.

In this first case, I have made a typo in the name of the .nav file. Now, when I click on “Execute” to start the solution, it almost immediately halts and reports “Error: no nav data” in the error window near the bottom of the window. If only RTKLIB would always be as straightforward as this!

Next, I intentionally make a typo in the rover observation file and click on “Execute”. Again, the solution halts almost immediately but in this case the error message in the error window is much less useful. Not only does it not tell me which file has a problem, but it also suggests there is something specific wrong with the input data, not that the file is completely missing. Fortunately, clicking on another tiny unlabeled square (circled in red below) brings up the trace file, assuming trace debug was enabled as described above. This tells us exactly what went wrong, that the input rover observation file could not be opened.

For the next example, I will correct the typo in the file name but then set the location in the base rinex file header to all zeros. This is another common problem and usually happens because the rinex file was generated by RTKCONV or other application from a binary receiver output file that did not contain navigation messages. Without navigation information, RTKCONV can not estimate the receiver location and sets the approximate position field in the rinex header to all zeros. Again, the solution halts almost immediately, and the error message is identical to the previous case. However, looking at the trace file, there is now no mention of a file open error. To understand the difference between these two trace files, it’s important to know that the entries in the trace file beginning with 2 are warnings, and those beginning with 1 are errors. RTKLIB treats the missing file as a warning since there may be other observation files that are present, and doesn’t get an actual error until it looks for the base position information. With just the error, it is impossible to tell these two case apart, but by looking at the warnings as well, it is fairly easy to differentiate them.

Another fairly common error is to use the wrong base observation file. If the base observations do not overlap the rover observations in time, then the solution will run all the way through the data, without error, but the solution status will always be Q=0. Here’s the trace output for this example. I aborted the run before it completed, after it was obvious that something is wrong which is why the error windows shows “aborted”. The “age of differential” is the time between rover observation and closest base observation so again in this case, looking at the trace file makes the problem much more obvious

If the wrong navigation file is used, the solution will again run to the end with no error, and the status will remain Q=0, just as above, but this time the trace file will report “no common satellites”

Another mistake I make fairly often, is when I am specifying the precise base location rather than using the approximate location in the rinex header, and I forget to change it when moving to a different data set with a different base location. In this case, the solution will run without error, but the fix status will remain in float (Q=2) for the entire solution. There are several reasons why you might never get a fix, including just poor quality data, but in this case when I look at the trace file I see extremely large outliers right from the first sample. If all the satellites are reporting outliers and they are as large as this, that almost always means that the base location has been specified incorrectly.

These are only a few examples but probably cover over 80% of cases I look at where people are unable to get a solution. Most of the rest are caused by low quality data.

If you have other examples of common failures that I haven’t covered here, please describe them in a comment, along with the signature seen in the trace file.

A first look at the u-blox ZED-F9P dual frequency receiver

The new low cost dual frequency receiver from u-blox, the ZED-F9P, is just now becoming available for purchase for those not lucky enough to get early eval samples from u-blox.  CSGShop has a ZED-F9P receiver in stock for $260 which seems quite reasonable, given that it is only $20 more than their NEO-M8P single frequency receiver.

Even better, Ardusimple is advertising an F9P  receiver for 158 euros (~$180) + 20 euros shipping , although their boards won’t ship until January.  As far as I’m aware of, this is actually less than anybody today is selling the M8P receiver for today!

Of course, this is still a fair bit more than a u-blox M8T single frequency receiver without an internal RTK engine, which is available from CSGShop for $75, but the F9T will be coming out next year also without internal RTK engine, which should bring down the price for the lowest cost dual frequency receivers.

Unfortunately I am not one of the lucky ones who got eval boards directly from u-blox yet.  However, I do have two prototype boards from Gumstix, given to me by them for evaluation.  Gumstix offers both off-the-shelf boards and semi-custom boards designed from their libraries of circuits.  I haven’t worked with them directly but it looks like an interesting and useful concept.  The F9P boards from Gumstix won’t be available for sale until at least Feburary next year but I thought I would share the results of some initial testing.  From a performance perspective, I would expect these boards to be similar to F9P boards from other suppliers.

For a first look, I chose to compare the F9P to an M8T for one of my typical driving-around-the-neighborhood exercises.  I looked at both the internal real-time F9P solution and the RTKLIB solutions, both real-time and post-processed.

Experiment Setup:

For the base stations, I connected a CSGShop M8T receiver and a Gumstix F9P  receiver through an antenna splitter to a ComNav AT330 dual frequency antenna on my roof.  Since RTKLIB doesn’t yet fully support the receiver commands needed to setup the F9P, I used the most recent version (18.08) of the u-blox u-Center app run on a Windows laptop to configure the F9P receiver using the documentation on the u-blox website.  I then saved the settings to flash.  The receivers were connected to a laptop with USB cables and I broadcast the base observations over the internet on a couple of NTRIP streams using STRSVR and RTK2GO.com as I’ve described previously.  I configured the F9P to send RTCM3 1005, 1077, 1087, 1097, 1127, and 1230 messages which include base location, raw observations, and GLONASS biases.

For the most part the u-blox documentation is well written and this exercise was fairly straightforward, but I did run into a couple of issues.  First of all, when I plugged the F9P receiver into the laptop, Windows chose the standard Windows COM port driver instead of the u-blox GNSS COM port driver that it chose for the M8T receiver.  You can see this in the screen snapshot below where COM17 is the M8T and COM21 is the F9P.

Both drivers allow the user to set the baudrate in the properties menu available by right clicking on the device name.   With the u-blox driver, the baudrate setting doesn’t seem to matter which makes sense since it is a USB connection.  I have always left the u-blox driver baudrate at the default of 9600 baud without any issue.  With the windows driver, however,  I found that I had to increase the baudrate setting to 115200 to avoid data loss issues.  I have run into a similar problem before for sample rates greater than 5 Hz when the M8T is accessed through it’s UART interface and an FTDI converter is used to translate to USB, rather than communicating directly through it’s USB interface.  I verified though, that in this case the board is using the USB interface on the receiver and not the UART interface.   Not a big deal, and it may be unique to this board, but something to be aware of in case you run into a similar problem.

The second problem I ran into is that the F9P module seems to be sensitive to my antenna splitter, a standard SMA DC block and tee which I have used on many other receivers before without issue.  It works fine if the F9P power is blocked but if the M8T power is blocked, the F9P seems to detect the tee and shut off the antenna power.  Again, not a big deal, but something to be aware of.

For the rovers, I used a u-blox ANN-MB-00 dual-frequency antenna for the F9P receiver.  This is the antenna u-blox provides with its F9P eval units.  I had planned to split this antenna signal to both receivers as I usually do, but I ran into the problem described above, and not fully understanding the issue yet, ended up using a separate Tallysman TW4721 L1 antenna for the M8T receiver.  Both antennas were attached directly to the car roof which acted as a large ground plane.

I used a hot spot on my cellphone to stream the NTRIP base station observations from the phone to a laptop and then to the F9P receiver and to two instances of RTKNAVI, one for each rover receiver.

Streaming the base observations to the F9P, while simultaneously logging the internal RTK solution and the raw rover observations, and also sending the raw rover observations to RTKNAVI, all over a single serial port can be challenging since only a single application can be connected to the serial port at one time.  Fortunately RTKLIB has a little trick to deal with this.  If the “Output Received Stream to TCP Port” box is checked in STRSVR and a port number specified as shown below, all data coming from the other direction on the serial port will be redirected to a local TCP/IP port.  This data  can then be accessed by any of the other RTKLIB apps as a TCP Client with server address “localhost” using the specified port number.

I set up the F9P rover to output both raw observation and navigation messages (UBX-RXM-RAWX/ UBX-RXM-SFRBX) and solution position messages( NMEA-GNGGA).  RTKNAVI then logged all of these messages to a single log file.  RTKCONV and RTKPLOT can both extract the messages they need from this file and ignore the rest so combining them was not an issue.

The NMEA-GNGGA messages from the F9P default to a resolution of 1e-7 degrees of latitude and longitude which works out to roughly 1 cm.  For higher resolution you can increase the resolution of the GNGGA message by setting a bit in the UBX-CFG-NMEA message.   Unfortunately I did not realize the resolution issue until after I collected the data and so my results for the internal F9P solution for this experiment were slightly deteriorated by the lower resolution.

I used the most recent demo5 b31 code to calculate all of the RTKLIB solutions.  Both the demo5 and the 2.4.3 versions of RTKLIB have been updated to translate the new dual frequency u-blox binary messages.  The demo5 solution code will process all the dual frequency observations but I don’t believe 2.4.3 code is able to process the E5b Galileo measurements yet.  The RTKLIB 2.4.2 code however does not have any of these updates.

The demo5 code updates made in the recent B30/B31 versions are based on the updates from the 2.4.3 B30 code but include some modifications to the u-blox cycle slip handling that I had previously added to the demo5 code for the M8T.  Since the demo5 code is primarily aimed at low cost receivers I also made some changes to make the E5b frequency a little easier to specify and faster to process.

To run the RTKNAVI F9P real-time solution, the only significant change I needed to make to the default M8T config file was to change the frequency option from “L1” to “L1+L2+E5b”.  I should have also changed the base station position to “RTCM Antenna Position” to take advantage of the F9 base station RTCM 1005 base location messages but neglected to do this.  This caused the RTKNAVI solution to differ from the F9P solution by small constant values due to the approximate base location used in the RTKNAVI solution.  I later used exact base locations for the RTKLIB post-processing solutions to verify that the different solutions did in fact all match.

Once I had everything set up, I then drove around the local neighborhood, emphasizing the streets with most challenging sky views since I knew both receivers would perform well and be difficult to distinguish if the conditions were not challenging enough.

Results:

I first converted the binary log files to observation files using RTKCONV and verified that the F9P was logging both L1 and L2 measurements for GPS, GLONASS, and Galileo.  I had the Bediou constellation enabled as well but as I verified later, there were no fully operational Bediou satellites overhead when I collected the data.

Here is an plot of the L1 observations for the M8T on the left and the F9P on the right.  I have zoomed into just two minutes during some of the more difficult conditions to compare the two.  The red ticks are cycle slips and the grey ticks are half cycle ambiguities.

First, notice that the F9P does not log observations for the SBAS satellites, while the M8T does, giving the M8T a couple more satellites to work with.  However, what’s also interesting, and I don’t know why, is that the F9P collected quite good measurements from the Galileo E27 satellite, while the M8T did not pick up this one at all.  Of course the F9P also got a second set of measurements from the second frequency on each satellite and so overall ended up with nearly twice as many raw observations as the M8T.

Also notice that the F9P reports somewhat less cycle slips and many less half cycle ambiguities than the M8T.  Some of this might be because of the different antennas, but particularly the large difference in half cycle ambiguities suggests that u-blox has made other improvements to the new module besides just adding the second frequencies.

Another thing to notice is the number of Galileo satellites.  If you compare these plots to earlier experiments I’ve posted, you’ll notice there are more Galileo satellites now as more and more of them are starting to come online.  The extra satellites really help the M8T solutions because as you can see, they tend to have the highest quality observations through the most difficult times.  Again I don’t know why this is.  It doesn’t appear to be as true for the F9P though.

Next I looked at the real-time solutions.   First, the RTKLIB solutions with RTKNAVI for both receivers.  For the full driving route, the M8T solution had a 77.3% fix rate and the F9P solution had a 96.4% fix rate.  Here is a zoom into the most challenging part of the drive, an older neighborhood with narrower streets and larger trees, the M8T is on the left, and the F9P on the right.  Fixed solutions are in green and float in yellow.  Clearly here the F9P significantly outperformed the M8T.

The F9P internal solution did even better with a 99.2% fix rate, as shown in the plot below.  All three solutions agreed within 2 cm horizontal, a little more in vertical, and none of them showed any sign of any false fixes.

I didn’t do any static testing to characterize time to first fix as I sometimes do, but for this one run the RTKLIB time to first fix for the M8T was 18 seconds while the RTKLIB F9P solution reached first fix in 6 seconds.  In both cases, RTKLIB was started after the hardware had time to lock to the satellites and acquire navigation data for all satellites.  The demo5 RTKLIB code has an additional fix constraint based on the kalman filter position variance to minimize false fixes while the filter is converging and so this can sometimes affect time to first fix.  I had increased this parameter to 0.1 meter for this experiment since the large number of measurements reduces the chance of a false fix.  This constraint did not limit the M8T time to first fix but it did so for the F9P, meaning the F9P would have reached first fix even faster if this constraint were opened up more.   I can’t tell what the equivalent number would be for the internal F9P solution from this data since it had already been running and achieved a fix before I started logging the data but generally the F9P seems to acquire first fix very quickly.

Next I post-processed both data sets with RTKLIB using the combined-mode setting to run the kalman filter both forwards and backwards over the data.  This noticeably improved the results, bringing the fix rate for the M8T up from 77.3% to 96.1% and the F9P fix rate from 96.4% to 98.8%.

Conclusion:

Obviously this is not enough data to make any definitive conclusions, but so far I am very impressed with the F9P!  Both the raw observations and the internal RTK solutions for the F9P look as good as anything I’ve seen from receivers costing many times what this one cost.

If anybody would like to look at the data from this experiment more closely, I have uploaded it to here.  I should mention that all the fix rates I specify in this post and other posts won’t exactly match the fix rates in the raw solutions, since I adjust the data start and end times to be consistent between data sets and to start after all solutions have achieved first fix.  I believe this is the fairest way to compare multiple solutions, especially when there is a mix of internal and RTKLIB solutions

Also, I’d like to thank Gumstix again for making these modules available to me for evaluation!

Update: 12/2/18:

Reviewing the config files I used for this experiment I discovered that, while I had intended the real-time and post-processing config files to be identical, there were in fact some small differences between them.  One difference in particular, that appears to have affected the results as described above, is that I reduced the minimum number of consecutive samples required to hold ambiguities (pos2-arminfix) from 100 to 20 for the post-processed config files.  A value of 100 corresponds to 20 seconds at the experiment’s 5 Hz sample rate which is a value I have typically used.  However, with lower ambiguity tracking gain (pos2-varholdamb=0.1) and the increase in observations coming from including Galileo, the chances of false fixes is reduced and I have been tending to use lower values of arminfix in more recent experiments.   Reducing this value appears to explain a large part of the jump in percent fix for the M8T between real-time and post-processing, rather than the switch from forward-only to combined that I attribute it to above.  These differences only affect the comparisons between RTKLIB real-time and post-processed results, and not between the M8T and the F9P since the config files were consistent between the two receivers.

This was only intended to be a quick first look at the F9P.  It will require more data and more analysis to properly characterize the F9P so I  won’t try to do that here but I will share the table shown below which includes a few cases I have run since the original post.  I hope to dig into the details in future posts.

	Fix percent
	Real-time	Post-process	Post-process	Post-process
	ARMIN=100	ARMIN=100	ARMIN=20	ARMIN=20
	forward-only	forward-only	forward-only	combined
M8T/RTKLIB	77.3%	81.2%	96.0%	96.1%
F9P/RTKLIB	96.4%	99.1%	99.3%	98.8%
F9P internal	99.2%

One last point worth making is that while at first glance the post-process fix percent increase from M8T=96.0% to F9P=99.3% may not sound that significant, it is in fact a factor of nearly six if you consider it as a decrease in float from 4.0% to 0.7%.

Event logging with RTKLIB and the u-blox M8T receiver

Event logging is a nice feature that has been available in the Emlid version of RTKLIB for a long time.  In the latest version of the demo5 code (b29e), I have ported this feature from their open-source code repository.  Their version is specifically for the u-blox M8T receiver but I have extended it to support the Swiftnav receiver as well.  I mentioned this feature in my previous post and had a couple requests for more information, hence this post.

Both the u-blox and Swiftnav receivers have hardware/firmware to capture the precise time an external pin changes state and send out a binary message with this information.  The RTKLIB event logging code decodes these messages and logs the events to the rinex file.  The events in the rinex file are then used in post-processing to generate a position log containing an interpolated position for each timing event. The most popular use for this feature is probably to record camera shutter times but it can also be used for other purposes such as marking survey locations in the data stream.

Here is an example of a drone flight from a data set containing events that I downloaded from the Emlid forum.  On the left is the ground track of the standard position solution plotted with RTKPLOT.  It includes one point for every rover observation epoch.  On the right is a plot of the event positions from the new event position file.  In this case there is one point for every event which gives precise locations for each camera image.  This is very useful information when processing the images.

Here are the positions of the two plotted on top of each other, green dots are the rover observation epochs from the position file and the blue dots are the events from the event position file.  As you can see from the plot, the event positions are interpolated from the observation epochs.

 

There is information in the Emlid and Swiftnav documentation on how to connect an external trigger to their hardware so I won’t cover that here.

Instead, I will go through an example using an M8T receiver from CSGShop.  I will also use this example to try and validate this feature since there has been some discussion on the Emlid forum about potential issues that as far as I can tell have not been completely resolved on the forum.

The CSGShop M8T receiver comes in several variations.  To use event logging you will need to choose a board that provides access to the external interrupt pins.  You can use either EXINT0 or EXINT1.  For this experiment I also use the TIMEPULSE pin to provide triggers for the event logging.  Here is an image of the receiver and the interface pins.

The goal of this experiment is to generate events for which I know their precise timing so I can use them to validate the RTKLIB event logging results.  To do this, I configured the M8T TIMEPULSE output for a period of two seconds and a falling edge that occurs at 0.2 seconds, all in GPST time.  I then connected the TIMEPULSE output pin to the EXTINT1 input pin so that each state change of the output pulse will be recorded as an event.  Although the M8T will record both rising edges and falling edges, RTKLIB is setup to record only the falling edges.

To configure the timing pulse, I used the u-blox u-center app to setup the UBX-CFG-TP5 command as shown below.

I then enabled the UBX-TIM-TM2 messages which the receiver uses to output the event information.  Next, I opened the table view in u-center and configured it to log GPS time, and the rise and fall times for EXTINT1.  This information is extracted from GPRMC and TIM-TM2 messages.  As you can see the falling edges of the pulse are occurring at exactly 0.2 seconds on the even seconds in GPS time so it looks like we have correctly configured the output pulse

Now that I have external events occurring at precisely known times, I can use these to test the RTKLIB code.   The u-blox example command files that I include with the demo5 executables already are setup to enable the UBX-TIM-TM2 messages, so there is no need to make any changes there.

The next step is to collect some base and rover data using the modified receiver as rover.  I did that, and then converted the raw .ubx files to rinex using the new demo5 version of RTKCONV.  The events appear with a time stamp followed by a 5 in the next field to indicate an external event as shown below.  The zero in the last field indicates it is a valid time mark.

The observation epochs are occurring every second, so notice that the event is being logged out of sequence with a one sample delay.  I did not see this with the Emlid data set example described above.  However, I do see the same delay  if I use the Emlid code to convert the binary file instead of my code.  I don’t know if the Emlid hardware has somehow been configured to avoid this sequencing issue or whether it can occur on the Emlid hardware as well.  I’ll get back to this in a minute.

Next I ran RTKPOST to calculate a position solution.  With the new code changes, a *.events.pos file is generated in addition to the *.pos file.  It is the same format as the *.pos file but contains the event positions instead of the observation epoch positions.  Note, that it will be generated for absolute solutions (XYZ,LLH) LLH but not for relative (ENU) solutions.

I first did this with the Emlid code and got the following result when plotting both the position file and event position file.

The events are occurring at the correct times, but note that unlike the previous example, the positions are not being correctly interpolated between the two closest observation epochs.  In fact, if you look carefully you will see they are being extrapolated from the two previous observation epochs.  This is most obvious in the N-S axis points and is occurring because the events are being logged out of sequence.

To fix this, I modified the interpolation code to use the nearest observation epochs even when the event logging was delayed by one sample.  Here is the result using the latest demo5 b30 code.

Looking at the time stamps from the position log and the event position log, shown below, you can see that the observation epochs are occurring on the integer seconds and the events are occurring 0.2 seconds later on the even seconds, all in GPST time, just as we set them up to occur and verified with u-center.

So I don’t fully understand why the time stamps are appearing out of sequence with the CSGShop M8T data and not in the Emlid M8T data.  It may be that Emlid has configured the hardware somehow so this can not happen.  If this is true, then there should be no issue using the Emlid RTKLIB code with Emlid data but be careful using it with data from other hardware.  If anybody has any additional insight into this discrepancy please leave a comment.

I should also mention that all these code changes are in the core code so are present in both the command line apps as well as the GUI apps.  The most recent demo5 executables (b29e) do not contain the fix for interpolating delayed events and will function the same as the Emlid code.  The Github respository does have this fix.  The fix will also be in the demo5 b30 executables which I hope to release soon.

Comparison of the u-blox M8T to the u-blox M8P

I recently acquired a couple of  u-blox C94-M8P receivers and so I am now able to do a direct comparison between the u-blox M8T and M8P modules, something I’ve been wanting to do for a while.

I believe the hardware between the two receivers is identical, the differences are only in firmware.  The most significant difference in firmware is that the M8P includes an internal RTK solution.  In order to squeeze this into the firmware, however, they had to remove support for the Galileo and SBAS constellations, so this is another fairly significant difference.

Cost, of course, is also different.  The M8T receivers I usually use are available from CSGShop for $75.  CSG sells an M8P receiver for $240 or you can buy a kit with two C94-M8P eval boards from u-blox for $400.  The C94-M8P boards also include on-board radios.

In this post, I will describe how I used RTKLIB and a cell phone hot spot to connect the M8Ps rather than using the internal radios.  I will also compare the RTKLIB solutions for a pair of M8T receivers with the internal u-blox RTK solution for a pair of M8P receivers. I hope to compare the RTKLIB solution to the internal solution for a pair of M8Ps in a future post.  If you are mostly interested in the M8T to M8P comparison results, you can jump directly to the end of this post.

To set up the experiment I first connected both a C94-M8P receiver and a CSG M8T receiver to the GPS survey antenna on my roof using a signal splitter.  I then connected both receivers to a laptop with USB cables.  I configured the M8P using u-center following instructions in the C94-M8P Setup Guide with a few modifications.  First of all, I normally use an NTRIP caster over a cell phone hot spot for my real-time data link so didn’t want to bother with the on-board radios.  I disabled the radios following the instructions in the setup guide.  This involves removing a plastic cover, soldering a wire to a capacitor, drilling a hole in the plastic cover to run the wire, then plugging the other end into a pin on the connector.  It is also possible to do this by connecting an external voltage and ground to the correct pins on the connector, but a simple jumper option would have been a lot more convenient.

The setup instructions are intended to run the setup and solution output messages over the USB port while running the RTCM raw observation messages between receivers over the UART interface.  The UART interface is internally connected to the radios.  Since I am not using the radios, I wanted to run all communication over the USB interface to avoid extra cables.  To do this, I disabled the UART interface and configured the USB interface for UBX messages in and RTCM messages out using the UBX-CFG-PRT configure command from u-center.  Following the C94-M8P setup instructions, I then specified a fixed base station location with the UBX-CFG-TMODE3 command and enabled 1005,1077,1087, and 1230 RTCM messages which include the raw observations, base station location, and GLONASS biases.

I setup the base station M8T receiver as I usually do to output the raw observations using the UBX-RXM-RAWX messages.

I then started two copies of the RTKLIB STRSVR app on the base station laptop and streamed both sets of base observations to an NTRIP server as I’ve described in an earlier post using the free RTK2GO.com community NTRIP server.  With this setup, I can receive the NTRIP streams anywhere I can get cell phone coverage, using a cell phone hot spot and a laptop connected to the two rovers and I can test over much longer distances than the radios would allow.

One thing to be aware of is that there are separate versions of the receiver firmware for the M8P base and M8P rover.  I didn’t realize this at first.  It turned out that both of my receivers came loaded with the rover firmware so I spent an embarrassingly long time unsuccessfully trying to configure one of them as a base station.  Once I realized the problem, I was able to fairly quickly download the base firmware from the u-blox website, then use u-center to download the base station firmware to the receiver.

Next, I attached the two rover receivers to a laptop using USB cables and to a single antenna using a signal splitter.  To make the results more comparable to some of my other recent experiments I used the same GPS-500 L1/L2 antenna from SwiftNav that I used for those experiments.  This is a more expensive antenna than generally would be used with these receivers but I wanted to avoid mixing differences between antennas with differences between receivers.

The M8P rover does not need much configuration since it will by default process any incoming RTCM messages as inputs to an RTK solution.  I did configure the USB port  to support NEMA, UBX, and RTCM messages in and UBX and NMEA messages out.  For the static rover test, I left the position message output rate at the default 1 Hz, but increased it to 5 Hz for the dynamic rover test.  I suspect this only affects the message output rate, and not the solution itself since the solution is probably run at a faster internal rate.

The only other setting I modified in the M8P rover receiver was the dynamic model of the navigation mode using the UBX-CFG-NAV5 message.  I set it to “Pedestrian” for my static rover test and “Automotive” for my moving rover test.

I then started another copy of STRSVR on the rover laptop to stream the M8P base station data from the NTRIP server to the M8P rover over the USB COM port.

At this point, getting the solution output messages from the M8P rover is a little tricky.  Only a single user can attach to a Windows COM port at one time and the STRSVR app is not fully bi-directional.  One solution might be to use u-center to stream the NTRIP data and receive the rover messages but I did not verify if this is possible.

Instead, I used a feature of STRSVR that allows you to pipe the data coming from the other direction to a TCP/IP port which can then be processed by either another copy of STRSVR, or plotted with RTKPLOT, or used as an input to RTKNAVI, or all three at once.  Below, on the left, I show how I modified the STRSVR setup that streams the base data from the NTRIP client to the rover by checking the “Output Received Stream to TCP Port” and selecting port 1000.  On the right, I show the STRSVR setup necessary to stream this incoming data from the rover to a file.  This is a convenient work-around for the problem of only being able to connect one user at a time to a COM port.

In my case I configured the rover receiver to send both the M8P internal solution position using NMEA messages and the raw observations using UBX messages so that I could post-process the raw observations later with RTKLIB.  In general though, it is only necessary to send the NMEA messages.  I configured the rover to send GGA and RMC NMEA messages, but others will work as well.

For the M8Ts, I ran a real-time RTKLIB solution using RTKNAVI and my normal kinematic solution configuration for both static and moving rover tests.  As usual, I used the demo5 version of RTKLIB for this test.

So, how did they compare?  First of all, a few differences to consider between the solutions.   The M8P solution only uses GPS and GLONASS satellites since the SBAS and Galileo satellites have been disabled in the M8P firmware as I mentioned earlier.  This should give the M8T solution an advantage.  The M8P also has limited processing power relative to the laptop running the RTKLIB solution, so this may also give the M8T solution an advantage.  On the other hand, I’m sure that u-blox has lots of smart people that know all the internal details of the hardware which should give the M8P solution an advantage.

For my first comparison, I did a a static rover test with the antenna located on a tripod a few meters from the house and partially obstructed by trees.  To avoid different starting conditions between the two receivers, I connected both receivers to the antenna and waited till they both converged to fixed solutions before starting the test.  I then disconnected the antenna for about 30 seconds, and then reconnected it.  This will cause both receivers to lose lock with all satellites and the solutions will have to re-converge from scratch so this should be a fair comparison.  I disconnected and reconnected the antenna several times to compare the times to re-acquire fix and the accuracies of the fixes.

The results are plotted below, the M8P internal solution on the top, and the real-time RTKLIB M8T solution on the bottom.  I re-connected the antenna five times.  Once, the internal M8P solution re-acquired fix slightly quicker than the RTKLIB M8T solution, once the M8T re-acquired fix slightly quicker than the M8P.  The other three times, the M8T re-acquired fix significantly quicker than the M8P.   The M8P times to re-acquire varied from 52 to 220 seconds.  The M8T took from 50 to 76 seconds.

The M8T RTKLIB solutions had no false fixes, the M8P internal solutions had false fixes with several meter errors for over five seconds in two of the five tests.  The false fixes are circled with red in the plot below.

The E-W and N-S axes matched within one centimeter between the two solutions.  The U-D axis at first appears to differ by 21.5 meters between the two solutions but this is because I specified the RTKLIB vertical solution to be relative to the ellipsoid, while the M8P solution is relative to the geoid.  If I subtract the geoid to ellipsoid offset (available in the GGA message from the M8P), then the two solutions match in the vertical axis as well.

Overall, I would have to say that in this particular run,  the RTKLIB M8T solution out-performed the M8P internal solution by a fairly significant amount.

One other observation about the M8P solution is that the precision reported (at least in the GGA message) is only to one centimeter in the horizontal axes and 10 cm in the vertical accuracy.  I didn’t see any obvious way to get outputs with better precision than this, at least with the NMEA messages.  [Note 8/16/18: It turns out that the UBX-CFG-NMEA message can be used to enable high precision mode to fix this.  Thanks to Charles Wang for pointing this out in a comment]

Next I mounted the same antenna on the roof of my car and drove around the neighborhood.  The sky view varied from nearly unobstructed to moderate tree canopy.  Here are the results of that test, the MP internal RTK solution on the left, and the M8T real-time RTKLIB solution on the right.

The fix rate for the M8P solution was 52.4%.  The M8T solution was noticeably better with a 95.2% fix rate.

Here’s a plot of the difference between the two solutions (green indicates both solution are fixed, yellow indicates that at least one is float).  The U-D axis differences are fairly large, mostly because of the 10 cm vertical precision of the M8P position messages.  There is also a fairly large discrepancy between the two solutions around 22:55:00.  This corresponds to a 15 second gap in base station data, probably due to a loss of cell phone reception.  This seems to be a large error for such a short base outage but I was not able to narrow down the cause beyond this or determine which receiver contributed more to this combined error.  This probably deserves more investigation in a future post.

Again, the real-time RTKLIB M8T solution appeared in this test to be better than the internal M8P solution.

The most likely reason the M8T is getting a better solution is that it is using more satellites.  Here are the rover raw observations for the M8P on the left, and the M8T on the right.  With the GPS and GLONASS constellations, both receivers are seeing 13 satellites.  With Galileo and SBAS, the M8T is seeing an additional 7 satellites.  One of these Galileo satellites is not fully operational yet and so is not used in the solution, but that still gives 6 extra satellites, nearly 50% more than the M8P solution.

If the extra satellites are what is improving the M8T solution, then running a solution with the M8T data without the SBAS and Galileo satellites should give a solution similar to the M8P solution.

This is easy to do, so let’s try it.  Here’s a post-processed solution for the M8T data using only the GPS and GLONASS satellites.

Fix rate is 66.5%, reasonably close to the 52.4% fix rate from the M8P internal solution.  This suggests that a large part of the improvement for the M8T solution is coming from the additional satellites and not from any differences in the solution algorithms.

As in all my comparisons, this is intended to be one user’s initial experience, and not any kind of rigorous comparison test.  Please interpret these results with that in mind.  If you have experiences that differ from these I would be very interested to hear about them.

Glonass Ambiguity Resolution with RTKLIB Revisited

To get a high precision fixed solution in RTKLIB we need to resolve the integer ambiguities that come from the carrier phase measurements.  Resolving the integer ambiguities for the GLONASS satellites is more challenging than resolving them for the other constellations.  This is because, unlike the other constellations, the GLONASS satellites all transmit on slightly different frequencies.  This introduces an additional bias error in the receiver hardware.

These hardware biases are constant, generally the same for all receivers from the same manufacturer, are proportional to carrier frequency and are similar between L1 and L2.

In the demo5 version of RTKLIB, there are four choices for how to handle GLONASS ambiguity resolution (AR). I will cover all four briefly, but then focus on the “autocal” setting which I have enhanced in the most recent version (b29c) of the demo5 code.

Off:  If Glonass  AR is set to “Off”, then the raw measurements from the Glonass satellites will be used for the float solution but ambiguity resolution will be done only with satellites from the other constellations.  If you are not using the demo5 version of RTKLIB, this is usually your only choice when using receivers from different manufacturers for the rover and the base.  However, you are giving up a significant amount of information by ignoring the GLONASS ambiguities and so I would not recommend this setting if you are using the demo5 code, unless of course your receivers don’t support the Glonass satellites.

On:  If Glonass AR is set to “On” then RTKLIB will treat the Glonass ambiguities the same as the ambiguities from the other constellations and will not make any attempt to account for the additional hardware biases.  Use this setting if your base and rover receivers are from the same manufacturer, since in this case, the biases will cancel and can be ignored.  There are also some cases in which different manufacturers have equal or nearly equal biases as we will see later, in which case you can also use this setting.  This is your best solution for dealing with Glonass ambiguities.  I always try to use matched receivers for base and rover if possible.

Fix-and-Hold: This is an option I have added to the demo5 code for Glonass AR.  It is an extension to the “fix-and-hold” method used for other constellations but instead of using the additional feedback to track the ambiguities, it uses it to null out the hardware biases.  I recommend this setting when using the demo5 code with unmatched receivers.  It takes advantage of the additional information in the Glonass ambiguites most of the time.  However, fix-and-hold is not enabled until after a first fix has been achieved, and so the Glonass ambiguities are not available until then.  This can mean longer time to first fix and less robustness compared to the “On” option, so don’t use this option for matched receivers.

Auto-cal:  This option adds additional states to the kalman filter to estimate the receiver hardware biases as a function of carrier frequency, one state for L1, another for L2.  In previous experiments I had not had any success with it.  Recently, however, I discovered that if I adjusted the filter settings, it can be effective for a zero baseline case, where base and rover are both connected to the same antenna so that almost all other errors are completely cancelled.  With a little more experimentation I also found that for short baselines it can also be effective if the kalman filter state is pre-set to something close to the final value before the solution is started.  It will then usually converge to the correct bias value.  However, there is currently no mechanism in the code to adjust any of these values, so I have not found this mode to be useful in its current implementation.

To make the auto-cal option more flexible, and hopefully more useful, I made a few modifications to it in the b29c code.  I added the capability to pre-set the initial state value and also to adjust the internal filter settings, specifically the initial variance and process noise for this state.  The units for the state, and hence for the initial value are in meters per frequency channel and values generally are within +/-5 cm per channel.  I used some existing config parameters that are currently unused to reduce the amount of code I needed to change.  Unfortunately it means that the names are not as descriptive as they could be.  The new config parameters are:

pos2-arthres2 = relative GLONASS hardware bias in meters per frequency slot
pos2-arthres3 = initial variance of GLONASS hardware bias states
pos-arthres4 = process noise for GLONASS hardware bias states

Bias values have been published for some of the most popular geodetic quality receivers but are generally not available for lower-cost or less popular receivers.  Here is a table of values from a paper published by Lambert Wanninger in 2011 for nine receiver manufacturers.

I was able to verify these results for Trimble, Leica, and Novatel, but I found a much lower value for Septentrio so I suspect the biases may have changed in their newer receivers.

To demonstrate the modified autocal option, I will start with a zero baseline case between a ComNav receiver and a Tersus receiver.  It is easiest to measure the hardware biases in the zero baseline case because most other errors will cancel and the hardware biases will be the dominant error.  In this case, I have significantly reduced the initial variance setting from the original value of 1.0 to 1E-7 and increased the process noise from 1E-6 to 1E-3.

I have run the solution several times with the initial bias value set to different numbers between -.05 and 0.06.  Here are the results for both L1 and L2.

The convergence occurs just after first fix is achieved.  If a fix is not achieved, then the state will not converge as you can see above for the 0.06 example.   In this case, the initial value was too far from the correct value and prevented getting a fix.  As you can see, all the other cases converged towards a single value around -.022, both for L1 and for L2.

Another way to visualize the error in the initial value is to look at the GLONASS residuals after first fix is achieved.  The plot below shows the GLONASS L1 carrier phase residuals  for different initial values, for 0.03 on the left, -0.05 in the middle, and what I believe is the correct value for this receiver combination of -.022 on the right.

Here are the same plots for the L2 carrier phase residuals.

Through a slightly tedious process, I am fairly easily able to iterate the residuals down to near zero for different pairs of receivers in my possession.  Note that this gives me the relative difference in biases for each receiver pair, and not absolute values for each receiver, unlike Wanniger’s table which is for absolute biases.

Extending the table to receivers used in nearby CORS stations is a little more challenging because the initial bias value needs to be fairly close to get a first fix and hence a convergence, but still possible if the base station is not too distant.   I found data sets that included CORS data from Leica, Novatel, Trimble, and Septentrio receivers.  Using the above procedure to iterate the residuals down to near zero, I was then able to extend my table and make the values absolute by choosing the unknown offset to make my bias pairs align with Wanninger’s table.  This is the resulting table I created.

[Note: table updated 3/16/20:]

ComNav               =   2.3 cm
Leica                      =   2.3 cm
Novatel                 =  2.3 cm
Septentrio           = -0.3 cm
Spectra Physics =  0.0 cm
SwiftNav              =  0.0 cm
Tersus                   =  0.0 cm
Topcon                 =  0.0 cm
Trimble                = -0.7 cm
u-blox M8T        = -3.2 cm
u-blox F9P          =  0.0 cm

To generate an initial value for the bias state from this table for an RTKLIB solution, subtract the base station bias from the rover bias, then divide by 100 to convert from centimeters to meters.  This value can then be used to set the “pos2-arthres2” config parameter in the config file.  For the RTKPOST and RTKNAVI GUI option menus I have labeled this “Glo HW Bias”.

To test this code on an independent set of data after generating the table, I used a data set recently sent to me by a reader.  It consists of a u-blox  M8T receiver for rover and Leica receiver just a few kilometers away for base, and was collected in Europe.  The rover position was static but I ran the solution in kinematic mode to make the solution a little more challenging and to make any errors in the solution more visible.

To generate the correct config value for RTKLIB I  subtracted the Leica bias of 2.3 cm from the above table from the u-blox bias of -3.2 cm to get a relative bias between receivers of -5.5 cm or -0.055 m.  I added “pos2-arthres2=-0.055” to the config file and then ran the solution four times, with pos2-gloarmode set to “off”,”fix-and-hold”,”autocal”, and “on”.  Although I left the bias value set for all runs it is ignored unless gloarmode is set to autocal.

Here are the times to first fix, the number of satellite pairs used for the initial fix, and the number of satellite pairs being used for fix after 10 minutes.

	Time to	# sat pairs used	# sat pairs used for
GLO AR mode	first fix	for initial fix	fix after 10 min
OFF	4:10	7	7
Fix&Hold	4:10	7	11
Autocal	1:05	14	14
On	6:47	14	14

As you would expect, the time to first fix for gloarmode=”off” was the same as “fix-and-hold” since “fix-and-hold” does not use the GLONASS satellites for initial fix.  After 10 minutes it was still only including four of the GLONASS satellites in the ambiguity resolution which was a little unusual, typically I would have expected more GLONASS satellites to be included.

With gloarmode=”autocal”, the time to first fix was reduced from 250 seconds to 65 seconds and the number of satellites included in the first fix increased from 7 to 14., both significant improvements.

The most surprising thing in this data is that when gloarmode was set to “on” it acquired a fix at all.  In many similar cases it will never get a fix.  The GLONASS carrier phase residuals after initial fix were very high though as can be seen below.  The left plot is with gloarmode set to “on”, and the right plot is with it set to “autocal”.

The ambiguity resolution ratio was also much higher when autocal was enabled as can be seen below (yellow/green=autocal, olive/blue=on) which improves robustness.

The large residuals did not affect the solution position, as the two solution did not differ by more than 2 mm at any time.  The autocal solution however is much more robust in the sense that it is less likely to lose fix.

Although I have found the results with autocal enabled are generally excellent with relatively short baselines (<10 km), I have found the results less encouraging for longer baselines (>25 km).  In these case I have found that I often get better results with pos-gloarmode set to “fix-and-hold” then I do with “autocal”.  I don’t understand exactly why this is, but suspect that the fix-and-hold correction is more general and may be correcting for more than just the GLONASS hardware biases.

The code changes for this feature are included both in my Github repository and in the newest (demo5 b29c) executables available to download from the rtkexplorer website.   If you choose to experiment with this feature, please let me know if you find any errors in my table, or can add values for any additional receivers.

[Note 6/17/18:  I had a issue with uploading the executables to the website.   If you downloaded them prior to 6/17/18, please download again to get the updated version.] 

Using SSR corrections with RTKLIB for PPP solutions

If you have been following recent announcements in precision GNSS, you may have been hearing a lot about SSR (State State Representation).  SwiftNav recently announced their Skylark corrections service, and u-blox announced a partnership with Sapcorda to provide correction service for their upcoming F9 receivers.  Both of these services are based on SSR corrections.

So, what is SSR?  Very briefly, it refers to the form of the corrections.  In traditional RTK with physical base stations or virtual reference stations (VRS), the corrections come in the form of observations in which all of the different error source are lumped together as part of the observation.  This is referred to as OSR (Observation Space Representation).  In SSR corrections, the different error source (satellite clocks,  satellite orbits, satellite signal biases, ionospheric delay, and tropospheric delay) are modeled and distributed separately.  There are many advantages to this form but what seems to be driving industry towards it now is that it allows the current VRS model where each user requires a unique data stream with observations tailored for their location to be replaced with a single universal stream that can be used by all observers.  This is a requirement if the technology is going to be adopted for the mass-market automotive industry for self-driving cars, since it is not practical to provide every car with it’s own data stream.

For more detailed information on SSR, Geo++ has a one page summary here and IGS has an 18 minute video presentation on the topic here.  Both are excellent.

Below is an image I borrowed from the IGS presentation which shows the flexibility of the SSR format.  It is intended to show how the same SSR data stream can be used globally for PPP quality corrections and also regionally for RTK quality corrections but it is also a good visual for understanding the message details I describe below.

The RTCM standards committee is still in the process of finalizing the messages used to send the different correction components.  They have split the work into three phases.  Phase 1 includes the satellite clock, orbit, and code biases.  Phase 2 includes satellite phase biases and vertical ionosphere corrections, and phase 3 includes ionospheric slant corrections and tropospheric corrections.

There are several real-time SSR streams accessible for free today.  Unlike the paid services, they do not contain enough detailed regional atmospheric corrections to use as a replacement for a VRS base but they can easily be used for static PPP solutions.

I used the CLK93 stream from CNES for an experiment to test how well RTKLIB handled the SSR corrections.  It includes the Phase 1 and Phase 2 RTCM messages but not the Phase 3 messages.  Here is the format of the the messages in the CLK93 data stream:

You can register for free access to the CLK93 (and other) streams from any of these locations:

• USA: CDDIS and IGS Central Bureau
• Europe : Bundesamt fuer Kartographie und Geodaesie
• China: The GNSS Research Center, Wuhan University and The ShangHai Astronomical Observatory (SHAO)
• Australia: Geoscience Australia

Unfortunately, RTKLIB currently only supports the Phase 1 RTCM messages and even this is not complete in the release version.  I have gone through the code and made a few changes to make the Phase 1 SSR functional and have checked those changes into the demo5 Github repository.  I also added some code to handle the mixed L2 and L2C observations from the ComNav and Tersus receivers.  After a little more testing I plan to release this code into the demo5 executables, hopefully in the next week or two.

With only phase 1 measurements, the RTKLIB PPP solutions will work much better with dual frequency receivers than with single frequency receivers.  This is because single frequency receivers require ionospheric corrections for longer baselines.  For this reason, I did not bother with collecting any single frequency data.  Instead, I collected both L1/L2C data with a Swiftnav Piksi Multi receiver and L1/L2/L2C data with a ComNav K708 receiver and a Tersus BX306 receiver.

RTKLIB is usually used to calculate PPP solutions without SSR corrections but this requires downloading multiple correction files for orbital errors, clock errors, and code bias errors and it is usually done with post-processing rather than real-time, after the corrections are available.  With SSR, the process is simpler because the solution can be done real-time and there is no need to download any additional files.  It does, however, require access to the internet to receive the real-time SSR data stream from an NTRIP caster.  The solution can be calculated real-time or the SSR corrections and receiver observation streams can be recorded and the solution post-processed.

To enable the use of SSR corrections in RTKLIB, you need to set the “Satellite Ephemeris/Clock (pos1-sateph) input parameter to either “Broadcast+SSR APC” or “Broadcast+SSR CoM”.  Note that CoM stands for Center of Mass and APC for Antenna Phase Center.  They refer to the reference point for the corrections.  The CLK93 corrections are based on antenna phase centers.

To generate my PPP solution I set the solution mode to “PPP-Static”,  ephemeris/clock (pos1-sateph) to “brdc+ssrapc”, ionosphere correction (pos1-ionopt) to “dual-freq”, and troposphere correction (pos1-tropopt) to “est-ztd”.  I also enabled most of the other PPP options including  earth tides,  satellite PCVs, receiver PCVs, phase windup, and eclipse rejection.

RTKLIB PPP solutions don’t support ambiguity resolution so the ambiguity resolution settings are ignored.  I specified the satellite antenna file as “ngs14.atx” which is the standard antenna correction file and is available as part of the demo5 executable package.  I also needed to include the CLK93 data stream as one of the inputs in addition to the receiver observations (and navigation file if post-processing).

I collected a couple hundred hours of observations with the SwiftNav receiver connected to a ComNav AT-330 antenna on my roof with moderately good sky visibility.  I then ran many four hour static solutions over randomly selected data windows.  I also collected a small amount of raw data from a ComNav K708 receiver and a Tersus BX306 receiver.

Below is a typical 12 hour static solution for a SwiftNav and a ComNav receiver.  The SwiftNav solution is in green and the ComNav solution is in purple.  Zero in these plots represents an online PPP solution from CSRS from data collected over a month earlier.  In this particular example, the SwiftNav solution is slightly better but this was not always the case.

 

Here is a shorter data set from a Tersus BX306 receiver.  With the relatively small amount of Tersus and ComNav data I collected, I did not notice any differences in convergence rates or final accuracy between any of the three receivers.

The solutions generally all converged to less than 6 cm of error in each axis after 4 hours with one or two minor exceptions that seemed to involve small anomalies at the day boundary.  Since the RTKLIB PPP solutions don’t include ambiguity resolution they do take longer to converge but the eventual accuracy should be similar.

I’ve uploaded some of the raw observation data for the different receivers and the CLK93 data stream as well as the config file that I used for the solution here.

This seems like a good start and I hope that RTKLIB will support phase 2 and phase 3 corrections in the future.

Swiftnav experiment: Improvements to the SNR

In my previous couple of posts, I evaluated the performance of a pair of dual freqeuncy SwiftNav Piksi multi receivers in a moving rover with local base scenario.  I used a pair of single frequency u-blox M8T receivers fed with the same antenna signals as a baseline reference.

It was pointed out to me that the signal to noise ratio (SNR) measurements of the rovers were noticeably lower than the bases, especially the L2 measurements and that this might be affecting the validity of the comparison.  This seemed to be a valid concern so I spent some time digging into this discrepancy and did indeed find some issues.  I will describe the issues as well as the process of tracking them down since I think it could be a useful exercise for any RTK/PPK user to potentially improve their signal quality.

Previously , in another post, I described a somewhat similar exercise tracking down some signal quality issues caused by EMI from the motor controllers on a drone.  In that case, though, the degradation was more severe and I was able to track it down by monitoring cycle slips.  In this case, the degradation is more subtle and does not directly show up in the cycle slips.

Every raw observation from the receiver generally includes a signal strength measurement as well as pseudorange and carrier phase measurements.  The SwiftNav and u-blox receivers both actually report carrier to noise density ratio (C/NO), rather than signal to noise ratio (SNR) but both are measures of signal strength.  They are labelled as SNR in the RTKLIB output, so to avoid confusion I will refer to them as SNR as well.  I will only be using them to compare relative values so the difference isn’t important for this exercise, but for anyone interested, there is a good explanation of the difference between them here.  Both are logarithmic values measured in dB or dB-Hz so 6 dB represents a factor of two in signal strength.

Since the base and rover have very similar configurations we would expect similar SNR numbers between the two, at least when the rover antenna is not obstructed by trees or other objects.  I selected an interval of a few minutes when the rover was on the open highway and plotted SNR by receiver and frequency for base and rover.  Here are the results, base on the left and rover on the right.  The Swift L1 is on the top, L2 in the middle, and the u-blox L1 on the bottom.  To avoid too much clutter on the plots, I show only the GLONASS SNR values, but the other constellations look similar.

Notice that the L1 SNR for both rovers is at least 6 dB (a factor of 2) lower than the base, and the Swift L2 SNR is more like 10 dB lower.  These are significant enough losses in the rover to possibly affect the quality of the measurement.

The next step was to try and isolate where the losses were coming from.  I set up the receiver configurations as before and used the “Obs Data” selection in the “RTK Monitor” window in RTKNAVI to monitor the SNR values in real time for both base and rover as well as the C/NO tracking window in the Swift console app.  I then started changing the configuration to see if the SNR values changed.  The base and rover antennas were similar but not identical so I first swapped out the rover antenna but this did not make a difference.  I then moved the rover antenna off of the car roof and onto a nearby tripod to see if the large ground plane (car roof) was affecting the antenna but this also did not make a difference.  I then removed the antenna splitter, but again no change.

Next, I started modifying the cable configuration between the receivers and my laptop.  To conveniently be able to both collect solution data and be able to collect and run a real-time solution on the raw Swift observations, I have been connecting both a USB serial cable and an ethernet cable between the Swift board and my laptop.  My laptop is an ultra-slim model and uses an etherent->USB adapter cable to avoid the need for a high profile ethernet connector.  So, with two receivers and my wireless mouse, I end up having more USB cables than USB ports on my computer and had to plug some into a USB hub that was then plugged into my laptop.

The first change in SNR occured when I unplugged the ethernet cable from the laptop and plugged it into the USB hub.  This didn’t affect the L1 measurements much but caused the Swift L2 SNR to drop another 10 dB!  Wrong direction, but at least I had a clue here.

By moving all of the data streams between Swift receiver and laptop (base data to Swift, raw data to laptop, internal solution to laptop) over to the ethernet connection I was able to eliminate one USB serial port cable.  This was enough to eliminate the USB hub entirely and plug both the USB serial cable from the u-blox receiver and the ethernet->USB cable from the Swift receiver directly into the laptop.  I also plugged the two cables into opposite sides of the laptop and wrapped the ethernet->USB adapter with aluminum foil which may have improved things slightly more.

Here is the same plot as above after the changes to the cabling from a drive around the neighborhood.

I wasn’t able to eliminate the differences entirely, but the results are much closer now.  The biggest difference now between the base configuration and the rover configuration is that I am using a USB serial cable for the base, and a ethernet->USB adapter cable for the rover so I suspect that cable is still generating some interference and that is causing the remaining signal loss in the rover.  Unfortunately I can not run all three streams I need for this experiment over the serial cable, so I am not able to get rid of the ethernet cable.

I did two driving tests with the new configuration, similar to the ones I described in the previous posts.   One was through the city of Boulder and again included going underneath underpasses and a parking garage.  The second run was through the older and more challenging residential neighborhood.  Both runs looked pretty good, a little better than the previous runs but it is not really fair to compare run to run since the satellite geometry and atmospheric conditions will be different between runs.  The relative solutions between Swift and u-blox didn’t change much though, which is probably expected since the cable changes improved both rovers by fairly similar amounts.

Here’s a quick summary of the fix rates for the two runs.  The fix rates for the residential neighborhood look a little low relative to last time but in this run I only included the most difficult neighborhood so it was a more challenging run than last time.

	Fix rates
	City/highway	Residential
Swift internal RTK	93.60%	67.50%
Swift RTKLIB PPK	93.70%	87.90%
U-blox RTKLIB RTK	95.70%	92.80%
U-blox RTKLIB PPK	96.10%	91.10%

Here are the city/highway runs,  real-time on the top, post-process on the bottom with Swift on the left and u-blox on the right.  For the most part all solutions had near 100% fix except when recovering from going underneath the overpasses and parking garage.

Here are the same sequence of solutions for the older residential neighborhood.  This was more challenging than the city driving because of the overhanging trees and caused some amount of loss of fix in all solutions.

Here’s the same images of the recovery after driving under an underpass and underneath a parking garage that I showed in the previous post.  Again, the relative differences between Swift and u-blox didn’t change much, although the Swift may have improved a little.

Overall, the improvements from better SNR were incremental rather than dramatic, but still important for maximizing the robustness of the solutions.  This exercise of comparing base SNR to rover SNR and tracking down any discrepancies could be a useful exercise for anyone trying to improve their RTK or PPK results.

Underpasses and urban canyons

[Update: 4/17/18:  Although I don’t think it changes the results of this experiment significantly, there was an issue with apparent interference from a USB hub and ethernet cable on the rover setup during this testing.  See the next post for more details. ]

In my last post I demonstrated fairly similar fix rates and accuracies between an M8T single-frequency  four-constellation solution and a SwiftNav Piksi dual-frequency two-constellation solution.

One advantage often mentioned for dual frequency solutions for moving rovers is that their faster acquisition times should help when fix is lost due to a complete outage of satellite view caused by an underpass or other obstruction.  This makes sense since the dual frequency measurements should allow the ambiguities to be resolved again more quickly.

Since my last data set included several of these types of obstructions I thought it would be interesting to compare performance specifically for these cases.

To create the Google Earth images below I used the RTKLIB application POS2KML to translate the solution files to KML format files and then opened them with Google Earth.

Here are the raw observations for the first underpass I went under, Swift rover on the left, M8T rover on the right.  In this case there was an overhead sign just before the underpass which caused a momentary outage on all satellites followed by about a two second outage from the underpass, followed by a period of half cycle ambiguity as the receivers re-locked to the carrier phases.

Here’s the internal Swift solution for the sign/underpass combo above at the top of the photo and a second underpass at the bottom of the photo.  For the first underpass, the solution is lost at the sign, achieves a float solution (yellow) after about 9 seconds, then re-fixes (green) after 35 seconds.

Here’s the RTKLIB post-processed solution (forward only) for the Swift receivers with fix-and-hold low tracking gain enabled as described in my previous post.  It looks like a small improvement for both underpasses.  The solution loses fix at the sign but in this case maintains a float solution until the underpass.

Here’s the RTKLIB post-processed solution (same config) for the M8T receivers.  Notice the no-solution gaps after the underpasses are shorter.  In this case, for the upper underpass, a solid fix was re-achieved after about 21 sec.

Here’s a zoom in of the M8T solution (yellow dots) for the lower underpass.  If the position were being used for lane management it looks like the float solution would probably be accurate enough for this.  The other yellow line with no dots is the gap in the Swift solution.

Here’s a little further down the road.  At this point the Swift solution achieves a float position at about the same time the M8T solution switches to fix.  Lane management would clearly be more difficult with the initial Swift float solution.

Next, I’ll show a few images from another underpass.  In this case I drove under the underpass from the left, turned around, then drove under the underpass again from the right.  The Swift internal solution is on the left, the Swift RTKLIB solution in the middle, and the M8T RTKLIB solution on the right.  Notice that the time to re-acquire a fix is fairly similar in all three cases.

Here is zoom in of the two Swift solutions, they are quite similar.

Here is a zoom-in of the M8T RTKLIB solution.  Again, the float solution is achieved very quickly, and appears to be accurate enough for lane management.

My last test case was a combination urban canyon and parking structure.  In the photo below, I drove off the main street to the back of the parking garage, underneath the pedestrian walkway, into the back corner, then underneath the back end of the garage and then back to the main street.  I would consider this a quite challenging case for any receiver.

Here are the raw observations.

Here are the three solutions, again the Swift internal is on the left, the Swift RTKLIB in the center, and the M8T RTKLIB on the right.

 

Here is an image of the Swift internal solution.

Here is an image of the Swift RTKLIB solution

And here is an image of the M8T RTKLIB solution

In this case, the M8T RTKLIB solution appears to be the best.

So, this experiment seems to show that a dual frequency solution will not always handle satellite outages better than single frequency solutions.  In this case, the extra Galileo and SBAS satellites in the M8T solution seem to have helped a fair bit, and the M8T solution is, at least to me, surprisingly good.

If anyone is interested in analyzing this data further, I have uploaded the raw data, real-time solutions, and config files for the post-processed solutions to the sample data sets on my website, available here.  I should mention that there is an unexplained outage in the Swift base station data near the end of the data set.  This could have been caused by many things, most of them unrelated to the Swift receiver, so all the analysis in both this post and the previous post were done only for the data before the outage.

 

 

 

 

 

 

 

 

 

 

Improved results with the new SwiftNav 1.4 firmware

I last took a look at the SwiftNav Piksi Multi low-cost dual-frequency receiver back in November last year when they introduced the 1.2 version of FW.  They are now up to a 1.4 version of firmware so I thought it was time to take another look.  The most significant improvement in this release is the addition of GLONASS ambiguity resolution to the internal RTK solutions but they also have made some improvements in the quality of the raw observations.

I started with a quick spin around the neighborhood on my usual test route.  The initial results looked quite good, so for the next test I expanded my route to include a drive to and around Boulder, Colorado, a small nearby city of just over 100,000.  The route included some new challenges including underpasses, urban canyons, higher velocities, and even a pass underneath a parking structure.  This is the first time I have expanded the driving test outside my local neighborhood.

My test configuration was similar to previous tests.  I used a ComNav  AT330  antenna on my house roof for the base station, and a SwiftNav GPS-500 antenna on top of my car for the rover. I split the antenna signals and in both cases, fed one side to a Piksi receiver and the other side to a  to a u-blox M8T single frequency receiver.   I ran an internal real-time RTK solution on the Piksi rover and an RTKNAVI RTK real-time solution on the M8T rover.  The M8T receivers ran a four constellation single frequency solution (GPS/GLONASS/Galileo/SBAS) to act as a baseline while the Piksi receivers ran a two constellation (GPS/GLONASS) dual frequency solution.  Both rovers were running at a 5 Hz sample rate and both bases were running at a 1 Hz sample rate.  The distance between rover and base varied from 0 to just over 13 km.  The photos below show different parts of the route.

Here are the real-time solutions for the two receiver pairs, internal Swift on the left, and RTKNAVI M8T on the right.

Both solutions had similar fix rates (79.9% for Swift, 82.6% for M8T) and in both cases the float sections occurred for the most part either in the older neighborhood with larger trees (top middle) of after underpasses (bottom left).  The higher velocity (100 km/hour) on the highway (center) did not cause any trouble for either solution.

Based on a comparison of the two solutions, accuracy was relatively good for the fix sections of both solutions.  Below on the left, is the difference between both solutions for points where both solutions had a fix.  In the center and right are plots of both solutions (Swift internal=green,M8T RTKNAVI=blue) for the two locations with the longest duration discrepancies of any magnitude.  Both look like false fixes by the Swift internal solution, based on the discontinuities.  Overall, though the errors between the two were reasonably small and of short duration.

Post-processing the Swift data with RTKLIB produced the solution on the left below with an 85.5% fix rate and a good match to the M8T solution.  The difference between both solutions for the fixed point is shown on the right.  This solution was run with continuous ambiguity resolution.

 

For more challenging environments like this I often add some tracking gain to the ambiguities by enabling “fix-and-hold” for the ambiguity resolution mode but setting the variance  of the feedback (input parameter pos2-varholdamb) to a fairly large number (0.1 or 1.0) to effectively de-weight the feedback and keep the tracking gain low.  For comparison, the default variance for fix-and-hold mode feedback is 0.001 which results in quite a high tracking gain.  I find that with the low tracking gain, I generally do not have an issue with fix-and-hold locking on to false fixes.

Running RTKLIB solutions for Swift and M8T with this change (fix-and-hold AR enabled, pos2-varholdamb=1.0) improved the fix ratio for the Swift RTKLIB solution from 85.5% to 91.1% and the M8T RTKLIB solution from 82.6% to 92.6% with no apparent degradation in accuracy.

Using a combined solution instead of a forward solution (only a choice for post-processing) improved the fix ratios even further, again with no apparent degradation in accuracy.  The Swift RKLIB solution increased to a 96.2% fix rate and the M8T RTKLIB solution increased to 94.1%.

Overall, the Swift RTKLIB solutions were noticeably better and more consistent than in my previous test.  Considering the difficulty of the environment, I consider all of these solutions to be very good.

In my next post, I will look specifically at how the two receivers handled going through a narrow urban canyon, underneath three underpasses and underneath a parking structure.