RTKLIB Benchmarking: versions 2.4.2, 2.4.3, and demo5

It’s been about four years now since I created the demo5 branch of RTKLIB. During that time I have added a number of features and enhancements to the code with a focus on low-cost receivers (primarily u-blox) and moving rovers while at the same time keeping synced with the latest 2.4.3 code from the official RTKLIB code base. I thought it would be interesting to do a little benchmarking between the the two official versions of RTKLIB (2.4.2 and 2.4.3) and the demo5 code to see to what extent this evolution of the code has affected the results.

First of all, though, it’s probably worth a little discussion about versions 2.4.2 and 2.4.3. Source and executables for both versions are available on Github at https://github.com/tomojitakasu. The source is in the RTKLIB repository and the executables are in the RTKLIB_BIN repository. Both of these repositories default to the “master” branch which is the 2.4.2 code. This is what you will get unless you specifically request the “2.4.3” branch of the code. For several years, almost all the development activity was on the 2.4.3 branch and only very minimal changes were being made to the master branch. In Jan 2018, there was a merge of the 2.4.3 changes back to the master branch although it appears that not all of the changes in 2.4.3 were merged back into the master branch. Since then development has continued on 2.4.3 without another merge back to the master branch.

For most of my data analysis posts, I focus on a single data set, spending a fair bit of time to make sure I am only analyzing the usable parts of the data, possibly tweaking the configuration file for that specific data, and digging into any issues that crop up. In this case I didn’t do that. I picked nine raw data sets, all with u-blox M8T receivers and a moving rover, made no effort to filter out bad data, and used a single generic configuration file for all nine data sets. Eight of the data sets are those that I have previously uploaded to my website at http://rtkexplorer.com/downloads/gps-data/ and the ninth was from my most recent drive around the neighborhood with a u-blox M8T and an antenna on top of the car. I ran solutions for each of the three RTKLIB versions on all nine data sets. For each solution, I converted the data from u-blox binary to rinex using the same code version as I did for the solution, since the different codes will affect this conversion as well as the solution.

I ran the post-processed solutions in “combined” mode, meaning that the solution is run both forward and backward and the results are then combined. Not only does this tend to produce better results, but the results also have higher confidence since RTKLIB compares the forward and backwards solutions, sample by sample, and downgrades any sample where the solutions in both directions are fixed and the results differ by more than four standard deviations. This tends to do a good job of detecting and rejecting any false fixes in the results. However, it is not foolproof. If the solution is fixed in only one direction and float in the other, then there is no additional validation.

I used the same configuration settings for running each of the three RTKLIB versions on each of the data sets with a few exceptions. Versions 2.4.2 and 2.4.3 do not have the “arfilter” feature that automatically holds off new satellites until their phase bias estimates have converged enough to not break the ambiguity resolution so I increased the fixed hold off (arlockcnt) from 0 to 10 for the 2.4.2 and 2.4.3 codes. The outlier detection scheme is also different in the demo5 code from the other two versions, making it necessary to increase the outlier threshold (rejionno). In this case I increased the threshold from 1.0 to 30.0 for versions 2.4.2 and 2.4.3. Lastly, the 2.4.2 code runs very slowly if dynamics is enabled, so I turned dynamics off for this code.

The specific code versions for the experiment were: 2.4.2 p13, 2.4.3 b33 and demo5 b33b2.

I used fix percentage as a metric for the test. This isn’t a perfect metric because it doesn’t take into effect the accuracy of the non-fixed results and it can be affected by false fixes. Overall, though it is probably the best single metric for this kind of test, especially since running the solutions in combined mode should reject many of the false fixes.

The fix percentages for each test are listed below. The data set names correspond to the names of the sample data sets on my website.

Test	V 2.4.2	V 2.4.3	V2.4.3a	demo5	data set
1	37.7%	93.9%	96.3%	94.4%	union_0705
2	35.7%	9.8%	60.6%	85.5%	niwot2_car
3	52.4%	16.2%	89.0%	80.2%	drone_0414
4	85.8%	51.2%	93.8%	98.8%	m8t_niwot_0606
5	59.7%	21.4%	83.7%	97.1%	swift_m8t_road_0606
6	63.1%	10.0%	83.6%	99.0%	car_1114
7	54.6%	26.6%	73.3%	94.2%	car_0320
8	39.4%	11.2%	54.4%	98.1%	comnav_car
9	25.0%	9.2%	53.4%	98.1%	not uploaded
median	52.4%	16.2%	83.6%	97.1%

Experiment results in fix percentage

For the most part, the results match my expectations. Version 2.4.2 has the lowest fix percentage, 2.4.3 is in the middle, and demo5 has the highest. However, I ran into one very significant issue with the 2.4.3 code that I do not fully understand. In the table above, the “V2.4.3” column is the results using version 2.4.3 for both the conversion from raw binary to rinex as well as for the solution. As you can see, the fix percentages were very low for this test for all data sets except the first, significantly lower than even the 2.4.2 results. I did not fully debug this issue but the problem appears to be in the conversion from raw binary to rinex, not in the solution itself.

The V2.4.3a column is the results for running the 2.4.2 code for the raw binary to rinex conversion, and then the 2.4.3 code for the solution. This result is much more within my expectations for the 2.4.3 code. I suspect that the issue with the 2.4.3 rinex conversion is that when it is filtering out low quality observations it is not preserving the cycle-slips. RTKLIB can be very sensitive to unflagged cycle slips.

I am very curious if anyone who is a regular user of the 2.4.3 code can duplicate this result. As you can see from the first data set, it does not always occur, and is much less of a problem if the data does not have a large number of cycle slips, so it would need to be tested on a more challenging data set to see this issue.

Regarding the demo5 results, seven of the nine tests had over 94% fix rate and the median fix percentage was 97.1%. I consider this quite reasonable since all of these were fairly challenging data sets and most of them included at least a small amount of unusable data. The two data sets that did not perform as well (80-86% fix percentage) were both older. One did not include Galileo and the other was from a drone that had one particularly poor quality section of data. However both solutions had well over a 90% fix rate when run in the forward only direction which indicates the fixes in the combined solution were downgraded because of mismatches between the two directions. In one of these cases (test 3), the 2.4.3 solution obtained a higher fix rate than the demo5 code but it only got fixes in the forward direction, not in the backward direction so had no additional validation. Based on some discrete jumps in that solution, I suspect it would have also downgraded the fix percentage if it had achieved fix in the backwards direction.

Looking more closely at the cases where the demo5 solution points were downgraded to float for mismatches, it’s interesting because, at least at first glance, it appears that these were not false fixes, but discrepancies from using different combinations of satellites that were large enough to trigger the four standard deviation threshold. This is a little concerning and worthy of further investigation. Fortunately it only appears to occur when the data quality is fairly poor. However, this does emphasize the importance of insuring the best quality measurements possible, and not over-relying on RTKLIB to reject inaccurate solutions.

As always, I’d like to emphasize that these tests are intended only as one users snapshot of one fairly particular use case of RTKLIB and are not intended to be any kind of comprehensive analysis. Also, it’s important to understand that 99+% of the code in all versions of RTKLIB including the demo5 code are the result of many years of dedicated effort by Tomoji Takasu and his team at Tokyo University of Marine Science and Technology. My only contribution has been to add a few changes on top of this code to make it a little more focused on practical application for specific uses rather than a more generic academic tool.

Lastly, for reference, here’s a partial list of the most important configuration settings I used for this experiment for the demo5 code. The other two codes used the same settings with the exceptions I describe above.

pos1-posmode =kinematic # solution mode
pos1-soltype =combined # solution type (forward, backward, combined)
pos1-frequency =l1 # (l1, l1+l2, l1+l2+l5)
pos1-elmask =15 # min sat elevation to include in solution (deg)
pos1-snrmask_r =off # SNR mask rover (off, on)
pos1-snrmask_b =off # SNR mask base (off, on)
pos1-dynamics =on # add dynamic states to kalman filter (off, on)
pos1-navsys =15 # (1:gps+2:sbas+4:glo+8:gal+16:qzs+32:comp)
pos2-armode =fix-and-hold
pos2-gloarmode =on # Glonass AR mode (off, on, fix-and-hold, autocal)
pos2-bdsarmode =off # Bediou AR mode (off, on)
pos2-aroutcnt =50 # outage count to reset sat ambiguity (samples)
pos2-arminfix =50 # min # of fix samples to enable AR hold (samples)
pos2-rejionno =1.0 # phase bias outlier threshold (m)
pos2-maxage =30 # max age of differential (secs)
pos2-arthres =3.0 # minimum AR ratio for fix (m)
pos2-arthres1 =0.1 # max variance of position states to attempt AR (m)
pos2-varholdamb =0.1 # variance of fix-and-hold tracking feedback (cyc^2)
pos2-arfilter =on # automatic hold off for adding new sats to AR
pos2-arlockcnt =0 # fixed hold off for adding new sats to AR (samples)
pos2-minfixsats =4 # min sats required for fix
pos2-minholdsats =5 # min sats required for AR hold
pos2-mindropsats =10 # min sats required to drop sats from AR
stats-eratio1 =300 # ratio of input stdev of code to phase observations
stats-eratio2 =300 # ratio of input stdev of code to phase observations

Building a simple u-blox F9P data logger with a Sparkfun OpenLog board

Based on the number of views, one of my all time most popular posts is one I wrote nearly three years ago describing how to build a GPS data logger using a Raspberry Pi Zero. Although it specifically describes using a Pi Zero to do nothing except configure the receiver and log raw data to an SD card, the platform itself is quite general and can easily be extended to more complete options including running a real-time RTKLIB solution on the Pi. It is a little out of date now but I think it can still be useful.

However, if what you want is just a raw GPS data logger and nothing more, then, while the above choice may still be the lowest cost solution, it is not the simplest option to implement.

In this post I describe another inexpensive alternative using the very popular OpenLog data logger board available from Sparkfun which currently sells for $15.50. In this particular example, I have used it to log data from an Ardusimple u-blox F9P receiver but with a few minor modifications the design should work with many other popular receivers. For reasons described below it does require the board to have both a USB and a UART port available as well as easy access to 3.3 volts.

Here’s a couple of photos of the front and back of the Sparkfun OpenLog board. As you can see, it is only slightly larger than the size of the microSD card it uses to store the data.

Sparkfun OpenLog board, front and back

The chip on the front side of the board is an Arduino processor which comes pre-loaded with code to automatically log all incoming data from the RX/TX lines directly to the microSD card. The source code is open and available on Github, so if you need to modify it you can, but in most cases it should work fine as is.

To connect the OpenLog to the receiver is very simple. First connect VCC and GND on the OpenLog to 3V3_OUT and GND on the Ardusimple receiver. Fortunately, the pin spacing for the two boards is the same so I was able to use a two pin header to connect power and ground and rigidly mount the OpenLog at the same time. You can see the details in the two photos below. I also added a small piece of double sided foam tape between the two boards to strengthen the physical connection between them.

Back of Ardusimple board with OpenLog. The two boards are physically attached using the two pin header shown on right to connect power and ground. The top right cable is connected to the antenna and the bottom right cable is connected to a USB charger for power or a PC to configure the receiver.

Front of Ardusimple board. The GND/3.3V holes used to connect the two boards are visible just below the u-blox chip. The edge of the microSD card is visible just below the power LED

Then connect the RX/TX data lines on the Open Log to TX1/RX1 on the Ardusimple making sure to swap the two so that RX goes to TX1 and TX goes to RX1. On many boards this will be all you need, but the Ardusimple also has a IOREF input which selects the voltage levels of the GPIO pins. In this case we will connect that to 3.3 volts. You can see these three jumpers on the top photo above.

That’s it for the hardware. The next step is to configure the data logger. To do this you will first need to format a microSD card. Then create a file with the name config.txt, cut and paste the text below into this file and save it to the microSD card.

115200,26,3,0,1,1,0
 baud,escape,esc#,mode,verb,echo,ignoreRX

This is what the data logger uses when it powers up for configuration parameters. The only detail that matters in the above text is the first number which should be set to the same baud rate that UART1 on the receiver is configured to. The maximum baud rate that the OpenLog supports is 115200, so that is what I am using. For more information on the other numbers in this file or about the OpenLog board in general, Sparkfun has an excellent tutorial available here.

The last step is to configure the F9P receiver to output the appropriate messages to UART1. This is where things become much simpler by having both UART and USB ports available on the receiver board. To configure the receiver we will use the u-blox u-center app running on a PC and connected to the receiver through the USB port. Once the receiver is configured and the results saved to flash, we can then unplug the USB cable from the computer and plug it into a USB charger to provide power to the receiver and data logger while they are collecting data.

For this tutorial I will assume you have already downloaded the u-center app from u-blox and are somewhat familiar with using it to configure u-blox receivers. If not, there are many tutorials out there including some of my earlier posts that will help you.

One thing to be aware of is that the messages going out of the receiver to the USB port will generally be different from those going out to UART1, so just because you have the right messages coming out the USB port does not mean they will also be coming out of the UART port. Also be aware that if you use the “Messages View” in u-center to enable and disable messages, this method only affects the active port and so you will be enabling them for USB but not for UART1.

To enable and disable the messages for UART1, you can use the CFG-MSGOUT option from the “Generation 9 Configuration View” which is opened from the “View” tab on the main menu in u-center. I find this method nice for listing which messages are enabled and disabled but rather clunky for actually changing their status, so I used the MSG message from the “Configuration View” ( also opened from the “View” tab) to actually enable and disable the messages. I show this in the screenshot below. If you do it this way, make sure you issue a CFG-CFG message when you are done to save everything to flash.

There are many NMEA messages enabled by default on UART1. You will probably want to disable most or all of these to avoid using unnecessary UART bandwidth and microSD file space.

If you already have raw base observations streaming to the receiver and are using the internal RTK engine to calculate position solutions real-time then you will just need to enable the NMEA GLL messages to log position. These will give you the LLH positions in text format.

If in the more likely case that you are logging raw data for later post-processing, then you can enable either the appropriate RTCM3 messages or the UBX-RXM-RAWX messages to get the raw observations. To minimize the load on the logger, I would suggest using RTCM3 messages for the raw observations and not logging the navigation messages. You only need a single set of navigation messages and in most cases it is easier to log those on the base station. It is simplest to enable the same set of RTCM3 messages for both the USB and UART port. If you need to make these different for any reason, be aware that the F9P does not handle the RTCM3 end of epoch flag independently for both ports, so the last message in each epoch must be the same for both ports or you will have problems.

That should be it. At this point, every time you power up the receiver/logger, it should start a new log file on the microSD card and log all incoming messages from the receiver. The log file names will be in the format LOGxxxx.TXT where “xxxx” increments each run. Simply remove the microSD card from the OpenLog and plug it through an adapter into a PC to transfer the log files. Be sure to be careful when removing the microSD card from the OpenLog. The card locks in with a spring and needs to be pushed in to unlock it. Occasionally I have found that the card catches on the spring and needs to be gently coaxed out or you will break the spring as I found out the hard way.

If using RTKCONV to convert the raw data, you will need to specify the data type (UBX or RTCM3) since the file extension will not be correct for auto format detection.

Here’s a plot of some raw observations I collected this way, then converted to Rinex format with RTKCONV, and then plotted with RTKPLOT, using navigation messages collected separately on my base station.

RTKPLOT of observations collected with the OpenLog logger

The data logger performance is limited by a relatively small buffer size and it is possible that you will see missing samples in your data if the logger buffer overflows while writing to the microSD card. However I did not see any missing samples in my data while collecting dual-frequency GPS, Glonass, and Galileo RTCM3 observations at 5 HZ.

This is the simplest method I am aware of to turn an eval board into a relatively fully functional receiver but if anyone has an easier way, please leave a comment. All it needs now is a 3D printable box to protect it and enclose it from the weather to turn this into a general purpose low-cost dual receiver for real world data collection, at least for post-processing.

For another interesting option, check out this post from the Ardusimple website that describes combining an Ardusimple board with a bluetooth module and smartphone for real-time RTK.

A first look at the u-blox ZED-F9P dual frequency receiver

The new low cost dual frequency receiver from u-blox, the ZED-F9P, is just now becoming available for purchase for those not lucky enough to get early eval samples from u-blox.  CSGShop has a ZED-F9P receiver in stock for $260 which seems quite reasonable, given that it is only $20 more than their NEO-M8P single frequency receiver.

Even better, Ardusimple is advertising an F9P  receiver for 158 euros (~$180) + 20 euros shipping , although their boards won’t ship until January.  As far as I’m aware of, this is actually less than anybody today is selling the M8P receiver for today!

Of course, this is still a fair bit more than a u-blox M8T single frequency receiver without an internal RTK engine, which is available from CSGShop for $75, but the F9T will be coming out next year also without internal RTK engine, which should bring down the price for the lowest cost dual frequency receivers.

Unfortunately I am not one of the lucky ones who got eval boards directly from u-blox yet.  However, I do have two prototype boards from Gumstix, given to me by them for evaluation.  Gumstix offers both off-the-shelf boards and semi-custom boards designed from their libraries of circuits.  I haven’t worked with them directly but it looks like an interesting and useful concept.  The F9P boards from Gumstix won’t be available for sale until at least Feburary next year but I thought I would share the results of some initial testing.  From a performance perspective, I would expect these boards to be similar to F9P boards from other suppliers.

For a first look, I chose to compare the F9P to an M8T for one of my typical driving-around-the-neighborhood exercises.  I looked at both the internal real-time F9P solution and the RTKLIB solutions, both real-time and post-processed.

Experiment Setup:

For the base stations, I connected a CSGShop M8T receiver and a Gumstix F9P  receiver through an antenna splitter to a ComNav AT330 dual frequency antenna on my roof.  Since RTKLIB doesn’t yet fully support the receiver commands needed to setup the F9P, I used the most recent version (18.08) of the u-blox u-Center app run on a Windows laptop to configure the F9P receiver using the documentation on the u-blox website.  I then saved the settings to flash.  The receivers were connected to a laptop with USB cables and I broadcast the base observations over the internet on a couple of NTRIP streams using STRSVR and RTK2GO.com as I’ve described previously.  I configured the F9P to send RTCM3 1005, 1077, 1087, 1097, 1127, and 1230 messages which include base location, raw observations, and GLONASS biases.

For the most part the u-blox documentation is well written and this exercise was fairly straightforward, but I did run into a couple of issues.  First of all, when I plugged the F9P receiver into the laptop, Windows chose the standard Windows COM port driver instead of the u-blox GNSS COM port driver that it chose for the M8T receiver.  You can see this in the screen snapshot below where COM17 is the M8T and COM21 is the F9P.

Both drivers allow the user to set the baudrate in the properties menu available by right clicking on the device name.   With the u-blox driver, the baudrate setting doesn’t seem to matter which makes sense since it is a USB connection.  I have always left the u-blox driver baudrate at the default of 9600 baud without any issue.  With the windows driver, however,  I found that I had to increase the baudrate setting to 115200 to avoid data loss issues.  I have run into a similar problem before for sample rates greater than 5 Hz when the M8T is accessed through it’s UART interface and an FTDI converter is used to translate to USB, rather than communicating directly through it’s USB interface.  I verified though, that in this case the board is using the USB interface on the receiver and not the UART interface.   Not a big deal, and it may be unique to this board, but something to be aware of in case you run into a similar problem.

The second problem I ran into is that the F9P module seems to be sensitive to my antenna splitter, a standard SMA DC block and tee which I have used on many other receivers before without issue.  It works fine if the F9P power is blocked but if the M8T power is blocked, the F9P seems to detect the tee and shut off the antenna power.  Again, not a big deal, but something to be aware of.

For the rovers, I used a u-blox ANN-MB-00 dual-frequency antenna for the F9P receiver.  This is the antenna u-blox provides with its F9P eval units.  I had planned to split this antenna signal to both receivers as I usually do, but I ran into the problem described above, and not fully understanding the issue yet, ended up using a separate Tallysman TW4721 L1 antenna for the M8T receiver.  Both antennas were attached directly to the car roof which acted as a large ground plane.

I used a hot spot on my cellphone to stream the NTRIP base station observations from the phone to a laptop and then to the F9P receiver and to two instances of RTKNAVI, one for each rover receiver.

Streaming the base observations to the F9P, while simultaneously logging the internal RTK solution and the raw rover observations, and also sending the raw rover observations to RTKNAVI, all over a single serial port can be challenging since only a single application can be connected to the serial port at one time.  Fortunately RTKLIB has a little trick to deal with this.  If the “Output Received Stream to TCP Port” box is checked in STRSVR and a port number specified as shown below, all data coming from the other direction on the serial port will be redirected to a local TCP/IP port.  This data  can then be accessed by any of the other RTKLIB apps as a TCP Client with server address “localhost” using the specified port number.

I set up the F9P rover to output both raw observation and navigation messages (UBX-RXM-RAWX/ UBX-RXM-SFRBX) and solution position messages( NMEA-GNGGA).  RTKNAVI then logged all of these messages to a single log file.  RTKCONV and RTKPLOT can both extract the messages they need from this file and ignore the rest so combining them was not an issue.

The NMEA-GNGGA messages from the F9P default to a resolution of 1e-7 degrees of latitude and longitude which works out to roughly 1 cm.  For higher resolution you can increase the resolution of the GNGGA message by setting a bit in the UBX-CFG-NMEA message.   Unfortunately I did not realize the resolution issue until after I collected the data and so my results for the internal F9P solution for this experiment were slightly deteriorated by the lower resolution.

I used the most recent demo5 b31 code to calculate all of the RTKLIB solutions.  Both the demo5 and the 2.4.3 versions of RTKLIB have been updated to translate the new dual frequency u-blox binary messages.  The demo5 solution code will process all the dual frequency observations but I don’t believe 2.4.3 code is able to process the E5b Galileo measurements yet.  The RTKLIB 2.4.2 code however does not have any of these updates.

The demo5 code updates made in the recent B30/B31 versions are based on the updates from the 2.4.3 B30 code but include some modifications to the u-blox cycle slip handling that I had previously added to the demo5 code for the M8T.  Since the demo5 code is primarily aimed at low cost receivers I also made some changes to make the E5b frequency a little easier to specify and faster to process.

To run the RTKNAVI F9P real-time solution, the only significant change I needed to make to the default M8T config file was to change the frequency option from “L1” to “L1+L2+E5b”.  I should have also changed the base station position to “RTCM Antenna Position” to take advantage of the F9 base station RTCM 1005 base location messages but neglected to do this.  This caused the RTKNAVI solution to differ from the F9P solution by small constant values due to the approximate base location used in the RTKNAVI solution.  I later used exact base locations for the RTKLIB post-processing solutions to verify that the different solutions did in fact all match.

Once I had everything set up, I then drove around the local neighborhood, emphasizing the streets with most challenging sky views since I knew both receivers would perform well and be difficult to distinguish if the conditions were not challenging enough.

Results:

I first converted the binary log files to observation files using RTKCONV and verified that the F9P was logging both L1 and L2 measurements for GPS, GLONASS, and Galileo.  I had the Bediou constellation enabled as well but as I verified later, there were no fully operational Bediou satellites overhead when I collected the data.

Here is an plot of the L1 observations for the M8T on the left and the F9P on the right.  I have zoomed into just two minutes during some of the more difficult conditions to compare the two.  The red ticks are cycle slips and the grey ticks are half cycle ambiguities.

First, notice that the F9P does not log observations for the SBAS satellites, while the M8T does, giving the M8T a couple more satellites to work with.  However, what’s also interesting, and I don’t know why, is that the F9P collected quite good measurements from the Galileo E27 satellite, while the M8T did not pick up this one at all.  Of course the F9P also got a second set of measurements from the second frequency on each satellite and so overall ended up with nearly twice as many raw observations as the M8T.

Also notice that the F9P reports somewhat less cycle slips and many less half cycle ambiguities than the M8T.  Some of this might be because of the different antennas, but particularly the large difference in half cycle ambiguities suggests that u-blox has made other improvements to the new module besides just adding the second frequencies.

Another thing to notice is the number of Galileo satellites.  If you compare these plots to earlier experiments I’ve posted, you’ll notice there are more Galileo satellites now as more and more of them are starting to come online.  The extra satellites really help the M8T solutions because as you can see, they tend to have the highest quality observations through the most difficult times.  Again I don’t know why this is.  It doesn’t appear to be as true for the F9P though.

Next I looked at the real-time solutions.   First, the RTKLIB solutions with RTKNAVI for both receivers.  For the full driving route, the M8T solution had a 77.3% fix rate and the F9P solution had a 96.4% fix rate.  Here is a zoom into the most challenging part of the drive, an older neighborhood with narrower streets and larger trees, the M8T is on the left, and the F9P on the right.  Fixed solutions are in green and float in yellow.  Clearly here the F9P significantly outperformed the M8T.

The F9P internal solution did even better with a 99.2% fix rate, as shown in the plot below.  All three solutions agreed within 2 cm horizontal, a little more in vertical, and none of them showed any sign of any false fixes.

I didn’t do any static testing to characterize time to first fix as I sometimes do, but for this one run the RTKLIB time to first fix for the M8T was 18 seconds while the RTKLIB F9P solution reached first fix in 6 seconds.  In both cases, RTKLIB was started after the hardware had time to lock to the satellites and acquire navigation data for all satellites.  The demo5 RTKLIB code has an additional fix constraint based on the kalman filter position variance to minimize false fixes while the filter is converging and so this can sometimes affect time to first fix.  I had increased this parameter to 0.1 meter for this experiment since the large number of measurements reduces the chance of a false fix.  This constraint did not limit the M8T time to first fix but it did so for the F9P, meaning the F9P would have reached first fix even faster if this constraint were opened up more.   I can’t tell what the equivalent number would be for the internal F9P solution from this data since it had already been running and achieved a fix before I started logging the data but generally the F9P seems to acquire first fix very quickly.

Next I post-processed both data sets with RTKLIB using the combined-mode setting to run the kalman filter both forwards and backwards over the data.  This noticeably improved the results, bringing the fix rate for the M8T up from 77.3% to 96.1% and the F9P fix rate from 96.4% to 98.8%.

Conclusion:

Obviously this is not enough data to make any definitive conclusions, but so far I am very impressed with the F9P!  Both the raw observations and the internal RTK solutions for the F9P look as good as anything I’ve seen from receivers costing many times what this one cost.

If anybody would like to look at the data from this experiment more closely, I have uploaded it to here.  I should mention that all the fix rates I specify in this post and other posts won’t exactly match the fix rates in the raw solutions, since I adjust the data start and end times to be consistent between data sets and to start after all solutions have achieved first fix.  I believe this is the fairest way to compare multiple solutions, especially when there is a mix of internal and RTKLIB solutions

Also, I’d like to thank Gumstix again for making these modules available to me for evaluation!

Update: 12/2/18:

Reviewing the config files I used for this experiment I discovered that, while I had intended the real-time and post-processing config files to be identical, there were in fact some small differences between them.  One difference in particular, that appears to have affected the results as described above, is that I reduced the minimum number of consecutive samples required to hold ambiguities (pos2-arminfix) from 100 to 20 for the post-processed config files.  A value of 100 corresponds to 20 seconds at the experiment’s 5 Hz sample rate which is a value I have typically used.  However, with lower ambiguity tracking gain (pos2-varholdamb=0.1) and the increase in observations coming from including Galileo, the chances of false fixes is reduced and I have been tending to use lower values of arminfix in more recent experiments.   Reducing this value appears to explain a large part of the jump in percent fix for the M8T between real-time and post-processing, rather than the switch from forward-only to combined that I attribute it to above.  These differences only affect the comparisons between RTKLIB real-time and post-processed results, and not between the M8T and the F9P since the config files were consistent between the two receivers.

This was only intended to be a quick first look at the F9P.  It will require more data and more analysis to properly characterize the F9P so I  won’t try to do that here but I will share the table shown below which includes a few cases I have run since the original post.  I hope to dig into the details in future posts.

	Fix percent
	Real-time	Post-process	Post-process	Post-process
	ARMIN=100	ARMIN=100	ARMIN=20	ARMIN=20
	forward-only	forward-only	forward-only	combined
M8T/RTKLIB	77.3%	81.2%	96.0%	96.1%
F9P/RTKLIB	96.4%	99.1%	99.3%	98.8%
F9P internal	99.2%

One last point worth making is that while at first glance the post-process fix percent increase from M8T=96.0% to F9P=99.3% may not sound that significant, it is in fact a factor of nearly six if you consider it as a decrease in float from 4.0% to 0.7%.

Comparison of the u-blox M8T to the u-blox M8P

I recently acquired a couple of  u-blox C94-M8P receivers and so I am now able to do a direct comparison between the u-blox M8T and M8P modules, something I’ve been wanting to do for a while.

I believe the hardware between the two receivers is identical, the differences are only in firmware.  The most significant difference in firmware is that the M8P includes an internal RTK solution.  In order to squeeze this into the firmware, however, they had to remove support for the Galileo and SBAS constellations, so this is another fairly significant difference.

Cost, of course, is also different.  The M8T receivers I usually use are available from CSGShop for $75.  CSG sells an M8P receiver for $240 or you can buy a kit with two C94-M8P eval boards from u-blox for $400.  The C94-M8P boards also include on-board radios.

In this post, I will describe how I used RTKLIB and a cell phone hot spot to connect the M8Ps rather than using the internal radios.  I will also compare the RTKLIB solutions for a pair of M8T receivers with the internal u-blox RTK solution for a pair of M8P receivers. I hope to compare the RTKLIB solution to the internal solution for a pair of M8Ps in a future post.  If you are mostly interested in the M8T to M8P comparison results, you can jump directly to the end of this post.

To set up the experiment I first connected both a C94-M8P receiver and a CSG M8T receiver to the GPS survey antenna on my roof using a signal splitter.  I then connected both receivers to a laptop with USB cables.  I configured the M8P using u-center following instructions in the C94-M8P Setup Guide with a few modifications.  First of all, I normally use an NTRIP caster over a cell phone hot spot for my real-time data link so didn’t want to bother with the on-board radios.  I disabled the radios following the instructions in the setup guide.  This involves removing a plastic cover, soldering a wire to a capacitor, drilling a hole in the plastic cover to run the wire, then plugging the other end into a pin on the connector.  It is also possible to do this by connecting an external voltage and ground to the correct pins on the connector, but a simple jumper option would have been a lot more convenient.

The setup instructions are intended to run the setup and solution output messages over the USB port while running the RTCM raw observation messages between receivers over the UART interface.  The UART interface is internally connected to the radios.  Since I am not using the radios, I wanted to run all communication over the USB interface to avoid extra cables.  To do this, I disabled the UART interface and configured the USB interface for UBX messages in and RTCM messages out using the UBX-CFG-PRT configure command from u-center.  Following the C94-M8P setup instructions, I then specified a fixed base station location with the UBX-CFG-TMODE3 command and enabled 1005,1077,1087, and 1230 RTCM messages which include the raw observations, base station location, and GLONASS biases.

I setup the base station M8T receiver as I usually do to output the raw observations using the UBX-RXM-RAWX messages.

I then started two copies of the RTKLIB STRSVR app on the base station laptop and streamed both sets of base observations to an NTRIP server as I’ve described in an earlier post using the free RTK2GO.com community NTRIP server.  With this setup, I can receive the NTRIP streams anywhere I can get cell phone coverage, using a cell phone hot spot and a laptop connected to the two rovers and I can test over much longer distances than the radios would allow.

One thing to be aware of is that there are separate versions of the receiver firmware for the M8P base and M8P rover.  I didn’t realize this at first.  It turned out that both of my receivers came loaded with the rover firmware so I spent an embarrassingly long time unsuccessfully trying to configure one of them as a base station.  Once I realized the problem, I was able to fairly quickly download the base firmware from the u-blox website, then use u-center to download the base station firmware to the receiver.

Next, I attached the two rover receivers to a laptop using USB cables and to a single antenna using a signal splitter.  To make the results more comparable to some of my other recent experiments I used the same GPS-500 L1/L2 antenna from SwiftNav that I used for those experiments.  This is a more expensive antenna than generally would be used with these receivers but I wanted to avoid mixing differences between antennas with differences between receivers.

The M8P rover does not need much configuration since it will by default process any incoming RTCM messages as inputs to an RTK solution.  I did configure the USB port  to support NEMA, UBX, and RTCM messages in and UBX and NMEA messages out.  For the static rover test, I left the position message output rate at the default 1 Hz, but increased it to 5 Hz for the dynamic rover test.  I suspect this only affects the message output rate, and not the solution itself since the solution is probably run at a faster internal rate.

The only other setting I modified in the M8P rover receiver was the dynamic model of the navigation mode using the UBX-CFG-NAV5 message.  I set it to “Pedestrian” for my static rover test and “Automotive” for my moving rover test.

I then started another copy of STRSVR on the rover laptop to stream the M8P base station data from the NTRIP server to the M8P rover over the USB COM port.

At this point, getting the solution output messages from the M8P rover is a little tricky.  Only a single user can attach to a Windows COM port at one time and the STRSVR app is not fully bi-directional.  One solution might be to use u-center to stream the NTRIP data and receive the rover messages but I did not verify if this is possible.

Instead, I used a feature of STRSVR that allows you to pipe the data coming from the other direction to a TCP/IP port which can then be processed by either another copy of STRSVR, or plotted with RTKPLOT, or used as an input to RTKNAVI, or all three at once.  Below, on the left, I show how I modified the STRSVR setup that streams the base data from the NTRIP client to the rover by checking the “Output Received Stream to TCP Port” and selecting port 1000.  On the right, I show the STRSVR setup necessary to stream this incoming data from the rover to a file.  This is a convenient work-around for the problem of only being able to connect one user at a time to a COM port.

In my case I configured the rover receiver to send both the M8P internal solution position using NMEA messages and the raw observations using UBX messages so that I could post-process the raw observations later with RTKLIB.  In general though, it is only necessary to send the NMEA messages.  I configured the rover to send GGA and RMC NMEA messages, but others will work as well.

For the M8Ts, I ran a real-time RTKLIB solution using RTKNAVI and my normal kinematic solution configuration for both static and moving rover tests.  As usual, I used the demo5 version of RTKLIB for this test.

So, how did they compare?  First of all, a few differences to consider between the solutions.   The M8P solution only uses GPS and GLONASS satellites since the SBAS and Galileo satellites have been disabled in the M8P firmware as I mentioned earlier.  This should give the M8T solution an advantage.  The M8P also has limited processing power relative to the laptop running the RTKLIB solution, so this may also give the M8T solution an advantage.  On the other hand, I’m sure that u-blox has lots of smart people that know all the internal details of the hardware which should give the M8P solution an advantage.

For my first comparison, I did a a static rover test with the antenna located on a tripod a few meters from the house and partially obstructed by trees.  To avoid different starting conditions between the two receivers, I connected both receivers to the antenna and waited till they both converged to fixed solutions before starting the test.  I then disconnected the antenna for about 30 seconds, and then reconnected it.  This will cause both receivers to lose lock with all satellites and the solutions will have to re-converge from scratch so this should be a fair comparison.  I disconnected and reconnected the antenna several times to compare the times to re-acquire fix and the accuracies of the fixes.

The results are plotted below, the M8P internal solution on the top, and the real-time RTKLIB M8T solution on the bottom.  I re-connected the antenna five times.  Once, the internal M8P solution re-acquired fix slightly quicker than the RTKLIB M8T solution, once the M8T re-acquired fix slightly quicker than the M8P.  The other three times, the M8T re-acquired fix significantly quicker than the M8P.   The M8P times to re-acquire varied from 52 to 220 seconds.  The M8T took from 50 to 76 seconds.

The M8T RTKLIB solutions had no false fixes, the M8P internal solutions had false fixes with several meter errors for over five seconds in two of the five tests.  The false fixes are circled with red in the plot below.

The E-W and N-S axes matched within one centimeter between the two solutions.  The U-D axis at first appears to differ by 21.5 meters between the two solutions but this is because I specified the RTKLIB vertical solution to be relative to the ellipsoid, while the M8P solution is relative to the geoid.  If I subtract the geoid to ellipsoid offset (available in the GGA message from the M8P), then the two solutions match in the vertical axis as well.

Overall, I would have to say that in this particular run,  the RTKLIB M8T solution out-performed the M8P internal solution by a fairly significant amount.

One other observation about the M8P solution is that the precision reported (at least in the GGA message) is only to one centimeter in the horizontal axes and 10 cm in the vertical accuracy.  I didn’t see any obvious way to get outputs with better precision than this, at least with the NMEA messages.  [Note 8/16/18: It turns out that the UBX-CFG-NMEA message can be used to enable high precision mode to fix this.  Thanks to Charles Wang for pointing this out in a comment]

Next I mounted the same antenna on the roof of my car and drove around the neighborhood.  The sky view varied from nearly unobstructed to moderate tree canopy.  Here are the results of that test, the MP internal RTK solution on the left, and the M8T real-time RTKLIB solution on the right.

The fix rate for the M8P solution was 52.4%.  The M8T solution was noticeably better with a 95.2% fix rate.

Here’s a plot of the difference between the two solutions (green indicates both solution are fixed, yellow indicates that at least one is float).  The U-D axis differences are fairly large, mostly because of the 10 cm vertical precision of the M8P position messages.  There is also a fairly large discrepancy between the two solutions around 22:55:00.  This corresponds to a 15 second gap in base station data, probably due to a loss of cell phone reception.  This seems to be a large error for such a short base outage but I was not able to narrow down the cause beyond this or determine which receiver contributed more to this combined error.  This probably deserves more investigation in a future post.

Again, the real-time RTKLIB M8T solution appeared in this test to be better than the internal M8P solution.

The most likely reason the M8T is getting a better solution is that it is using more satellites.  Here are the rover raw observations for the M8P on the left, and the M8T on the right.  With the GPS and GLONASS constellations, both receivers are seeing 13 satellites.  With Galileo and SBAS, the M8T is seeing an additional 7 satellites.  One of these Galileo satellites is not fully operational yet and so is not used in the solution, but that still gives 6 extra satellites, nearly 50% more than the M8P solution.

If the extra satellites are what is improving the M8T solution, then running a solution with the M8T data without the SBAS and Galileo satellites should give a solution similar to the M8P solution.

This is easy to do, so let’s try it.  Here’s a post-processed solution for the M8T data using only the GPS and GLONASS satellites.

Fix rate is 66.5%, reasonably close to the 52.4% fix rate from the M8P internal solution.  This suggests that a large part of the improvement for the M8T solution is coming from the additional satellites and not from any differences in the solution algorithms.

As in all my comparisons, this is intended to be one user’s initial experience, and not any kind of rigorous comparison test.  Please interpret these results with that in mind.  If you have experiences that differ from these I would be very interested to hear about them.

Glonass Ambiguity Resolution with RTKLIB Revisited

To get a high precision fixed solution in RTKLIB we need to resolve the integer ambiguities that come from the carrier phase measurements.  Resolving the integer ambiguities for the GLONASS satellites is more challenging than resolving them for the other constellations.  This is because, unlike the other constellations, the GLONASS satellites all transmit on slightly different frequencies.  This introduces an additional bias error in the receiver hardware.

These hardware biases are constant, generally the same for all receivers from the same manufacturer, are proportional to carrier frequency and are similar between L1 and L2.

In the demo5 version of RTKLIB, there are four choices for how to handle GLONASS ambiguity resolution (AR). I will cover all four briefly, but then focus on the “autocal” setting which I have enhanced in the most recent version (b29c) of the demo5 code.

Off:  If Glonass  AR is set to “Off”, then the raw measurements from the Glonass satellites will be used for the float solution but ambiguity resolution will be done only with satellites from the other constellations.  If you are not using the demo5 version of RTKLIB, this is usually your only choice when using receivers from different manufacturers for the rover and the base.  However, you are giving up a significant amount of information by ignoring the GLONASS ambiguities and so I would not recommend this setting if you are using the demo5 code, unless of course your receivers don’t support the Glonass satellites.

On:  If Glonass AR is set to “On” then RTKLIB will treat the Glonass ambiguities the same as the ambiguities from the other constellations and will not make any attempt to account for the additional hardware biases.  Use this setting if your base and rover receivers are from the same manufacturer, since in this case, the biases will cancel and can be ignored.  There are also some cases in which different manufacturers have equal or nearly equal biases as we will see later, in which case you can also use this setting.  This is your best solution for dealing with Glonass ambiguities.  I always try to use matched receivers for base and rover if possible.

Fix-and-Hold: This is an option I have added to the demo5 code for Glonass AR.  It is an extension to the “fix-and-hold” method used for other constellations but instead of using the additional feedback to track the ambiguities, it uses it to null out the hardware biases.  I recommend this setting when using the demo5 code with unmatched receivers.  It takes advantage of the additional information in the Glonass ambiguites most of the time.  However, fix-and-hold is not enabled until after a first fix has been achieved, and so the Glonass ambiguities are not available until then.  This can mean longer time to first fix and less robustness compared to the “On” option, so don’t use this option for matched receivers.

Auto-cal:  This option adds additional states to the kalman filter to estimate the receiver hardware biases as a function of carrier frequency, one state for L1, another for L2.  In previous experiments I had not had any success with it.  Recently, however, I discovered that if I adjusted the filter settings, it can be effective for a zero baseline case, where base and rover are both connected to the same antenna so that almost all other errors are completely cancelled.  With a little more experimentation I also found that for short baselines it can also be effective if the kalman filter state is pre-set to something close to the final value before the solution is started.  It will then usually converge to the correct bias value.  However, there is currently no mechanism in the code to adjust any of these values, so I have not found this mode to be useful in its current implementation.

To make the auto-cal option more flexible, and hopefully more useful, I made a few modifications to it in the b29c code.  I added the capability to pre-set the initial state value and also to adjust the internal filter settings, specifically the initial variance and process noise for this state.  The units for the state, and hence for the initial value are in meters per frequency channel and values generally are within +/-5 cm per channel.  I used some existing config parameters that are currently unused to reduce the amount of code I needed to change.  Unfortunately it means that the names are not as descriptive as they could be.  The new config parameters are:

pos2-arthres2 = relative GLONASS hardware bias in meters per frequency slot
pos2-arthres3 = initial variance of GLONASS hardware bias states
pos-arthres4 = process noise for GLONASS hardware bias states

Bias values have been published for some of the most popular geodetic quality receivers but are generally not available for lower-cost or less popular receivers.  Here is a table of values from a paper published by Lambert Wanninger in 2011 for nine receiver manufacturers.

I was able to verify these results for Trimble, Leica, and Novatel, but I found a much lower value for Septentrio so I suspect the biases may have changed in their newer receivers.

To demonstrate the modified autocal option, I will start with a zero baseline case between a ComNav receiver and a Tersus receiver.  It is easiest to measure the hardware biases in the zero baseline case because most other errors will cancel and the hardware biases will be the dominant error.  In this case, I have significantly reduced the initial variance setting from the original value of 1.0 to 1E-7 and increased the process noise from 1E-6 to 1E-3.

I have run the solution several times with the initial bias value set to different numbers between -.05 and 0.06.  Here are the results for both L1 and L2.

The convergence occurs just after first fix is achieved.  If a fix is not achieved, then the state will not converge as you can see above for the 0.06 example.   In this case, the initial value was too far from the correct value and prevented getting a fix.  As you can see, all the other cases converged towards a single value around -.022, both for L1 and for L2.

Another way to visualize the error in the initial value is to look at the GLONASS residuals after first fix is achieved.  The plot below shows the GLONASS L1 carrier phase residuals  for different initial values, for 0.03 on the left, -0.05 in the middle, and what I believe is the correct value for this receiver combination of -.022 on the right.

Here are the same plots for the L2 carrier phase residuals.

Through a slightly tedious process, I am fairly easily able to iterate the residuals down to near zero for different pairs of receivers in my possession.  Note that this gives me the relative difference in biases for each receiver pair, and not absolute values for each receiver, unlike Wanniger’s table which is for absolute biases.

Extending the table to receivers used in nearby CORS stations is a little more challenging because the initial bias value needs to be fairly close to get a first fix and hence a convergence, but still possible if the base station is not too distant.   I found data sets that included CORS data from Leica, Novatel, Trimble, and Septentrio receivers.  Using the above procedure to iterate the residuals down to near zero, I was then able to extend my table and make the values absolute by choosing the unknown offset to make my bias pairs align with Wanninger’s table.  This is the resulting table I created.

[Note: table updated 3/16/20:]

ComNav               =   2.3 cm
Leica                      =   2.3 cm
Novatel                 =  2.3 cm
Septentrio           = -0.3 cm
Spectra Physics =  0.0 cm
SwiftNav              =  0.0 cm
Tersus                   =  0.0 cm
Topcon                 =  0.0 cm
Trimble                = -0.7 cm
u-blox M8T        = -3.2 cm
u-blox F9P          =  0.0 cm

To generate an initial value for the bias state from this table for an RTKLIB solution, subtract the base station bias from the rover bias, then divide by 100 to convert from centimeters to meters.  This value can then be used to set the “pos2-arthres2” config parameter in the config file.  For the RTKPOST and RTKNAVI GUI option menus I have labeled this “Glo HW Bias”.

To test this code on an independent set of data after generating the table, I used a data set recently sent to me by a reader.  It consists of a u-blox  M8T receiver for rover and Leica receiver just a few kilometers away for base, and was collected in Europe.  The rover position was static but I ran the solution in kinematic mode to make the solution a little more challenging and to make any errors in the solution more visible.

To generate the correct config value for RTKLIB I  subtracted the Leica bias of 2.3 cm from the above table from the u-blox bias of -3.2 cm to get a relative bias between receivers of -5.5 cm or -0.055 m.  I added “pos2-arthres2=-0.055” to the config file and then ran the solution four times, with pos2-gloarmode set to “off”,”fix-and-hold”,”autocal”, and “on”.  Although I left the bias value set for all runs it is ignored unless gloarmode is set to autocal.

Here are the times to first fix, the number of satellite pairs used for the initial fix, and the number of satellite pairs being used for fix after 10 minutes.

	Time to	# sat pairs used	# sat pairs used for
GLO AR mode	first fix	for initial fix	fix after 10 min
OFF	4:10	7	7
Fix&Hold	4:10	7	11
Autocal	1:05	14	14
On	6:47	14	14

As you would expect, the time to first fix for gloarmode=”off” was the same as “fix-and-hold” since “fix-and-hold” does not use the GLONASS satellites for initial fix.  After 10 minutes it was still only including four of the GLONASS satellites in the ambiguity resolution which was a little unusual, typically I would have expected more GLONASS satellites to be included.

With gloarmode=”autocal”, the time to first fix was reduced from 250 seconds to 65 seconds and the number of satellites included in the first fix increased from 7 to 14., both significant improvements.

The most surprising thing in this data is that when gloarmode was set to “on” it acquired a fix at all.  In many similar cases it will never get a fix.  The GLONASS carrier phase residuals after initial fix were very high though as can be seen below.  The left plot is with gloarmode set to “on”, and the right plot is with it set to “autocal”.

The ambiguity resolution ratio was also much higher when autocal was enabled as can be seen below (yellow/green=autocal, olive/blue=on) which improves robustness.

The large residuals did not affect the solution position, as the two solution did not differ by more than 2 mm at any time.  The autocal solution however is much more robust in the sense that it is less likely to lose fix.

Although I have found the results with autocal enabled are generally excellent with relatively short baselines (<10 km), I have found the results less encouraging for longer baselines (>25 km).  In these case I have found that I often get better results with pos-gloarmode set to “fix-and-hold” then I do with “autocal”.  I don’t understand exactly why this is, but suspect that the fix-and-hold correction is more general and may be correcting for more than just the GLONASS hardware biases.

The code changes for this feature are included both in my Github repository and in the newest (demo5 b29c) executables available to download from the rtkexplorer website.   If you choose to experiment with this feature, please let me know if you find any errors in my table, or can add values for any additional receivers.

[Note 6/17/18:  I had a issue with uploading the executables to the website.   If you downloaded them prior to 6/17/18, please download again to get the updated version.] 

Swiftnav experiment: Improvements to the SNR

In my previous couple of posts, I evaluated the performance of a pair of dual freqeuncy SwiftNav Piksi multi receivers in a moving rover with local base scenario.  I used a pair of single frequency u-blox M8T receivers fed with the same antenna signals as a baseline reference.

It was pointed out to me that the signal to noise ratio (SNR) measurements of the rovers were noticeably lower than the bases, especially the L2 measurements and that this might be affecting the validity of the comparison.  This seemed to be a valid concern so I spent some time digging into this discrepancy and did indeed find some issues.  I will describe the issues as well as the process of tracking them down since I think it could be a useful exercise for any RTK/PPK user to potentially improve their signal quality.

Previously , in another post, I described a somewhat similar exercise tracking down some signal quality issues caused by EMI from the motor controllers on a drone.  In that case, though, the degradation was more severe and I was able to track it down by monitoring cycle slips.  In this case, the degradation is more subtle and does not directly show up in the cycle slips.

Every raw observation from the receiver generally includes a signal strength measurement as well as pseudorange and carrier phase measurements.  The SwiftNav and u-blox receivers both actually report carrier to noise density ratio (C/NO), rather than signal to noise ratio (SNR) but both are measures of signal strength.  They are labelled as SNR in the RTKLIB output, so to avoid confusion I will refer to them as SNR as well.  I will only be using them to compare relative values so the difference isn’t important for this exercise, but for anyone interested, there is a good explanation of the difference between them here.  Both are logarithmic values measured in dB or dB-Hz so 6 dB represents a factor of two in signal strength.

Since the base and rover have very similar configurations we would expect similar SNR numbers between the two, at least when the rover antenna is not obstructed by trees or other objects.  I selected an interval of a few minutes when the rover was on the open highway and plotted SNR by receiver and frequency for base and rover.  Here are the results, base on the left and rover on the right.  The Swift L1 is on the top, L2 in the middle, and the u-blox L1 on the bottom.  To avoid too much clutter on the plots, I show only the GLONASS SNR values, but the other constellations look similar.

Notice that the L1 SNR for both rovers is at least 6 dB (a factor of 2) lower than the base, and the Swift L2 SNR is more like 10 dB lower.  These are significant enough losses in the rover to possibly affect the quality of the measurement.

The next step was to try and isolate where the losses were coming from.  I set up the receiver configurations as before and used the “Obs Data” selection in the “RTK Monitor” window in RTKNAVI to monitor the SNR values in real time for both base and rover as well as the C/NO tracking window in the Swift console app.  I then started changing the configuration to see if the SNR values changed.  The base and rover antennas were similar but not identical so I first swapped out the rover antenna but this did not make a difference.  I then moved the rover antenna off of the car roof and onto a nearby tripod to see if the large ground plane (car roof) was affecting the antenna but this also did not make a difference.  I then removed the antenna splitter, but again no change.

Next, I started modifying the cable configuration between the receivers and my laptop.  To conveniently be able to both collect solution data and be able to collect and run a real-time solution on the raw Swift observations, I have been connecting both a USB serial cable and an ethernet cable between the Swift board and my laptop.  My laptop is an ultra-slim model and uses an etherent->USB adapter cable to avoid the need for a high profile ethernet connector.  So, with two receivers and my wireless mouse, I end up having more USB cables than USB ports on my computer and had to plug some into a USB hub that was then plugged into my laptop.

The first change in SNR occured when I unplugged the ethernet cable from the laptop and plugged it into the USB hub.  This didn’t affect the L1 measurements much but caused the Swift L2 SNR to drop another 10 dB!  Wrong direction, but at least I had a clue here.

By moving all of the data streams between Swift receiver and laptop (base data to Swift, raw data to laptop, internal solution to laptop) over to the ethernet connection I was able to eliminate one USB serial port cable.  This was enough to eliminate the USB hub entirely and plug both the USB serial cable from the u-blox receiver and the ethernet->USB cable from the Swift receiver directly into the laptop.  I also plugged the two cables into opposite sides of the laptop and wrapped the ethernet->USB adapter with aluminum foil which may have improved things slightly more.

Here is the same plot as above after the changes to the cabling from a drive around the neighborhood.

I wasn’t able to eliminate the differences entirely, but the results are much closer now.  The biggest difference now between the base configuration and the rover configuration is that I am using a USB serial cable for the base, and a ethernet->USB adapter cable for the rover so I suspect that cable is still generating some interference and that is causing the remaining signal loss in the rover.  Unfortunately I can not run all three streams I need for this experiment over the serial cable, so I am not able to get rid of the ethernet cable.

I did two driving tests with the new configuration, similar to the ones I described in the previous posts.   One was through the city of Boulder and again included going underneath underpasses and a parking garage.  The second run was through the older and more challenging residential neighborhood.  Both runs looked pretty good, a little better than the previous runs but it is not really fair to compare run to run since the satellite geometry and atmospheric conditions will be different between runs.  The relative solutions between Swift and u-blox didn’t change much though, which is probably expected since the cable changes improved both rovers by fairly similar amounts.

Here’s a quick summary of the fix rates for the two runs.  The fix rates for the residential neighborhood look a little low relative to last time but in this run I only included the most difficult neighborhood so it was a more challenging run than last time.

	Fix rates
	City/highway	Residential
Swift internal RTK	93.60%	67.50%
Swift RTKLIB PPK	93.70%	87.90%
U-blox RTKLIB RTK	95.70%	92.80%
U-blox RTKLIB PPK	96.10%	91.10%

Here are the city/highway runs,  real-time on the top, post-process on the bottom with Swift on the left and u-blox on the right.  For the most part all solutions had near 100% fix except when recovering from going underneath the overpasses and parking garage.

Here are the same sequence of solutions for the older residential neighborhood.  This was more challenging than the city driving because of the overhanging trees and caused some amount of loss of fix in all solutions.

Here’s the same images of the recovery after driving under an underpass and underneath a parking garage that I showed in the previous post.  Again, the relative differences between Swift and u-blox didn’t change much, although the Swift may have improved a little.

Overall, the improvements from better SNR were incremental rather than dramatic, but still important for maximizing the robustness of the solutions.  This exercise of comparing base SNR to rover SNR and tracking down any discrepancies could be a useful exercise for anyone trying to improve their RTK or PPK results.

Underpasses and urban canyons

[Update: 4/17/18:  Although I don’t think it changes the results of this experiment significantly, there was an issue with apparent interference from a USB hub and ethernet cable on the rover setup during this testing.  See the next post for more details. ]

In my last post I demonstrated fairly similar fix rates and accuracies between an M8T single-frequency  four-constellation solution and a SwiftNav Piksi dual-frequency two-constellation solution.

One advantage often mentioned for dual frequency solutions for moving rovers is that their faster acquisition times should help when fix is lost due to a complete outage of satellite view caused by an underpass or other obstruction.  This makes sense since the dual frequency measurements should allow the ambiguities to be resolved again more quickly.

Since my last data set included several of these types of obstructions I thought it would be interesting to compare performance specifically for these cases.

To create the Google Earth images below I used the RTKLIB application POS2KML to translate the solution files to KML format files and then opened them with Google Earth.

Here are the raw observations for the first underpass I went under, Swift rover on the left, M8T rover on the right.  In this case there was an overhead sign just before the underpass which caused a momentary outage on all satellites followed by about a two second outage from the underpass, followed by a period of half cycle ambiguity as the receivers re-locked to the carrier phases.

Here’s the internal Swift solution for the sign/underpass combo above at the top of the photo and a second underpass at the bottom of the photo.  For the first underpass, the solution is lost at the sign, achieves a float solution (yellow) after about 9 seconds, then re-fixes (green) after 35 seconds.

Here’s the RTKLIB post-processed solution (forward only) for the Swift receivers with fix-and-hold low tracking gain enabled as described in my previous post.  It looks like a small improvement for both underpasses.  The solution loses fix at the sign but in this case maintains a float solution until the underpass.

Here’s the RTKLIB post-processed solution (same config) for the M8T receivers.  Notice the no-solution gaps after the underpasses are shorter.  In this case, for the upper underpass, a solid fix was re-achieved after about 21 sec.

Here’s a zoom in of the M8T solution (yellow dots) for the lower underpass.  If the position were being used for lane management it looks like the float solution would probably be accurate enough for this.  The other yellow line with no dots is the gap in the Swift solution.

Here’s a little further down the road.  At this point the Swift solution achieves a float position at about the same time the M8T solution switches to fix.  Lane management would clearly be more difficult with the initial Swift float solution.

Next, I’ll show a few images from another underpass.  In this case I drove under the underpass from the left, turned around, then drove under the underpass again from the right.  The Swift internal solution is on the left, the Swift RTKLIB solution in the middle, and the M8T RTKLIB solution on the right.  Notice that the time to re-acquire a fix is fairly similar in all three cases.

Here is zoom in of the two Swift solutions, they are quite similar.

Here is a zoom-in of the M8T RTKLIB solution.  Again, the float solution is achieved very quickly, and appears to be accurate enough for lane management.

My last test case was a combination urban canyon and parking structure.  In the photo below, I drove off the main street to the back of the parking garage, underneath the pedestrian walkway, into the back corner, then underneath the back end of the garage and then back to the main street.  I would consider this a quite challenging case for any receiver.

Here are the raw observations.

Here are the three solutions, again the Swift internal is on the left, the Swift RTKLIB in the center, and the M8T RTKLIB on the right.

 

Here is an image of the Swift internal solution.

Here is an image of the Swift RTKLIB solution

And here is an image of the M8T RTKLIB solution

In this case, the M8T RTKLIB solution appears to be the best.

So, this experiment seems to show that a dual frequency solution will not always handle satellite outages better than single frequency solutions.  In this case, the extra Galileo and SBAS satellites in the M8T solution seem to have helped a fair bit, and the M8T solution is, at least to me, surprisingly good.

If anyone is interested in analyzing this data further, I have uploaded the raw data, real-time solutions, and config files for the post-processed solutions to the sample data sets on my website, available here.  I should mention that there is an unexplained outage in the Swift base station data near the end of the data set.  This could have been caused by many things, most of them unrelated to the Swift receiver, so all the analysis in both this post and the previous post were done only for the data before the outage.

 

 

 

 

 

 

 

 

 

 

U-blox announces the F9 platform with L1/L2/L5 frequencies supported

https://www.u-blox.com/en/press-releases/u-blox-announces-u-blox-f9-robust-and-versatile-high-precision-positioning-technology

Using RTKLIB to process ComNav receiver data

Previously I collected some data with a pair of ComNav K708 receivers and compared the internal real-time ComNav position solution with a real-time RTKLIB solution from a pair of u-blox M8T receivers.  The results showed, at least for the moving rover case, that the two were quite similar.  In this post I will post-process the ComNav raw data with RTKLIB and compare the results both to the internal ComNav solution and the M8T post-processed RTKLIB solution .  The ComNav receiver configured to sell for sub-$1000 is setup to receive only GPS and GLONASS L1 and L2 frequencies so this is how I ran this test.  The M8T will also receive Galileo and SBAS satellites so I included these in the M8T solutions.

RTKLIB is not able to directly process raw binary ComNav data, but I had configured the receivers to output the raw data in RTCM3, then converted this to Rinex using RTKCONV, so this was not a problem.

Before we can get a solution however, we need to deal with the issue that the ComNav GPS L2 observations are a mix of L2 and L2C as seen in the RINEX observation data below.

RTKLIB handles multiple code types for the same frequency like this by choosing the code with higher priority as set in the code priority table in rtkcmn.c.  Unfortunately this choice is made for the full data set, not each individual observation, so the lower priority observations are thrown out even for epochs when the higher priority code observations are not present.  There is an option in the ComNav configuration to force all observations to L2 and this would have been one solution to the problem.  However, in theory, the L2C code should be more robust than the L2 code, so using L2C when we have the option as the ComNav default setup does is probably the right choice.

I was not able to find any simple fix for the RTKLIB code or configuration that would allow it to process both observation types so I simply edited the list of observation types in the RINEX file headers and changed “C2X L2X S2X” in the list to “C2W L2W S2W”.  With this change, all the observations are now described as L2 and none will be thrown out.  It would be a problem if we were mixing L2 observations from the rover with L2C observations from the base, but since both will be consistently L2 or L2C, the differences should cancel, and this should not cause any significant errors in the solution.

I looked at three data sets, all from a moving rover with a local base station.  The first data set is configured as described in my earlier post, with both the M8T and the ComNav rover receivers sharing the same antenna.  In the second two data sets, the rovers are connected to separate antennas. I ran with continuous ambiguity resolution set and GLONASS ambiguity resolution enabled for all solutions.  The only change I made from the default ComNav config settings was to change the “rtkquality” setting from “quick” to “normal” since I had trouble with false fixes when it was set to”quick”.

First of all, let’s compare the ComNav internal solution to the ComNav RTKLIB solution.  Here’s the internal solution on the left, and the RTKLIB solution on the right.  In this case RTKLIB had a little higher fix rate at 76.8% vs the internal solution at 68.8%.

What I found more surprising was that the internal solution did not stay fixed for the circles I drove in the parking lot from  23:37 to  23:40.  Normally I rarely have trouble maintaining 100% fix for this part of the route since there are very few trees here and the sky views are almost 100%.

Here’s a comparison of the difference between the two solutions.

I expect the difference to be near zero for all fixed sections and for the most part this is true.  There is an exception though between  23:32 and 23:33.   I don’t have an absolute reference to compare to so I can’t be absolutely certain which is correct and which is in error.  However I can compare to the M8T solution and in this case the ComNav RTKLIB solution nearly exactly matches the M8T solution so I am fairly certain that it is the internal ComNav solution that is wrong.  This error is fairly large and is most likely caused by a false fix.  Also, in general, the error in the U-D axis is fairly large in both cases relative to what I am used to, but is worse in the COMNAV internal solution.  Below is the difference between the RTKLIB u-blox and RTKLIB ComNav solutions on the left, and the difference between the RTKLIB u-blox and internal ComNav solutions on the right, both for the U-D axis only.

Here is the M8T RTKLIB solution for comparison.  Fix rate was 81.8%, just slightly higher than the ComNav RTKLIB solution.

One advantage of post-processing the data over a real-time solution is that solutions can be run in combined mode where the result is a combination of running the solution through the kalman filter forwards and backwards.  Here’s the combined mode results, ComNav on the left, and M8T on the right.

In this case both solutions were near 100% fix.  ComNav does sell post-processing software that would probably also let you do the same thing but I do not have a copy of it and don’t know how much it cost.

The results for the second two data sets with separate antennas showed similar differences to this one so I’ll include the plots here but won’t go into any detailed analysis.  In both sets of plots, the left is the ComNav internal solution, the middle is the ComNav RTKLIB solution, and the right is the M8T RTKLIB solution.

The biggest difference in these two data sets is that the M8T results were slightly worse relative to the ComNav results than in the above example but this is most likely because the antennas were separate and I used a low-cost single feed L1 antenna instead of the higher performance dual feed L1 antenna I prefer to use in these comparisons.  There is a good description here of why the dual-feed antenna should give better results.

It is always dangerous to conclude too much from a single experiment but this data does support what I have found in my other single frequency to dual frequency comparisons.  For short baselines with a pair of matched receivers and moving rovers, an L1+L2 GPS/GLONASS solution tends to be similar in performance to a L1-only GPS/GLONASS/SBAS/Galileo solution.   I expect this would be less true for longer baselines and stationary receivers.

New 1.2 Swift firmware with a moving rover

In the last couple of posts I compared the u-blox M8T receiver and the Swift Piksi Multi receiver for a stationary rover and an external base station using the latest 1.2 firmware from Swift.  I did this for both RTK and PPP solutions.  In this post I will look at a moving rover case with a pair of Piksi receivers and a pair of M8T receivers.

The new Swift firmware supports raw observations and float solutions for the GLONASS satellites, but does not yet support ambiguity resolution for GLONASS.  In the previous experiments the lack of GLONASS ambiguity resolution did not affect the comparison since an external (non-matching reciever) base was used. This meant that the RTKLIB solutions were not able to use GLONASS ambiguity resolution either.

In this experiment I did use matching receivers for the base data, so the RTKLIB solutions do include GLONASS ambiguity resolution and so should provide an advantage to the Swift internal solutions.  As I did last time, I will compare the real-time Swift RTK solution with post-processed RTKLIB solutions for the Piksi pair and the M8T pair.  In this case the M8T receivers are from CSG Shop and are running the 3.01 firmware so they include Galileo observations in addition to GPS, GLONASS, and SBAS.

In my previous moving rover experiments I shared a base antenna for the two base receivers but used separate antennas for the two rover receivers.  In this case I decided to also share the rover antenna between the two rover receivers to enable some more direct comparisons.  The base antenna is the same Swift GPS-500 antenna mounted on my roof that I used for the rover in the previous experiment.  The rovers are using a Tallysman TW7872 antenna mounted on top of a car.  In both cases I use a capacitively coupled splitter to isolate the two receivers and allow only one DC supply voltage to reach the antenna.  As usual, the data was collected while driving around my local neighborhood.

Whenever connecting two receivers to one antenna, or even locating to antennas close to each other there is always the concern that they may interfere with each other.  So, as a baseline, I first did a run with separate antennas for the rovers.  I used my inexpensive u-blox antenna for the M8T since it has a very long cable allowing me to separate both the receivers and the antennas by over a meter.  Here are the position solutions for the baseline test.  The Swift internal solution is on the left, the RTKLIB post-processed solution in the middle, and the M8T post-processed solution on the right.  As always, green is fixed points and yellow is float.

The post-processed solutions were both run forward-only and with continuous ambiguity resolution including GLONASS.  The percent fix for the three solutions were 69% for the Swift internal solution, 33% for the Swift RTKLIB solution, and 75% for the M8T RTKLIB solution.  I will use percent fix as a very coarse measure of goodness for now, later I will look at the solutions in more detail.  In this case I consider the Swift internal solution and the M8T RTKLIB solution roughly equal.  Both achieved 100% fix for a number of circles in a parking lot with open skies (22:30 to 22:35), and both struggled with partial fixes on the rest of the route which had more obstructed sky views driving along residential streets.  The Swift RTKLIB solution was noticeably worse and had 0% fix in the parking lot, and lower percent fix everywhere else.

I then re-ran a similar route a short time later, but this time with the M8T rover sharing the dual frequency antenna with the Swift rover, produced the following results in the same order as above.  The percent fixes were 53% for the Swift internal solution, 79% for the Swift RTKLIB solution, and 62% for the M8T RTKLIB solution.

Again I would consider the Swift internal and the M8T RTKLIB solutions roughly equal.  A different parking lot with open skies (1:43 to 1:48), gave no trouble to either solution, but both had only partial fix for the rest of the route.  In this case the Swift RTKLIB solution was noticeably better than the other two solutions which is interesting since last time it was noticeably worse.  This has been my experience in general with the Swift RTKLIB solutions, that they tend to be quite inconsistent.  With the previous firmware I traced some of the reason for this to unreported cycle slips.  I suspect the same is probably going on here but haven’t looked close enough to fully verify that.  As far as any interference occurring between the antennas, if it exists, it is too small to detect with this simple comparison.

Here are the raw observations for the two rovers for the second run, M8T on the left, Swift on the right.  Green lines are dual frequency, yellow are single frequency and red ticks are cycle slips.  There are 19 measurements available for the M8T solution and 22 for the Swift solution.  All are used for ambiguity resolution in the RTKLIB solutions but only the 11 GPS measurements are used for ambiguity resolution in the internal Swift solution.  The relatively cycle-slip-free region from 1:43 to 1:48 is from the parking lot where the sky views are open.

Since the exact same signal was fed into both receivers the observations can be compared more directly than in previous experiments.  Here is a zoom into the time in the parking lot and shortly after.   If you look carefully, you will notice that there are definitely more cycle slips reported in the M8T observations than in the Swift data.

We can verify if those cycle slips are real or not by plotting the double differences of the bias states from the RTKLIB statistics output file.  I selected two satellites that had more cycle slips in the M8T data, G03 and R04, and plotted their double differences relative to the most cycle-free satellite in their constellation.  In the plots below, the red lines are the double differences from the M8T bias states and the blue lines are the double differences from the Swift bias states.  The red circles are cycle slips reported in the observations and the discrete jumps are actual cycle slips.  The green x’s are half cycle invalid flags and not relevant to this particular example.

 

Both plots show good correlation between the reported cycle slips and the true cycle slips as well as confirming that their are more slips in the M8T observations than in the Swift observations.  So, at least in this example, the quality of the Swift observations is noticeably better than the M8T observations, even though both came from the same antenna.  Of course this might be expected since the Swift receiver, although low cost by dual frequency standards,  is still significantly more expensive than the M8T receiver.

The solutions however, don’t reflect this difference in observation quality.  The Swift internal solution appears to be no better, and possibly even a little worse than the RTKLIB M8T solution.  This is probably because the M8T solutions include GLONASS ambiguity resolution and the Swift solutions do not.  Once Swift adds this feature to their firmware I would expect the Swift internal solutions to be better than the M8T solutions.

The variability for the results from the Swift RTKLIB solutions are harder to explain and I don’t have any good answers for this yet.  I still do see cases of unreported cycle slips causing problems even with the new firmware and suspect this is a part of it.  I hope to investigate this further.

Another thing I wanted to look at is how good are the float solutions.  In Swift’s description of the new firmware they state that:

“Swift also suggests that users utilize the estimated accuracy fields
in navigation outputs for an indication of solution quality rather than using the
transition to RTK “fixed” mode as an indicator of solution quality, as the new
and improved float solution performance can often fulfill precision navigation
requirements.”

So, let’s take a look at the float portion of the internal Swift solution.  I loaded the .csv output file from the Swift internal solution into matlab, then plotted histograms for the horizontal and vertical accuracies for the fix and float solution points.  Here’s what it looks like:

Notice that the float accuracies are significantly larger than the fix accuracies but they are probably realistic unlike the RTKLIB float accuracies which I demonstrated to be at least a factor of two overly optimistic in a recent post.

Here is an example of the solution points from the above data set in which the RTKLIB solution re-converges more quickly than the Swift internal solution.  This allows us to evaluate the accuracy of the Swift float values.  The top plot is one component of the horizontal position, and the bottom plot is the vertical position. Yellow is the Swift float solution, Olive green is the RTKLIB float solution and blue is the RTKLIB fix solution.  From the beginning of the data to 22:47:14 both solutions are float.  The RTKLIB solution converges to a fix at 22:47:14 and the Swift internal solution converges to a fix at 22:47:37.  Since both solutions eventually converge to the same value I will assume the RTKLIB fix solution is correct.

 The errors in the float solutions appear to be similar between RTKLIB and Swift and consistent with the Swift estimates of accuracy.  So the Swift float solution does not appear to be any more accurate than the RTKLIB float solution but the Swift estimates of the accuracy do seem to be more realistic.  Three quarters of a meter vertical accuracy is better than a single point solution but I suspect it is too large to be useful for many applications.  Still, it is a noticeable improvement from the very noisy float values I saw in the internal Swift solution in my previous comparison.

So what does all this mean?  I would summarize by saying that the results for a pair of Swift receivers with the new firmware are noticeably better than the results in my previous comparison and so are definitely a step in the right direction.  At least in my particular example, though, it still only puts them roughly on par with the results from a pair of M8T receivers.  They certainly seem to have the potential to be better than the M8T’s and hopefully with further improvements from Swift and maybe improvements in RTKLIB as well, we will see this soon.

I did not include a single M8T or Swift receiver paired with an external CORS or similar base station example in this moving rover comparison but would expect the Swift receiver to outperform the M8T receiver in that case because  of a better overlap in satellite pairs.  Also keep in mind that the Swift receiver does have some definite advantages in static and long baseline experiments as I showed in my previous two posts, especially the ability to get accurate locations for the base station using PPP solutions.

As always I want to emphasize that these are only one users results in one particular configuration and other users experiences in other environments could be quite different.  I have uploaded the raw data and RTKLIB config files to the sample data section on the rtkexplorer.com website if anybody would like to explore further.

I am just getting my newly borrowed NavCom receivers up and running and have upgraded my Tersus receiver with their new firmware so I hope to have results for these other low-cost dual frequency receivers soon.