Real-time solutions with RTKLIB and NTRIP using a cell phone as data link

As I mentioned in an earlier post, I’ve recently acquired access to some low cost dual frequency receivers, specifically a Tersus Precis BX306 and a pair of Swift Piksi Multis.  I have been playing with them over the past few weeks and plan to share my experiences with them over a series of posts.

Both receivers provide internal RTK solutions as well as raw measurements that can be processed with RTKLIB.  I’m interested in how the RTKLIB solutions compare to the internal solutions as well as how both of these compare to solutions derived from single frequency data collected simultaneously with the dual frequency data.

The first issue I ran into with this experiment, however, is that both receivers will only provide an RTK solution for real-time data, neither have the capability to post-process previously collected data.  This meant that I needed a way to provide a real-time stream of dual frequency base station data to the receivers.  I wanted to be able to  do this while driving a car around the local area so I needed more range than a low cost set of radios would give.

Fortunately, I have fairly good cell phone coverage in this area so I was able to rely on my cell phone for the data link.  In this post I will explain how I did that, both for an external CORS reference station and for my own base station.  In both cases I used  NTRIP server/caster/clients to do this.  NTRIP is a protocol for streaming of DGPS or RTK correction data via the internet using TCP/IP.  The NTRIP server sends out the data to an NTRIP caster and the NTRIP client receives it. For more details, there is a good description here.

Using this setup I was able to run real-time solutions with RTKLIB as well as with the intenal RTK engines in the Swift and Tersus receivers.  Here’s a diagram from the RTKLIB manual showing the setup I used for running a real-time RTKLIB solution using RTKNAVI.  When I ran a Swift or Tersus solution, the configuration was similar, but the NTRIP caster streamed the base station data to STRSVR instead of RTKNAVI, and STRSVR then streamed it to the receiver where it was combined with the raw receiver observations to create an internal RTK solution.  Also missing in this diagram is the cell phone which should be in between the internet and the rover PC.


The amount of free base station reference data that is available online on a real-time basis is a fair bit more limited that what is available after the fact for post-processing.  Fortunately I was able to find a CORS reference station about 17 km away that is available real-time through the UNAVCO NTRIP caster.  The service is free if the data is used for educational purposes and appropriately attributed.   Most of their stations are on the west coast of the U.S. but they do have some scattered across the rest of the country as you can see in this map from their site.  There are other networks available in other parts of the world that can be found by searching online.


To access the UNAVCO data I had to request access through email but the process was very simple and within a couple hours of my request I was all setup with an account and password.

Once I had my account set up, I used RTKLIB on my laptop computer to collect the data from the internet and stream it to the rover receiver over a serial port.  If I were doing this experiment within range of a wireless router then I could leave the computer connected to the wireless.  In this case though, I wanted to roam outside the range of my home wireless.  To do this, I enabled a hot spot on my cell phone and logged into that with my computer.

I was able to access the raw observation data stream from the UNAVCO NTRIP caster directly using the NTRIP client option in RTKLIB.  If I had wanted to generate a real-time RTKLIB solution, I would have configured the input streams of RTKNAVI but in this case I want to stream the raw data directly to the receiver so it can use the observation data for it’s internal solution.  I did this using the STRSVR app in RTKLIB.  I specifed the “NTRIP Client” option as input type and then entered the information from my UNAVCO account into the “Ntrip Client Options” as shown below.


In this case I wanted the data from station P041 in RTCM3 format so I had to specify the Mountpoint as “P041_RTCM3”.  For other networks, the mountpoint details may be a little different.  Most NTRIP casters use Port 2101, and that was the case for this one.  For the STRSVR output type, I specified “Serial” and then configured the serial port options for whichever rover receiver I was using.  Before doing the configuration, I had connected the receiver to the laptop using a USB cable.

I then had to configure the receiver to tell it to get its base station data from the COM port and specify that it is in RTCM3 format.  The details for doing this on the two receivers are a little different but fairly straightforward in both cases.  You may also need to specify the exact base station location manually or the receiver may be able to get it from the data stream depending on the receiver and NTRIP stream details.

And that’s it.  With this configuration, either receiver was able to fairly quickly lock to a fixed RTK solution and continue to receive base data as long as I stayed in range of cell reception.  Any lag in the base station observations appeared to be less than a second.

That worked great for using an existing external reference as base station.  However, I also wanted to run another real-time experiment where I used one Swift receiver as base and the other as rover.   To do this, I needed to set up an NTRIP server to stream the data to  a caster on the internet as well as an NTRIP client to receive it.

I started by connecting the second Swift receiver to an old laptop with a USB cable and then downloading RTKLIB, the Swift console app,  and the right USB drivers.  The base station antenna is on top of my roof and the laptop is in the house so I was able to connect the laptop to the internet using my home wireless.

For the NTRIP caster, I found it convenient to use RTK2GO which is a community caster available for anyone to use at no cost.  To send the data to the caster, I used the “NTRIP Server” as the STRSVR output type and configured it as shown below.


Again, the port is 2101.  You can choose any name for the mountpoint.  If that name is already in use, then rtk2go will assign a suffix to it, so it is best to choose a name that is unlikely to already be in use.  The password at the current time is BETATEST but that may change from time to time so it’s worth verifying it is still correct.

For the STRSVR input, I selected “Serial” and specified the correct COM port for the base station receiver.  In this case the raw observations are in Swift binary format which RTKLIB does not support so it sends them unaltered.  If they were in a format that RTKLIB did support, then they could be converted to RTCM3 to reduce bandwidth and make them more easily usable by someone else not using a Swift receiver as rover.  You can specify the conversion to RTCM3 using the “Conv” menu on the STRSVR output.

Start STRSVR and your base station observations are now accessible to anyone in the world through!

On the rover side, the NTRIP client is set up as I previously described using STRSVR except you want to use the same caster/mountpint/password as you just did on the base station.  In this case the user-id is left blank.  Again, set the STRSVR output to “Serial” to send it to the receiver.   Then set up the receiver to get it’s base station data from the serial port and, in this case, specify that it is in the Swift Binary Protocol (sbp).  Start the receiver and it should fairly quickly get a fix.  If you are seeing baseline data but not a solution, then most likely you have not specified the base station location to the rover.

I was now able to drive around almost anywhere and get continuous real-time RTK solutions using either my own base station or the CORS reference station as base.  In the next post I will discuss some of the data I collected and analyzed.





A fix for the RTCM time tag issue

In my last post I described a problem with a loss of some of the raw measurement information caused by the lack of resolution in the time tags in the RTCM format.  Since the RTCM format is typically used to reduce bandwidth requirements in real-time applications, it is causing real-time solutions to fail when post-processing the same raw data without the translation to RTCM gives good results.  In this post I will describe a fix for this problem.

First of all I want to thank Felipe Nievinski, Igor Vereninov from Emlid, and Anthony Woolridge for their comments to the last post that pointed me to the solution.  They make this a collaborative effort between the U.S., Brazil, Russia, and the U.K!  It still amazes me how enabling the internet can be!

I’ll start by showing again this example of a RINEX output from an M8T receiver with the official raw measurement output (RXM_RAWX) and the debug raw measurement output (TRK_MEAS) enabled simultaneously.  I think  this provides a good insight to what is going on.  The RXM_RAWX message is the top 5 lines and the TRK_MEAS message is the bottom 5 lines for a single epoch.  The first line in each message is the time stamp and the following lines are the measurements for each satellite.  In the satellite measurements, the second column contains the pseduorange value.


The time stamp specifies the receiver time of the received signals and the sixth column is the number of seconds.  For the TRK_MEAS message these values are always aligned to round numbers based on alignment to the sample rate.  For example in this case the measurement rate was 5 Hz and all the time stamps occur on multiples of 0.2.  This is because they are based on the raw receiver clock without any corrections.

The time stamps from the RXM_RAWX messages however often differ from the round numbers by small arbitrary amounts.  This is because the receiver has estimated the error in its own clock and adjusted the measurements to remove this error.  In this case the estimate of clock error is 0.001 seconds and so the time stamp is adjusted by this value (18.8000000 to 18.7990000).

To keep the time stamps consistent with the other parts of the measurement, the clock error also needs to be removed from the psuedorange and carrier phase values since they are based on the difference in time between satellite transmission and receiver reception and will include any errors in the receiver clock.  We see from the above observations that the pseudorange measurement for satellite G24 has been adjusted from 22675327.198 to 22375547.970, a difference of 299779.228 meters.   The speed of light is 299792458 meters per second so the clock error of 0.001 seconds is equivalent to 299792.458 meters,  a value very close to the amount that the pseudorange was adjusted by.

A similar adjustment needs to be made to the carrier phase measurement as well but it is not as easy to see in this example because the carrier phase measurements are relative rather than absolute and the two messages in this case use different references.  The carrier phase measurements are in cycles, not meters, so the frequency of the carrier phase needs to be included in the translation from clock error to carrier phase cycles but is otherwise the same as the pseudorange adjustment.  In equation form, the adjustments are:

P = P -toff*c
L =L – toff*freq

where P=pseudorange, L=carrier phase, c= speed of light, and freq=carrier frequency

So, basically, the receiver is trying to help us out by removing its best estimate of the clock error from the measurements.  This is unnecessary since RTKLIB is quite good at estimating this clock error on its own, but by itself this adjustment does not cause a problem.

It is when the adjusted measurement is translated to RTCM that we get in trouble.  The resolution of the time stamps in the RTCM format is 0.001 seconds.  In this particular example it would not be an issue because the error is exactly 0.001 seconds or one count of the RTCM format.  Most of the time, however, this error is not an exact multiple of 1 millisec.

For example, here is a time stamp for the data set described in the previous posts.

> 2017  1 17 20 31 48.9995584  0  9

And here is the same time stamp after being translated to RTCM and then to RINEX

> 2017  1 17 20 31 49.0000000  0  9

As you can see, the clock adjustment was less than half a millisec so was completely lost in the roundoff to the RTCM format.  However, the adjustments the receiver made to the pseudorange and carrier phase are still present in those measurements.  We now have a problem because the clock correction is in part of the measurement and not the other pieces.  RTKLIB can not correct for this lack of consistency within the measurement.

So, how do we avoid this problem?  Fortunately, RTKLIB has an option to adjust the time stamps to round values using the same equations described above to adjust time stamp, pseudorange, and carrier phase to maintain consistency within the measurement.   I imagine it was put in specifically to solve this problem. We can invoke this option by adding “-TADJ=0.001” in the “Options” box in the “Conversion Options” menu in STRSVR or using the “-opt” option in the command line with STR2STR.  Note that this option needs to be set in the conversion from raw binary format to RTCM format, not the conversion from RTCM to RINEX.  It is possible to set this option when converting from RTCM to RINEX but this won’t help because the damage has already been done in the earlier conversion.

Unfortunately, there is a bug in the implementation of this option in RTKLIB, at least for the u-blox receivers, so by itself, this is not enough.  The problem is that invalid carrier phase measurements are flagged in RTKLIB by setting the carrier phase value to zero.  The time stamp adjustment feature adjusts these zero values slightly so they are no longer recognized as invalid.  They end up getting included in the output as valid measurements and corrupt the solution.

Fortunately, the fix for this bug is very simple.  Here is the code in the decode_rxmrawx() function in ublox.c that makes the adjustment:

/* offset by time tag adjustment */
if (toff!=0.0) {

If we add a check to the first line of code to skip the adjustment if the carrier phase is zero, then all is fine.

if (toff!=0.0&&cp1!=0) {

Below is the original solution after RTCM conversion on the left and with time tag adjustment and the bug fix on the right.  If you compare the solution on the right to the solution with no  RTCM correction in the previous post you will see they are nearly identical.


I am still wary of using RTCM because of its other limitations described in the last  post, particularly the loss of the half cycle invalid flag and the doppler information, but I believe this fix eliminates the most serious issue that comes from using RTCM.

I will release a new version of the demo5 code with this fix sometime in the next few days.  It will take a little while because I also want to include some other features that have been waiting in the pipeline.  If you want to try the fix right away, you just need to  modify the one line of code described above and rebuild.

Update 2/2/17:    I have taken Anthony Woolridge’s suggestion and modified the RTCM conversion code to automatically adjust the pseudorange and carrier phase measurements to compensate for any round off done to the time tag.  This means it is not necessary to set the time-tag adjust receiver option.  This change is currently checked into my Github page and I hope to post new executables in the next couple of days.

Limitations of the RTCM raw measurement format

In the last post I described a process to troubleshoot problems occurring in real-time solutions that are not seen in post-processing solutions for the same data.  I collected a data set demonstrating this issue, and traced the problem to the conversion of the measurement data from raw binary format to the RTCM format.  This conversion is typically done in real-time applications to compress the data and minimize bandwidth requirements for the base to rover real-time data link.  In this post I will look into that example in more detail and also explore some of the limitations of the RTCM format.

First, it is important to understand that the conversion to RTCM is not a lossless process. There are several ways in which information is lost in this process.  In some cases these losses are probably not significant but in other cases it is not so clear that is the case.

So let’s look at some of those differences.  We actually have three formats to compare here: the raw binary format from the u-blox receiver, the RTCM format, and the RINEX format.  Both the RTCM and RINEX formats contain less information than the raw binary format and information is lost when the conversion is made to either format.  The reason I include the RINEX format here is because in the post-processing procedure, the measurements, whether they come from the raw binary format or the RTCM format, must first be converted to RINEX format before being input into the solution.   What I see with my example data set that fails in real-time is that it looks good in post-processing if the raw measurements are converted directly from raw binary to RINEX but fail if the raw measurements are first converted to RTCM and then the RTCM is converted to RINEX.  Therefore it is very likely that there is something critical that is lost in the conversion to RTCM that is not lost in the conversion to RINEX.

The official RTCM spec is not freely available on the internet (it must be purchased), so I have relied on this document from Geo++ for the RTCM details.  Here is a chart of the most significant differences I am aware of between the three formats.  In the case of RTCM, these numbers apply only to the older 1002/1010 messages used by Reach and most other systems, not the newer MSM messages.

U-blox binary RINEX 3.0 RTCM 3.0
Psuedorange resolution double precison floating point 0.001 m 0.02 m
Carrier phase resolution double precison floating point 0.001 cycles = 0.2 mm 0.5 mm
Doppler resolution single precision floating point 0.001 Hz Not supported
Time stamp resolution double precison floating point 100 nsec 1 msec
Lock time 1 ms Lock status only Variable (> 1 ms)
Half cycle invalid Supported Supported Not supported


To figure out which (if any) of these differences is responsible for the failure I needed a way to run the solution multiple times, each run done with only a single difference injected into the conversion.

I already had a matlab script I had previously written previously to parse a RINEX observation file into a set of variables in the matlab space.  So I wrote a second script that goes the other way, from variables in memory to a RINEX observation file.  Once I had done this, I could read in the good RINEX observation file translated directly from the u-blox binary file, modify a single measurement type, write it back to a new RINEX observation file, then run this file through a solution.

My first guess was that it was the missing  “Half Cycle Invalid” flag that would prove to be the culprit since I have seen this before with the M8N receiver as described in this post.  Although I suspect that this probably is true in some cases, it did not make a difference with this data set.  My next suspect was the missing doppler measurements, since RTKLIB uses the doppler measurements when estimating the receiver clock bias, but again, it was not the case.  In the end it turned out to be my very last guess that made the difference and that was the time stamp resolution.  So much for me thinking I was starting to get the hang of this RTK stuff!  The differences were so small in the time stamps relative to the distance between them, that I had unconsciously  ignored them.  For example, the two first time stamps in the good measurements were 49.9995584 and 50.999584 but the time stamps in the failing measurements had been rounded off to 50.0000000 and 51.0000000.  Even after discovering that this round-off error makes a difference, it still is not obvious to me why this is true.  In any GPS solution, the receiver clocks are assumed to lack sufficient accuracy  to be relied upon without correction and the clock errors are one of the unknowns in the solution along with the three  position axes.  I don’t know why RTKLIB does not correctly estimate this error in its clock bias estimate and remove it.  Maybe one of you guys who has been doing this a lot longer than I have can explain this?

Just to be sure it wasn’t a fluke, I started the data processing at three different times in the data set, and I also ran additional solutions with the sign of the error in the time stamps reversed.  In every cases, regardless of sign, or starting location, the solution failed to get a fix when the error was present and succeeded when the error was not there.

I have read somewhere that more expensive receivers will typically align there time stamps to round numbers which would avoid the need for as much resolution.  The only expensive receivers I have access to are the CORS stations so I took a look at data from a couple of them.  Sure enough, it appears to be true that they do use round numbers for their time stamps.  If this is more generally true it might explain why the RTCM spec does not have sufficient resolution for the u-blox data but would work fine for more commonly used, higher priced receivers.

I was curious why the u-blox time stamps don’t occur at round numbers so took a look  at the hardware description spec.  I found this explanation

“In practice the receiver’s local oscillator will not be as stable as the atomic clocks to which GNSS systems are referenced and consequently clock bias will tend to accumulate. However, when selecting the next navigation epoch, the receiver will always try to use the 1 kHz clock tick which it estimates to be closest to the desired fix period as measured in GNSS system time”

I interpret this to mean that the receiver is aware of alignment error in its clock source relative to GPS system time, and it adjusts the time stamp values to  includes its estimate of that error.

Something else I am curious about but have not had time to investigate in any detail is how this issue is affected by differences between the RXM_RAWX measurements which are what is normally used with the M8T receiver, and the debug TRK_MEAS messages which also contain the raw measurements and are the only raw measurement messages available on the M8N receiver.  Looking at several data sets from the both the M8N and M8T, it appears that the TRK_MEAS time stamps for both receivers are aligned to round numbers  while the RXM-RAWX measurements are not aligned.  This means that the TRK_MEAS messages would not be affected by the lack of resolution in the RTCM format.   However, the TRK_MEAS measurements lack the compensation for inter-channel frequency delays in the GLONASS measurements and so would not be a good substitute.  Maybe it’s possible to combine the two into a single set of measurements?  The two include different references and clock errors so it is not obvious if that is possible. Below is an example of partial TRK_MEAS and RXM-RAWX outputs for the same epoch when both were enabled, TRK_MEAS on the top, and RXM_RAWX below.


Another avenue I considered is using the newer MSM messages (1077,1087)in the RTCM format instead of the current 1002/1010 messages that Reach and most other users are using.  These have higher resolutions for the pseudorange and carrier phase, and include doppler and half cycle invalid flags.  Unfortunately, the resolution for the time stamps does not seem to have changed, or if it has, it hasn’t changed enough to see a difference in the output for the small deltas in my example.

There also appears to be a bug in the RTKLIB implementation of the encode or decode of these messages which sometimes causes the number of integer cycles in the carrier phase measurements to be incorrect (the fractional part is fine).    This bug appears to be present in both the official 2.4.3 release and the demo5 code but some of the changes I have made to the u-blox translation in the demo5 code seem to have increased the frequency of these incorrect measurements.

Reach does use the MSM messages for the SBAS measurements although it does not need to since the 1002 message supports SBAS as well as GPS.   It is possible this could introduce a problem for users in North America where the WAAS satellites used for SBAS correction include carrier phase measurements.  Users in Europe would not see this problem because the EGNOS satellites used for SBAS correction in Europe don’t provide the carrier phase.  I did not see any corruption in the SBAS carrier phase measurements in the initial RTCM data in this example but after I enabled the 1077 and 1087 measurements, I did see corruption in the measurements in all three systems.

So, unfortunately this is still somewhat a work in progress and I don’t have any easy answer how to fix this.  I am hoping some of the experts out there can comment and help put some of the pieces of the puzzle together.

In the meantime I would suggest using the u-blox binary format for the base-rover data-link instead of the RTCM format.  The bandwidth requirements will be 2.5 to 3 time higher but some of this can be offset by reducing the measurement sample rate for the base station.

I believe a long term fix is going to require two things.  First of all a workaround to the time tag resolution issue described in this post.  But even with fixed, the half cycle valid flag and doppler information will still be lost.  I haven’t  done any tests to understand how critical the doppler measurements are, but I have demonstrated in the post I referenced above, that losing the half cycle valid flag can definitely degrade the solution.  Fortunately, the newer MSM RTCM messages do include both half cycle valid flag and doppler.  They do not appear to be usable until the bug in the encode/decode of the carrier phase data is fixed, so that will have to happen as well.

On the other hand, I suspect most real-time RTK systems do use RTCM and manage to live with its limitations so maybe I am overreacting here.  I would be interested in other people’s opinions and experiences with RTCM on u-blox or other receiver types.




Exploring differences between real-time and post-processed solutions.

I’ve had a few questions recently about differences showing up when the same set of raw data measurements are processed real-time and when they are post-processed.  Since I haven’t done a lot of real-time work I didn’t have a good answer to these questions, but it seemed like an interesting problem so I thought I would dig into a little bit.

In many cases, these differences can be traced to a poorly performing data link between base and rover that loses, delays or corrupts the base measurement data.  These problems are usually diagnosed fairly easily by looking at the “age of differential”  between base and rover or by seeing missing data in plots of the base observations.  My interest is not in these cases but rather where the data link is performing well and there is still a difference between the real-time and post-process solutions.

To troubleshoot real-time solutions is a little trickier than post-processing solutions because you may need a way to re-run the data through the real-time RTKLIB app (either RTKRCV or RTKNAVI) to recreate the problem.  The standard *.ubx log files do not contain enough information to do this since they contain only a time stamp for when the measurement was made and not when it was actually available to the solution.  There will usually be some delay between the two because of latencies in the data link between rover and base.  The post-processing solutions ignore this delay and simply align the two measurements assuming zero delay but we need to know what these delays are to recreate the real-time solution.

The real-time solution apps have an option in the input stream setup to read from a file instead of a real-time stream.  This allows you to re-run previously recorded log files but when doing this they require a *.ubx.tag file in addition to the *.ubx file to provide the latency information.   These *.ubx.tag files are generated automatically when you log real-time data if you select the appropriate option before you collect the data.  For RTKRCV, this is a “::T” appended on to the end of the log file name.  For RTKNAVI, it is checking the “Time Tag” box in the log stream options.  I recommend always enabling these options when you are running real-time solutions because the extra files are not very large and you never know when you are going to get something unusual in the data that you would like to investigate later.

Since none of the data sets I had been sent to look at contained tag files, my first step was to try and collect some data that looked good in post-processing but not in real-time with time tags enabled.  I chose to use my Emlid Reach receivers to do this, in part because it is easy to do real-time solutions with the onboard wireless, and in part because I wanted to try out their recently released 2.1.6 version of the RTKLIB code.  This version is a very close cousin to my demo5 code and contains all of its features (although many of them are not currently accessible through the Reachview GUI).

I first added or modified a couple of lines of code in the Reach startup files to save time tag versions of both the base and rover data on the rover, and the base data on the base.  I’ve added some notes at the bottom of this post on how I did it but I don’t necessarily recommend doing it yourself unless you are fairly comfortable with linux because it can be a little tricky to recover without reflashing the unit if you make a mistake.  I wanted to be able to collect data on the Reach units using the command line based RTKRCV app but use the GUI based RTKNAVI on my laptop to re-create the realtime run.  This is because RTKNAVI has a much nicer  interface with a lot more information available.  However, this meant that I needed to fix an incompatibility in the RTKLIB code between the time stamp formats of RTKRCV and RTKNAVI as described in the RTKLIB Github issue #99.  Using the fix recommended in the issue description,  I rebuilt the code on the Reach unit to create a new str2str executable with this fix incorporated.

With these changes, I can collect measurement data that gives me the option to run post-process solutions or re-created real-time solutions.  In addition, these can be run either with measurements made before or after the data link and raw binary to RTCM conversion.  This gives me quite a bit of capability  to investigate where a potential problem might be occurring.

To test this setup, I first collected some static data with both base and rover exposed to open skies.  I got all three sets of data and tag files and using these I was able to re-run the data using RTKNAVI.  Both real-time and post-processed solutions got a fix fairly quickly and the two solutions were very similar.  So, nothing interesting to look at in this example.

Next I placed both base and rover on my back patio, just a few meters away from the house and partially blocked by a large tree, knowing that this would be a more stressful measurement environment.  I may have just got lucky, but the very first data set I collected gave me multiple fixes in post-processing but none in real-time as shown below (post-process on the left, real-time on the right).  The two loss of fixes are caused by me restarting the data collection on the Reach rover.  In this case I ran the post-processing solution using the base data collected on the base in raw binary format (*.ubx), not the data after it had been converted to RTCM and transmitted to the rover (*.rtcm)  since this is the way post-processing is usually done.


Next I ran a second post-processing solution, this time using the raw measurement file saved in RTCM format on the rover.  This time there was no fix and the solution looked nearly identical to the real-time solution plotted above.   So somewhere between when these two data files were saved, the problem is occurring.  Note that in this case I was able to do all this without the time tag files or re recreating a real-time run but I imagine this capability will be helpful in future analysis.

I had monitored the age of differential while collecting the data and after collecting the data I plotted the base observations to verify there was no missing data.  This suggests that the data link was working fine.  So my next guess was that the conversion from raw binary measurements to RTCM format might be the cause of the problem.  In real-time solutions, the base data is typically translated to RTCM before transmitting over the data link to the rover to compress the data and reduce bandwidth requirements on the data link, and this is the default configuration of the Reach units.   The amount of compression will vary depending on the details of the data but in this case the RTCM file (*.rtcm) was about one third as large as the raw binary file (*.ubx).  Some of this is lossless compression but not all of it so there is potential for degrading the solution with this translation.

The next step was to isolate the effects of the RTCM translation from any effects from the data link latency.  I did this by using the STRSVR app to translate the raw binary base data saved on the base station to RTCM format.  I configured the conversion options to use the same RTCM messages as used by Reach.  ran this data through a post-process solution.  Sure enough, just converting the undelayed raw binary data to RTCM was enough to break the solution.  That means, at least for this case, we can ignore any effect of the data link delays and focus on the RTCM conversion.

Note that the post-processing apps require all the measurement input files to be in RINEX format.  This means that both the raw binary files and the RTCM files are converted to RINEX first using RTKCONV first as part of the post-processing procedure.  One thing to be aware of when using RTKCONV to convert from RTCM to RINEX is the signal mask input options.  The default signal mask has all observation types selected and if left this way it will cause the file header to be incorrect.  If you do not de-select all the extra observation types you will see this in your observation file header


The number of observations is 8 instead of 4 and there are extra observation types listed.  This will confuse RTKLIB and it will not interpret the rest of the file properly. Specifically it will not pick up any of the GLONASS observations.  It won’t flag an error but it will cause all the GLONASS measurements to be left out of your solution.  The signal mask button is on the options page as shown below.  You want to un-check all options except “1C”.


This post is already getting fairly long so I will put off to the next post the rest of the story including discussion about what is actually lost in the translation to RTCM and why it caused this particular example to fail.  In general, though, it is important to understand there are real losses in this translation and that they may affect the quality of your solution.  If you have the bandwidth to transfer the raw binary format instead of the RTCM format I would recommend you consider doing that.  If you don’t have the bandwidth, I would suggest you consider the trade-offs from reducing the base sample rate enough so that you are able to transfer the measurements in raw format.  As I mentioned above, in this example the raw binary file was about three times as large as the RTCM file.



Notes on how I set up the Reach to collect extra data.  There may be a more elegant way to do this but I just wanted a quick hack.  Please be careful if you try to do this yourself and be sure to back up any files before modifying them:

RTKLIB has a “::T” option to record the time-tags but I don’t believe Reach supports this option.  I got around this by adding extra instances of str2str initiated from a function call I added to the “reach_setup” script in the /usr/bin folder.  This, and all the instructions below assume you are running the 2.1.6 version of Reach code.

 I added the call right before the call to “reachview” in the “reach_setup” script as shown in blue below.  I did this on the rover receiver assuming it is getting the base measurements through a data link.
ncat -k -l 2000 < /dev/ttyMFD1 > /dev/ttyMFD1 &
#start logging data files with time stamps
# Run ReachView
led set_color green

I created the “reach_time_logs” script in the /usr/bin folder and put in the following lines of code

# Log u-blox data to file with time stamp logs
# find unused file name
while [ -f $fname ]; do
    let “i=i+1”
# start data collection from rover
/usr/bin/RTKLIB/app/str2str/gcc/str2str_tag  -in tcpcli://localhost:2000 -out $fnameR::T &
# start data collection from base
/usr/bin/RTKLIB/app/str2str/gcc/str2str_tag  -in tcpcli:// -out $fnameB::T &

This finds an unused filename and saves the measurements and the tags for both the rover and base data.  You will need to modify the specified input stream for the base data to match what you are using.  You can look at the inpstr2-type and path in the /usr/bin/RTKLIB/app/rtkrcv/rtk.conf file for the exact format.  You might be able to use the RTKLIB wildcards instead to create the file name but I just copied this code from my PiZero logger which doesn’t update the clock.  I don’t know if on the Reach the clock has been updated yet at this point in the start-up.

I also had to modify the str2str app to make the time-tags compatible with RTKNAVI.  I used the bug fix recommended in Github issue #99.  I recommend debugging by re-running the data through RTKNAVI (on a Windows machine) rather than RTKRCV because it has a much nicer interface with much more info available.  If you decide you want to re-run the data through RTKRCV you will either need to rebuild it with the bug fix or collect the data with the unmodified str2str.  I think it’s unlikely that you will see different solutions between RTKRCV and RTKNAVI assuming they are both configured the same.
Rename the modified str2str executable to str2str_tag and leave it in the /usr/bin/RTKLIB/app/str2str/gcc folder .  Use the chmod +x command to make this file and the “reach_time_logs” file both executable.
I also modified the base receiver and saved the base data in ubx format before it was converted to rtcm so I could compare before and after to see if the conversion or data link might be causing problems.  You can use the same modifications described above, except delete the last two lines in the “reach_time_logs” script.
With these changes in place, the units will automatically save time-tagged data to a new file every time they are turned on.
After collecting data, the data files will be in the /home/root/logs folder.  The file names will be basexx.rtcm, basexx.rtcm.tag, roverxx.ubx, and roverxx.ubx.tag where “xx” will increment every time you run until you delete the old files.  To run them through RTKNAVI, just specify files in the input stream and check the time tag box.
You then have the options of running post-process or simulated real-time with measurements either before or after the data link/RTCM conversion.  This should give you a fair bit of insight into where the problem is occurring.
 I had a bit of trouble with files I edited getting corrupted after  a power cycle (maybe because I was using WinSCP through the wireless) so I suggest using the “reboot” or “shutdown” commands to avoid problems.  Also be sure to make copies of the files before you edit them.  At one point I corrupted the “reach_setup” script and then could only access the Reach by using the instructions in the Software Development section of the QuickStart guide.  Another time the /etc/reachview/stable_config.json disappeared and I had to restore it.

Adding a radio link

In the last post I described setting up RTKNAVI in a simple configuration with both receivers connected directly to a laptop. While this is a good way to become familiar with RTKNAVI, it is not a useful configuration for actual measurement since the rover can’t rove for more than a few feet before running out of cable.

In this post I will describe adding a pair of HobbyKing SiK V2 Telemetry radios to separate the base from the laptop and rover. These radios are based on the same open-source design as the 3DR radios previously made by 3DRobotics and sell for $33 dollars for the pair. They are supposed to be good to up to about 300 m with the supplied antennas. There is a 915 Mhz version and a 433 Mhz version available, you will need to choose the one that is legal in your location. Both transceivers have both a USB connector and a UART connector. We will use the USB connector to connect one radio to the laptop and the UART connector on the other radio to connect to the GPS receiver. Here’s what they look like coming out of the box.

The first thing I did after opening up the package was to screw the antennas onto the transceivers since it is possible to damage the radios if they are accidentally powered up without the antennas attached.

To create the base station, I connected one of my Ublox M8N receivers to the radio and to a USB battery pack by cutting and reconnecting the cables that came with the devices.  I connected VCC for all 3 cables together, and the same for all 3 GND wires. I then connected RX to TX and TX to RX between the GPS receiver and the radio. This is what it looked like when I was done.


If you haven’t already set the baud rate on the GPS receiver it is possible to set it through the radios but it is probably easier to do it beforehand with the receiver connected directly to the laptop. In my case, I had previously set it to 115K from the RTKNAVI demo in the previous post and continued to use that baud rate for this exercise.

I then plugged the second radio into the laptop using a USB cable. I also plugged the second GPS receiver, which will be the rover, into a second USB port on the laptop, using an FTDI board to convert from UART to USB as I’ve described before.


Next I downloaded MissionPlanner, an open-source software package developed for drone users. I used this to configure the radios. It’s fairly straightforward and there’s some good documentation here to help you through it so I won’t go through all the details. This is the configuration that I ended up using after a little experimentation:


It is important to match the baud rates for the different pieces of the link. Set the kilo-baud rate (and the port number) for the laptop com port up in the top right corner. This needs to match the “Baud” setting for the local radio on the left. The “Air Speed” setting is the kilo-baud rate the radios operate at, and the two radios (local and remote) need to have the same value. The “Baud” setting on the remote radio must match the kilo-baud rate of the base GPS receiver.

Often when I changed these settings, it was difficult for me to get the complete link working again and I had to fiddle with it. Sometimes this meant clicking on “Save Settings” more than once, sometimes I would restart the Mission Planner app, sometimes the RTKNAVI app, and at least once I had to reboot the laptop. This was all rather frustrating and I don’t really know which steps helped and which didn’t, but once I stopped changing the settings, things seemed to be more stable.

You will need to be careful not to overwhelm the data link with too much data. In the previous demo I had reduced the base station sample rate to 1 Hz which is where I left it for this exercise.  In many cases, people convert the raw measurement data to RTCM format to reduce its size before sending it over the radio but this is not an option in this case because the receiver won’t output the raw measurements in RTCM format and we do not have a CPU in the base station to do the conversion.  As long as we are careful not to exceed the bandwidth of the radio link this should be OK although our rover distances may be limited since higher data rates are supposed to decrease the range of the radio.

At this point you should be able to communicate with the GPS receiver in the base station through the radio link. I started up the Ublox u-center eval software at this point just to verify that I could communicate in both directions. Make sure you disconnect or close it when you are done, or it will prevent RTKNAVI from accessing the com port.

Once you have established the radio link is working, you should be able to startup RTKNAVI and follow the instructions from the previous post to configure and run it. The only difference will be that you will probably find the radio is using a different com port than the GPS receiver so you will need to change that in the Input data stream menu.

I placed my base station on a tripod for convenience and to get the radio antenna further off the ground. I used a 8” pizza pan (88 cents at Walmart) for a ground plane. Here’s a photo of the assembled base station.


I placed the radio underneath the ground plane and the antenna pointed down in case that helped reduce possible interference between the radio and the GPS receiver but I did not do any testing to evaluate how effective this was. I probably should have also mounted the USB battery pack underneath as well just to keep things cleaner but didn’t get around to it.

I then mounted the other radio and GPS receiver antennas on top of my car to use as the rover. As I do for all my data sets, I started the data collection and then remained stationary until I got a fix. Typically this takes about 3 or 4 minutes and that is what happened in all of my runs. After starting RTKNAVI, I opened two plot windows. In the first I selected “Gnd Trk” and in the second I selected the “Nsat” plot option because this option includes a plot of age of differential, the delay in time between the rover measurement and the base station measurement. When close to the base station the age of differential remained between 0.2 and 1.2 seconds which makes sense since the base station is sampling every second and there will be a short delay for the radio link. As I got further from the rover I started to see this number increase as the radio link started to breakdown and I started to lose base observations. Here is the plot with the age of differential shown in the middle window.


Here is the ground plot and position plot from the same run.


In general, I seemed to start losing the radio link at about 100 meters. This is less than the 300 meters I was expecting, but maybe optimization of the radio settings and antenna locations would help. I did spend a little time adjusting these without seeing much difference in the results, but it was far from an exhaustive effort.

Here’s another short run where I drove out 350 meters and back showing age of differential and position. In this case I again lost the radio connection at about 100 meters and the age of differential increased all the way to the “Max Age of Diff” option (75 sec) without losing fix. It then regained a fix immediately after the age of differential dropped back below 75 seconds.


In another run, I reduced the base station sampling rate from 1 Hz to 0.2 Hz and also reduced the air speed setting of the radio from 64 to 16 to see if this would affect either the range of the radios or the reliability of the solution. I did not find it made much difference to either one. I did lose the fix after exceeding the max age of differential in this run but that may just be because I exceeded it for a longer time than in the previous example. Here is the age of differential and position plots for this run:


Overall, the radios were a little frustrating to configure, and their range was a little disappointing, but otherwise the experiment was a success.

Getting started with RTKNAVI

[Update 11/25/16:  See here for a more recent version of this post]

Up until now I have used RTKLIB entirely for post-processing previously collected data and have not tried to process any data real-time. Now that RTKNAVI, the real-time GUI version of RTKLIB, is successfully compiling in the 2.4.3 b17 release of RTKLIB, I decided to give it a try.

I first had to update the GUIs to add all of my additional input configuration parameters and options. I started a new “Demo5” branch in my Github repository with these changes. While I was at it, I also updated the RTKPOST GUI for post-processing so both applications now support all of what was available previously in my code only in the RNX2RTKP CUI version. I’ve uploaded the executables and they are available here along with all the input configuration file and receiver startup files I used for this exercise.

As a starting point I chose to connect both M8N receivers directly to my laptop PC. This is not a very useful configuration, since the rover can only travel as far as the USB cable extends, but it greatly simplifies things, avoiding have to deal with radios or other real-time links while getting started. I connected both receivers to my laptop USB ports using FTDI boards to translate from UART to USB as I’ve previously described in this post. We’ll connect a pair of  3DR 915 Mhz radios later to make this a useful setup.

Below I will describe how I set up and ran RTKNAVI in the hope it will be useful to other people just starting out. I will assume you are using M8N receivers and my version of the code but much of this will also apply to the most recent 2.4.3 release version of code and to other receiver types as well.

When you first bring up RTKNAVI it should look something like this:


In the top left corner you will see what version of code you are running. If you are running my code, you should see the demo4 (I need to update this to demo5) tag. In the top right corner are the menus for setting up the input, output, and log streams. We will start here.

Click on the “I” button to bring up the “Input Stream” menu. The red ovals below show what we need to change here. Check the boxes for both the rover and the base station, set both Types to “Serial” and the “Format” for both to “u-blox”. Plug the rover receiver USB cable into the laptop, then click the “Opt” button for the rover to bring up the “Serial Options” menu. Click on the arrow next to the “Port” box and select the com port for the rover. It should be the only choice at this point. Set the baud rate to match the GPS receiver, then click on OK. Plug in the USB cable for the base station receiver and then click the “Opt” button for the “Base Station”. Set the “Port” and baud rate as you did for the rover.


Next, click the “Cmd” button for the rover to specify the commands that RTKNAVI will use to initialize the GPS receiver. Click the “Load” button in the “Serial/TCP Commands” pop-up and select the “m8n_rover_5hz.cmd” file. (You should have downloaded this when you downloaded the executables). Do the same for the base, but choose the “m8n_base_1hz.cmd” file. These files will configure the rover to output raw GPS, GLONASS, and SBAS measurements at 5 Hz, and the base station at 1 Hz. We run the base station at a lower sample rate since it is not moving and later we will need to relay this information over a real-time link which may have limited bandwidth. Check both boxes to enable the “Commands at startup” and the “Commands at shutdown”, then click OK to close the two windows. If you are not using the M8N receivers you will need to provide your own startup files.


Next we’ll configure the output stream to send the solution to a file. Click the “O” button to open the “Output Streams” pop-up. Check the box next to “Solution 1” to enable the ouput stream, set the “Type” to “File” and the “Format” to “E/N/U-Baseline”. This will format the output to give us the distance between rover and base. Enter a file name and path in the “Output File Path” box.


Next we will set up the log files. Although these are not necessary to run RTKNAVI, they are very useful for debugging any issues that may come up later. Check the boxes for both “Rover” and “Base Station”, and set both “Types” to “File”. Enter file names for both logs. They will be in raw ublox format so I give them a “.ubx” extension. If you check the “Time-Tag” box, you will be able to re-run the log files with RTKNAVI. If you don’t check this box, you can still re-run the logs, but only with one of the post-processing apps (RTKPOST or RNX2RTKP)


OK, that should take care of all of the data streams. Next we will set up the solution configuration options. Click on the “Options” button in the bottom row of buttons. Select the “rtknavi_5hz_m8n.conf” file as shown below.


Again, you should have downloaded this file with the demo5 executables. We’ll go over the details of these settings in this file later, for now I’ll just mention that the solution mode is set to “Static-start”. This option is only available in the demo5 code and will assume the rover’s location is stationary (“Static” mode) until a fixed solution is achieved, at which point it will assume the rover is moving (“Kinematic” mode). In this exercise we could use “Static” mode instead of “Static-start” if we didn’t plan to move the rover, or “Kinematic” mode if we did.

There is no need to enter the base station location if we are only concerned with relative distance between the rovers which is what we are doing in this exercise.  The configuration file specifies that we will use the measured “Single” position for the base station location.  I have limited the number of averages for the base station location to one because allowing the base to move while we are running the solution can cause it to converge quite slowly.  If you were trying to calculate absolute position with any accuracy you would need to enter accurate coordinates of the base station in the position sub-menu.

At this point, before we setup all the output windows, it would be a good time to verify the receivers are communicating properly with the laptop. I suggest using the ublox evaluation software, u-center, to do this. Open u-center, connect to each receiver, check it’s baud rate is correct and monitor the packet output window for a few seconds. You should see readable NMEA text messages if everything is working right for each receiver. You can read this post for more details on using u-center. If you need to change a baud rate, don’t forget to save the configuration to the receiver so it will come up correctly after a power-cycle. Also don’t forget to disconnect u-center from both receivers, or close it when you are done or it will prevent access to the com ports when you start RTKNAVI.

Coming back to RTKNAVI, the last thing to set up the output windows. Clicking on the two arrows in the top right corner will cycle through various options for the main display window. The right arrow cycles through plot types and the left arrow through sub-types. I’ve chosen “Rover:Base SYS SNR (db Hz)” here which allows us to see signal strengths for all satellites for both rover and base. Satellites are colored by system (GPS, GLONASS, SBAS) and only satellites with sufficient quality in both receivers to be used in the solution are colored, the others are grayed out. Clicking on the small box above the “Start” button brings up additional monitor windows. Each window allows you to choose what that window will monitor. For this exercise, we will click the box three times to open three windows and set them to “RTK”,“Obs Data”,and “Error/Warning”. You can re-size and move around the windows to make all of them visible at the same time as I have done in the screen capture below.  The  example below shows what the screen looks like after hitting start, at the moment your boxes should be mostly empty.


Once you’ve done this, we are almost ready for the big moment! First check your antennas, make sure both have open views of the sky with no nearby obstructions and you have ground planes under both antennas. If all looks good, go ahead and click the “Start” button in RTKNAVI.  The monitor windows should start to fill with information and hopefully look something like the example above. In this case, the GPS receivers should have had enough time to converge to an internal solution while we were verifying them above and before we pushed “Start”, but if you skipped that step let the receivers run for a minute or two after power-up before clicking the “Start” button.

OK, now let’s check a few things to make sure everything is working right. First look at the main display window, shown in the upper left corner above. If you are in North America you should see colored bars for both receivers for all three systems (GPS, GLONASS, and SBAS). In Europe the SBAS satellites will be grayed out since the EGNOS SBAS satellites don’t broadcast range information.

Next check the observation monitor window, shown on the far right above. You should see valid pseudorange (P1) and carrier-phase (L1) measurements for both receivers (1 and 2). You should see both GPS (Gxx) and GLONASS (Rxx) and possibly SBAS (Ixx) in the list, again for both receivers. The rover observations should update five times per second and the base observations one time per second. If they are not continuously changing, something is wrong with your setup.

Next, check the Error/Warning monitor window shown on the bottom left above. In the first couple minutes you will see “large residual” errors and “position variance too large” errors and maybe a few “slip detected” errors as the solution converges. This should switch to mostly “ambiguity validation failed” errors as the solution converges but before the ambiguities are resolved. If you are getting frequent occurrence of any other message, then something is probably wrong and needs to be investigated. If not, the ambiguity resolution ratio (AR ratio) listed in the main display window and in the Error/Warning messages will fluctuate up and down but eventually should reach 3.0 at which point the solution status in the main window should switch from “Float” to “Fixed” and hopefully stay there. For me, this typically occurs in less than 5 minutes but this number will vary depending on your configuration. At that point the “Positioning mode” in the RTK window should switch from “Static-start” to “Kinematic” and now if you like you should be able to move the rover antenna without losing lock.  Make sure you don’t block the antenna’s view of the sky when moving it.  Of course the movement will have to be pretty limited since both receivers are hard-wired to the laptop.  (Note: while writing this tutorial I noticed that the RTK window is incorrectly displaying“Moving-base” instead of “Static-start” before the switch over. This is a bug I must have created when I added the “Static-start” mode but it only affects the display window and not any functionality. I’ll plan on fixing this in my next code update)

Assuming you’ve got a fix, then I would suggest playing around with all the display options since there are many of them. Below I show a screen capture after achieving a fix. I’ve switched the main display over to baseline so it shows the distance between the two antennas. I’ve also clicked the “Plot” button to start real-time plotting of the solution and let it run for 5 or 10 minutes. You can see the solution is moving around by at least a couple of centimeters. If I had run the solution as “Static” instead of “Static-start” this variation would be much smaller but I would not be able to move the rover around.  I also suspect if I had waited a few more minutes before collecting this data, the errors would be smaller.


Hopefully everything goes well and you quickly get an accurate fix. If you don’t get a fix, I would first check the error/warning window, see if there are any clues there. If not, and all the other checks I mentioned above look good, then the next step would be to look at the log files we enabled in the data stream menus. They will be in raw ublox format but can easily be converted to RINEX observation and navigation files with the RTKCONV GUI. Plot the observation files using RTKPLOT and look for cycle-slip issues or other quality problems with the measurements. Check that there are no missing or extra observations. If they look good, the next step would be to post-process the log files with RTKPOST to see if that makes a difference. If all that looks good and you still can’t get a fix, send me a copy of the raw data files and I’ll take a look.

Good luck!!

In the next post I will add the radios to make this a more useful experiment.