A fix for the RTCM time tag issue

In my last post I described a problem with a loss of some of the raw measurement information caused by the lack of resolution in the time tags in the RTCM format.  Since the RTCM format is typically used to reduce bandwidth requirements in real-time applications, it is causing real-time solutions to fail when post-processing the same raw data without the translation to RTCM gives good results.  In this post I will describe a fix for this problem.

First of all I want to thank Felipe Nievinski, Igor Vereninov from Emlid, and Anthony Woolridge for their comments to the last post that pointed me to the solution.  They make this a collaborative effort between the U.S., Brazil, Russia, and the U.K!  It still amazes me how enabling the internet can be!

I’ll start by showing again this example of a RINEX output from an M8T receiver with the official raw measurement output (RXM_RAWX) and the debug raw measurement output (TRK_MEAS) enabled simultaneously.  I think  this provides a good insight to what is going on.  The TRK_MEAS  message is the top 5 lines and the RXM_RAWX message is the bottom 5 lines for a single epoch.  The first line in each message is the time stamp and the following lines are the measurements for each satellite.  In the satellite measurements, the second column contains the pseduorange value.

The time stamp specifies the receiver time of the received signals and the sixth column is the number of seconds.  For the TRK_MEAS message these values are always aligned to round numbers based on alignment to the sample rate.  For example in this case the measurement rate was 5 Hz and all the time stamps occur on multiples of 0.2.  This is because they are based on the raw receiver clock without any corrections.

The time stamps from the RXM_RAWX messages however often differ from the round numbers by small arbitrary amounts.  This is because the receiver has estimated the error in its own clock and adjusted the measurements to remove this error.  In this case the estimate of clock error is 0.001 seconds and so the time stamp is adjusted by this value (18.8000000 to 18.7990000).

To keep the time stamps consistent with the other parts of the measurement, the clock error also needs to be removed from the psuedorange and carrier phase values since they are based on the difference in time between satellite transmission and receiver reception and will include any errors in the receiver clock.  We see from the above observations that the pseudorange measurement for satellite G24 has been adjusted from 22675327.198 to 22375547.970, a difference of 299779.228 meters.   The speed of light is 299792458 meters per second so the clock error of 0.001 seconds is equivalent to 299792.458 meters,  a value very close to the amount that the pseudorange was adjusted by.

A similar adjustment needs to be made to the carrier phase measurement as well but it is not as easy to see in this example because the carrier phase measurements are relative rather than absolute and the two messages in this case use different references.  The carrier phase measurements are in cycles, not meters, so the frequency of the carrier phase needs to be included in the translation from clock error to carrier phase cycles but is otherwise the same as the pseudorange adjustment.  In equation form, the adjustments are:

P = P -toff*c
L =L – toff*freq

where P=pseudorange, L=carrier phase, c= speed of light, and freq=carrier frequency

So, basically, the receiver is trying to help us out by removing its best estimate of the clock error from the measurements.  This is unnecessary since RTKLIB is quite good at estimating this clock error on its own, but by itself this adjustment does not cause a problem.

It is when the adjusted measurement is translated to RTCM that we get in trouble.  The resolution of the time stamps in the RTCM format is 0.001 seconds.  In this particular example it would not be an issue because the error is exactly 0.001 seconds or one count of the RTCM format.  Most of the time, however, this error is not an exact multiple of 1 millisec.

For example, here is a time stamp for the data set described in the previous posts.

> 2017  1 17 20 31 48.9995584  0  9

And here is the same time stamp after being translated to RTCM and then to RINEX

> 2017  1 17 20 31 49.0000000  0  9

As you can see, the clock adjustment was less than half a millisec so was completely lost in the roundoff to the RTCM format.  However, the adjustments the receiver made to the pseudorange and carrier phase are still present in those measurements.  We now have a problem because the clock correction is in part of the measurement and not the other pieces.  RTKLIB can not correct for this lack of consistency within the measurement.

So, how do we avoid this problem?  Fortunately, RTKLIB has an option to adjust the time stamps to round values using the same equations described above to adjust time stamp, pseudorange, and carrier phase to maintain consistency within the measurement.   I imagine it was put in specifically to solve this problem. We can invoke this option by adding “-TADJ=0.001” in the “Options” box in the “Conversion Options” menu in STRSVR or using the “-opt” option in the command line with STR2STR.  Note that this option needs to be set in the conversion from raw binary format to RTCM format, not the conversion from RTCM to RINEX.  It is possible to set this option when converting from RTCM to RINEX but this won’t help because the damage has already been done in the earlier conversion.

Unfortunately, there is a bug in the implementation of this option in RTKLIB, at least for the u-blox receivers, so by itself, this is not enough.  The problem is that invalid carrier phase measurements are flagged in RTKLIB by setting the carrier phase value to zero.  The time stamp adjustment feature adjusts these zero values slightly so they are no longer recognized as invalid.  They end up getting included in the output as valid measurements and corrupt the solution.

Fortunately, the fix for this bug is very simple.  Here is the code in the decode_rxmrawx() function in ublox.c that makes the adjustment:

/* offset by time tag adjustment */
if (toff!=0.0) {
fcn=(int)U1(p+23)-7;
freq=sys==SYS_CMP?FREQ1_CMP:
(sys==SYS_GLO?FREQ1_GLO+DFRQ1_GLO*fcn:FREQ1);
raw->obs.data[n].P[0]-=toff*CLIGHT;
raw->obs.data[n].L[0]-=toff*freq;
}

If we add a check to the first line of code to skip the adjustment if the carrier phase is zero, then all is fine.

if (toff!=0.0&&cp1!=0) {

Below is the original solution after RTCM conversion on the left and with time tag adjustment and the bug fix on the right.  If you compare the solution on the right to the solution with no  RTCM correction in the previous post you will see they are nearly identical.

I am still wary of using RTCM because of its other limitations described in the last  post, particularly the loss of the half cycle invalid flag and the doppler information, but I believe this fix eliminates the most serious issue that comes from using RTCM.

I will release a new version of the demo5 code with this fix sometime in the next few days.  It will take a little while because I also want to include some other features that have been waiting in the pipeline.  If you want to try the fix right away, you just need to  modify the one line of code described above and rebuild.

Update 2/2/17:    I have taken Anthony Woolridge’s suggestion and modified the RTCM conversion code to automatically adjust the pseudorange and carrier phase measurements to compensate for any round off done to the time tag.  This means it is not necessary to set the time-tag adjust receiver option.  This change is currently checked into my Github page and I hope to post new executables in the next couple of days.