RTKLIB: Missing data samples

Update 7/3/16:  A couple of things to be aware of before reading this post:

  • I inadvertently had the base and rover observation files switched as input parameters to rnx2rtkp when I ran the solution.  When rerun with the moving receiver as the rover, the problem described here does not occur.
  • I have removed this change from the latest code since it also turns out to have some undesirable side effects which are explained in another update at the bottom of the post.


The demo3 code is looking pretty clean on the last data set so it’s time to collect some more challenging data. I went back to the previous location but this time instead of running back and forth on the top road in the picture below, I drove around the full loop. The trees are closer on the lower road, and driving on both sides alternates between obstructing different sets of satellites. In a couple of spots, the road passes underneath tree branches and causes a momentary loss of nearly all the satellites.


Here’s the observation data for the base on the left and the rover on the right, where the red ticks are cycle slips reported by the receiver. The base receiver was located in the middle of the field with an unobstructed view of the sky.


There are no cycle slips at the beginning and end of the rover observations because, as in all my data sets, I left the rover stationary with unobstructed skies during these times. I use the initial stationary period to allow the RTKLIB solution to get a reliable fix before the rover starts moving and the stationary period at the end is to help with verification of the solution. I20 is an EGNOS SBAS satellite but because the EGNOS satellites don’t provide ranging information, it is not used in the solutions.

Below is a plot of the solution using the current demo3 code. Position is on the left. The yellow sections indicate float positions where RTKLIB was not able to get a fix. On the right is a plot of velocity, zoomed in a little to show some detail. There is an obvious problem here with all the large spikes in the velocity.


So what is causing these velocity spikes? Zooming in further and looking at the observations and the positions shows a couple of things.


First of all, the rover observation data plotted on the left shows that the velocity spikes, which in this case occur at 7:04:09.6 and 7:04:11.6, are occurring for epochs where there are no valid satellite observations. Second, the corresponding positions shows that these are obviously incorrect as well. Specifically, the reported position for epochs with no valid rover data appear to be based on the last valid data and don’t reflect any movement of the rover since then. What is most concerning is that although these samples have large errors (about half a meter in this case) RTKLIB reports them as valid fixes.

For now I will not worry about why I am getting the missing data samples, but instead use them as an opportunity to understand why RTKLIB is responding so poorly to the missing data.

Digging into the code a little bit sheds some light on to what is going on. RTKLIB does check the ages of the two observations and internally records the difference between them. However, unless the age of the data exceeds the input parameter “pos2-maxage”, it does very little with this information. The solution is calculated from the last valid data and not flagged in any way to indicate it’s poor quality.

So, how do we fix this? First of all, let’s go into a little more background on how the kalman filter works since if used properly it is very good at handling situations like this. Also, let’s assume that the receiver dynamics option is enabled, since we will need the additional states it provides. If you are running RTKLIB without receiver dynamics enabled, you might want to read my last post where I describe a change to make this feature run much faster.

In the last post I briefly explained how enabling the receiver dynamics feature in RTKLIB extends the kalman filter to include receiver velocity and acceleration states. I also described how this improves the initial position estimate RTKLIB makes at the beginning of each sample. However, as I mentioned before, the final position estimate is based on a combination of this initial prediction estimate and a second estimate calculated from the carrier phase measurements for that sample. Most of the time the final position estimate is derived primarily from the measurement-based estimate and the accuracy of the initial prediction estimate does not have much influence on the final result.

This is because the relative weighting between the two estimates depends on our best guess of the variances of each. It is important to understand that every input to the kalman filter has two components. The first is the actual value of the measurement or prediction. The second component is an estimate of the variance of that value. Large variances mean the uncertainty of the measurement is high, small variances indicate low uncertainty, or high accuracy. If you are more familiar with standard deviations, just remember that the variance is the square of the standard deviation. If the variance estimates are accurate, and the measurement distributions are well behaved, then the kalman filter will select the optimal combination of the measurement and the prediction. For example, if we know the measurements are noisy, the measurement variances will reflect this and the kalman filter will rely more on the predicted position, if we know the rover can accelerate rapidly in unknown directions between samples, the prediction variances will reflect this, and the kalman filter will rely more on the measurements.

So how does RTKLIB determine the variances for the measurements and the predictions? The variances for the measurements are a function of the two input parameters “stats-errphase” and “stats-errphaseel” as well as the satellite elevations. The variances for the predictions are calculated internally to the kalman filter and are also a function of the input parameters “stats-prnaccelh”, stats-prnaccelv” and the time between samples. I’ll leave the details to another time, but for now, what is important to understand is that normally the variances of the measurements are much smaller than the variances of the predictions. Hence the final estimate is weighted heavily towards the measurements and the predicted position matters much less.

Back to our problem. What is happening in the case of a missing sample, is that RTKLIB is feeding measurements into the kalman filter that it knows have large uncertainty because of their age, but does not modify the variances to make the kalman filter aware of this possible error. 

So how do we fix the problem? The best solution would probably be simply to not feed this measurement into the kalman filter since it is redundant to the previous one and is not providing any additional information. It turns out the architecture of the code does not lend itself to this fix easily, or at least I did not find an easy way to do this.

What I found to be a simpler fix was to adjust the variance of this measurement to reflect its large uncertainty but otherwise let the code continue through its normal path. Multiplying the estimated velocity with the time since the last good measurement gives a rough estimate of the error. Adding this to the variance is not exactly correct but is good enough for what we are trying to do, and is a much better estimate than what we do now, which is nothing. Since the variance of the measurement will now be much larger than the variance of the prediction, the final estimate will be virtually equal to the prediction, which is what we want.

It is much better to overestimate the variance of a measurement than to underestimate it. Overestimating a variance just means less information is extracted from that measurement than could be. Underestimating a variance, on the other hand, can be very detrimental to the solution since we are telling the filter we have high confidence in a measurement that is very likely wrong.

I added these lines of code to the ddres() function that calculates the double-difference residuals to accomplish what I described above.

/* adjust measurement error variances for missing samples if have velocity estimate */
if (dt>0&&rtk->opt.dynamics) {

Ri and Rj are the variances of the carrier phase measurements from the base and the rover respectively.

Here is the velocity and position for the solution run with the above code change. Both the spikes in the velocity and the discontinuities in the position are now gone.


Often times, the variances of the measurements tend to get ignored or at least under-emphasized. This is usually because they can be hard to estimate and it is not as obvious how they affect the solution, but I think this example shows how they can be important and should not be overlooked. Improving the accuracy of the variance estimates is also a way to improve the quality of the solution and is something I hope to come back to soon.

I have started a new demo4_b12 branch in my GitHub repository which I have included this change. I have also merged in updates from the main RTKLIB repository bringing my code up to date with the 2.4.3 b12 level.  

I have also uploaded this new raw data set into the argeles2_car folder in my library of data sets.  One of the changes in this newer version of RTKLIB is that the executables have been removed and relocated to a separate repository.  I have copied the executables that I am using to here.

Update 7/3/16:  I’ve removed this change from the demo4_b12 code for the time being.  There’s a couple things I’ve realized since writing this post.  First of all, I inadvertently had the base and rover observation files switched as input parameters to rnx2rtkp when I ran the solution.  The velocity spikes disappeared when I corrected this since RTKLIB does not do updates for missing rover observations.  In theory, the problem still exists if there are missing base observations and the base is moving, but this would only happen in a moving base situation.  

I also found that this change causes problems when the base station sample rate is slower than the rover sample rate.  This is quite common in practice since the base station is usually not moving and so there is no need to sample it’s position as frequently. The lower sample rate is treated by this change as many missing samples which is what it effectively is.   Unfortunately, my change tends to overestimate the measurement variances for the missing samples as I mentioned in the description above.  This was not a problem when it was a small number of missing samples, but when it is the majority of the samples, then it is a problem and the solution is degraded.


4 thoughts on “RTKLIB: Missing data samples”

  1. Ah yes you are completely right: the base station receiver experiences a power cycle!
    The clock bias changes, changing all the code phases but this does not matter as RTKLIB handles well clock jumps. However, when the base restarts the carrier phase measurements are arbitrarily reset so the number of cycles is totally uncorrelated with the last good epoch. That explains it, thank you so much!
    I also had the feeling that resetting the phase bias states might not have been enough: I will take a look to which other states I should be resetting. Thank you very much again for your insight! Michele


  2. Hello, thanks again for the great work.
    I think the problem I experienced could also be correlated with the above, see my post on FOSS-GPS:

    I am suspecting also your development branch would not recover cleanly, but would you please be so kind to take a look at it?
    I feel that the same problem would come back in many real-life scenarios and so it would be worth fixing in a clean way.

    For your reference, the dataset is here

    All the very best,


    1. Hi Michele. I think you are probably right that these two issues are related. I downloaded your data and confirmed my change not only does not fix your problem but actually makes things even worse. I’ve removed the change from my code until I understand better what is going on. I agree it would be nice to have a clean fix for this and will see if I can add anything to what you have already done.


      1. Hi Michele. It looks like there may be more going on in your base station data than just some missing samples. Possibly some sort of end of month clock reset? There are large jumps in all of the raw pseudorange and carrier phase measurements after the gap which look like they were caused by a large jump in the receiver clock. This causes jumps in all of the phase-biases. Your fix to reset all the states looks like a reasonable way to handle this. The only thing I can suggest is that you are resetting all the states, not just the phase-bias states. Most of the time this probably doesn’t make any difference, but if you had some of the other states enabled for atmospheric correction or IC biases, you might not want to reset those.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s