Exploring differences between real-time and post-processed solutions.

I’ve had a few questions recently about differences showing up when the same set of raw data measurements are processed real-time and when they are post-processed.  Since I haven’t done a lot of real-time work I didn’t have a good answer to these questions, but it seemed like an interesting problem so I thought I would dig into a little bit.

In many cases, these differences can be traced to a poorly performing data link between base and rover that loses, delays or corrupts the base measurement data.  These problems are usually diagnosed fairly easily by looking at the “age of differential”  between base and rover or by seeing missing data in plots of the base observations.  My interest is not in these cases but rather where the data link is performing well and there is still a difference between the real-time and post-process solutions.

To troubleshoot real-time solutions is a little trickier than post-processing solutions because you may need a way to re-run the data through the real-time RTKLIB app (either RTKRCV or RTKNAVI) to recreate the problem.  The standard *.ubx log files do not contain enough information to do this since they contain only a time stamp for when the measurement was made and not when it was actually available to the solution.  There will usually be some delay between the two because of latencies in the data link between rover and base.  The post-processing solutions ignore this delay and simply align the two measurements assuming zero delay but we need to know what these delays are to recreate the real-time solution.

The real-time solution apps have an option in the input stream setup to read from a file instead of a real-time stream.  This allows you to re-run previously recorded log files but when doing this they require a *.ubx.tag file in addition to the *.ubx file to provide the latency information.   These *.ubx.tag files are generated automatically when you log real-time data if you select the appropriate option before you collect the data.  For RTKRCV, this is a “::T” appended on to the end of the log file name.  For RTKNAVI, it is checking the “Time Tag” box in the log stream options.  I recommend always enabling these options when you are running real-time solutions because the extra files are not very large and you never know when you are going to get something unusual in the data that you would like to investigate later.

Since none of the data sets I had been sent to look at contained tag files, my first step was to try and collect some data that looked good in post-processing but not in real-time with time tags enabled.  I chose to use my Emlid Reach receivers to do this, in part because it is easy to do real-time solutions with the onboard wireless, and in part because I wanted to try out their recently released 2.1.6 version of the RTKLIB code.  This version is a very close cousin to my demo5 code and contains all of its features (although many of them are not currently accessible through the Reachview GUI).

I first added or modified a couple of lines of code in the Reach startup files to save time tag versions of both the base and rover data on the rover, and the base data on the base.  I’ve added some notes at the bottom of this post on how I did it but I don’t necessarily recommend doing it yourself unless you are fairly comfortable with linux because it can be a little tricky to recover without reflashing the unit if you make a mistake.  I wanted to be able to collect data on the Reach units using the command line based RTKRCV app but use the GUI based RTKNAVI on my laptop to re-create the realtime run.  This is because RTKNAVI has a much nicer  interface with a lot more information available.  However, this meant that I needed to fix an incompatibility in the RTKLIB code between the time stamp formats of RTKRCV and RTKNAVI as described in the RTKLIB Github issue #99.  Using the fix recommended in the issue description,  I rebuilt the code on the Reach unit to create a new str2str executable with this fix incorporated.

With these changes, I can collect measurement data that gives me the option to run post-process solutions or re-created real-time solutions.  In addition, these can be run either with measurements made before or after the data link and raw binary to RTCM conversion.  This gives me quite a bit of capability  to investigate where a potential problem might be occurring.

To test this setup, I first collected some static data with both base and rover exposed to open skies.  I got all three sets of data and tag files and using these I was able to re-run the data using RTKNAVI.  Both real-time and post-processed solutions got a fix fairly quickly and the two solutions were very similar.  So, nothing interesting to look at in this example.

Next I placed both base and rover on my back patio, just a few meters away from the house and partially blocked by a large tree, knowing that this would be a more stressful measurement environment.  I may have just got lucky, but the very first data set I collected gave me multiple fixes in post-processing but none in real-time as shown below (post-process on the left, real-time on the right).  The two loss of fixes are caused by me restarting the data collection on the Reach rover.  In this case I ran the post-processing solution using the base data collected on the base in raw binary format (*.ubx), not the data after it had been converted to RTCM and transmitted to the rover (*.rtcm)  since this is the way post-processing is usually done.


Next I ran a second post-processing solution, this time using the raw measurement file saved in RTCM format on the rover.  This time there was no fix and the solution looked nearly identical to the real-time solution plotted above.   So somewhere between when these two data files were saved, the problem is occurring.  Note that in this case I was able to do all this without the time tag files or re recreating a real-time run but I imagine this capability will be helpful in future analysis.

I had monitored the age of differential while collecting the data and after collecting the data I plotted the base observations to verify there was no missing data.  This suggests that the data link was working fine.  So my next guess was that the conversion from raw binary measurements to RTCM format might be the cause of the problem.  In real-time solutions, the base data is typically translated to RTCM before transmitting over the data link to the rover to compress the data and reduce bandwidth requirements on the data link, and this is the default configuration of the Reach units.   The amount of compression will vary depending on the details of the data but in this case the RTCM file (*.rtcm) was about one third as large as the raw binary file (*.ubx).  Some of this is lossless compression but not all of it so there is potential for degrading the solution with this translation.

The next step was to isolate the effects of the RTCM translation from any effects from the data link latency.  I did this by using the STRSVR app to translate the raw binary base data saved on the base station to RTCM format.  I configured the conversion options to use the same RTCM messages as used by Reach.  ran this data through a post-process solution.  Sure enough, just converting the undelayed raw binary data to RTCM was enough to break the solution.  That means, at least for this case, we can ignore any effect of the data link delays and focus on the RTCM conversion.

Note that the post-processing apps require all the measurement input files to be in RINEX format.  This means that both the raw binary files and the RTCM files are converted to RINEX first using RTKCONV first as part of the post-processing procedure.  One thing to be aware of when using RTKCONV to convert from RTCM to RINEX is the signal mask input options.  The default signal mask has all observation types selected and if left this way it will cause the file header to be incorrect.  If you do not de-select all the extra observation types you will see this in your observation file header


The number of observations is 8 instead of 4 and there are extra observation types listed.  This will confuse RTKLIB and it will not interpret the rest of the file properly. Specifically it will not pick up any of the GLONASS observations.  It won’t flag an error but it will cause all the GLONASS measurements to be left out of your solution.  The signal mask button is on the options page as shown below.  You want to un-check all options except “1C”.


This post is already getting fairly long so I will put off to the next post the rest of the story including discussion about what is actually lost in the translation to RTCM and why it caused this particular example to fail.  In general, though, it is important to understand there are real losses in this translation and that they may affect the quality of your solution.  If you have the bandwidth to transfer the raw binary format instead of the RTCM format I would recommend you consider doing that.  If you don’t have the bandwidth, I would suggest you consider the trade-offs from reducing the base sample rate enough so that you are able to transfer the measurements in raw format.  As I mentioned above, in this example the raw binary file was about three times as large as the RTCM file.



Notes on how I set up the Reach to collect extra data.  There may be a more elegant way to do this but I just wanted a quick hack.  Please be careful if you try to do this yourself and be sure to back up any files before modifying them:

RTKLIB has a “::T” option to record the time-tags but I don’t believe Reach supports this option.  I got around this by adding extra instances of str2str initiated from a function call I added to the “reach_setup” script in the /usr/bin folder.  This, and all the instructions below assume you are running the 2.1.6 version of Reach code.

 I added the call right before the call to “reachview” in the “reach_setup” script as shown in blue below.  I did this on the rover receiver assuming it is getting the base measurements through a data link.
ncat -k -l 2000 < /dev/ttyMFD1 > /dev/ttyMFD1 &
#start logging data files with time stamps
# Run ReachView
led set_color green

I created the “reach_time_logs” script in the /usr/bin folder and put in the following lines of code

# Log u-blox data to file with time stamp logs
# find unused file name
while [ -f $fname ]; do
    let “i=i+1”
# start data collection from rover
/usr/bin/RTKLIB/app/str2str/gcc/str2str_tag  -in tcpcli://localhost:2000 -out $fnameR::T &
# start data collection from base
/usr/bin/RTKLIB/app/str2str/gcc/str2str_tag  -in tcpcli:// -out $fnameB::T &

This finds an unused filename and saves the measurements and the tags for both the rover and base data.  You will need to modify the specified input stream for the base data to match what you are using.  You can look at the inpstr2-type and path in the /usr/bin/RTKLIB/app/rtkrcv/rtk.conf file for the exact format.  You might be able to use the RTKLIB wildcards instead to create the file name but I just copied this code from my PiZero logger which doesn’t update the clock.  I don’t know if on the Reach the clock has been updated yet at this point in the start-up.

I also had to modify the str2str app to make the time-tags compatible with RTKNAVI.  I used the bug fix recommended in Github issue #99.  I recommend debugging by re-running the data through RTKNAVI (on a Windows machine) rather than RTKRCV because it has a much nicer interface with much more info available.  If you decide you want to re-run the data through RTKRCV you will either need to rebuild it with the bug fix or collect the data with the unmodified str2str.  I think it’s unlikely that you will see different solutions between RTKRCV and RTKNAVI assuming they are both configured the same.
Rename the modified str2str executable to str2str_tag and leave it in the /usr/bin/RTKLIB/app/str2str/gcc folder .  Use the chmod +x command to make this file and the “reach_time_logs” file both executable.
I also modified the base receiver and saved the base data in ubx format before it was converted to rtcm so I could compare before and after to see if the conversion or data link might be causing problems.  You can use the same modifications described above, except delete the last two lines in the “reach_time_logs” script.
With these changes in place, the units will automatically save time-tagged data to a new file every time they are turned on.
After collecting data, the data files will be in the /home/root/logs folder.  The file names will be basexx.rtcm, basexx.rtcm.tag, roverxx.ubx, and roverxx.ubx.tag where “xx” will increment every time you run until you delete the old files.  To run them through RTKNAVI, just specify files in the input stream and check the time tag box.
You then have the options of running post-process or simulated real-time with measurements either before or after the data link/RTCM conversion.  This should give you a fair bit of insight into where the problem is occurring.
 I had a bit of trouble with files I edited getting corrupted after  a power cycle (maybe because I was using WinSCP through the wireless) so I suggest using the “reboot” or “shutdown” commands to avoid problems.  Also be sure to make copies of the files before you edit them.  At one point I corrupted the “reach_setup” script and then could only access the Reach by using the instructions in the Software Development section of the QuickStart guide.  Another time the /etc/reachview/stable_config.json disappeared and I had to restore it.

23 thoughts on “Exploring differences between real-time and post-processed solutions.”

  1. Hi All,
    In one of my testcase i have a huge difference between real-time and post processed solutions.
    In fact, the real time is suspicious (the floating solution have results > 2km that the real solution, fixed more the 800m !!!) but I did not find what is wrong.
    The base is a rtcm stream, the rover an ublox m8t stationary.
    I could provide testcase datas if someone is interested in my case !


    1. Additional comment, the post processed solution is more “conventionnal” (floated solution < 10m, fixed solution < 20cm). Configuration file is the same for both solution.
      If I replay solution using tiletag feature I have the same incoherant result than in real time.


    2. Hi Christophe. Your fix rate is very low and your fix times are very short. This suggests that they are likely false fixes. From the plots, it appears that the discrepancy between solutions is occurring right at the beginning of the solution while the kalman filter is converging. This may be because the two solutions have different initial conditions, possibly the base station location is specified differently between the two? The quality of your rover observations looks fairly good so I would have expected a better solution. I would check your config settings, especially your base station location.


  2. Hi rtklibexplorer !
    First thank you for all your job on rtklib it is a hug help.
    You mentionned that you appplied the bug fix in the Github issue #99 to make readable the .tag for rtknavi. Is it this one ? :
    typedef struct { /
    time struct /
    time_t time; /
    time (s) expressed by standard time_t /
    attribute ((aligned (8)))double sec; /
    fraction of second under 1 s */
    } gtime_t;

    I personnaly try to read under rtkrcv (linux compiler) observations with *.tag recorded from strsvr (windows compiler). Of course I did not succeed to read the stream from windows properly. I have date issues on my log file and frozen rtkrcv.
    Do you know if the bug fix #99 can be modified to read windows *tags under linux ?

    Best regards,


    1. Hi RCT. In the current windows RTKLIB code, the time tag feature in RTKNAVI is not working due to some recent changes ported in from the main release code. Since the windows tags don’t work under windows I am not surprised they don’t work under linux either. I would recommend disabling time tags until the feature is fixed. The time tags feature enables the age of differential to be incorporated into the post-processed solution to better emulate a real-time solution but unless your age of differential is large (say >2 sec) I don’t think you will find it makes much difference.


      1. Hi rtklibexplorer. Thank you for your answer. It could explain the incompatibilities between Windows and Linux stream. As I need my saved observations to be run in real time I have no choice but using the tag file. I tried to find the issue in the code and my rtkrcv always freeze at the first call of the following function: “timeset(gpst2utc(timeadd(svr->.sol.time,tt)));” on the “rtkrcv.c” file.
        By lack of time I gave up with linux and used demo version of embarcadero to compile rtknavi and it worked well.


    1. Hi Anthony. RTKLIB doesn’t have this capability built-in but it could be added fairly easily with a simple external app. I also believe the Emlid version of RTKLIB does have an event logging feature that could be used for this.


  3. Hi all

    In which part of the RTKRCV code should I modify to indicate by blinking LED the “—–” “SINGLE” “FLOAT” “FIX” solution type? Exemple: LED G= Solution FIX , LED Red = Solution SINGLE



  4. Hi all,

    I am using a reach unit as a rover with NTRIP corrections (real-time). All is working well (after some time…) but I see that there is a growing delay in the solution updates coming through the tcp server link from rtkrcv. I am comparing the time of the computer running my application (connecting to rtkrcv via tcp) and the gps time that is sent from rtkrcv (after doing the necessary corrections of course). initially the delay is around 0.2-0.3 seconds, but with time (during the float and later fix) it grows, I have seen it go up to as far as 15 seconds.

    First I thought this was caused by the quite high receiver frequency I was using: 10Hz. Perhaps the real-time solution or the connection to my system could not keep it up. The connection with the ntrip seems ok, the ‘age of differential’ is never more than 1 second. So I lowered the frequency gradually to 1Hz, but I am still getting growing delays, they only grow slower as if it depends on the number of cycles that are passed (so it only takes 10 times as long to get to the 15 seconds).

    Another thing I see is that the delay only goes up (in general, there are minor ups and downs), but never back down.

    I can see the processor of the reach unit (Edison) is not doing much (4% user) and I can’t imagine that the WiFi link I am using cannot keep up with the 1Hz (actually not even with the 10Hz).

    I hope anyone has suggestions as to what may cause this behaviour, could the subject of this post cause this? The corrections from NTRIP are in RTCM (3.0) format, but I cannot change this.

    Kind regards,
    Dimitri Willemse


    1. Hi Dimitri. Anton Chy just submitted a fix for a memory leak he found in the demo5 code. I believe the same issue is probably in the Reach code. This would be a more likely explanation for what you are seeing than the problem with RTCM conversion. You can find the fix in the Github repository for the demo5 code.


    2. Hi, can you just please explain how you were able to fix a point using NTRIPclient (realtime). i ve been seriously struggling with it. i just click a rover, single option ( a receiver used), then NTRIPcaster host, ID, password, mountpoint were done then i clicked on start. please, is/are there anything missing?


      1. Hi Lucas. I describe connecting RTKLIB to an NTRIP caster in this post. The details will vary a bit between different NTRIP casters but this should be enough to get you started. Some NTRIP casters do require an SSL certificate in which case RTKLIB won’t support them … maybe this is what you are running into?


        1. hi friend, i am using it for academic purpose. it was just sent to my mail d NTRIP caster host address and 2101, after certified registration with IGS. would i ve to source for any other useful data from IGS before i can be able to fix a point using realtime service with just a receiver?


  5. Hi Tim!

    Thank you again for this post!

    i am a owner of reach and did see an mention problems with reach last months.
    my problem was that reach was able to get fix with best conditions but i lost fix after running longer times..

    if i simultaneous got raw data from both reach units and run it with rtknavi on my pc i was able to hold fix for a long time.

    for me now i have a proof that i wasn´t wrong with this….



    1. Hi Andreas. I would be careful about assuming the problem in your data is identical to what I found in my data. They may be the same problem, or they may be different but both be related to the RTCM conversion, or they may be completely different issues. RTK is a complicated system, and there are many,many different things that can go wrong.


    2. Hi,
      I also see similar problems between realtime and post-processing on Reach Ublox. I have been following you and this blog to see what may come of it.

      I think you and RTKExplorer are on the right track. A fresh look is often needed to get the answer. Thanks for being so dedicated to improving RTKLIB.


  6. Great post re Strsvr RTCM conversion errors!
    I first mentioned these errors back in May 2015 on FOSS-GPS under the obscure title BINR to RTCM conversion, but confirmed it also applied to UBX to RTCM conversion too.
    This might give you a clue as to where the errors occur. It was picked up by David Kelly who kindly confirmed it existed as a problem. It seems no one else to my knowledge has picked up the baton until you now. I really hope you can get to the bottom of it as I’m sure you will.
    I too found the only solution at present is to send raw ubx data over the link. I tend to use tcpip links over the mobile phone network which fortunately does give sufficient bandwidth to not worry about data compression. Radio modems would be a problem though hence a solution would be very welcome. I’m sure this problem must have bugged many people before now too.
    Keep up the good work, and I eagerly await your next post hopefully with a solution!


    1. Hi Anthony. Thanks for the vote of confidence! Unfortunately I have not yet been able to come up with a solution. I’m hoping you or one of the other GPS old-timers can help figure it out. I did go back and read the previous discussion you mentioned and there are some similarities between the two but there are also some differences as you’ll see when you read part 2. (I just published it)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: