Idea

laurenceh suggested Jul 24 2020 at 4:15 PM

Setting time without internet connection on Rpi Venus OS

As I will be using my Venus OS Rpi3B+ on my boat (Lady's Smock) I will not always have an internet connection when I switch the Raspberry Pi on, and the Raspberry Pi does not have an RTC.

I wonder if others have this problem in remote or mobile (motorhome/boat) installations?

In my case I will always be using GPS and should be more likely to have a GPS data fix than an internet connection.

The script here sets the data and time from the GPS data. It waits around 100 seconds for the GPS device to be found (or start up) and collects up to a hundred lines from the GPS device looking to check that

(1) It has a satellite fix

(2) It can provide the data and time

Once decoded it automatically sets the Rpi date and time according to the GPS data.

It has been tested with Venus OS 2.57 in July 2020.

I will look at adding it to /data/rc.local so that it runs at start up. I would not at this stage suggest running it continuously to provide updates as it might conflict with the NTP server system used by default in Venus OS.

The location of the GPS device "/run/serial-start/gps/ttyACM0" is hard wired at present and may need to be changed if you are using an NMEA GPS interface.

It relies on GPGGA and GPRMC data packets from my GPS device - I presume these are pretty standard across devices.

#!/bin/bash
N=0;
FILE=/run/serial-starter/gps/ttyACM0
echo "Waiting for GPS data"
while [ ! -e $FILE ] && [ $N -lt 100 ];
do
    sleep 1;
    ((N++))
    echo -n "."
done;
sleep 1;
 
if [ $N -ge 99 ]; then
   echo " GPS data not available is the device plugged in"
   exit 0;
fi
 
echo
echo "Device attached waiting for data"
 
input=$FILE
N=0
 
# check if GPS has FIX
LINE1='^\$..GGA,[0-9\.]+,[0-9\.]+,[NS],[0-9\.]+,[EW],([0-2])'
# Catch lines with date and time info
LINE2='^\$GPRMC'
 
FIX=""
 
# extract date and time from GPS packet
 
function extract {
[[ $1 =~ ^\$GPRMC,([0-9]{4})([0-9]{2})\.[0-9]{2},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}), ]]
  COMMAND="${BASH_REMATCH[5]}${BASH_REMATCH[4]}${BASH_REMATCH[3]}${BASH_REMATCH[1]}.${BASH_REMATCH[2]}"
  date -s $COMMAND
exit 0
}
 
while IFS= read -r line && [ $N -lt 100 ]
do
#  echo "$line"
  [[ $line =~ $LINE1 ]] && FIX=${BASH_REMATCH[1]} && echo "GPS device has a fix"
  [[ $line =~ $LINE2 ]] && [[ $FIX -eq 1 ]] && extract $line
  ((N++))
done < "$input"

laurenceh commented Jul 25 2020 at 12:38 PM

This now works from /data /rc.local on my Raspberry Pi.

Add the following lines to /dat/rc.local

if test -f /data/venus-rpi-setup/gps.sh; then
 /data/venus-rpi-setup/gps.sh &>/var/volatile/log/gps-time &
fi

Clearly you can change the location where you store the script and where you want the log to go.

Notes:

1) Don't forget the & at the end of the line it is important to make sure it runs as a background process otherwise it stops the rest of the initialisation.

2) You can't rely on standard output to log the output information. The standard boot output logging which goes to /var/log/boot can be shut down before the script terminates so you lose output messages.

3) I changed this line in the original script to read 200 lines as sometimes the script terminated before finding the data it needed in the GPS data stream.

while IFS= read -r line && [ $N -lt 100 ]

Change to:

while IFS= read -r line && [ $N -lt 200 ]

It would be nice if Victron added something like this into the start up sequence as they already detect cases where there in no hardware clock and should be able to detect if there is no route to an NTP server.

natteverf commented Sep 20 2022 at 3:38 PM

Hi
does this script still work ok?
I get multiple "GPS device has a fix" followed by "date: invalid date '.'"

johnny-brusevold commented · Sep 20 2022 at 10:52 PM

Tested it and got it working with some changes due to my usb-gps (ttyUSB3) using the $GNRMC string.

I didn't get "date -s $COMMAND" to work either, so I changed the code a bit.

Pi3b+ venus-os 2.90

#!/bin/bash
N=0;
FILE=/run/serial-starter/gps/ttyUSB3
echo "Waiting for GPS data"
while [ ! -e $FILE ] && [ $N -lt 100 ];
do
    sleep 1;
    ((N++))
    echo -n "."
done;
sleep 1;
 
if [ $N -ge 99 ]; then
   echo " GPS data not available is the device plugged in"
   exit 0;
fi
 
echo
echo "Device attached waiting for data"
 
input=$FILE
N=0
 
# check if GPS has FIX
LINE1='^\$..GGA,[0-9\.]+,[0-9\.]+,[NS],[0-9\.]+,[EW],([0-2])'
# Catch lines with date and time info
LINE2='^\$GNRMC'
 
FIX=""
 
# extract date and time from GPS packet
function extract {
[[ $1 =~ ^\$GNRMC,([0-9]{2})([0-9]{2})([0-9]{2})\.[0-9]{3},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}),.* ]]
  COMMAND="date -s '20${BASH_REMATCH[6]}-${BASH_REMATCH[5]}-${BASH_REMATCH[4]} ${BASH_REMATCH[1]}:${BASH_REMATCH[2]}:${BASH_REMATCH[3]}'"
  eval $COMMAND
exit 0
}
 
while IFS= read -r line && [ $N -lt 200 ]
do
  [[ $line =~ $LINE1 ]] && FIX=${BASH_REMATCH[1]} && echo "GPS device has a fix"
  [[ $line =~ $LINE2 ]] && [[ $FIX -eq 2 ]] && extract $line
  ((N++))
done < "$input"

0 ·

natteverf johnny-brusevold commented · Sep 22 2022 at 3:18 PM

Thanks for your help

I am not getting an error now, but i have no idea if it is really working, i created the rc.local entry as you said and there is an output in /var/volatile/log/gps-time of multiple "GPS device has a fix" after a restart

0 ·

johnny-brusevold natteverf commented · Sep 22 2022 at 6:18 PM

If everything is ok, you get this when you run the bash script from terminal, and the same in /var/volatile/log/gps-time

root@raspberrypi2:~# /data/GpsClock/gps-clock.sh
Waiting for GPS data
 
Device attached waiting for data
GPS device has a fix
Thu Sep 22 17:28:00 UTC 2022

You probably have to edit line 33 $GxRMC to match your string from gps

Mine is $GNRMC for multi (GPS, Galileo and GLONASS)

$GNRMC,174721.000,A,5993.123983,N,01074.878083,E,0.01,148.82,220922,,,D*7C

And this code picks data from gps string

[[ $1 =~ ^\$GNRMC,([0-9]{2})([0-9]{2})([0-9]{2})\.[0-9]{3},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}),.* ]]

You get $RxRMC data from your GPS.

cat /dev/your-GPS |grep RMC

0 ·

natteverf johnny-brusevold commented · Sep 23 2022 at 8:12 PM

ok, the output of /var/volatile/log/gps-time is this:

Waiting for GPS data
 
Device attached waiting for data
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix

which i guess looks like it is working.
one of my $RxRMC data lines looks like this:

$GPRMC,201015.000,A,5222.7424,N,00458.1954,E,0.00,356.24,230922,,,D*60

do i need to change lines 27 and 33 to match?

0 ·

natteverf natteverf commented · Sep 24 2022 at 11:37 AM

I changed line 33 but got the same results, so i changed line 27 also and it is working properly now :-)

Thanks @Johnny Brusevold for your help with this.
I am not a scripting guru, but would it not be possible to make the script a bit more agnostic by somehow saying G*RMC? Or are there more options that can appear there? I am also thinking about maybe detecting the usb port because mine has changed for some reason sometimes on a restart. Also a Pi3 w venus-os 2.90 large

0 ·

laurenceh commented Oct 07 2022 at 6:19 PM

I wrote the original code and guess I can fix it.

I have a GitHub repository and my new services package of services there has the fix for the date command format in the line that starts COMMAND=

Basically I wrote the code in 2019, at that time a date format yymmdd was sufficient but somewhere after 1/1/2020 the format was changed to yyyymmdd so the code needed to “20” to be added to the command ( it should now work until 1/1/2100 by which time non of us will care ;-). I’m Not sure if the format change was just as the year changed or because of a change in the base code of Venus OS.

Either way new code on GitHub published a few months ago fixed that.

The code looping saying GPS has a fix, is NOT an indication of working. The code is just looping saying it has a fix but still can’t find the line with the time/date information. This is probably due to the noted GxRMC problem.

I’ll fix the code on GitHub to recognise GxRMC but others will need to test it.

natteverf commented · Nov 01 2022 at 8:30 PM

I am happy to test of course :-)

What is the GitHub repository?

1 ·

laurenceh natteverf commented · Nov 14 2022 at 6:04 PM

I have updated the script here

https://github.com/LHardwick-git/Victron-Dbus-Services/blob/main/Rpi-GPS-time/gps-time.sh

To accept different GPS speakers so it should accept xxRMC.

The NMEA Spec is here

https://gpsd.gitlab.io/gpsd/NMEA.html#_rmc_recommended_minimum_navigation_information

Currently untested but the change was trivial.

0 ·

natteverf laurenceh commented · May 30 2023 at 4:32 PM

I know it's been a long while, i was suffering from terrible interference issues with my GPS that caused my Pi3 to randomly freeze up, finally resolved using some ferrite cores around the usb cable, amazing what a difference that makes!

I have tested your "agnostic" script, unfortunately it doesn't work and produces the constant "GPS device has a fix after x lines" until it times out.

0 ·

natteverf natteverf commented · May 30 2023 at 5:26 PM

It looks like the data lines are a slightly different format, I have no idea in the world how to fix that

GNRMC,165306.00,A,5222.74129,N,00458.18549,E,0.020,,300523,,,D*6F

0 ·

natteverf natteverf commented · May 30 2023 at 6:38 PM

I am proud of myself! I used bits of both of your scripts and one minor change to the REMATCH and it works!

                                                                                                    
#!/bin/bash
#add this line to /data/rc.local
#/data/gps-time.sh > /var/log/gps-date &
# This script read the GPS data from a USB connected dongle and sets the Rpi
# time and date from the GPS data received.
 
N=0;
FILE=/run/serial-starter/gps/ttyACM0
echo "Waiting for GPS data"
while [ ! -e $FILE ] && [ $N -lt 50 ];
do
    sleep 1;
    ((N++))
    echo -n "."
done;
sleep 5;
 
if [ $N -ge 17 ]; then
   echo " GPS data not available is the device plugged in"
   exit 0;
fi
 
echo
echo "Device attached waiting for data"
 
input=$FILE
N=0
 
# check if GPS has FIX
LINE1='^\$..GGA,[0-9\.]+,[0-9\.]+,[NS],[0-9\.]+,[EW],([0-2])'
# Catch lines with date and time info
LINE2='^\$..RMC'
 
FIX=""
 
# extract date and time from GPS packet
 
function extract {
[[ $1 =~ ^\$..RMC,([0-9]{2})([0-9]{2})([0-9]{2})\.[0-9]{2},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}),.* ]]
  COMMAND="date -s '20${BASH_REMATCH[6]}-${BASH_REMATCH[5]}-${BASH_REMATCH[4]} ${BASH_REMATCH[1]}:${BASH_REMATCH[2]}:${BASH_REMATCH[3]}'"
  eval $COMMAND
exit 0
}
 
while IFS= read -r line && [ $N -lt 200 ]
do
  [[ $line =~ $LINE1 ]] && FIX=${BASH_REMATCH[1]} && echo "GPS device has a fix"
  [[ $line =~ $LINE2 ]] && [[ $FIX -eq 2 ]] && extract $line
  ((N++))
done < "$input"
echo "Timed out no date/time information in 2000 lines from GPS reciver"

0 ·

25 People are following this idea.

Idea

Setting time without internet connection on Rpi Venus OS

3 Comments

Your Opinion Counts

IDEA DETAILS

Related Ideas

Modifications Space

Node Red Space