Idea

laurenceh suggested Jul 24, '20

Setting time without internet connection on Rpi Venus OS

As I will be using my Venus OS Rpi3B+ on my boat (Lady's Smock) I will not always have an internet connection when I switch the Raspberry Pi on, and the Raspberry Pi does not have an RTC.

I wonder if others have this problem in remote or mobile (motorhome/boat) installations?

In my case I will always be using GPS and should be more likely to have a GPS data fix than an internet connection.

The script here sets the data and time from the GPS data. It waits around 100 seconds for the GPS device to be found (or start up) and collects up to a hundred lines from the GPS device looking to check that

(1) It has a satellite fix

(2) It can provide the data and time

Once decoded it automatically sets the Rpi date and time according to the GPS data.

It has been tested with Venus OS 2.57 in July 2020.

I will look at adding it to /data/rc.local so that it runs at start up. I would not at this stage suggest running it continuously to provide updates as it might conflict with the NTP server system used by default in Venus OS.

The location of the GPS device "/run/serial-start/gps/ttyACM0" is hard wired at present and may need to be changed if you are using an NMEA GPS interface.

It relies on GPGGA and GPRMC data packets from my GPS device - I presume these are pretty standard across devices.

#!/bin/bash
N=0;
FILE=/run/serial-starter/gps/ttyACM0
echo "Waiting for GPS data"
while [ ! -e $FILE ] && [ $N -lt 100 ];
do
    sleep 1;
    ((N++))
    echo -n "."
done;
sleep 1;

if [ $N -ge 99 ]; then
   echo " GPS data not available is the device plugged in"
   exit 0;
fi

echo
echo "Device attached waiting for data"

input=$FILE
N=0

# check if GPS has FIX
LINE1='^\$..GGA,[0-9\.]+,[0-9\.]+,[NS],[0-9\.]+,[EW],([0-2])'
# Catch lines with date and time info
LINE2='^\$GPRMC'

FIX=""

# extract date and time from GPS packet

function extract {
[[ $1 =~ ^\$GPRMC,([0-9]{4})([0-9]{2})\.[0-9]{2},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}), ]]
  COMMAND="${BASH_REMATCH[5]}${BASH_REMATCH[4]}${BASH_REMATCH[3]}${BASH_REMATCH[1]}.${BASH_REMATCH[2]}"
  date -s $COMMAND
exit 0
}

while IFS= read -r line && [ $N -lt 100 ]
do
#  echo "$line"
  [[ $line =~ $LINE1 ]] && FIX=${BASH_REMATCH[1]} && echo "GPS device has a fix"
  [[ $line =~ $LINE2 ]] && [[ $FIX -eq 1 ]] && extract $line
  ((N++))
done < "$input"

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

laurenceh commented Jul 25, '20

This now works from /data /rc.local on my Raspberry Pi.

Add the following lines to /dat/rc.local

if test -f /data/venus-rpi-setup/gps.sh; then
 /data/venus-rpi-setup/gps.sh &>/var/volatile/log/gps-time &
fi

Clearly you can change the location where you store the script and where you want the log to go.

Notes:

1) Don't forget the & at the end of the line it is important to make sure it runs as a background process otherwise it stops the rest of the initialisation.

2) You can't rely on standard output to log the output information. The standard boot output logging which goes to /var/log/boot can be shut down before the script terminates so you lose output messages.

3) I changed this line in the original script to read 200 lines as sometimes the script terminated before finding the data it needed in the GPS data stream.

while IFS= read -r line && [ $N -lt 100 ]

Change to:

while IFS= read -r line && [ $N -lt 200 ]

It would be nice if Victron added something like this into the start up sequence as they already detect cases where there in no hardware clock and should be able to detect if there is no route to an NTP server.

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

natteverf commented Sep 20, '22

Hi
does this script still work ok?
I get multiple "GPS device has a fix" followed by "date: invalid date '.'"

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

johnny-brusevold commented · Sep 20, 2022 at 10:52 PM

Tested it and got it working with some changes due to my usb-gps (ttyUSB3) using the $GNRMC string.

I didn't get "date -s $COMMAND" to work either, so I changed the code a bit.

Pi3b+ venus-os 2.90

#!/bin/bash
N=0;
FILE=/run/serial-starter/gps/ttyUSB3
echo "Waiting for GPS data"
while [ ! -e $FILE ] && [ $N -lt 100 ];
do
    sleep 1;
    ((N++))
    echo -n "."
done;
sleep 1;
 
if [ $N -ge 99 ]; then
   echo " GPS data not available is the device plugged in"
   exit 0;
fi
 
echo
echo "Device attached waiting for data"
 
input=$FILE
N=0
 
# check if GPS has FIX
LINE1='^\$..GGA,[0-9\.]+,[0-9\.]+,[NS],[0-9\.]+,[EW],([0-2])'
# Catch lines with date and time info
LINE2='^\$GNRMC'
 
FIX=""
 
# extract date and time from GPS packet
function extract {
[[ $1 =~ ^\$GNRMC,([0-9]{2})([0-9]{2})([0-9]{2})\.[0-9]{3},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}),.* ]]
  COMMAND="date -s '20${BASH_REMATCH[6]}-${BASH_REMATCH[5]}-${BASH_REMATCH[4]} ${BASH_REMATCH[1]}:${BASH_REMATCH[2]}:${BASH_REMATCH[3]}'"
  eval $COMMAND
exit 0
}
 
while IFS= read -r line && [ $N -lt 200 ]
do
  [[ $line =~ $LINE1 ]] && FIX=${BASH_REMATCH[1]} && echo "GPS device has a fix"
  [[ $line =~ $LINE2 ]] && [[ $FIX -eq 2 ]] && extract $line
  ((N++))
done < "$input"

0 ·

natteverf johnny-brusevold commented · Sep 22, 2022 at 03:18 PM

Thanks for your help

I am not getting an error now, but i have no idea if it is really working, i created the rc.local entry as you said and there is an output in /var/volatile/log/gps-time of multiple "GPS device has a fix" after a restart

0 ·

johnny-brusevold natteverf commented · Sep 22, 2022 at 06:18 PM

If everything is ok, you get this when you run the bash script from terminal, and the same in /var/volatile/log/gps-time

root@raspberrypi2:~# /data/GpsClock/gps-clock.sh
Waiting for GPS data

Device attached waiting for data
GPS device has a fix
Thu Sep 22 17:28:00 UTC 2022

You probably have to edit line 33 $GxRMC to match your string from gps

Mine is $GNRMC for multi (GPS, Galileo and GLONASS)

$GNRMC,174721.000,A,5993.123983,N,01074.878083,E,0.01,148.82,220922,,,D*7C

And this code picks data from gps string

[[ $1 =~ ^\$GNRMC,([0-9]{2})([0-9]{2})([0-9]{2})\.[0-9]{3},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}),.* ]]

You get $RxRMC data from your GPS.

cat /dev/your-GPS |grep RMC

0 ·

natteverf johnny-brusevold commented · Sep 23, 2022 at 08:12 PM

ok, the output of /var/volatile/log/gps-time is this:

Waiting for GPS data

Device attached waiting for data
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix

which i guess looks like it is working.
one of my $RxRMC data lines looks like this:

$GPRMC,201015.000,A,5222.7424,N,00458.1954,E,0.00,356.24,230922,,,D*60

do i need to change lines 27 and 33 to match?

0 ·

natteverf natteverf commented · Sep 24, 2022 at 11:37 AM

I changed line 33 but got the same results, so i changed line 27 also and it is working properly now :-)

Thanks @Johnny Brusevold for your help with this.
I am not a scripting guru, but would it not be possible to make the script a bit more agnostic by somehow saying G*RMC? Or are there more options that can appear there? I am also thinking about maybe detecting the usb port because mine has changed for some reason sometimes on a restart. Also a Pi3 w venus-os 2.90 large

0 ·

laurenceh commented Oct 7, '22

I wrote the original code and guess I can fix it.

I have a GitHub repository and my new services package of services there has the fix for the date command format in the line that starts COMMAND=

Basically I wrote the code in 2019, at that time a date format yymmdd was sufficient but somewhere after 1/1/2020 the format was changed to yyyymmdd so the code needed to “20” to be added to the command ( it should now work until 1/1/2100 by which time non of us will care ;-). I’m Not sure if the format change was just as the year changed or because of a change in the base code of Venus OS.

Either way new code on GitHub published a few months ago fixed that.

The code looping saying GPS has a fix, is NOT an indication of working. The code is just looping saying it has a fix but still can’t find the line with the time/date information. This is probably due to the noted GxRMC problem.

I’ll fix the code on GitHub to recognise GxRMC but others will need to test it.