question

p0lar avatar image
p0lar asked

Venus PI3 offline cache of data?

I have a Pi3 installed and it collects data great. when I go offline camping I was thinking that it would store this in a DB to upload then the internet returned but it does not. am I missing something? should this work? is there any way to get it to work? can I l log to a file and upload?


Venus OS
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

4 Answers
alexheylin avatar image
alexheylin answered ·

I also expected this to work but it doesn't. I think on the Cerbo(?) you need to insert a MicroSD card for the data to be stored on. As my Pi has plenty of storage, I suspect all that's really required is a mod to either link the file system directly, or create a data store partition (to emulate the separate card) and get that mounted appropriately. Perhaps when I'm less busy I'll have a play, and if I get it working I'll submit a PR.

I think logging to file and manually uploading is supported in some situations. However I've not seen and VRM GUI that suggests how to do this. I think it may be in the Cerbo manual though.


2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Warwick Bruce Chapman avatar image
Warwick Bruce Chapman answered ·

We experience the same issue with logging devices that we have built using Raspberry Pi's. We have expanded to ensure that all available SD card space can be used by Venus OS but for some reason on Pi, it does not replay logs when it comes back online.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

benc-ipind avatar image
benc-ipind answered ·

Did you get anywhere with this @AlexHeylin ?

@p0lar

@Warwick Bruce Chapman


I'm digging into this more over here:


https://community.victronenergy.com/questions/145288/gaps-in-logging-on-rpi-venus-os.html?childToView=161940#answer-161940

2 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Warwick Bruce Chapman avatar image Warwick Bruce Chapman commented ·
I suspect this is related to the Pi not having a clock. When it loses power and is restarted, time is set to 1970-01-01 00:00:00 until updated by NTP. I suspect initial records are submitted but with that timestamp.
0 Likes 0 ·
benc-ipind avatar image benc-ipind Warwick Bruce Chapman commented ·

I resolved that by installing an RTC module.

1666210434334.pngThe issue is the coded 48hr internal storage limitation. There doesn't appear to be a way to turn it off or change the limit.


To add the RTC I just went and picked up a DS3231 RTC module and have got it working. You can also use a DS1307, but the DS3231 is more accurate.


To make the RTC module work on Venus OS. The DS3231 modules work with the DS1307 kernel module.

Download the required overlays:

ds1307 rtc overlay download: https://github.com/PiNet/PiNet-Boot/raw/master/boot/overlays/ds1307-rtc-overlay.dtb


i2c-rtc overlay download: https://github.com/PiNet/PiNet-Boot/raw/master/boot/overlays/i2c-rtc-overlay.dtb

Copy the .dtb files to /u-boot/overlays

Add the following to [all] section of /u-boot/config.txt

#ds1307 rtc interface - lives on the i2c bus
dtoverlay=i2c-rtc,ds1307-rtc>

Save the changes

Go to /data and create an rc.local file then edit it and add the following line:

echo ds1307 0x68 > /sys/class/i2c-adapter/i2c-1/new_device
hwclock -s

Save and exit

Make the file executable

chmod 755 rc.local

Install the RTC kernel module

opkg install kernel-module-rtc-ds1307

Reboot and run the following command to check the RTC time

hwclock -r


0 Likes 0 ·
1666210434334.png (254.8 KiB)
alexheylin avatar image
alexheylin answered ·

@benc_ipind sort of - I tried to reproduce it recently (after a few updates to Venus) and it seems to work now but I need to be patient. It might take 15-30 mins after the Wi-Fi / Internet connection comes back online. I've had my Internet drop out for a few days a few times in the last couple of months, and as long as I don't reboot the Pi it seems to eventually pick up and send the data out. Streaming the data out can take 30-120 minutes depending how long it's been offline. I think previously it wasn't reconnecting to the Wi-Fi / didn't realise the Internet connection was available again.

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

benc-ipind avatar image benc-ipind commented ·
Sorry - I meant with getting it to write the logs to the unused storage on the SD card?


32GB is about as small a card as you can reliably get these days, so you end up with 20GB of 'wasted' space.


0 Likes 0 ·

Related Resources

Victron Venus OS Open Source intro page

Venus OS GitHub (please do not post to this)

Additional resources still need to be added for this topic