question

swisssolar avatar image
swisssolar asked

vrmlogger-backlog - file too big / missing data after splitting the file

Good day

We retrieved a backlog file of 1.12GB original file, see here: https://www.dropbox.com/s/tk9lnq1lh91cntb/vrmlogger-backlog.sqlite3?dl=0 , the zip file here: https://www.dropbox.com/s/j92anffr9993n3j/vrmlogger-backlog.sqlite3.zip?dl=0 (82.7MB)

We were splitting this file and imported 2 files, but are missing the very important battery SOC data.



Here the 2 files after the split:

- https://www.dropbox.com/s/kkalkyxwoa5qyq7/vrmlogger-backlog_olddata.zip?dl=0

- https://www.dropbox.com/s/l4j2nelsmfckzns/vrmlogger-backlog.zip?dl=0

Thanks for any advise and support.

VRM
1545109063041.png (96.6 KiB)
2 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

jon-lowbank avatar image jon-lowbank commented ·

How did you split the file? I have split a file with "Split to volumes bytes" in 7-zip but VRM tells me it is not a .sqlite3 format when I upload. My file is over 2GB

0 Likes 0 ·
stuart-macdonald avatar image stuart-macdonald jon-lowbank commented ·
I also have a 1.9GB logger data file, that once compressed is still over the 64MB threshold. I am interested to know how the file can be divided into 2.
0 Likes 0 ·
3 Answers
Guy Stewart (Victron Community Manager) avatar image
Guy Stewart (Victron Community Manager) answered ·

That might be working as intended. Full VRM data is only stored for 6 months. Consumption and Production summery data is stored for 5 years, but that doesn't include battery states of charge.

See the section in the VRM FAQ here - How long is the data stored for?

I have some ideas for how this behaviour could be improved myself, and have been thinking about it for a while.

Feel free to express your experience into the editable google doc so we can work together to make it better.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Guy Stewart (Victron Community Manager) avatar image
Guy Stewart (Victron Community Manager) answered ·

You may also wish to try using the database conversion feature of VictronConnect for Windows or Mac.

https://www.victronenergy.com/live/victronconnect:start#importing_and_converting_a_gx_product_family_database_file

Though I have never used it for a dataset so big, so please let me know if it works. It might take an hour of computer thinking 💭

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

pesi avatar image
pesi answered ·

I have problems on my CerboGX. I noticed that one file occupied the entire memory. Is that normal?

'''

root@einstein:~# df

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/root 963716 835760 59296 93% /

devtmpfs 465376 4 465372 0% /dev

tmpfs 515040 908 514132 0% /run

tmpfs 515040 564 514476 0% /var/volatile

/dev/mmcblk1p5 1134336 1117916 0 100% /data

tmpfs 515040 908 514132 0% /service

root@einstein:~# ls -la /data/db

drwxr-xr-x 2 www-data root 4096 Apr 21 04:27 .

drwxr-xr-x 17 root root 4096 Apr 14 16:06 ..

-rw-r--r-- 1 root root 1105637376 Mar 9 08:04 vrmlogger-backlog.sqlite3

root@einstein:~#

'''

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.