question

vulture avatar image
vulture asked

Grafana refresh rate <10 seconds?

I followed the Victron Grafana w/ Docker instructions, and set the refresh rate in Grafana to 1sec. Even though Grafana seems to be querying every second, the values only update every 10 seconds or so. I want to have a live display like in the Venus remote console. Has anyone gotten this working at a 1 second refresh rate? Any suggestions for me to try? I tried looking in the chrome dev console and it seems like queries are not taking more than 50ms. Is there some other influx or MQTT limit setting I'm unaware of?

grafana
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

5 Answers
loxley avatar image
loxley answered ·

From what you're describing, it would appear to be an ingestion rate issue, rather than a Grafana refresh rate issue.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

dunnp avatar image
dunnp answered ·

Are you doing a local connection to the Venus device or via VRM?

I have one system connected via VRM set for 15sec refresh rate and Grafana is pretty slow updating. I haven’t timed it but attributed it to VRM lag.

I have another with a direct connection but haven’t tried it at anything faster than 5sec. I’ll try it in the morning.

One other observation: runnng on a low powered PC (Intel NUC), the browser struggles to keep up. CPU runs at 100% at 5sec or less Grafana refresh rate.

Phil

2 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

vulture avatar image vulture commented ·

Local connection, although VRM is being sent data from my venus device separately. I have a Rasberry PI 2 B+ taking ve.direct inputs, which is connected over ethernet to my router. I have a Rasberry Pi 4 connected to my Wifi running Grafana and taking the MQTT packets. CPU usage on the grafana pi seems to average 50% with 1 sec refresh rate and never reaches 100.

0 Likes 0 ·
dunnp avatar image dunnp vulture commented ·

Yep - confirmed the same slow response on a direct local connection. I had VictronConnect open beside Grafana and the lag was 10 sec or so with Grafana set to 1 sec. Nowhere near ‘live’.

I’ll watch this thread with interest to see if there is a way to improve the response time.

Phil

0 Likes 0 ·
vulture avatar image
vulture answered ·

@mvader looks like another person in the thread has reproduced this. Have you encountered this, or do you have any advice before I start down the road of debugging, forking etc?

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

marekp avatar image
marekp answered ·

Any solution to this "10s" problem?

2 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

marekp avatar image marekp commented ·

The funny thing is that VRM portal is refreshing info every 2 seconds (over Internet), but my local connection to GX device every 10 seconds. :)

This local connection was suppose to be much faster then VRM over Internet.

What gives?

0 Likes 0 ·
Guy Stewart (Victron Community Manager) avatar image Guy Stewart (Victron Community Manager) ♦♦ marekp commented ·

The data on the web coming direct from realtime MQTT, and not via writing to the influx database and then reading it by Grafana.

Those two extra write/read hops add a delay.

See here for a local near-realtime MQTT dashboard made by another community member that doesn't use a database, but the same MQTT stream as the VRM realtime dashboard, except over LAN.

0 Likes 0 ·
Guy Stewart (Victron Community Manager) avatar image
Guy Stewart (Victron Community Manager) answered ·

Hi @vulture

Have you adjusted the grouped interval time and changed it from 10s to 1s?

cleanshot-2020-11-16-at-092309.jpg

There is a few second delay in displaying the data as it goes from dbus -> MQTT -> influxdb -> Grafana display. I get 1 second resolution data points like this after the write to database, query the database delay.

The write -> query -> display is an inherent part of this database design model - and if you want 'push' display data refresh delay, rather than 'pull' - you might be better looking at the MQTT stream directly - https://github.com/victronenergy/venus-html5-app

cleanshot-2020-11-16-at-101103.jpg


cleanshot-2020-11-16-at-092309.jpg (337.9 KiB)
cleanshot-2020-11-16-at-101103.jpg (165.1 KiB)
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.