question

hobohome avatar image
hobohome asked

Venus MQTT BMS JSON Data Errors

Hi,

I have just installed a BYD battery pack onto my off grid system, replacing my ageing Lead Acid batteries. I have run into an issue with monitoring using the MQTT capability of the Venus GX.


Once the BYD battery is connected, the Venus creates a number of new MQTT topics - these include quite a few indicating fault conditions and one called "Connected" (in my case its full topic path is N/9884e3aed7f6/battery/512/Connected).

This topic delivers a payload of {"value": 1} - which is a valid JSON formatted string - great.

However, if the RS485 cable between the venus and the BYD BMS is interrupted, you would expect this value to report {"value": 0} - sadly it does not. The payload becomes NULL (ie nothing) and this is NOT valid JSON. This causes the JSON decoder (at the subscriber end) to throw en exception and causing my alerts to fail.

If values are going to be sent using JSON encoding, ALL values should be JSON encoded.

I have no way of simulating the other fault topics so I can not say if they do the same.

BMSVenus GX - VGXMQTT
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

3 Answers
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

Hi @Hobohome, thanks for reporting.

There are two issues I think:

1) that “connected” topic is a bit misleading. It originates from back in the days when we kept a dbus service online when a device was disconnected. Now we take the whole driver offline.

2) because of (1), we can’t make it send a 0. Instead it should send what we call a “DBUS-INVALID”, but then in a json valid manner.

If you have an idea; welcome to test and develop a solution & put in a patch or PR.

Background info is here:

https://www.victronenergy.com/live/open_source:ccgx:d-bus

And source code of dbus-mqtt driver is on our github. Make sure to read mqtt related issues in the venus repo.

1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

hobohome avatar image hobohome commented ·

Thanks @mvader (Victron Energy Staff) for the info - I understand.

I would really like to contribute to the project, but I have zero experience with python (I am a C++ / PHP programmer).

Question : Given that the "disappearance" of the BYD battery will lead to a shutdown after a few minutes, is there some way (via MQTT) that I can tell that this has happened?

I also think that this should be flagged as an error and notified via VRM.

I am seeing short duration (~5 seconds) disconnects about 4 times each day - this seems likely to be a BYD issue to me. Is this normal?

0 Likes 0 ·
hobohome avatar image
hobohome answered ·

Just an update on this for anyone who is interested.

The "connected" topic is now obsolete (thanks @mvader (Victron Energy Staff) for making that clear).

I have developed some code for my automation and monitoring system (OpenHab) that receives data from the root topic for the BYD battery, it then checks if it is valid JSON. If it is, it updates the values accordingly.

If any value is NOT valid JSON (ie a NULL or nothing), this is an indicator that the Venus has removed the driver for the BYD battery (because it has failed to communicate for a time???) . At this point, I flag a WARNING, wait a few seconds, then (if no valid data has arrived) change the warning to a FAULT.

In my system I am seeing about 5 disconnects per day, each of a duration of just a few seconds. Im not yest sure if this is normal, as unless you were monitoring the MQTT topics, you would not be aware of these disconnects.

If asked to guess, I would suggest that this is caused by the BYD BMU not communicating in the time-frame allowed OR the tolerance within the Venus is too tight.

I wonder if @BYD_service could comment?


8 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·

Hi @Hobohome, could you check the log of the byd?


Cat /data/log/dbus-canbusbms/* | tai64nlocal



(Directory names might be a bit different for real)

0 Likes 0 ·
hobohome avatar image hobohome mvader (Victron Energy) ♦♦ commented ·

root@beaglebone:/data/log# cat can-bus-bms.can1/*|tai64nlocal

[Lines removed]

2019-04-28 22:00:59.122768500 *** CCGX booted (4) ***

2019-05-26 06:00:26.383961500 can-bus-bms v0.15

2019-05-26 06:01:42.833095500 can-bus-bms v0.15

2019-05-26 06:04:21.084033500 *** CCGX booted (5) ***

2019-05-26 06:04:37.141800500 can-bus-bms v0.15

2019-05-26 06:18:47.087098500 *** CCGX booted (5) ***

2019-05-26 06:19:03.138891500 can-bus-bms v0.15

2019-05-26 06:30:39.087873500 *** CCGX booted (5) ***

2019-05-26 06:30:55.060035500 can-bus-bms v0.15

2019-05-26 06:37:23.093328500 *** CCGX booted (5) ***

2019-05-26 06:37:39.912613500 can-bus-bms v0.15

2019-05-26 22:02:38.721677500 can-bus-bms v0.15

2019-05-26 06:37:13.075219500 *** CCGX booted (4) ***

2019-05-30 03:50:13.202966500 can-bus-bms v0.15

2019-06-03 11:59:23.595196500 can-bus-bms v0.15

2019-06-04 01:39:01.023030500 *** CCGX booted (5) ***

2019-06-04 01:39:15.367295500 can-bus-bms v0.15

2019-06-04 01:47:43.005758500 *** CCGX booted (5) ***

2019-06-04 01:47:55.884030500 can-bus-bms v0.15


0 Likes 0 ·
hobohome avatar image hobohome mvader (Victron Energy) ♦♦ commented ·

From MQTT log

@400000005cf5cdf01130b414 -------- dbus_mqtt, v1.12 is starting up --------

[lines Edited]

@400000005cf5cdf2085366ac INFO:dbus_mqtt:[Scanning] service: com.victronenergy.solarcharger.ttyO2

@400000005cf5cdf23a1fbce4 INFO:dbus_mqtt:[Scanning] service: com.victronenergy.solarcharger.ttyO4

@400000005cf5cdf32ccdf68c INFO:dbus_mqtt:[Scanning] service: com.victronenergy.system

@400000005cf5cdf334c08e04 INFO:dbus_mqtt:[Scanning] service: com.victronenergy.qwacs

@400000005cf5cdf337e69d94 WARNING:dbus_mqtt:[Scanning] com.victronenergy.qwacs does not provide an item listing

@400000005cf5cdf4023c2e84 INFO:dbus_mqtt:[Scanning] service: com.victronenergy.logger

@400000005cf5cdf406b8e0ec INFO:dbus_mqtt:[Scanning] service: com.victronenergy.fronius

@400000005cf5cdf4097e61bc INFO:dbus_mqtt:[Scanning] service: com.victronenergy.settings

@400000005cf5cdf415ea198c INFO:dbus_mqtt:[Scanning] service: com.victronenergy.solarcharger.ttyUSB0

@400000005cf5cdf51638f50c INFO:dbus_mqtt:[Scanning] service: com.victronenergy.battery.socketcan_can1

@400000005cf5cdf51a7e7e5c INFO:dbus_mqtt:[Scanning] service: com.victronenergy.vebus.ttyO5

@400000005cf5cdf52d289d94 INFO:dbus_mqtt:[Scanning] service: com.victronenergy.temperature.builtin_adc5_di0

@400000005cf5cdf52faabc64 INFO:mqtt_gobject_bridge:[Init] Connecting to local broker

@400000005cf5cdf5306c4744 INFO:dbus_mqtt:[Connected] Result code 0

@400000005cf5cdf6308f7afc INFO:dbus_mqtt:[Message] Connected to cloud broker

@400000005cf5ce1603bd4c5c INFO:dbus_mqtt:[KeepAlive] Received request, publishing restarted

@400000005cf617d3056f5d74 INFO:dbus_mqtt:[OwnerChange] Service disappeared: com.victronenergy.battery.socketcan_can1

@400000005cf617d90e59f534 INFO:dbus_mqtt:[Scanning] service: com.victronenergy.battery.socketcan_can1


Last two lines repeat many times suggesting that the battery is disappearing regularly








0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ hobohome commented ·

Hi, yes that MQTT log is a good one to look at.

The reason for the service being taken offline is that the dbus-canbusbms driver concludes that the battery is no longer connected. Something with a time out; indeed. We'll check if we see the same thing on another system.

Note that this won't have the highest priority on our side; since functionally all works well.

0 Likes 0 ·
hobohome avatar image hobohome mvader (Victron Energy) ♦♦ commented ·

There may be another factor at play here. Last night (for the second time - its only two weeks old) we had a system shutdown - the BYD B-Box giving an error of "Internal Failure" . One of the batteries was showing an error light and would not reset (and the Quattro would not Invert). While I await a replacement, I have re-addressed the remaining 3 batteries and we are running on 3/4 of our storage.

Interestingly, the BYD reported SOC% went from 50% to 65% when I brought the system back online - this suggests that the faulty battery was not reporting its full capacity.

I am left wondering if that faulty battery has been responsible for the unexpected disconnects. Time will tell!

0 Likes 0 ·
hobohome avatar image hobohome mvader (Victron Energy) ♦♦ commented ·

A further update on this issue.

It is NOT related to a faulty battery (now disconnected, awaiting replacement).

The BYD battery driver is being taken off-line about 3 - 4 times per day for a few seconds each time.

To reproduce the problem:

BYD firmware is - Module V2_V4-4-11 (4-Dev-2018) BMU V2_V4-23_2-Apr-2019 (the disconnect issue was also evident in previous firmware)

Venus is 2.30 (the disconnect issue was also evident in firmware 2.23)

Assuming that the Venus is not getting a check-in from the BYD BMU within a timeout period - it may just be a case of increasing the tolerance?

I realise that this is not a high priority, but it would be nice to resolve, especially if it is just a case of raising a tolerance value.

Please just let me know if I can be of any help in testing a solution.

0 Likes 0 ·
hobohome avatar image hobohome mvader (Victron Energy) ♦♦ commented ·

@mvader (Victron Energy Staff) Just wondering if this fix is still on the to-do list?

The BYD disconnections are still happening every few hours (latest Venus OS)

0 Likes 0 ·
Rene Mullie avatar image Rene Mullie commented ·

hi HoboHome,

I've read al the topics concerning MQTT here, on MQTT I am still a NOOB (although I have some sensors based on MySensors over MQTT up&running)

I would like to connect my Rpi based Venus to my OpenHab Rpi setup, I am struggeling to get the basics working... Based on your postings I thing you ar quite a Pro on this.

Would you be so kind to share a HowTo based on the OpenHab/MQTT 2.4 bindings/things for setting up basic connection between Venus and OpenHab?

I think there is quite some interest in this from the Victron as wel the OpenHab community.

0 Likes 0 ·
byd-service avatar image
byd-service answered ·

Hello @hobohome,


I will not comment on the details of the communication as @mvader is the expert and he will let us know if something needs to be modified.


What I can say is that even if this were happening in the background we wouldn't notice as it doesn't have any apparent impact in the function of the system.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.