question

Daniel Weber avatar image
Daniel Weber asked

Venus OS 3.20 and MQTT fails!

Hi,

i´ve updated to Venus OS 3.20 and since no MQTT traffic comes via LAN!

I´ve have also checked the cerbo gx with mqtt explorer. I can only see $SYS, N/mySerial/system/0/serial but nothing else.

Previous Version was 3.14 and it worked all fine!

regards

Daniel


Venus OS
2 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Daniel Weber avatar image Daniel Weber commented ·

OK, reminder to myself... its a venus os 3.20 problem!

FW roleback to 3.14 and everything is fine in mqtt explorer and all related systems after venus os.


with 3.20 mqtt-explorer-with-venus-os-320.png


back with 3.14

mqtt-explorer-with-venus-os-314.png



0 Likes 0 ·
ckvsoft avatar image ckvsoft commented ·
I went back to version 3.14.
With 3.20 the connection works, but there are a lot of delays and timeouts in the responses. From version 3.10 to 3.14 everything worked without any problems.
It doesn't matter whether I try it locally on the Raspberry or via the VRM. As of 3.20 the responses do not work as expected.
 
                  
0 Likes 0 ·
9 Answers
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

UPDATED 2024-04-20, Venus OS v3.31 release - fixes empty username login issue
UPDATED 2024-02-27, after Venus OS v3.22 release.

Hey all,

An update regarding all MQTT issues that I've seen reported:

1) 2024-02-19: official release of Venus OS v3.21; which fixes the issue concerning writing negative numbers over the MQTT protocol.

2) 2024-02-27: official release of Venus OS v3.22; which fixes the issue concerning systems where the GX receives many writes over the MQTT protocol. Fixes the issues reported by @Drogos and @christian.gurk; except something with three phase systems that we're looking into.

3) 2024-04-20: official release of Venus OS v3.31, scheduled next Monday, fixes the issue that while anonymous login with no user name works fine, using an empty username did not. For further details, see here: https://github.com/victronenergy/venus/issues/1257.

4) @GITWeber reported "i´ve updated to Venus OS 3.20 and since no MQTT traffic comes via LAN! I´ve have also checked the cerbo gx with mqtt explorer. I can only see $SYS, N/mySerial/system/0/serial but nothing else.". I suspect the issue there is something to do with keep alives.

3 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

heman180 avatar image heman180 commented ·

I am still struggling to connect Home Assistant (running on a PI) to connect/receive MQTT traffic from Cerbo after Venus OS 3.21 upgrade.

MQTT Explorer message is "disconnected from server". Port 1883 is used. Obviously a role back to 3.14 is working.

0 Likes 0 ·

Hey all, Venus OS v3.22 was released officially just now. I have updated above accepted answer to latest status.

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ mvader (Victron Energy) ♦♦ commented ·
Hey all, since Venus OS v3.31, currently in beta we have v3.31~4, the empty username login is fixed. See above updated message for details.


That was the last known issue aka regression related to FlashMQ - all is in the green now!

0 Likes 0 ·
Matthias Lange - DE avatar image
Matthias Lange - DE answered ·

Maybe related to this:

1707849880330.png

https://github.com/victronenergy/venus/issues/1098

3.20 was in beta for over 4 months.


1707849880330.png (19.4 KiB)
1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Daniel Weber avatar image Daniel Weber commented ·

Matthias thanks a million for information. But this will be the next winterproject...

Now my pv-projects are starting.


Grüße

Daniel

0 Likes 0 ·
drdiesel avatar image
drdiesel answered ·

I am currently reading a ton of data via this method, just a snip of the code. Any idea if the changes associated with MQTT and/or 3.20 might break this functionality?


import mysql.connector
import time
from pymodbus.constants import Defaults
from pymodbus.constants import Endian
from pymodbus.client.sync import ModbusTcpClient as ModbusClient
from pymodbus.payload import BinaryPayloadDecoder
 
Defaults.Timeout = 25
Defaults.Retries = 5
client = ModbusClient('192.168.2.23', port='50

result = client.read_input_registers(840, 1)
decoder = BinaryPayloadDecoder.fromRegisters(result.registers, byteorder=Endian.Big)
batteryvoltage=decoder.decode_16bit_uint()
print("Battery voltage: {0:.2f}V".format(batteryvoltage/10.0))
1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

What I see in your code looks like ModbusTCP and that is not related to MQTT.
1 Like 1 ·
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

Hey all,

This must be something specific:

- html5 app uses MQTT and works fine in v3.20

- gui-v2 uses MQTT and works fine in v3.20

- VRM real time info uses MQTT and works fine in v3.20

- the MQTT related changes have been in beta for months and I know many beta users that use it for various things; and it works for them.


But for in your case its clear that something broke or something changed, so now we need to find out what.


How are you sending your keep alives?

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hey again @GITWeber , I don’t think that v3.30~6 fixes your issue.


My guess is that its related with how you send you keep alives. If you send those at all: that second screenshot doesn’t show any keepalive.


See here for documentation: https://github.com/victronenergy/dbus-flashmq/blob/master/README.md

1 Like 1 ·
eeghauptmann avatar image
eeghauptmann answered ·
Hi, i have also a Problem with Mqtt since Update 3.20 keepalive works fine for me …If i send a set Point for AC/L1 that is positiv zb 500 W it works fine but if send -500w nothing happens ..


Ess is on external Controll



7 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hey all, an update, there is indeed a bug:


Writing negative values indeed doesn't work, in some cases.

More details later - we're working on this and there will be a fix shortly.

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ mvader (Victron Energy) ♦♦ commented ·

Hi @EEG.Hauptmann , your issue is fixed per v3.30~6. Available as beta, see here:

https://community.victronenergy.com/questions/262520/venus-os-v3304-available-for-testing.html


Can you verify that it is also fixed for you?

0 Likes 0 ·
pelements avatar image pelements mvader (Victron Energy) ♦♦ commented ·

Hi @mvader (Victron Energy) thank,s for your fast Answer


I install it on my Privat Installation for testing I will give you a report tomorrow

Are you there any Problems with BETA Version becouse i have to fix it on 2 customers Instalations ..

ps EEG Hauptmann is the Account of My Comany



0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ pelements commented ·
Hi @p.elements , I’m not aware of any problems with the beta version.


Once we have confirmation, we’ll probably release this fix real soon as v3.21 official release. Ie next week or week thereafter.

0 Likes 0 ·
pelements avatar image pelements mvader (Victron Energy) ♦♦ commented ·

Hi @mvader (Victron Energy) i test it on my System i now it locks good if i find any problem i will post it

screenshot-2024-02-15-at-22-13-06-revpi-regelung-d.pngThanks

0 Likes 0 ·
hominidae avatar image hominidae mvader (Victron Energy) ♦♦ commented ·
...for what its worth, I have no problem with publishing negative numbers in ESS, like grid-setpoint when using a mqtt bridge on v3.20
0 Likes 0 ·
pelements avatar image pelements hominidae commented ·

Hello that,s two different Thing’s .

Publish the Gridsetpoint was not the Problem , Publish /vebus/276/Hub4/L1/AcPowerSetpoint negative was the Problem .

0 Likes 0 ·
ceriw avatar image
ceriw answered ·

Hi All,

I had a similar issue https://community.victronenergy.com/questions/262793/mqtt-fail-to-connect-320.html

But that post has now been locked. I've since installed 3v30~6 and the issue still persists, so it looks like my issue is a similar bug, but not the same one.1708094787254.png1708094762299.png1708094903605.png


Roll back to v3.14 and all works again, with no other changes

1708094950342.png

1708095072360.png

1708095113732.png


Thanks


1708094762299.png (16.3 KiB)
1708094787254.png (12.6 KiB)
1708094903605.png (8.7 KiB)
1708094950342.png (11.0 KiB)
1708095072360.png (15.4 KiB)
1708095113732.png (61.2 KiB)
5 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hey @Ceri.w is it ok if an engineer logs in?


I see you have remote support enabled.


Is it your system called “home”?

0 Likes 0 ·
ceriw avatar image ceriw mvader (Victron Energy) ♦♦ commented ·
Hi,

yes, sure.


Thanks

Ceri

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ ceriw commented ·

Done, the device you’re using seems to not comply to one of the MQTT protocol spec requirements, this is the error we see in the logs:


[ERROR] Unspecified or non-MQTT protocol error: Username flagged as present, but it's 0 bytes.. Removing client.


That's an explicit check the spec requires. And our new broker service is apparently more strict in that than the old one we had in Venus OS.

I recommend contacting the company or people behind the device that tries to connect.

0 Likes 0 ·
Show more comments
drogos avatar image
drogos answered ·

I also have issues with the newest version, so rolled back for now. When using the newest version with my simple node.js script it works sometimes, for a while. Then I need to reboot the ccgx, and it might start working again, or I have to reboot once more. If Victron want the script, or access to the rpi that is running it, so they can test, let me know how to share.

13 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hey @Drogos is it OK if we login to your system? I see you have Remote Support enabled.
1 Like 1 ·
drogos avatar image drogos mvader (Victron Energy) ♦♦ commented ·
Of course! Keep in mind, I am now running the previous version. (The new version also seems to trigger my inverter to turn on when I reboot the ccgx, the old one does not mess with inverter state by it's own) I am available in teams / discord / etc if you want quick communication / access to the internal rpi that is running my custom script, etc, etc. Anything I can make available for field testing, I am in :)
0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ drogos commented ·

Hey @Drogos fixed per v3.30~7 ,available as public beta per just now.


Details here: https://community.victronenergy.com/questions/262520/venus-os-v3304-available-for-testing.html.



1 Like 1 ·
drogos avatar image drogos mvader (Victron Energy) ♦♦ commented ·
Awesome! Installed. Worked on initial install! Will let it run a while and see if it behaves nice, and get back to you with result :)
0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ drogos commented ·
Yes, please do!
0 Likes 0 ·
drogos avatar image drogos mvader (Victron Energy) ♦♦ commented ·

Done some testing on the newest version, and it seems stable with one inverter connected. How ever, if I connect multiple inverters, mqtt stops working again. Also, after reboot, the inverter is automatically turned on, even if it was off before the reboot. This might be different issues tho, not belonging in this thread. For the record, these are issues that was introduced with the 3.20 version.

0 Likes 0 ·
beliffm avatar image beliffm drogos commented ·
Same here. With neither V3.21 nor V3.30~7 MQTT works in a 3Phase / Inverter environment
0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ beliffm commented ·
Hey both @Drogos and @beliffm , thanks for the reports, we'll look into that.


0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ mvader (Victron Energy) ♦♦ commented ·
Hey @Drogos can we login somewhere to see the 3 phase system not working?


ps. we tried to contact you on the email address that is in your profile, but doesn't work.

0 Likes 0 ·
drogos avatar image drogos mvader (Victron Energy) ♦♦ commented ·
Hmm, that is strange! :o The email should work just fine. I checked my span folder also, just in case, but see nothing. Try again is all I can think of?


You can log into my system as much as you please :) I do however not have a three phase system. I have multiple inverters of different power that is all on their own "bus" and not linked in any way. They serve different purposes. atm, I only have one connected to make sure mqtt is stable. But, at your request, I can connect the others and also open up the raspberry pi with the nodejs script that is doing the mqtt calls. Just unfortunate that the email does not work. Very strange, and I can see no reason for it :/

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ drogos commented ·
Ok I've sent an email just now @Drogos
0 Likes 0 ·
tehre avatar image tehre mvader (Victron Energy) ♦♦ commented ·
i just updated my cerbo to 3.22, 3 phase 5k multi.

Let me know if you want to connect with me.

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ tehre commented ·
Hey @tehre thanks.


Are you having the problem? What problem exactly?


And from what kind of system are you writing to MQTT on the GX?

0 Likes 0 ·
tehre avatar image
tehre answered ·

@mvader (Victron Energy)

i tried to use the gui-beta, so enabled the MQTT. Doenst work. Might be due to the same issue. Nothing else specific for MQTT is currently in place.


I just wanted to provide you a 3p env with latest cerbo version if this might help to investigate.

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hi @tehre , ok thanks. That must be another issue, ie not what is being discussed here on this thread.


Gui-beta works well, also in three phase systems. We have many three phase test systems where it works well; as do customers. Concluding, I'm not sure what your issue, and can't help myself with that at the moment; please await some further beta versions - thanks!

0 Likes 0 ·
Daniel Weber avatar image
Daniel Weber answered ·

Hi all,

thanks for being in touch for fixing MY problem.

I´ve a three phase system here and i use the mqtt data in my smarthome system (knx with edomi).

We have many internet outages here and in this moment mqtt sends no data via mqtt to edomi. Since i´ve figured it out, i´ve fixed it with edomi by triggering it with a keepalive to the cerbo gx every 60 seconds.

I will clearly say i dont use username and password between edomi and cerbo gx! In Edomi its possible to set username and password. But where in the cerbo gx device could i set this? I can imagin with ssh and putty... addon comment: Better to know where to set in Venus OS 3.22 to use it with flashMQ!

Whats not clear to me is flashMQ active parallel to mosquitto on the cerbo gx device in Venus OS >3.14? Are there two MQTT systems on it? One for VRM and one for personal use?

I am not afraid to test it here at my site. I will update 2 of my 8 victron customersites in the next two weeks because of "normal" installation without smarthome or other implementations. In June i have to build a "outback" site here in Germany and there i must use mqtt data for an intelligent loadmanagement.


regards

Daniel


1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hi again!


> We have many internet outages here and in this moment mqtt sends no data via mqtt to edomi. Since i´ve figured it out, i´ve fixed it with edomi by triggering it with a keepalive to the cerbo gx every 60 seconds.

> I will clearly say i dont use username and password between edomi and cerbo gx! In Edomi its possible to set username and password. But where in the cerbo gx device could i set this? I can imagin with ssh and putty... addon comment: Better to know where to set in Venus OS 3.22 to use it with flashMQ!


I'm not sure if I fully understand what I you want to explain. What I do know is that, to work around the passwordless connection issue, see also above answer in green, that happens on certain systems, setting a (dummy) password in the connecting software, in your case edomi/knx, is sufficient. Its it *not* needed to set the same password in the GX.

Further details in above accepted answer (green) as well as in the MQTT on Venus OS documentation, see the migration chapter.


> Whats not clear to me is flashMQ active parallel to mosquitto on the cerbo gx device in Venus OS >3.14? Are there two MQTT systems on it? One for VRM and one for personal use?

There is one broker, FlashMQ. Per v3.20 we've removed Mosquitto and replaced it with FlashMQ.

0 Likes 0 ·

Related Resources