Other solution all the way down

OK since I was having the same issue but the answers were a bit cryptic here is the solution. Most of this information and other things can be found on the main GitHub page https://github.com/victronenergy/dbus-mqtt

Make sure MQTT is turned on in the CCGX service menu.

Commands included are for windows, they are almost identical for other systems.

1- Find the right server

Victron changed the mqtt servers from a single mqtt.victronenergy.com to multiple instance mqttXX.victronenergy.com where XX is a number from 0-127.

To be able to access your data, first step is to figure out which broker to talk to. For this you need you VRM portal ID, you can get that from the VRM portal in settings, on CCGX or by polling the local MQTT.

With the VRM portal ID you shove it in the code described here : https://github.com/victronenergy/dbus-mqtt#determining-the-broker-url-for-a-given-installation

Here is an online version of the script, just replaced the portal id string at the top with yours and it will print the right broker:

https://onlinegdb.com/ByqR27fEL

OK, so now you have a number and you know which MQTT broker to connect to.

Fun Fact : if you connect to mqtt.victronenergy.com and not mqttXX.victronenergy.com it will succeed but there won't be any data beyond the serial no matter what you do.

2 - Download the certificate file from here : https://github.com/victronenergy/dbus-mqtt called

venus-ca.crt

3 - Connect to the Victron VRM MQTT with the following command

mosquitto_sub.exe -v -I myclient_ -t '#' -h mqttXX.victronenergy.com -u <your_username> -P <your_password> -p 8883 --cafile .\venus-ca.crt

Replace the XX with your broker and your username and password (those you use to connect to the VRM portal) with yours. make sure to point to the certificate file downloaded in step 2.

Alternatively use the GUI MQTT.fx, settings are pretty much the same, enable SSL/TLS and select TLSv1.2 and the CA certificate file from step 2.

OK so now you are connected and can read a single value, the VRM portal ID.

Where is your data you ask? Well that the whole keep-alive thing. The CCGX isn't reporting any MQTT values back to VRM until you wake it up even though you turned on the setting on the CCGX.

4 - Ok so let's poll the CCGX locally every 5 seconds to keep him awake.

On the github page they give a nice script, basically publish a message with nothing in it regularly.

https://github.com/victronenergy/dbus-mqtt#keep-alive

Here is my powershell equivalent

while (1){  .\mosquitto_pub.exe -m ' ' -t 'R/YOUR_PORTAL_ID/system/0/Serial' -h LOCAL.IP -p 1883 ; sleep 5;}

To do this you need to be on the same local network as the CCGX and enter its IP address after -h

If you are not on the same LAN you can spam the online MQTT for similar results, values are identical as those from step 3:

while (1){ .\mosquitto_pub.exe -m ' ' -t 'R/YOUR_PORTAL_ID/system/0/Serial' -h mqttXX.victronenergy.com -u <your_user> -P <your_password> -p 8883 --cafile .\venus-ca.crt ; sleep 5;}

Now all of a sudden you're going to get flooded with messages on the MQTT subscription from step 3. \o/

You can't stop the polling or after 60 seconds the data is going to stop, find a way to keep that script running on some machine.

My needs are to poll once every 5 minutes from my HassIO installation, will now see how to tie it all nicely together.

Can someone explain the whole keep-alive design decision?

That MQTT isn't on by default I get, but if I want MQTT on why should I have to spam the server to keep it going? Wouldn't an only local always on MQTT or VRM with a set interval reporting like the VRM make more sense?

[UPDATE AND SOLUTION]

OK, so want you want to control the MQTT to do things nicely?

So, really I don't want to be running a polling script that also spams victron MQTT when all I want is to know the percentage of charge of my battery...

This is definitely hacky and won't survive a firmware update, I'll look into making something more solid later but it gives you full control of MQTT behavior.

1 - Get super user and SSH into your Venus

2 - Change MQTT bridges

vi /usr/sbin/start-mosquito

2.a - Add your own MQTT bridge file

Add this line so that it will copy your own MQTT bridge configuration file, change cloud_mqtt_bridge.conf to something that makes you happy. We will create this file later.

ln -sf /data/conf/mosquitto.d/cloud_mqtt_bridge.conf /run/mosquitto/cloud_mqtt_bridge.conf 

2.b [OPTIONAL] stop updating victron VRM:

If you want to get rid of the Victron VRM MQTT and setup your own or just have it ignore the VRM on local polling comment this line:

#    ln -sf $bridge_conf $bridge_link 

So this looks something like Before:

if [ "$(get_setting /Settings/Services/MqttVrm)" = 1 ]; then                                     ln -sf $bridge_conf $bridge_link                                                     else                                                                                          rm -f $bridge_link                                                                       fi   

And After having also disabled victron:

if [ "$(get_setting /Settings/Services/MqttVrm)" = 1 ]; then                                 #ln -sf $bridge_conf $bridge_link                                                           ln -sf /data/conf/mosquitto.d/cloud_mqtt_bridge.conf /run/mosquitto/cloud_mqtt_bridge.conf else                                                                                         rm -f $bridge_link                                                                       fi   

3 - Make the MQTT keep alive always

Edit this file:

vi /service/dbus-mqtt/run  

And add the --keep-alive 0 option at the end

Before:

exec softlimit -d 100000000 -s 1000000 -a 100000000 /opt/victronenergy/dbus-mqtt/dbus_mqtt.py --init-broker

After:

exec softlimit -d 100000000 -s 1000000 -a 100000000 /opt/victronenergy/dbus-mqtt/dbus_mqtt.py --init-broker --keep-alive 0  

4 - Create your own bridge configuration:

Create the file you named in step 2.a, in my case:

vi /data/conf/mosquitto.d/cloud_mqtt_bridge.conf

Add to it your mqtt bridge config, this one is for cloudmqtt

connection cloudmqtt address mXX.cloudmqtt.com:17342 remote_username <Username> remote_password <Password> clientid <ID> try_private false start_type automatic topic N/VRMID/system/0/Dc/Battery/Soc out

Ok so, change username password, address... to fit your config and then we need to chose what is going to get sent.

[option 1] sync everything to your new broker, ok if you are doing this locally or have no bandwidth limit on the connection + broker

topic # out

[option 2] Or choose the items you're interested in, add as many lines as you want

topic N/VRMID/system/0/Dc/Battery/Soc out

Conclusion

So this will solve most issues as you can configure however you like.

This has no impact on the VRM portal functionality and also if you disable report MQTT to Victron is not spammy.

For me this solves all my needs, getting the 2-3 values I wanted for home assistant to take some decisions. If you want to control the CCGX from MQTT just add some in parameters.

I do recommend removing the victron MQTT bridge unless you need it as it will send a significant amount of data every second. Or if you want to use their MQTT only export the parameters you really need instead of everything.