question

Quick update, I added support for Tank level sensors. I have run some simulated tests and all is looking like it works OK with both temperature and tank sensors.

I'm running a demo mode on the Venus device and getting some DeviceInstance number overlaps with demo devices. Hopefully that is just a quirk of the demo mode not registering devices correctly with the dbus. When I get some time I'll look into hooking up a tank level sensor to my Arduino.

Hi,
thank you very much for your work and the documentation on github.
I will try to replicate it with a Tasmota or espEasy device.
My goal is to register a Sonoff device via MQTT as PV inverter.
Best regards,
Sebastian

Hi StundenBlume, Have you figured out which paths on the dbus you would need to push values to for your use case?

Hi,
first I will to follow your example. Than I will look into that.
But before all that I have to figure out why my venusOS don't recognize my Pylontech battery.

here is my experience: Installation without problem but I didn't get it to work.
What I did:

• Install like you documented, no problem
• reboot
• connected to venusOS with MQTT client id "b815"
• subscribed to "device/#"
• published to "device/b815/Status" payload "{ "clientid": "b815", "connected": 1, "version": "v1.0 ALPHA", "services": {"t1": "temperature"} }"
• no response under "device/#", don't get the "DeviceInstance message"
• published to "device/b815/DeviceInstance" Payload "{"t1": 5}"

I expected to see "W/<portal id>/temperature/<device instance>/Temperature" to show up.

Did I messed something up?

Hi Stundenblume, thanks for trying it out and giving feedback.

First thing to check is that the dbus-mqtt-devices service is running, from the ssh command line use

$ svstat /service/dbus-mqtt-devices

if the service is running, then next thing to check is the log with the command:

$ more /var/log/dbus-mqtt-devices/current

It should contain something like this:

@400000006238ead134c233e4 INFO:device_manager:Received device status message {'clientId': 'fe001', 'connected': 1, 'version': 'v1.0', 'services': {'t1': 'temperature'}}
@400000006238ead134c25324 INFO:device:**** Registering device: fe001, services: {'t1': 'temperature'} ****
@400000006238ead134c25edc INFO:device:Registering Service temperature for client fe001
@400000006238ead134c26a94 INFO:device_service_config:About to open config file
@400000006238ead136d95fcc INFO:device_service:Unregistered mqtt_fe001_t1 from dbus
@400000006238ead136df10d4 INFO:device_service:Unregistered mqtt_fe001_t1 from dbus
@400000006238ead136ea9ddc INFO:device_service:Unregistered mqtt_fe001_t1 from dbus
@400000006238ead13755bbbc INFO:device_service:Registering service temperature for client fe001 at path com.victronenergy.temperature.mqtt_fe001_t1
@400000006238ead13903b20c INFO:settingsdevice:Setting /Settings/Devices/mqtt_fe001_t1/ClassAndVrmInstance does not exist yet or must be adjusted
@400000006238ead13a94dd44 INFO:vedbus:registered ourselves on D-Bus as com.victronenergy.temperature.mqtt_fe001_t1
@400000006238ead13ac572c4 INFO:device_service:Registered Service com.victronenergy.temperature.mqtt_fe001_t1 under DeviceInstance 1
@400000006238ead13ad8d79c INFO:device_manager:publish {'t1': '1'} to device/fe001/DeviceInstance, status is 0

If you can have ssh open in another window, then

$ tail -f /var/log/dbus-mqtt-devices/current 

is a useful way to monitor the driver

I highly recommend using MQTT-Explorer to monitor the queues while debugging and if you are doing anything with MQTT.

http://mqtt-explorer.com/

Finally, if you have re-installed more than once, make sure there is only one line in your rc.local for dbus-mqtt-devices.

$ more /data/rc.local 

edit: for anyone else reading this, please also make sure that /data/rc.local is executable with

$ chmod +x /data/rc.local

thanks for you recommendation "mqtt-explorer" a very nice software and a lot more suitable than the mqtt-box extension for chrome.

$ svstat /service/dbus-mqtt-devices

gives me

/service/dbus-mqtt-devices: unable to chdir: file does not exist

$ more /data/rc.local

gives me

ln -s /data/drivers/dbus-mqtt-devices-0.2.1/bin/service /service/dbus-mqtt-devices

It looks like rc.local is not executed.

after

$ ln -s /data/drivers/dbus-mqtt-devices-0.2.1/bin/service /service/dbus-mqtt-devices

I get from

$ svstat /service/dbus-mqtt-devices

/service/dbus-mqtt: up (pid 1407) 83405 seconds, normally down

$ more /var/log/dbus-mqtt-devices/current

gives me than

@40000000623b94c31d1c2024 *** starting dbus-mqtt-devices ***
@40000000623b94c50d69c4ec -------- dbus_mqtt_devices, v0.10 is starting up --------
@40000000623b94c50d69e044 INFO:logger:Loglevel set to INFO
@40000000623b94c50da15b64 INFO:mqtt_gobject_bridge:[Init] Connecting to local broker
@40000000623b94c50fbacd54 INFO:device_manager:[Connected] Result code 0
@40000000623b976917ae010c *** CCGX booted (0) ***
@40000000623cdd411a882ed4 *** starting dbus-mqtt-devices ***
@40000000623cdd4308321f24 -------- dbus_mqtt_devices, v0.10 is starting up --------
@40000000623cdd4308323694 INFO:logger:Loglevel set to INFO
@40000000623cdd4308665e24 INFO:mqtt_gobject_bridge:[Init] Connecting to local broker
@40000000623cdd430b0cae94 INFO:device_manager:[Connected] Result code 0

after connecting to MQTT with client id

b815

and publishing

{ "clientid": "b815", "connected": 1, "version": "v1.0 ALPHA", "services": {"t1": "temperature"} }

to

device/b815/Status

I can see with

$ tail -f /var/log/dbus-mqtt-devices/current

@40000000623ce73705437efc INFO:device_manager:Received device status message {'clientid': 'b815', 'connected': 1, 'version': 'v1.0 ALPHA', 'services': {'t1': 'temperature'}}
@40000000623ce737071d6f1c *** starting dbus-mqtt-devices ***
@40000000623ce738327fe8ec -------- dbus_mqtt_devices, v0.10 is starting up --------
@40000000623ce73832800444 INFO:logger:Loglevel set to INFO
@40000000623ce73832af1324 INFO:mqtt_gobject_bridge:[Init] Connecting to local broker
@40000000623ce73834ff3ce4 INFO:device_manager:[Connected] Result code 0

but don't get a device instance.

Hi StundenBlume, Have you figured out which paths on the dbus you would need to push values to for your use case?

If I use a EM24 as 1 phase pvinverter it looks like this:

Hope this is helpful.

I don't know what the minimal information is to work properly.

Yes, as dafyddhirst pointed out, the /data/rc.local file does need to be set to executable with a
chmod +x /data/rc.local

Then when you reboot dbus-mqtt-devices will autostart. Without this the links will be wiped by Venus on reboot.

I just noticed that you used my fe001 clientid in the payload, so the registration response would have been published to

1. device/fe001/DeviceInstance

The MQTT clientid is not as important as the one in the payload.

You should see the registration request being processed in the in the log file.

edit: I origially wrote deviceid when I meant clientid and Status when I meant DeviceInstance, corrected both now.

Looking at the screen shot of the MQTT queues you just posted, I am not sure you actually need my device driver. The device you want to update is already in the dbus under device id 41. Assuming those properties are writable, you should just be able to publish messages to the relevant write topic. You can test that using MQTT explorer.

Sorry I can't figure out how to probably quote here.

I used my device id, that was a copy/paste problem just here. You can see the correct device id in the log file.

I don't see the registration request being processed in the in the log file.

This is all I get

root@raspberrypi2:~# tail -f /var/log/dbus-mqtt-devices/current
@40000000623ce6492d7eef3c INFO:logger:Loglevel set to INFO
@40000000623ce6492db0ec1c INFO:mqtt_gobject_bridge:[Init] Connecting to local broker
@40000000623ce649302af49c INFO:device_manager:[Connected] Result code 0
@40000000623ce73705437efc INFO:device_manager:Received device status message {'clientid': 'b815', 'connected': 1, 'version': 'v1.0 ALPHA', 'services': {'t1': 'temperature'}}
@40000000623ce737071d6f1c *** starting dbus-mqtt-devices ***
@40000000623ce738327fe8ec -------- dbus_mqtt_devices, v0.10 is starting up --------
@40000000623ce73832800444 INFO:logger:Loglevel set to INFO
@40000000623ce73832af1324 INFO:mqtt_gobject_bridge:[Init] Connecting to local broker
@40000000623ce73834ff3ce4 INFO:device_manager:[Connected] Result code 0
@40000000623ce98d1ceb3c84 *** CCGX booted (0) ***

For the pvinverter
This EM24 is just borrowed from someone. I think it would be nice to implement a own pvinverter without adding a original one first.

Unfortunately it looks like it is crashing. We see message saying "received device status message" immediately followed by a line saying "starting dbus-mqtt-devices". I think that is daemontools automatically restarting the service after a crash.

I'll run some tests using your data and see which log files might contain the error from the crash. I am launching my boat tomorrow so I will be tied up with that for a couple of days.

I just noticed a small but important difference:
{ 'clientid': 'b815', 'connected': 1, 'version': 'v1.0ALPHA', 'services': { 't1': 'temperature'}}

it should be clientId not clientid (uppercase i)

I should put some better validation in the code.

Good morning,
thank you, with "clientId" it works like expected. I copied the line from github.
Look like good weather to spend some time with the boat!
Best regards,
Sebastian :)

Awesome. Thanks for letting me know.

I will try to add a pvinverter by editing services.yml

Until now I didn't tray to to modify your services.yml.

I tried some other things:
1. writing to an existing EM24, defined as pvinverter

W/.../pvinverter/41/CustomName {"value": "test"}

works.

2. writing to

W/.../pvinverter/41/Ac/L1/Power {"value": 1}

or

W/.../pvinverter/41/Ac/L1/Current {"value": 1}

does not work.

So I think it's not writable.

Further I found this information from Victron:

com.victronenergy.pvinverter

/Ac/Energy/Forward     <- kWh  - Total produced energy over all phases
/Ac/Power              <- W    - Total power of all phases, preferably real power
/Ac/L1/Current         <- A AC
/Ac/L1/Energy/Forward  <- kWh
/Ac/L1/Power           <- W
/Ac/L1/Voltage         <- V AC
/Ac/L2/*               <- same as L1
/Ac/L3/*               <- same as L1

/Ac/Current            <- A AC - Deprecated
/Ac/Voltage            <- V AC - Deprecated

/Ac/MaxPower           <- Max rated power (in Watts) of the inverter
/Ac/PowerLimit         <- Used by the Fronius Zero-feedin feature, see ESS manual.
                          Path must be invalid or non-existent for PV Inverters
                          that do not implement Zero Feed-in power control.

/ErrorCode             <- 0=No Error
/FroniusDeviceType     <- Fronius specific product id list
/Position              <- 0=AC input 1; 1=AC output; 2=AC input 2
/StatusCode            <- 0=Startup 0; 1=Startup 1; 2=Startup 2; 3=Startup
                          3; 4=Startup 4; 5=Startup 5; 6=Startup 6; 7=Running;
                          8=Standby; 9=Boot loading; 10=Error

I wrote a python script which wrote to all topics, but without success.
So I think I don't understand how to add a new pvinverter to the victron system.

I will try to understand how you did it with the temperatur and tank sensors.

I got a little farther.

I added the following to services.yml

pvinverter:
  CustomName:
    default: "My PV Inverter"
    persist: true
  ErrorCode:
    description: "0=No Error"
    persist: true
    default: 0
    min: 0
    max: 1
  Position:
    description: "0=AC input 1; 1=AC output; 2=AC input 2"
    persist: true
    default: 0
    min: 0
    max: 2
  StatusCode:
    description: "0=Startup 0; 1=Startup 1; 2=Startup 2; 3=Startup 3; 4=Startup 4; 5=Startup 5; 6=Startup 6; 7=Running; 8=Standby; 9=Boot loading; 10=Error"
    persist: true
    default: 7
    min: 0
    max: 10

When I write to

device/a04/Status { "clientId": "a04", "connected": 1, "version": "v2.3", "services": {"pv1": "pvinverter"} }

Partial success: A new pvinverter is created.

But I don't know how to add "subcategories" like

/Ac/Power

If I add the following to services.yml

  Ac:
    Power:
      description: "W"

I get:

Any hint how to get the expected structure?

Ac/Power:

did the trick.

services.yml now looks like this:

[...]

pvinverter:
  CustomName:
    default: "My PV Inverter"
    persist: true
  ErrorCode:
    description: "0=No Error"
    persist: true
    default: 0
    min: 0
    max: 1
  Position:
    description: "0=AC input 1; 1=AC output; 2=AC input 2"
    persist: true
    default: 0
    min: 0
    max: 2
  StatusCode:
    description: "0=Startup 0; 1=Startup 1; 2=Startup 2; 3=Startup 3; 4=Startup 4; 5=Startup 5; 6=Startup 6; 7=Running; 8=Standby; 9=Boot loading; 10=Error"
    persist: true
    default: 7
    min: 0
    max: 10
  Ac/Energy/Forward:
    description: "kWh"
  Ac/Power:
    description: "W"
  Ac/L1/Current:
    description: "A"
  Ac/L1/Energy/Forward:
    description: "kWh"
  Ac/L1/Power:
    description: "W"
  Ac/L1/Voltage:
    description: "V"
  Ac/L2/Current:
    description: "A"
  Ac/L2/Energy/Forward:
    description: "kWh"
  Ac/L2/Power:
    description: "W"
  Ac/L2/Voltage:
    description: "V"
  Ac/L3/Current:
    description: "A"
  Ac/L3/Energy/Forward:
    description: "kWh"
  Ac/L3/Power:
    description: "W"
  Ac/L3/Voltage:
    description: "V"
  Ac/MaxPower:
    description: "W"

Current state using this services.yml content:

[...]

pvinverter:
  CustomName:
    default: "PV Inverter MQTT"
    persist: true
  Position:
    description: "0=AC input 1; 1=AC output; 2=AC input 2"
    persist: true
    default: 0
    min: 0
    max: 2
  Role:
    default: "pvinverter"
  Ac/Energy/Forward:
    description: "kWh"
  Ac/Power:
    description: "W"
  Ac/MaxPower:
    description: "W"
  Ac/L1/Current:
    description: "A"
  Ac/L1/Energy/Forward:
    description: "kWh"
  Ac/L1/Power:
    description: "W"
  Ac/L1/Voltage:
    description: "V"
  Ac/L2/Current:
    description: "A"
  Ac/L2/Energy/Forward:
    description: "kWh"
  Ac/L2/Power:
    description: "W"
  Ac/L2/Voltage:
    description: "V"
  Ac/L3/Current:
    description: "A"
  Ac/L3/Energy/Forward:
    description: "kWh"
  Ac/L3/Power:
    description: "W"
  Ac/L3/Voltage:
    description: "V"

I had to manually set

W/xxx/pvinverter/2/Role {"value": "pvinverter"}

But the "Role" makes no difference on the next behavior.

Registered device

But not here shown

EM24 device has units

MQTT device has no units

EM24 has "setup page", MQTT not

EM24

MQTT

How Do I handle a reboot of venusOS?

After a reboot a existing temperature sensor was gone.

After sending the status massage to the driver again it worked normal. Is that the correct procedure?

I just published a new release v0.3.1 on github with better validation of the status message, a new DBus registration topic and some bug fixes.

https://github.com/freakent/dbus-mqtt-devices

Thanks for you really nice software! :)

What is the proper way to update? just execute setup, or do I have the uninstall or remove something?

Just install the new version as per the instructions, but before rebooting just edit the /data/rc.local and remove the line that for the previous version.

Just for future readers. It works with "pvinverter" and "grid" services.
If I have everything running I will write a howto in this forum and will post a link here.

Thanks @gone-sailing for this brilliant and extreme useful driver! It opens a world of until now unusable sources for sensors.

Edit:
I will close now this tab in my browser, I don't get any notifications if something happens here. So maybe a PM is necessary to get in contact.

@stundenblume Thank's very much for the feedback. I have rolled your pvinverter changes into a v0.4.0 pre-release if you want to give it a try before I release a 0.4.1 more generally.

https://github.com/freakent/dbus-mqtt-devices

@gone-sailing
Ok, I will try it in the afternoon.

This looks amazing. I'd like to copy pvinverter data from one Venus install on a Pi in the garden across to the main Cerbo that has everything else attached to it.

I think this should be possible via Node Red? I have Node Red running on another Pi but I'm a total noob with it. I can see both installs in MQTT explorer fine.

I've ran through the Install and Setup notes on Github and now I'm thoroughly confused as to how to register the pvinverter on the Cerbo.

Hi @gone-sailing ,

thx for the nice software :) I'll try to implenet an oil sensor for oure home. This would run on an esp32 with deep_sleep and should only publish the remaining level all 12h. My mqtt code connects publishes messages and disconnects, but at disconnecting the device is removed from the function _remove_device, do you know a way to publish something that this would not happen at disconnect?

thx and regards

Kotty

Hi Kotty, I’d need to see how you are using MQTT to be sure, but you must use the registration process each time your device starts. The dbus will remember your device and give you the the same Device Id. If you have any problems open a new issue on the GitHub repository. It’s easier to track there than on this forum.

I have a smart grid meter (eHZ DD3) which has a D0 optical interface and provides high-res voltage, power and energy data once per second. A D0-compatible optical sensor is commercially available under the name "Hichi IR WiFi" for a very resonable price. The sensor contains an ESP8266_1M and is normally intended to run a Tasmota-based firmware, that supplies the readings to a MQTT broker.

Using this hardware, I have built a firmware using ESPHome, that supplies the data to the Cerbo running it's stock MQTT broker. The firmware is built to satisfy the expectations of dbus-mqtt-devices. After a few hours of coding, I am now able to see the readings of my D0 meter on my Cerbo (no battery and no Multiplus connected yet):

As D0/SML meters are very common here in Germany, maybe somebody else might find the ESPHome source code useful. I have put my work on Github: https://github.com/hsteinhaus/esph-gadgets/blob/master/esphome/d0-meter.yaml

@gone-sailing Thanks for publishing your driver - it was very helpful to get my meter data into the Cerbo!

Awesome, I'm glad it was of use to you. It sounds like a good project.

@gone-sailing thanks for this - just to say i've got it working on my boat now for temperature and pressure. My plan is to add more sensors now for the alternators, fridges and seawater. I also rebuilt the client to get it working on the WROOM-ESP32s I've got kicking around. After a bit of trial and error it's working now.... Great work, thank you!

@gone-sailing
Dear Gone sailing,
I am really a complete newby but I really want to learn this.
I would like to display the weather forcast on my Cerbo GX.
For this I want to use the Openweather API with the GPS coordinates.
This gives back a JSON message that I would like to display. I understood that I will have to pubisch this JSON message on the DBUS.
I am completely lost where to start to figure out this further. I know how to grab data from the DBUS, but I have no idea how to get it there. MQQT, Node RED other method??
Would appriciate if you could give me some directions.
Sorry for the newby question.....

@paul-kassenaar

The GX devices are not general purpose displays, you can not just display any sort of data you wish. There are a list of devices types that VenusOS supports. Look up the docs for the Venus Modbus TCP interface. I’m pretty sure weather forecast is not on that list. This driver just simplifies using the same device types over MQTT. Sorry, you were probably expecting something more helpful.

@gone-sailing

Yes you are right I was hoping on more a direction where I could work on because I saw a project from Itsme ( https://community.victronenergy.com/questions/186357/add-a-data-field-to-a-dbus-device.html) were he somehow managed to get the city name with an API call in combination with the GPS coordinates I assume and displayed the cityname on the screen.

Anyhow thank you for the very quick reaction

Bye from Holland

Took some time but got it to work: