question

mvader (Victron Energy) avatar image
mvader (Victron Energy) asked

Victron Bluetooth BLE Protocol announcement

UPDATE 2023-2-18: make sure to also read here https://community.victronenergy.com/questions/187303/victron-bluetooth-advertising-protocol.html


Hello Bluetooth-protocol-craving-developers!

Its finally here, a Victron published protocol to get data from our devices over Bluetooth Low Energy aka BLE.

By implementing this protocol, you will be able to read Victron product data such as Voltages, Currents, Power, and so forth.

Each of those values is made available as a GATT characteristic. Which is the normal way of doing that in the official Bluetooth Low Energy protocol. The advantage of doing it that way is that its relatively straight forward to use it. For example you could use a readily available app such as the nRF Connect App by Nordic (screenshot below). And also its possible to query data straight from the command line in Linux.

The details:

  • This feature adds a new BLE Service. Its a new BLE service, and its a different one than the BLE Services we use ourselves for VictronConnect communication and firmware updates.
  • The UUID for this new service is: 65970000-4bda-4c1e-af4b-551c4cf74769
  • This protocol (and its BLE service) is not the same protocol as used by VictronConnect.
  • A keep alive is required. Without that, the device will automatically disconnect after one minute.
  • For now, this new feature is available only for the SmartShunt only. It was added in SmartShunt firmware version v2.31. Also, you need VictronConnect v5.42 (or later) to enable this feature in the settings, see screenshot.

Instructions:

  1. Use a SmartShunt, other devices don't support this yet.
  2. Use VictronConnect v5.42 or newer.
  3. Update the SmartShunt to version v2.31 or later.
  4. Connect to the SmartShunt using VictronConnect, and enable this protocol (screenshot below)
  5. Power cycle the SmartShunt
  6. Refresh the services cache on your host (ie. phone/tablet/laptop/raspberrypi). This is done by removing the SmartShunt from the list of bonded devices, and then re-add it. This is necessary because Android, Bluez/Linux and so forth all cache the list of available services.


Now its up and running and available. On a phone, try the Nordic nRF Connect App and look for the service with the UUID ending at 4769 as listed above.

On linux, use the gattool. I found a nice primer about it: How to Get Started with Bluetooth Low Energy on Linux. And there must be many more to be found on the internet.

To find out which GATT characteristic is what, you can read the "Characteristic User Description" parameters. Currently I don't have a list of all UUIDs and what they mean.

Now, a bit more detail on the keep alive:

  • The default time-out is configered to 60 seconds.
  • The UUID for the Keep-alive characteristic is 6597ffff-4bda-4c1e-af4b-551c4cf74769.
  • Writing 0xFFFF to it disables the auto connection, resulting in a connection forever.
  • We strongly recommend not to use that 0xFFFF in a production environment. During VictronConnect development we learned the hard way that Bluetooth stacks can end up being in weird states, on the host as well as in our product; and one good way to make it robust it is to make the host send a keepalive; and make the device issue an internal disconnect when it no longer receives that keep-alive. This then resets all states and resets the rest of the stack, so its in a connectable state again.


Last, how to get the data? BLE supports multiple methods. First there is the obvious read command, which allows you to read a value, and the host (GATT Client) then sends the read command to the Victron product (GATT Server), after which the response including the data is returned.

The other method is to have the host (GATT Client) subscribe to notifications. Our implementation supports these notifications, and how they work is that whenever a value changes, its automatically sent by the Victron product to your host. Note that notifications are not acknowledged.

That is all for now. We are aiming to write this up in a more complete and technical document, but I can't promise when that is completed.

All the best, Matthijs


Picture one: the setting to enable this "third party implementation"-protocol:



1625169925253.png


Picture two, the Nordic nRF Connect App showing the voltage (0x2005 = 0x0520 = 13.12V). Note how the app Allows you to retrieve the data, retrieve the user description, as well as receive other information about the characteristic:


1625171683174.png


Picture three: how to enable notifications in the Nordic nRF Connect App.

1625171748346.png


Bluetooth
1625169925253.png (40.8 KiB)
1625171683174.png (531.0 KiB)
1625171748346.png (327.9 KiB)
12 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

robbo avatar image robbo commented ·

How do you update the firmware from the file? I don't get the Firmware update option for the Bluetooth Interface if I connect via cable (I only see the "Product" one that's on v4.07), and it won't connect via Bluetooth on my Linux machine and my iOS devices I can't select a file.


0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ robbo commented ·
Hi Robbo, I added a link to my comment above, to the manual; please check that
0 Likes 0 ·
robbo avatar image robbo mvader (Victron Energy) ♦♦ commented ·

Cheers, now singing on v2.31.

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ robbo commented ·
Hi and now I’m curious ;o) did it work?
0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·
Hi all, a heads-up: we're currently working on broadcasting all the most relevant data. Which is for a lot of systems probably a lot easier (and more stable) to work with, rather than connecting to the GATT service. More information about it later; might still take one or two months but then at least you already know.
0 Likes 0 ·
hoechst avatar image hoechst mvader (Victron Energy) ♦♦ commented ·
Thanks for the feature, seems to be very helpful for our case as well. Any update on the timeline especially for other products, i.e. the SmartSolar line?
0 Likes 0 ·
knauer avatar image knauer hoechst commented ·

@mvader (Victron Energy):

I'd also be interested in this. It's more than a year ago now, may I ask for some timing estimation für GATT for SmartSolar devices? I'm developing a new software for mobile home systems and like to meet the right descision whether to go via reverse engineering projects like https://github.com/birdie1/victron, https://github.com/vvvrrooomm/victron and https://github.com/Olen/VictronConnect, or via advertising protocol (which is missing some interesting values - waiting for these?) or wait for the GATT option for SmartSolar.

Thank You very much!

0 Likes 0 ·
anglo77 avatar image anglo77 mvader (Victron Energy) ♦♦ commented ·
Hi @mvader (Victron Energy) - just wondering how the broadcasting was coming along and when that may be available?

Thanks :)


0 Likes 0 ·
cawe-technologies avatar image cawe-technologies mvader (Victron Energy) ♦♦ commented ·
Hi @mvader (Victron Energy) , so will this Broadcasting packet be made unencrypted like everyone is asking, or at least a way for the user the make it available if needed unencrypted?
0 Likes 0 ·
yoctosam avatar image yoctosam commented ·
I would also like to be able to interpret the Advertisement data from the BMV712 and the SmartSolar MPPT. The use case for this is that I cannot receive the bluetooth data from these devices unless I am standing right next to (or inside) my van. I'd like to be able to read the data from inside the house using a custom device mounted on the house next to the van.
0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·

Two more pieces of information:

1. I made a mistake, the firmware version needed is not release cq. not shipped with VictronConnect at the moment. Here is a link to the file you need:

https://www.dropbox.com/s/08c5mbwdr4rdl2q/smartshunt52_ble_v2.31-gatt.bup?dl=0

Install it using the instructions here: https://www.victronenergy.com/media/pg/VictronConnect_Manual/en/firmware-updates.html#UUID-705fe0d3-30d8-e6e6-aad1-856f4ac6e3b3

2. Here is a list of the characteristics, (my apologies for the formatting).

UUID

Characteristic

Access

Type

Scale

Unit

Special values

6597ffff-4bda-4c1e-af4b-551c4cf74769

Keep-alive

Write/Read

un16

0,001

Seconds

0xFFFF

keep-alive forever

6597eeff-4bda-4c1e-af4b-551c4cf74769

Consumed Ah

Read/Notify

sn32

0,1

Ah



6597ed8e-4bda-4c1e-af4b-551c4cf74769

Power

Read/Notify

sn16

1

W

0x7FFF

not available

6597ed8d-4bda-4c1e-af4b-551c4cf74769

Voltage

Read/Notify

sn16

0,01

V

0x7FFF

not available

6597ed8c-4bda-4c1e-af4b-551c4cf74769

Current

Read/Notify

sn32

0,001

A

0x7FFFFFFF

not available

65970fff-4bda-4c1e-af4b-551c4cf74769

State of charge

Read/Notify

un16

0,01

%

0xFFFF

not available


Here is the same list, but then as a picture:

1625216203367.png


-1 Like -1 ·
1625216203367.png (27.6 KiB)
ripnc avatar image ripnc mvader (Victron Energy) ♦♦ commented ·

Using an Android tablet - file appeared to add to the library without issue but when I click to Update (from 2.30) I see nothing in the library. I get the "lightbilb" and the message saying I'm running the most current version. I have the option to use a laptop but I'm participating in a beta so I'd like to see if I can get the Android to work.

0 Likes 0 ·
27 Answers
trip-tracks avatar image
trip-tracks answered ·

Thanks, @mvader (Victron Energy).
And thanks for the detailed/informative approach.

Looks like a great start. Looking forward to getting other devices connected - the Orion-Tr Smart DC-DC Chargers would be a great target since they don't have a ve.direct interface ;)

4 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·
hahaha; yes. but also then you still won't see its output current; you're aware of that right?
0 Likes 0 ·
trip-tracks avatar image trip-tracks mvader (Victron Energy) ♦♦ commented ·

No, I wasn't. I know a SmartShunt on the output should work though ;) And with ve,direct

Sorry, not trying to undermine any of your great work!

You know it would be great to have a way to connect this into DVCC and for use with canbus BMS. The missing link to controlling all the charging,,,,,
Oh, and a 12-48V DC-DC charger, thanks! ;)

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ trip-tracks commented ·
no worries, no need for apologies! but its the common thing everyone wants from their Smart Orion (output current measurement) which it doesn't measure. So no matter the protocol.. its not there. Until we have redesigned the product ofcourse, but that will take a long long time.


I don't think we'll be using bluetooth soon for DVCC and such btw.


Here its time to sleep, have a good day!

0 Likes 0 ·
hjohnson avatar image hjohnson mvader (Victron Energy) ♦♦ commented ·
Bit of an old thread, but what I'd love to be able to do is a) do temperature compensation on the Orion, and b) adjust its voltage on the fly, based on the data from a Smart Shunt or BMV-7xx. Our plan is to use an Orion to maintain the starting battery on our boat, and it'd be nice to have a little more than on/off control via the relay input.
0 Likes 0 ·
rspott avatar image
rspott answered ·

Does this mean my Cerbo might be able to display data from my 6 Victron Energy Smart LiPo Battery 12.8V?

The phone app is neat but requires me to be physically present and click back and forth too many times to get status.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

birdie avatar image
birdie answered ·

Thanks Matthijs for this great announcement,

I have added a python script to retrieve the data to my repository with my reverse engineering from the older protocol: https://github.com/birdie1/victron

It is already working better and more stable than all the stuff before. Would be great to get more information on this way on the smart shunt. And of course if you add the new protocol to the other devices as well.

Looking forward to more changes like this!

Regards

Jonas

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·
Hi Jonas,


you're welcome! Wrt SmartShunt and more information: we don't have the intention to add anything.


Matthijs

0 Likes 0 ·
Philip Barclay avatar image
Philip Barclay answered ·

Is there a characteristic for the secondary voltage connection?

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hi

At the moment, this data is not available in a characteristic.

Kind regards,

Thiemo van Engelne

0 Likes 0 ·
David avatar image
David answered ·

Hello! Will GATT be enabled for the BMV-712? I got my hopes up when I read the changelog entry for today's 5.48 release at https://www.victronenergy.com/live/victronconnect:beta

Specifically:

  • BMV-712 & SmartShunt:
    • ...
    • Show GATT switch only when connected via Bluetooth
    • Mark GATT as experimental feature
    • ...

So, I gleefully updated to 5.48 firmware on my battery banks, but found that the Victron Connect app (on both Android and macOS) does not show a "Bluetooth GATT service" switch under the Product Info page.

screenshot-20211007-114033.png

For background: I made battery bank suitcases for personal emergency preparedness, and I chose BMV-712 monitors on each Pelican case to have the display available in the field. It would be fantastic if I may also monitor the batteries wirelessly to enable more unattended operation such as with Python & BLE GATT to automate dump loads and send battery level notifications without compromising case waterproofing for VE.direct cables or having to reconnect such cables during battery bank swaps.

Thank you,

David


2 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

jtlbean avatar image jtlbean commented ·

I have the same question as David here. Is this something we could expect to see enabled on the BMV-712? Really excited to integrate this functionality into my home-brew monitor system. The change log seems to indicate it was enabled with 5.48. However this appears not to be the case (unless I'm doing something wrong). I'll swap out with the Smart Shunt if support for the BMV is not in the future plans.

Thanks!


1 Like 1 ·
nirgo avatar image nirgo commented ·

Was it determined that this was an erroneous changelog entry for the BMV-712?

v5.48 (Released officially)

BMV-712 & SmartShunt:

  • Show GATT switch only when connected via Bluetooth

  • Mark GATT as experimental feature

0 Likes 0 ·
vinthewrench avatar image
vinthewrench answered ·

I posted some sample code in C that uses the Gattlib from (https://github.com/labapart/gattlib) to get some values from the smartshunt. O hardcoded the deviceID in the sample for now.

You still have to use bluetoothctl to connect and pair, I havent written that in C yet. but this might help.

my code is at https://gist.github.com/vinthewrench/4d2046c3d3cc0bae7e2fa54bdfb89dac

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

felixsteinbeis avatar image
felixsteinbeis answered ·

Hello Matthijs, hello Victron,

Any news when the new GATT service will also be available on the BMV-712?
And will the new GATT service also be available on the Phoenix Smart IP43 Charger?

I hope it will not take much longer.
Thank you very much.

Best regards
Felix

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

wolfgang-huse avatar image
wolfgang-huse answered ·

Hello! I started to create a BLE Client on ESP32, displaying Data works quite well on a M5Stack, next step will be forwarding Data as MQTT, maybe LoRa... i will share code after some additional testing...ve.jpg

Seems like in the meantime there are more Characteristics available under the Service, is there a newer documentation available ?

12:15:36.1600: discovered characteristics for service 65970000-4BDA-4C1E-AF4B-551C4CF74769: (
    "6597FFFF-4BDA-4C1E-AF4B-551C4CF74769",
    "65970FFF-4BDA-4C1E-AF4B-551C4CF74769",
    "6597ED8D-4BDA-4C1E-AF4B-551C4CF74769",
    "6597ED8E-4BDA-4C1E-AF4B-551C4CF74769",
    "6597ED8C-4BDA-4C1E-AF4B-551C4CF74769",
    "6597EEFF-4BDA-4C1E-AF4B-551C4CF74769",
    "6597ED7D-4BDA-4C1E-AF4B-551C4CF74769",
    "6597EDEC-4BDA-4C1E-AF4B-551C4CF74769",
    "65970382-4BDA-4C1E-AF4B-551C4CF74769",
    "65970383-4BDA-4C1E-AF4B-551C4CF74769",
    "65970FFE-4BDA-4C1E-AF4B-551C4CF74769"

ve.jpg (1.8 MiB)
5 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

gainestr avatar image gainestr commented ·
I am extremely interested in your project. I would like to set up a similar system in my RV with an M5Stack from Adafruit. Please keep us updated on your project as you progress.
1 Like 1 ·
mike4nl avatar image mike4nl commented ·
I would like to work with you. I work now for a long time with STM32, esp8266 and the esp32. I coded a lot for the TASMOTA (github) to communicate and for sensors like UV and Sun Power on square meter.

I use also a industrial PLC from B&R for my home automation, the cp1585 and a lot of bus hardware (PowerLink, X20) and digital and analog i/o's.

1 Like 1 ·
kowi avatar image kowi commented ·
I'm also very interested. Do you have any code to share yet?
0 Likes 0 ·
andrew-gerngross avatar image andrew-gerngross commented ·

Characteristic "6597ed7d-4bda-4c1e-af4b-551c4cf74769" appears to be the long-sought secondary battery voltage. Don't know about the other new ones.

0 Likes 0 ·
Bruno avatar image Bruno commented ·
Hi, I'm really interested in your implementation, as I'm trying to gather values from a BMV 712 through BLE, from an ESP32 device. Would you have a repository, Gist or anything to see how it is implemented on your side ?

Thanks !

0 Likes 0 ·
muhkuhve avatar image
muhkuhve answered ·

I am still a little confused. I got a SmartSolar MPPT 75/15 pv charging controller.. is there any way to get at least voltage/current from PV or to battery using my esp32, rpi or anything else?

No advertising firmware for SmartSolar so far? Couldn't find out behind what uuid these values may wait..

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

knauer avatar image knauer commented ·

Yes, for these parameters You have this: https://community.victronenergy.com/questions/187303/victron-bluetooth-advertising-protocol.html

But load output and other stuff is missing yet.

0 Likes 0 ·
netsrac27 avatar image
netsrac27 answered ·

Great stuff. The support for public Bluetooth address was actually the reason why I dropped my BMV-702 and replaced it with a SmartShunt.

Would actually love to see GATT services on the Smart Battery Sense. Together with the voltage and the temperature you could somehow estimate the charging status of the starter battery.


2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

wolfgang-huse avatar image
wolfgang-huse answered ·

@mvader (Victron Energy) I have some more questions on current implementation...

I am working on a auto-discovery based on BLE Advertised Services and i only receive a different Service UUID (cd55a389-880b-425b-b167-81ed6a15e913 )from the later used for the characteristics.

Can i rely on this UUID for Discovery? What about Manufacturer-Data (e102100289a3) is this fixed to the Smart Shunt Device ?

Regards,

Wolfgang

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hi @Wolfgang Huse

It would advice to rely on the Manufacturer Data. This is indeed (mostly) fixed for the SmartShunt. "e12110" is fixed and is the same for all Victron products. The following "02" might change in the future and/or between products. The "89a3" is fixed for the SmartShunt as this is the so called product id (in little endian order), which is unique for every product type.

In the future the content of the manufacturer data can be extended, but this first part will stay the same, maybe except for that "02" byte.

Kind regards,

Thiemo van Engelen

1 Like 1 ·
deadrabbit avatar image
deadrabbit answered ·

Hi,


i'm new to Bluetooth Low Energy and i want to develop a small OLED-Display which displays the battery values.


First point i'm struggeling: @mvader (Victron Energy) published a firmware version 2.31. On my smartShunt is v2.37, so i'm not able to downgrade.


But i have the GATT setting in my VictronApp and enabled it. But with nrfConnect i get different characteristics as described above.


For example i get the UUID 68c10001-b17f-4d3a-a290-34ad6499937c

screenshot-2022-05-21-15-51-17-947-nonordicsemiand.jpg

How can i access the SoC or Voltage characteristic?


Thanks!




3 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hi @deadrabbit

When you remove the pairing of the SmartShunt in the Android Bluetooth menu (or use "Remove bond information" in nrfConnect), is it then still not showing the correct service?

Remember that you need to pair again to the SmartShunt after this, so make sure that you know the PIN code or the PUK code to be able to reset it back to "000000".

With kind regards,

Thiemo van Engelen

0 Likes 0 ·
deadrabbit avatar image deadrabbit Thiemo van Engelen (Victron Energy staff) ♦ commented ·
Hi @Thiemo van Engelen (Victron Energy staff)


thanks for your reply. Your hint was helpful, now i'm able to the Service UUID as described above.

BW


0 Likes 0 ·
gainestr avatar image gainestr commented ·
I am interested in your progress of your project. I would very much like to do the same as you with a display that I can mount in my RV. Any progress updates?
0 Likes 0 ·
netsrac27 avatar image
netsrac27 answered ·

Will there be a chance that we see GATT services on other devices? I could also go with the new broadcasting data….if this wouldn’t be encrypted…

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

tsax6010 avatar image
tsax6010 answered ·

If had been a full year now. Any update @mvader (Victron Energy) on GATT support for the BMV-712?

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

felixsteinbeis avatar image
felixsteinbeis answered ·

Hi Matthijs,

I've been looking for a solution to read out my BMV-712 via Bluetooth for over a year now.

My hope was the new GATT service, but unfortunately it is not yet available for the BMV-712 (but I keep hoping it will come). Now my second hope is the new advertising data. These contain almost all the info I need.

However, the data are encrypted. How can I decrypt the data? I can read the advertising data and see a timestamp. But I am missing more information about encryption and encoding.

Alternatively, an option to use a configurable encryption key only for the advertising data that is independent of the six digit key and the pairing/bondig before.

I really hope Victron can provide the info so I can finally read my BMV-712 with Bluetooth.

Thank you very much.

Many greetings
Felix

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

cawe-technologies avatar image cawe-technologies commented ·
Also waiting for a way to get the info from the advertising data. Understand the security concerns they have.
0 Likes 0 ·
melbnz avatar image
melbnz answered ·

concur with above comments - it would be great a year on to have this functionality in BMV-712, smart solar chargers and DC-Dc. Here’s hoping, thanks

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

gainestr avatar image
gainestr answered ·

I’m still hoping that someone has developed a nice little BLE client on a M5Stack or similar to receive this info and will publish it in one of these threads.

I like the idea of having a display similar to the default Venus OS dashboard/Cerbo GX but using BLE instead of having it wired. Anyone out there have any solutions yet? I don’t always run my router when I’m boondocking to save on the inefficiency of the inverter so a BLE option would be perfect. I like the victron connect app but would love to see more of a gui and easy to see graphical display with additional information like in gui mods.

2 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

jkirchheimer avatar image jkirchheimer commented ·
I built a BLE based Dashboard device for my LIONTron Lithium Batterie in my camper, because there is none to buy. I started the project with using M5 Products (Stack and the e-paper one). As soon as I opened a connection to a BLE Device, the Devices rebooted after a random period of time between 10 and 120 seconds. I worked on it for a few weeks until I decided to give it a try with different hardware. Switched to a Raspberry Pi Zero W and it worked first try. I might did something wrong, even that I followed tons of documentations and examples for BLE on ESP32, but it seems as if the M5 BLE Stack isn't the most stable one.... :-(
0 Likes 0 ·
gainestr avatar image gainestr jkirchheimer commented ·
Good to know. Just looking for a nice easy implementation and one that is based off of BLE and not Wi-Fi since I only run Wi-Fi when I need it.
0 Likes 0 ·
jkirchheimer avatar image
jkirchheimer answered ·

Hey @mvader (Victron Energy)

Any chance to get some Informations about the BLE GATT Protocol for Smart MTTP, now that it's rolled out into the Victron Connect Apps? Would love to integrate the Solar Charger into my diy dashboard in my camper, that I had to build myself, because there is no Hardware Display for Liontron Batteries.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

embeddediot avatar image
embeddediot answered ·

Has anyone had any luck with implementing the Keep-alive characteristic on a victron orion? I have tried a few fixes by now but whatever i do it still disconnects within a minute.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

lvjoe avatar image
lvjoe answered ·

I have been trying to get an M5Stack Core2 to connect to the SmartShunt. Similar to the screenshot above by wolfgang-huse, but the SmartShunt seems to require Bluetooth Pairing.

Is this pairing required or is there anyway around it? Including reseting the PIN back to all zero's. Using a GATT tool on my iphone, I am able to connect to other Bluetooth GATT devices without pairing, but I am also not sure if this is just the way those devices work compared to the SmartShunt GATT support.

.(Hope wolfgang-huse shares his code. Otherwise I will share mine if I ever get it working)

4 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

wolfgang-huse avatar image wolfgang-huse commented ·

sorry, being long time not here... find my repo here: https://github.com/wolfganghuse/smartshunt-ble

It also contains pairing-mechanism...


1 Like 1 ·
Hi @Lvjoe


Yes, we made bluetooth pairing using a pin code mandatory for security/privacy reasons and at this time there is no way around it.

Kind regards,

Thiemo van Engelen

0 Likes 0 ·
cawe-technologies avatar image cawe-technologies Thiemo van Engelen (Victron Energy staff) ♦ commented ·
Hi @Thiemo van Engelen (Victron Energy staff) , is there any intention of securely sending advertisement packets to other devices/apps?
0 Likes 0 ·
netsrac27 avatar image netsrac27 cawe-technologies commented ·
Well, the advertisement data is there and you can read it. Unfortunately it‘s encrypted with some kind of rolling code (or there is a sequence number in the dataset). So “all“ we need is decode this data.


I would love to have access, because Victron seems to go forward with advertisement data instead of GATT…and it‘s not a bad idea, because you don‘t need to connect to a device which will give you a wider range.

I would even accept if there is a switch which allows to send unencrypted data, because it‘s read only and I don‘t care if someone could read my battery SOC.

2 Likes 2 ·
nirgo avatar image
nirgo answered ·

Just a :+1: for the GATT feature in the BMV-712.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

keabarnes avatar image
keabarnes answered ·

Python script to fetch all data:

main.py.txt

My env:

  1. Python 3.10
  2. Rename main.py.txt to main.py
  3. pip install bleak
  4. python main.py

mainpy.txt (1.3 KiB)
1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

xsilvergs avatar image xsilvergs commented ·
When I run main.py I get a bunch of errors, can you give more info please?
0 Likes 0 ·
davidliddell avatar image
davidliddell answered ·

There seems to a huge amount of interest in an open BLE protocol to access Victron's devices but Victron (thank you mvader for answering many of the questions) has declined to make the protocol public. This has forced many of us to reverse engineer the protocol. This of course is a waste of our time and may cause issues with the operation of our Victron gear which will in turn mean a support load on Victron. At the start of this post mvader announced support for an open BLE protocol but I can find no mention of this in Victron main site and it is only available on the SmartShunt, does anyone know better? ve.connect is a great app and in general Victron should be applauded for their open attitude. I can't quite see why they decided to use BLE as transport layer (which seems to go against the whole philosophy of BLE) rather than advertising a service with characteristics as they have done in mvaders announcement.

I am building a large BMS system using Victron MTTPs (a mixture of SmartSolar and BlueSoler with BT dongles) and other devices. I need to monitor the solar charger currents etc. If the new protocol were available on the MTTPs (and the dongles) this would be great. BUT I also need to be able to turn the Solar charging ON and OFF which I dont think it supported in the new protocol (some documentation on the protocol would be great). Currently I use ve..bus serial interface (I have to use the HEX version to be able to control the devices) which works great but needs a lots of wires and pins. I have even considered building my own ve.bus dongle with an open BLE interface (I already have the code for the serial end!). This isn't a bad solution but rather unnecessary! My worry is that over time Victron will depreciate the serial ve.bus in favour of BLE and the work to build the dongle will be in vain!

Other peoples thoughts on this would be much appreciated. My email is david@liddell.com

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

netsrac27 avatar image netsrac27 commented ·
Not sure if Victron should allow write access to their devices. I get your point that you want to control your MPPTs, but I think this can be tricky. But making those devices accessible via VRM and allow (controlled) switching and adjustment would be great.


BUT....I don't see a reason why reading data should not be public. I love the way they use the advertising package to send the data. That's all I need. Why does this need to be encrypted? Any secret data you share there? Probably not! And you still have the option to disable this feature in the settings.

Victron is offering their VenusOS as open source, they support to install it yourself. Not sure why they prevent the reading of the data.

I hope one day, they will also support BLE with VenusOS. It doesn't make any sense to me that you still need to physically wire the devices.

Unfortunately we can not do more than wait and see what happens....



0 Likes 0 ·
Bruno avatar image
Bruno answered ·

Hi, can I find an Arduino/ESP32 implementation to get Victron products values through Bluetooth (BMV712, MPPT Smart Solar, etc) ?

Any Github repo or anything ?

Thank you !

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

fabians avatar image
fabians answered ·

Hi,

Thanks for the documentation.

I wrote an ESPHome component to fetch the data.

https://github.com/Fabian-Schmidt/esphome-victron_ble

Cheers

4 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

gainestr avatar image gainestr commented ·
Thank you for this. I'm hoping someone can help me with integration on a M5Stack Core2 Controller. Seems like the possibility for a remote display that uses BLE connectivity is finally in the future. I don't even own a M5Stack yet but I'll be hitting the Add to Cart button this weekend. I do see that @Wolfgang Huse has some code up but I'm certain that with this new information, it will get updated sometime in the future. Again, thanks for your work on this and sharing.


1 Like 1 ·
theterminalman avatar image theterminalman gainestr commented ·

If you want to roll your own, I have a stripped-down ESP32 example that reads and decodes the BLE advertisement beacons sent out by SmartSolar controllers:

https://github.com/hoberman/Victron_BLE_Scanner_Display

I've also written code for the very nice M5StickC and M5StickCPlus modules that uses their integrated graphics hardware to display data from the BLE beacons.

https://github.com/hoberman/Victron_BLE_Scanner_Display

Does that help?

0 Likes 0 ·
Bruno avatar image Bruno commented ·
Thanks for this implementation !

What would be even better would be to extract a general-purpose lib we can use for any ESP project, including ESPHome or any Arduino-compatible project.

What about this ? I may contribute actively to this project, in fact :)

0 Likes 0 ·
kleiner-schelm avatar image kleiner-schelm commented ·
Hello,

i have the Device RS Smart 48/6000 Solar. I want to get this Device in home Assistant with victron ble. In home Assistant i have installed ESPHome. I tryed to programm a esp32 with ESPHome. But i get no connection from esp32 to Home Assistat and no connection from esp32 to the victron RS Smart 48/6000 Solar. In the yaml Code i did not see the parameter to acitve Wlan to Router. It failed to programm the Wlan access Code in your yaml code. Can you explain, how i can get the connection with victron ble to Home assistant ?

I am old german, my english ist not perfect. I hope you understand thist and can help to solve my problem.

i get connection with ve.direct and esp8266 from Victrom Smart 48/6000 to Home assistant i miss any parameter who not shown in home assistant.

If anybody can help me, thanks for this.

0 Likes 0 ·
evangrun avatar image
evangrun answered ·

Hi. Has anyone figured out on how to keep the ble Keep-alive characteristic going? I tried several things (resending, increasing values, etc,...) but the device keeps disconnecting after exactly one minute.

What value do we need to write to the characteristic for keep-alive/watchdog?

14 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Frank avatar image Frank commented ·
Would be interesting to learn how it works.

I also tried it and my device also disconnects.
Any ideas ? Thx

0 Likes 0 ·

Hi @evangrun

What are you writing to the keep alive characteristic? The value should be 2 bytes being a 16 bit little endian unsigned integer. The value is the number of milliseconds a count down timer that performs a disconnect when it expires is reset to. Writing the value 0xFFFF disables the keep alive timer for that connection session.

Kind regards,

Thiemo van Engelen

0 Likes 0 ·
evangrun avatar image evangrun commented ·

Hi Thiemo;

This is what I observed (and semi works for me now).

a)
Start I write value 1 to the characteristic.
After a while (no pattern) the shunt will respond with a value. Always seems to be 255. If it doesnt respond it gives back no value (eg no bytes)
When I write the respond value +1 back, eg 256, it does not disconnect for a couple of minutes. I have to write that value back after every other characteristic it sends me. Interval disconnect seems random. Sometimes 15 min. Sometimes 2 min. Etc.

b) Always add 1 to the value it returns.
The shunt will then start responding with what seems random numbers (eg 74273, 31, etc). It disconnects after 1 or 2 min.

c) write no value: it disconnects after 1 min

d) write 0xffff. it disconnects after 1 min.

So pattern
Me shunt
1 -> 255
256 -> 256
256 -> 256
256 -> 256

Me shunt
1 -> 255
256 -> 256
257 --> 21830
21831 -> 743
744 -> xxx

Note that I can almost always reconnect after 1 min (unless the BLE stack gets its panties confused, where it takes like 20 tries, with random "software caused a connection abort")

(edit: I did check my endianess, and some other things) and everything works fine there.

Thanks for responding.
Erik

0 Likes 0 ·
Thiemo van Engelen (Victron Energy staff) avatar image Thiemo van Engelen (Victron Energy staff) ♦ evangrun commented ·

Hi @evangrun

Make sure that you are using a write request and not a write command. I just tested it and then it does properly respond to 0xFFFF by disabling the disconnect timer.

Kind regards,

Thiemo van Engelen

0 Likes 0 ·
evangrun avatar image evangrun Thiemo van Engelen (Victron Energy staff) ♦ commented ·

Hi @Thiemo van Engelen (Victron Energy staff)


Ill check again with 0xffff, it might be a good test to check my connection stability both on HW and SW. I am using a request in the call, In fact, when I use a command the shunt will disconnect immediatly.

Did you have some time to test the other values I am getting? I tailed some connections from my logfile just now:

2023-11-14 12:05:12:521 INFO [Device] Connected to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)
2023-11-14 12:09:19:370 INFO [Device] Disconnected from 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)
2023-11-14 12:09:19:457 INFO [Device] Connecting to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)
2023-11-14 12:09:28:621 INFO [Device] Connected to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)
2023-11-14 12:11:10:161 INFO [Device] Disconnected from 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)
2023-11-14 12:11:10:681 INFO [Device] Connecting to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)
2023-11-14 12:11:20:367 INFO [Device] Connected to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)
2023-11-14 12:16:58:531 INFO [Device] Disconnected from 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)
2023-11-14 12:16:58:627 INFO [Device] Connecting to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)
2023-11-14 12:17:07:509 INFO [Device] Connected to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)

In this case it disconnects after 4 min, then 3 min, then 5 min, then 1 min. I still dont see a pattern.

Thanks again

Erik

0 Likes 0 ·
Thiemo van Engelen (Victron Energy staff) avatar image Thiemo van Engelen (Victron Energy staff) ♦ evangrun commented ·

Hi @evangrun

I have no idea where these disconnect come from. Without any more logging on the sent and received data, it is also impossible to say. What I do find a bit strange is that it takes approx 10 seconds between the connecting and the connected. This does not have to take that long.

Kind regards,

Thiemo van Engelen

0 Likes 0 ·
evangrun avatar image evangrun Thiemo van Engelen (Victron Energy staff) ♦ commented ·
HI @Thiemo van Engelen (Victron Energy staff) ,


Ill make a debug log today with some extra information on when I and what I send for the watchdog characteristic, and when it disconnects.

What I can confirm is that sending 0xffff does work, but only after about 5-10 times sending it. When I send 0xffff, the device does not disconnect anymore. For the first few times, it seems it is just not accepted. I didnt notice that before.

It does conclude me that the disconnect is started from the device (properties), and not the linux ble stack (but I will keep it running to be sure to confirm that).

As for the 10 seconds: my connections are serialized on a thread (among other things). There's other ble connections running also (renogy, can bus, etc). The 10 seconds is to prevent a connect flood storm. When a device disconnects, I prevent a new connect to that device for 10 seconds. Could reduce that time, but works ok for me now.

Thanks, more later today. Thanks.

Erik

0 Likes 0 ·
Thiemo van Engelen (Victron Energy staff) avatar image Thiemo van Engelen (Victron Energy staff) ♦ evangrun commented ·

Hi @evangrun

Are you communicating directly via the Linux BLE stack or are you using the Bluez dbus API?

Kind regards,

Thiemo van Engelen

0 Likes 0 ·
evangrun avatar image evangrun Thiemo van Engelen (Victron Energy staff) ♦ commented ·

hi @Thiemo van Engelen (Victron Energy staff) ;

Over dbus. Running on a raspberry pi 4 (4 MB). Dbus daemon version: D-Bus Message Bus Daemon 1.12.28.

My writes are serialized, but not stalled, or delayed. Except for the connect() routines, everything is async (connect() needs pair() sometimes, and agent doesnt like async).

Some more info on the disconnect in the log below:

I added an entry when I trigger the watchdog (sending 257). You can see it disconnects at 12:43:57:244. Just before, I send a watchdog ( at 12:43:57:244). its hours::min::sec::milliseconds. The time from the send watchdog is taken right after the send. The time from disconnect() when it disconnects.

I did some more testing also, with some other values. When I send a value (like 2048) to the watchdog it will disconnect immediately. I have to send it received/read value + 1.

Now, on endianess: that routine is used in several other things, and unless I missed something badly (which is always possible), it works correctly I think.

But when I send 1, the shunt responds with 256. Which is one, just other endianess.

When I send 257, it responds with 257. Not to question you guys, but are your parsing routines ok? ;)

Again, when I send it 2, or 2048 or anything like that -> disconnect. I can confirm 0xffff works.

Logging below:
2023-11-15 12:35:58:094 INFO [Device] Connecting to 'SmartShunt HQ2303UCHFV' (D7:BB:B7:F4:BF:D5)

2023-11-15 12:35:58:419 INFO [Device] Connected to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)

2023-11-15 12:43:57:244 INFO [Device] Disconnected from 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)

2023-11-15 12:43:57:244 INFO [Victron] Disconnect received. Last send watchdog: 257 at time 2023-11-15 12:43:57:244.

2023-11-15 12:43:57:265 INFO [Device] Connecting to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)

2023-11-15 12:43:57:434 INFO [Device] Connected to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)

2023-11-15 12:43:57:934 INFO [Device] Disconnected from 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)

2023-11-15 12:43:57:934 INFO [Victron] Disconnect received. Last send watchdog: 1 at time 2023-11-15 12:43:57:611.

2023-11-15 12:44:07:069 INFO [Device] Connecting to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)

2023-11-15 12:44:07:356 INFO [Device] Connected to 'SmartShunt HQ2303UCHFV' (xx:xx:xx:xx:xx:xx)

--> Now stays connected for a bit, sending it 257

0 Likes 0 ·
Show more comments
dtschneiderlein avatar image
dtschneiderlein answered ·

Is there any way to read out consumed or charged energy (Wh)?

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.