question

nafets avatar image
nafets asked

Victron Bluetooth BLE protocol?

Hi,

I am looking for a possibility, to grab data from some Victrons (BMV, MPPT) via BLE for further usage in homeautomation.


Victron said, they will not publish BLE documentation

@mvader (Victron Energy Staff) answered me, i should post this Topic here...


I started looking at the BLE Interfaces of BMV712 and 3 different MPPTs

Till now i found, that the Service UUIDs and the Properties (of the 4 devices) are _absolute_ identical !


I started sniffing the BT Communication between ‚Victron Connect‘ and the devices...

The Logs are actually available in TexasInstruments SmartRF sniffer format (.psd)


I would apreciate finding somebody, to share Knowledge.

I am sure, there are some people outside, also interested in this topic.

And perhaps we can coordinate it here in this Thread ?


Regards stefan

BluetoothVE.Direct
13 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

olen avatar image olen commented ·

Hi,

I am currently working on a project to monitor various solar energy devices, by converting their proprietary protocols to MQTT (and JSON) to push them to a remote server and monitor them using whatever application I want.
I appreciate that Victron is pretty open in most of its software, but it is a shame that the Bluetooth API is still closed. I cant really see why it would be more difficult to keep a stable BT API than a stable Serial API, but lets not argue over that here.

I am more than willing to help decoding the API, and if you already have done some decoding it would be a good starting point. We can take that discussion elsewhere if it annoys Victron that we do it here.
I have already successfully reverse enginered the BLE protocols used by Meritsun and SolarLink, so I think we should be able to get this one as well...

1 Like 1 ·
olen avatar image olen olen commented ·

I now have initial support for VE Direct (BT dongle) devices in python.

What I am currently able to read from my inverter (a phoenix 12) is:
- Input voltage
- Output voltage
- Current

I can also remotely turn the power on/off/eco

I have not completely reverse engieered all of the protocol, and I might have misinterpreted some data, but what I get seems plausible. Also, I have no idea if other devices use completely different parameters.

If anyone is interested, look me up on github, or send me a direct message here.

0 Likes 0 ·
dtschneiderlein avatar image dtschneiderlein olen commented ·

Hi Guys, I'm looking for someone who can help me "writing" an Android app, using App Inventor 2 that shows SmartShunt values Voltage, Current, Power, Energy and time on one simple display. I made something with App Inventor and BLE before, but I don't think I really understand this BLE things. This is a commercial project, so I would pay for it. olen? I had a look on your GitHub project. Do you think this could work?

0 Likes 0 ·
dewitpj avatar image dewitpj commented ·

Wait....so we still doing this even thou Victron is going to release a linux version ?

0 Likes 0 ·
nafets avatar image nafets dewitpj commented ·

Suggest, to wait for the released version and have a look, whether data export (of collected data) would be possible or not

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ nafets commented ·

The linux version is 100% the same app as you know it elsewhere. Same code base, same everything, except for some platform specific issues.


but, as question for our ble protocol come up now and then, lets not go off topic here by talking about VC for linux :-)

0 Likes 0 ·
dewitpj avatar image dewitpj mvader (Victron Energy) ♦♦ commented ·

My point was if the Linux version is coming, why even bother with decoding the protocol? You can get the information (aka software) for Mac and Windows and long term data is on VRM.

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·

Hi. Thanks, this is a better place than the venus os development list.

I took the liberty to change the title of your question a little bit; to make it more clear.

Do note that I’m a bit “split” here: as also explained elsewhere I am (still) convinced that publicly documenting our bluetooth protocol is not a good idea. We won’t be able to support paying customers with their implementations, and neither the more individual user aiming to make this work.

The protocol has been designed by us, and is complex: for example, we chose to *not* make a protocol in which each value has its own uuid. Opposite to the common heart rate monitor and other BLE examples that float around the web. We use just a few UUIDs, and they’re used for transport.

It uses CBOR and VREGs.

As just me Matthijs, I wish you lots of fun and success; and in my function in Victron I wish you the same ofcourse, but do note the warning above; and sorry I cant “just” publish the docs here.

-1 Like -1 ·
rk31 avatar image rk31 mvader (Victron Energy) ♦♦ commented ·

"We won’t be able to support paying customers with their implementations"

I don't think we're asking for support of customer-implemented software.. But for those of us who have a need to publish how to do basics with your BLE devices.

We have a need for remote/off-grid visibility into solar performance, and while the 100/20's used currently are good when you're within 10 feet (BT range...), I won't be adding anymore if we can't use a mini-pc or rasp-pi to remotely get battery voltage/solar production.

So, +1 for documenting even a hello-world example of reading some data from these devices.

thanks

1 Like 1 ·
nafets avatar image nafets mvader (Victron Energy) ♦♦ commented ·

Thanks for your explanations, Mathijs !


Wouldn‘t it be perhaps a good idea, to add a Public Part for BLE Communication ? Maybe readonly



I have nearly 10 Victrons installed in my basement (in different rooms) and its not so easy to connect each one with a cable.

IMHO, BLE communication would perfectly fit in this scenario.

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ nafets commented ·

what are they doing in all those basements? ;o).


Yes, that would be nice. We'll look further into using BLE for data logging, (or GX-device to normal product communication) somewhere in the future. For now, focus lies elsewhere.

0 Likes 0 ·
nafets avatar image nafets mvader (Victron Energy) ♦♦ commented ·

In fact, its 3 different DC systems. 2x 12v and 1x 48v, due to my structural conditions and positioning of panels.


I would appreciate BLE datalogging very much !

Do you have a rough idea, when this might be ?

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ nafets commented ·

Hi. No, sorry. No idea.

-1 Like -1 ·
11 Answers
jeremy-f avatar image
jeremy-f answered ·

That's a shame. We use MPPT regulators in remote parts of Australia for road condition signage. We were hoping to read the data from the regulators to save people having to drive a 2500 km round trip. I hope you reconsider allowing third party read access for Voltage and Current in the future.


3 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Ole Saether avatar image Ole Saether commented ·

BLE is short range :-)

I have several Victron devices and I read them over the Internet 110 km away from the devices. Most Victron MPPT regulators have a VE.Direct port or CAN-bus port that you can connect to third party devices. The spec for the VE.Direct protocol is available on the Victron web site:
https://www.victronenergy.com/support-and-downloads/whitepapers

0 Likes 0 ·
Show more comments
saxofon avatar image
saxofon answered ·

One more with need of reading out values via BLE, so please don't hesitate too long with this ;-)

For my usecases, I'd like to

Smart battery : read temp, package voltage and cell voltages.

Smart shunt : read temp, voltage, current and soc.

Smart battery protect : read/write load on/off.

For example, when doing the initial charging of the Victron smart batteries, it would be perfect to be able to log cell voltages along the process and store it as documentation of work done.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

krajan avatar image
krajan answered ·

I have a BMV602 (battery monitor) in my boat. I use an Arduino to log all the data from the batteries to an SD-card. I can take the data right in to Excel, but not more extravagant than that(!). And since the BMV is an older version everything goes over TTL serial. Extreamly easy to handle and I love that Victron made that available (thank you!!).

So I really would like to have something similar to go with my charger over BT. It's a bit tricky for me to reverse engineer the handshakes and data format so I would be enormous grateful to have anything (even a little) helpful to start with.

I took a class in a half semester at the university studying BlueTooth back in 2002, and I still remember some of the tech :D


/Krajan

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

graham-stinson avatar image
graham-stinson answered ·

Upvoting this as well. We use iPhones as stockpile inventory monitoring cameras, powered by solar, controlled by vicron bluesolar 75/15 MPPT. Our cameras connect to our portal, where we monitor our entire systems. It would be incredibly helpful for us to be able to monitor our solar power status in the same way.

Our intent would be to connect each camera to the MPPT controller via Bluetooth within the same app that monitors the stockpiles, and upload that status data to our portal for remote system monitoring. That would give us literal days of notice between the time that one of our power components goes down and the time that the camera runs out of power.

Any knowledgeable suggestions on how we might do this would be greatly appreciated. Thanks!

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

olen avatar image
olen answered ·

I have now pushed my findings and a working example for monitoring data from a Phoenix inverter by python here: https://github.com/Olen/VictronConnect

Feel free to use it as you want.

1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

timo-kovanen avatar image timo-kovanen commented ·

It looks like Victron BLE notifications uses same vregs as VE.Can (with some undef. vregs also). 4th and 5th byte are vreg value, 6th. datatype and rest depending datatype.

0x41 un8

0x42 sn16

0x44 un32

0x58 array (next byte len)

Messages can continue over multiple notifications.


0 Likes 0 ·
fenix avatar image
fenix answered ·

i used 'Home Assistant " to integrate Victron data....i also include other sensors...


1608645398860.png


1608645398860.png (97.4 KiB)
1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

rk31 avatar image rk31 commented ·

This is nice, thanks for sharing.

Do you have the code on github or somewhere? Are you getting the data from the BT connection or hardwired?

0 Likes 0 ·
whnz avatar image
whnz answered ·

Hey @mvader (Victron Energy),

Just wanted to add my 2 cents here - I too would love the Bluetooth protocol doc published.

It seems you guys really hit the nail on the head with open sourcing everything. One of the reasons I keep buying more and more Victron gear :)

It does seem to defy a bit of logic that you can open source VE.Direct HEX Protocol, and not get your developers slammed with support requests, but can't do the same with bluetooth.

Please - we promise not to complain when then format changes later on, and we promise not to flood your support lines for help with my DIY BLE reader!

It seems exactly the same as everything else - no support (at all!) and may change at any time.

Cheers

/Matt


1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

whnz avatar image whnz commented ·

Just to add an example of how I would use this - 10 cars parked in a garage. Smart Battery Sense in each. Custom Pi / ESP to display all the battery voltages and alert when one has a low battery.

0 Likes 0 ·
timo-kovanen avatar image
timo-kovanen answered ·

For advertised data, just read notifies from [306b0003-b081-4037-83dc-e59fcc3cdfd0] and [306b0004-b081-4037-83dc-e59fcc3cdfd0] combined as a stream.

Message format is 08-00-19-VREG-CBOR (data as little endian).

Some examples from Blue Smart IP22 24|16 (3):

080019ed8d428d0a vreg 0xed8d (Channel 1 voltage, voltage in 0.01V, sn16), cbor 0x42 (byte string, len=2), value 0x8d0a (little endian 2701 > 27.01V)

080019ed8f420c00 vreg 0xed8f (Channel 1 current, current in 0.1A, sn16), cbor 0x42 (byte string, len=2), value 0x0c00 (little endian 12 > 1.2A)

080019ed8c44d8040000 vreg 0xed8c, cbor 0x44 (byte string, len=4), value 0xd8040000 (little endian 1240)

080019edd5428d0a vreg 0xedd5 (Charger voltage, voltage in 0.01V, un16), cbor 0x42 (byte string, len=2), value 0x8d0a (little endian 2701 > 27.01V)

080019edd7420c00 vreg 0xedd7 (Charger current, current in 0.1A, sn16), cbor 0x42 (byte string, len=2), value 0x0c00 (little endian 12 > 1.2A)

080019012044bc0c0400 vreg 0x0120 (Uptime, uptime in seconds, un32), cbor 44 (byte string, len=4), value 0xbc0c0400 (265404 seconds)

080019eddb42d002 vreg 0xeddb (Charger internal temperature, temperature in 0.01 degrees centigrade, sn16), cbor 0x42 (byte string, len=2), value 0xd002 (little endian 720 > 7.20C)

Vregs are based on vecan public documentation.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

wcn00 avatar image
wcn00 answered ·

If anyone can share sniffer or wireshark recordings of bluetooth interactions with the bluetooth shunt device I would be happy to collaborate on creating a simple app that records voltage and current on the shunt. Getting just those two values should be possible. I've already got my raspberry pi talking to the multiplus charger inverter and I get everything but battery current there. I would like to be able to present a comprehensive dashboard about battery usage, cycling, temp etc on the head unit in my RV. I'm happy to share all that for folks with similar equipment.

I'm also going to install 600w of solar next summer soon as I figure out how to attach things securely to the roof of my RV.
Happy trails everyone.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

birdie avatar image
birdie answered ·

We did some work on reverse engineering the ble protocol.

We have two different repositories with the results.

This is for development, It contains a wireshark dissector and some python scripts: https://github.com/vvvrrooomm/victron/

This is one is more ready for usage, You can output the data to syslog or mqtt to integrate it in homeassistant or other: https://github.com/birdie1/victron


We had 4 devices:

- Smart Shunt

- SmartSolar MPPT 100/30

- Orion Smart

- Phoenix 12-800VA -> Serial only


This is already reverse engineered and working:

- SmartShunt: Charge Status, Time to go, Voltage, Current, Power, Starter (second battery), all the history values like deepest discharge and stuff, setting some of the settings like capacity

- SmartSolar: Power, Voltage, Current. The history values are difficult. We have the hex values and can decode it, but we don't know yet what we need to send as init sequence. The smartsolar just doesn't send out the history values. Some settings are available as well.

- OrionSmart: I am not sure what is working correctly right now, but I think some settings like shutdown and start voltage can be already set.


And again to the Victron Company, please release the protocol description, it is just so unnessessary work to reverse engineer it. Like always in open source, I think there are many people out there who are willing to put some work and create nice scripts and help people integrate all these devices into homeautomation system.

4 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·

Nice work - and sorry to not make life easier for you - I think I explained the why behind that elsewhere. Either in this thread or another one.


But I do have some good news on this: we’ve been working on a new way to get Victron data via BLE: by having simple GATT attributes. One for voltage, one for current and so forth.

I expect to announce that including a test firmware for the SmartShunt within 4 to 6 weeks from now; hopefully even sooner. Other products will follow.

Announcement will be on modifications section here in community.

Have a good Sunday, Matthijs

-1 Like -1 ·
birdie avatar image birdie mvader (Victron Energy) ♦♦ commented ·

Yes, you explained it, but I still don't get it. Nobody expect to have a perfect non changing bluetooth protocol/api. It is ok if you just change it again. But if you release to protocol description (or at least the source code of the VictronConnect app) it will be very easy to change our scripts as well. But if we need a restart of reverse engineering, it will just cost us a creazy amount of extra time.

1 Like 1 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ birdie commented ·

We’ve published other complex protocols in the past, and the result was not good.

People/DYI/professional customers, sales, etc. will expect support; and the only people on our side that are able to give such support properly are those that I want to work on new products & features , not give support on complex protocols to either overcome lack of quality in documentation or lack of experience of the person working to implement it.


Anyway; its a choice and for now its made and I’m not going to change my mind on it. Sorry.

I really think an easy to implement protocol serves everyone best, and static GATT attributes how we’re going for that - and its almost finished.

-1 Like -1 ·
birdie avatar image birdie mvader (Victron Energy) ♦♦ commented ·

The simpler GATT attributes sounds at least good. Looking forward to test the new firmwares once its ready.

0 Likes 0 ·
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·
1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

david-ross avatar image david-ross commented ·
Any plans to expand BLE to the Orion chargers or any other device?
0 Likes 0 ·