question

Al avatar image
Al asked

VenusOS Bug?: BMS Lost: Alarm, Low Battery: Alarm, Error Code #67 - No BMS = Quattro Shutdown

I think it's a VenusOS bug that shuts down the Quattro when a BMS is removed or the communication is lost, or is this expected behaviour?

I get all the alarms in the title and a Quattro shutdown if I remove a BMS or if the communication is lost, it is exceptionally frustrating for off-grid systems.

It seems the Quattro's logic is to alarm a low battery and shutdown when a BMS stops communicating, but it's falsely reporting a low battery which is wrong. I'm not sure which of the three alarms actually causes the shutdown, but this is very unhelpful logic when off-grid, for ESS systems they can go into Passthrough but we cannot easily.

This is from at least VenusOS v2.80 to v3.30

I use the Dbus-SerialBattery driver, and if a driver update doesn't work with Venus, or the communication stops for whatever reason, unplugged cable, driver crash etc, then the Quattro turns off AC out.

Generally everything is fine and stable, but I've now had this multiple times, where it goes into a loop with me trying to update/ roll back the dbus-serialbattey driver before the system shuts down again, power off, lights off, I lose wifi, can't SSH in, until I can either fix it or re-detect VE.BUS System.

If it's deliberate logic to protect the batteries, I would suggest just stopping charging or some kind of 'limp mode', but certainly allow discharge / inverting to power AC loads, the BMS can still protect the batteries even if VenusOS isn't communicating with it anymore. In a critical system having AC out is more important than trying to protect the batteries with VenusOS when they have their own BMS anyway.

I would love this logic to be changed.

Thanks

P.S. I use CerboGX, GuiMods, Dbus-SerialBattery, BatteryAggregator, 3 x Batteries with JK BMS's

MultiPlus Quattro Inverter ChargerVenus OSBMSshutdownserialbattery
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

5 Answers
nickdb avatar image
nickdb answered ·

This is as designed for system safety. If you have a managed battery and the BMS is not seen in the timeout window, the system will shutdown. Several posts already on the subject.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Al avatar image
Al answered ·

Ok, I do genuinely appreciate all the work that's gone into making VenusOS and Victron products so great, but this is not desirable behaviour. There is no 'low battery', so it should not alarm, let alone shut down the whole system.

Maybe this is a hangover from Victron's own BMS managed batteries, but I would guess that's a minority use case scenario, or for people with a grid connection using ESS, but for the rest of us with our own BMS managed batteries it is a direct safety concern.

For system safety I would disagree that it's desirable behaviour, turning off AC Out on the whole system when off grid is by definition more of a safety risk. Maybe for an ESS it doesn't matter but off-grid it does. Think boats, navigation, critical equipment, fridges, computers, Internet connection, sewage pumps, bilge pumps...

As I mentioned above the BMS's are there for system safety, that's their role, VenusOS is not, it should not turn off AC Out for 'safety' just because the communication link has stopped.

To highlight how out of step this logic is:

Do Multi's shut down when VE.CAN MPPT's cable is unplugged?

Do Multi's shut down when VE.Bus from Cerbo to Multi is unplugged?

The answer is no, as it would be unnecessary, probably more unsafe and a hindrance, because it's only the communications.

So why shut down when only the battery communication is unplugged? It's not a direct risk or sign of something else dangerous imminently happening. Both the other scenarios have similar possible impacts to system safety and yet they don't shut the system down.

Every time it has shut my system down, it has been a massive hindrance, I was in the dark with no power yesterday, with no internet trying to SSH in and figure out what was wrong, rebooting the Cerbo until the power went off again, multiple times as I didn't have internet long enough in between the shutdowns to do anything. In the end re-detecting the VE.BUS in the Quattro to forget the BMS and run the system dumb is the only way to keep AC Out on long enough to properly fix.

It affects many peoples systems as noted in past posts, even Dbus-SerialBattery has an option to bypass this logic, but sadly if the driver itself isn't working correctly the Victron default of 60 seconds kicks in.

It would be great if this was re-thought.


1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Al avatar image Al commented ·

Here is the Dbus-SerialBattery driver way of dealing with a BMS disconnect:


--------- BMS disconnect behaviour ---------
 Description: Block charge and discharge when the communication to the BMS is lost. If you are removing the BMS on purpose, then you have to restart the driver/system to reset the block.
False: Charge and discharge is not blocked on BMS communication loss for 20 minutes, if cell voltages are between 3.25 V and 3.35 V. Else the driver block charge and discharge after 60 seconds.
True: Charge and discharge is blocked on BMS communication loss, it's unblocked when connection is established again or the driver/system is restarted. This is the Victron Energy default behaviour.
BLOCK_ON_DISCONNECT = False


It at least tries to give 20 minutes before allowing the system to shut down, but still I'm not sure that's ideal for many, being able to select an: On / Off / Timer option would be a good way to deal with it directly in VenusOS.

0 Likes 0 ·
Alexandra avatar image
Alexandra answered ·

@Al

In a system where under DVCC the battery is controlling the system (through can). And the BMS over can looses communication you want your system to just keep on going? what is there is a problem at the battery level? With lifepo4 the energy there is massive.

What is the point of BMS Control if the system will just work without it or ignore it? So a fake feature?

PS it is not immediate that the system shuts down, it actually waits about the length of time it takes for a GX reboot. Otherwise firmware updates reboot on no contact and other things like that would be impossible. It is a well thought out safety feature.

Mppts stop charging when loss of Comms to the system. The inverter is still connected to the GX and the can BMS so will continue.

Inverters shut down when the BMS is removed. Or when their Comms are removed.

If you don't want the behaviour, under DVCC you can remove BMS control on the newer firmware. That way this is not an issue anymore if you still want to have it connected to monitor and have information only from the battery.

Don't forget the #67 will reappear until you re program the mppts to stand alone (set to default and program voltages and chemistry)

7 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Alexandra avatar image Alexandra ♦ commented ·

I am running JK BMS and don't have the problem you described.

1 Like 1 ·
Al avatar image Al commented ·

@Alexandra

"In a system where under DVCC the battery is controlling the system (through can). And the BMS over can looses communication you want your system to just keep on going?"

Yes.

If I do a risk assessment on just losing the BMS communication, and another on losing all power, I know which has the greater risks. Again, fine for people with a grid connection using ESS and Passthrough, but it's a sledgehammer to crack a nut for DVCC off grid users.

I shouldn't need to emphasize here, but we are talking about a simple communication loss, we're not ignoring LV or HV battery alarms, that's literally what the BMS is for, that will shut off power at the battery level whether there are communications or not.

Of course the system should still alarm, but I'm not sure why it's considered a good thing to shut off AC Out after 60 seconds on a simple communication loss, if concern for the LFP currents was so high it should cut off immediately.

The fact that every component in the system: Inverter / mppt's etc should all be programmed for the correct voltage settings, discharge and charge rates in case of communication loss is already a secondary safety, the third safety backup being the BMS's own shut off limits.

It's simply an unnecessary hindrance, not really based on evaluating the risk scenarios clearly.

0 Likes 0 ·
Al avatar image Al Al commented ·

Please don't take this as a dig at yourself or Victron, I just don't think the current way is well considered.

I think your own system is on a yacht, so off grid? You're lucky you've not had an OS or driver update, damaged cable, failed RS485 adaptor, mistakenly unplugged RJ45, electrical interference etc, which can all cause BMS communication to fail and shut off the system.

Lets say you're sailing into a new marina, and the house battery powered all loads onboard, bow thruster, radio to marina office, electric inboard, navigation lights.. would you want to shut off all power whilst navigating into harbour, just because a communication cable came loose from a rogue wave?

What is the likely cause of a communication failure that would necessitate a whole system shutdown? Maybe only:

1. Battery fire (Unlikely with LFP)

2. BMS failure

1. = Highly unlikely, but a fire would most likely cause a short circuit and should trip the T class or similar fuse, the BMS will go over or under voltage or sense a short itself and also shut down, both will protect the battery.

2. = The BMS could fail closed circuit, if that's from over current, then likely the fuse will also blow, if not then an alarm for communication loss is all that's needed to alert the user to see what the problem is before the battery is over charged or drained, but as above all other equipment already have the correct settings to keep within the battery parameters anyway.

As both the above scenarios are so unlikely and already have mitigations in place, versus the common and benign causes that regularly and are most likely to trigger the shutdown behaviour, the inconvenience is really unnecessary.

0 Likes 0 ·
Alexandra avatar image Alexandra ♦ Al commented ·

It is not a simple communication loss, it is a loss of control. For some manufacturers they want the system to shut down on poss of Comms.


For the officially unsupported BMS there is an option to remove control under DVCC. So do that. Still allows Comms for information but not the shut down on Comms loss. The solution for you is available use it.

Lots of other BMS do not have the issue, so the issue is with the JK. They have to adjust their side.

I am not taking it personally I am saying there is already a solution built in for the JK BMS that works. I have used in on some systems already. Smooth sailing so far.

0 Likes 0 ·
Al avatar image Al Alexandra ♦ commented ·

Ok, I think we'll all have to agree to disagree :)


DIY battery users like myself who use a serial connection for closed loop control of our JK, Daly, JBD, etc BMS and use the SerialBattery driver, or ESP or even the later CAN 'Inverter' JK BMS and Seplos, Pace etc, use it specifically for BMS control for DVCC, voltage setpoints and alarms in Venus. Permanently removing BMS control means we lose that, it's not a solution. It is nothing specific to JK.

Temporary loss of control from the BMS is much lower risk than complete loss of power. (Remember there are already the fall back settings in all other devices, and the BMS itself for ultimate safety) All we need is an alarm, we don't need Venus to make that ultimate system safety decision for us.

OK maybe some officially supported batteries want the system to shutdown 'for safety' fair enough they can force enable like the DVCC settings, whether the safety risk analysis is valid seems a moot point, but for 'unsupported' BMS's we should have the option, like we also have with the other settings in DVCC.

There are many ways to lose comms it's also impossible to destroy a battery from loss of comms when the BMS and Victron system are set up correctly and appropriate fusing is used. *Caveat - If the BMS has no self shut off ability or relay/contactor it may be advisable to allow comms loss to power off - but that's a rare use case now.

The bigger picture here is protecting the whole power supply to the Boat, Home, Campervan, Business, not just a blanket reaction to a communication loss.

0 Likes 0 ·
nickdb avatar image nickdb ♦♦ Al commented ·
If communication losses are so common and exceed the timeout, around 15 minutes or so iirc, then something is wrong with your system.

I cannot recall one comms loss in years across multiple sites. Maintenance is easily completed in the timeout window.

Like I said, I have had 2 as a result of BMS failures, and in that case the system needs to go off, in one of these the battery turned itself off anyway.

This should not be a problem for anyone if their pack is working properly.

0 Likes 0 ·
Al avatar image Al nickdb ♦♦ commented ·

I've not had any BMS failures, my own packs are all working fine.


Lets not conflate lack of communications and system failures, and it's normally a software driver Mod issue when trying a new update, so granted not a Victron concern, but there's a simple solution for the many of us using unsupported BMS's and the various mods.


I have had problems with TTL/RS485 to USB connectors stop comms

I've had DC interference on cat5 cables stop comms

I've had VenusOS update that stopped the SerialBattery driver functioning stop Comms

I've had a SetupHelper/GuiMods update not re-install the BatteryAggregator stop comms


Each one of these situations causes the inverter to shut off, none are a system risk, but all need time to test or try to find a solution, all whilst the power is off.

I look after 3 other Pylontech systems, and so far they have all been fine, supported batteries are great, but they do not need to use serial connections or other mods.


0 Likes 0 ·
nickdb avatar image
nickdb answered ·

You are not considering something.

Comms loss is not a usual event. It might be more common with DIY batteries, but in a commercial battery you’re not losing a BMS unless there is a bigger problem, and in that case, you want the system off.

This caters for most users and is required for safety. I have had this happen only twice and in both cases the system needed to be down.

As mentioned by Alex, as a DIY battery you have an option to bypass this behaviour.

While it is inconvenient, when you have spent a small fortune on a supported pack, you really want that investment and warranty protected.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

captkrisp avatar image
captkrisp answered ·

Throwing in my thoughts.

Since dbus-serialbattery is written in python and pyserial has a very well known problem of communication failures, specifically with lower baud rates, the primary problem is pyserial.

The code could be structured to keep serial ports continuosly open, which will reduce the pyserial problem but not completely eliminate it. In fact, there was a piece of code added to dbus-serialbattery a couple years back to accomplish this, and was helpful to some users, but was removed for a reason that I no longer remember.

It would be far better to migrate the dbus-serialbattery code to a lower level language. C++ would be an excellent choice since it is known by many developers. Even PHP would be a better choice due to it's more stable serial comms... even though its execution would be slower than python.

Cap

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.