question

scoraigwind avatar image
scoraigwind asked

Does GX firmware 2.80 have a bug with respect to MPPT connection to BMS?

My Venus updated to 2.80. The power went off, and now my MPPTs do not work. Error #67 no BMS. Same thing happened to a customer at the same time (3 am in the night). How do we fix this?

Venus OSBMSfirmware update
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

9 Answers
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

Dear all, today we released Venus OS v2.82; which fixes this problem.

Full details for the release are in the change log, see https://professional.victronenergy.com..


Symptoms of the issue included MPPT Error #67 - No BMS, and in case no AC in is available even an inverter shutdown.


For more details, please read my earlier post.

2 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·
ps. I accepted this answer to make sure it shows up on top. And lastly: thank you again all for the reports and help in getting this sorted quickly!
0 Likes 0 ·
sean-lepoidevin avatar image sean-lepoidevin mvader (Victron Energy) ♦♦ commented ·

An update to the solution we found to our version of this error. We found a Victron community thread that is in German, explaining that replacing the cerbo solved the issue. One week after replacing the cerbo, the error has not been present and system has been stable since.

We suspect there are Cerbos with some kind of error than can't be fixed with firmware updates or any of the solutions discussed in this thread. I hope this helps others that find themselves in this error situation.

Bitte um Hilfe, Error Code #67 & Low Battery Alarm / trotz geladener Batterie - Victron Community (victronenergy.com)

0 Likes 0 ·
kwunhung avatar image
kwunhung answered ·

I have 5 systems at the same time having this problem this morning.Another one with no problems is v2.71

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

kevgermany avatar image
kevgermany answered ·

Thanks for reporting. This is being checked by Victron.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

scoraigwind avatar image
scoraigwind answered ·

In case anyone else has this issue, I resolved it by resetting to factory defaults and putting settings back into the MPPTs. Error 67 went away.

3 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

kevgermany avatar image kevgermany ♦♦ commented ·
Thanks, really helpful. Much better than reverting to the old version.
0 Likes 0 ·
Oliver avatar image Oliver commented ·
In that case you removed the Mppt from the BMS control. No external control will happen. You can xcheck by pulling it's communication cable. It will not stop the charging. Whereas it should do so on external control loss.
0 Likes 0 ·
scoraigwind avatar image scoraigwind commented ·
I did revert because I do need the BMS to work with my system but I am considering trying 2.82.
0 Likes 0 ·
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

Hi both @scoraigwind & @kwunhung is it ok if we login remotely to those systems? Thank you, Matthijs

3 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

scoraigwind avatar image scoraigwind commented ·
Yes that is fine with me. You will find they are working OK now. But you can see the history. My own home system is here on VRM 985dadd09b54
0 Likes 0 ·
kwunhung avatar image kwunhung commented ·

Yes, but how?

0 Likes 0 ·
kwunhung avatar image kwunhung kwunhung commented ·

id is ok?

id= d41243049c58

0 Likes 0 ·
scoraigwind avatar image
scoraigwind answered ·

Is there some way to block this automatic upgrade of firmware? I did not like the power failure in the middle of the night.

4 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hi @scoraigwind automatic update of firmware is -not- a default setting, it has to be enabled by user.


*some older systems might have had it enabled by default, but that has changed a couple of years ago.


0 Likes 0 ·
@scoraigwind

You'll find that option in the firmware settings on your GX device. Go to Firmware/Online Updates/Auto Updates and set it to Check only.

0 Likes 0 ·
Al avatar image Al commented ·

Yep @scoraigwind, you can turn off auto updates in the GX.

Settings > Firmware > Auto Update > 'Disabled' or 'Check only' screenshot-20220201-111751.png

0 Likes 0 ·
nickdb avatar image nickdb ♦♦ commented ·
It is also very easy to just boot back into the older (saved) version on the GX.

Perhaps you can share what specific batteries/BMS you are using?

Many of us have been beta testers and did not spot this issue, and my installs seem to have updated fine with Pylon/BSL, so would be interesting to know if this is specific to a BMS or a more general issue.


0 Likes 0 ·
scoraigwind avatar image
scoraigwind answered ·

Thanks for all the input. I am using BYD batteries, older B-boxes. It is good to know that it is possible to revert to the older firmware, but now I fixed it I will stay with 2.80.


I wonder why the inverter shut down? That is very annoying. If the inverter has to shut down for the GX firmware upgrade then it should not be done automatically in the night.

4 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

nickdb avatar image nickdb ♦♦ commented ·

This is not normal at all. Likely the lack of BMS/comms resulted in the battery shutting off.

Darth Vader will get to the bottom of it.

Countless numbers of updates are done non-disruptively.

Those of us that beta tested must have been through nearer 30 updates, with minor issues along the way, none that severe.

0 Likes 0 ·
raymiller avatar image raymiller nickdb ♦♦ commented ·
I also had a power failure using MulitPlus2 and Pylontech 2500 24V system during the manual updating of the OS and as you stated this is very unusual. The power outage was not inconvenient for me but not expected, for an update where a power outage is expected would be good to have prior warning.


But I did note the system Pylontech BMS now has control of the charging MPPTs without me doing anything. So no problems.

Thanks to everyone for all the hard work that goes into programming updates.


0 Likes 0 ·
kevgermany avatar image kevgermany ♦♦ commented ·

It's pretty normal for electronic hardware to have to shut down for an OS or firmware update. Windows, Linux, IOS, MacOS, Unix all need it at times. The issue is control. And that's done by notifications or allowed to happen automatically. So more a case of awareness than shortcomings.

Sorry it caught you, there was a huge amount of testing, but as always some bugs slip through. Lets see what they find caused the problem.


0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·
Hi, yes, the shutting down of the inverter is a safety feature: it didn't get any comms from the BMS anymore, and as such no longer knew what state the battery would be in.

Thank you for the extra details @scoraigwind , also others that posted. It helped.

Test version that fixes this is available, see below post.

0 Likes 0 ·
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

Hi both,

This issue affects systems that have a can-bus connected BMS, like a BYD or Pylontech, and at the same time have another battery monitor, or the inverter/charger itself, selected to be the main battery monitor in Settings -> System Setup.

Like here:

1643715754804.png


In such configuration, the GX would "underwater" control the Inverter/charger and MPPTs with data from BMS and at the same time use data from the other selected Battery Monitor.

Per v2.80 we changed this: when the BMS is not selected to be the main battery monitor, it no longer uses its information to control anything. It doing so before lead to (other..) issues, hence the change.

This change explains the Error #67 and the Shutdown of the Inverter: they were being controlled by a BMS, and then after the updated to v2.80 no longer received BMS data. In that case a safety feature kicks in, shutting them down to make sure they are not charging a battery while the BMS for example died or was accidentally disconnected.

There are a few immediate solutions:

1. Select the BMS as the main battery monitor in Settings -> System setup.

2. Revert to v2.73; see Settings -> Firmware -> Stored backup firmware.

3. Select the BMV, or Inverter/charger; accept that as such the BMS is no longer "in control" of the MPPT and Inverter/charger, and also make sure to restart/reset Inverter/charger, as well as set the MPPT to no longer be BMS-Controlled.


Now with above as the bare facts, I do understand and can think of a few reasons to want a BMV installed next to a BMS, one being that the SOC of the BMS is not accurate enough - even though a bit "corner cases". We're internally looking further at this - and welcome to post why you have such setups;

I saw one system with a CAN-bus connected BMS where the inverter/charger is selected as the battery monitor: why is that?

And several systems with CAN-bus connected BMS, and a BMV at the same time - and it selected as the main battery monitor. Why?


My apologies that this happened like this!

Matthijs



1643715754804.png (61.9 KiB)
6 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

scoraigwind avatar image scoraigwind commented ·
hi and thanks for the explanation.

I really do need the BYD battery to control the inverter and the MPPTs. But the battery SOC monitor in the battery is very bad. So I do need the BMV for SOC.


Please could you tell me how to revert to the old firmware, so that I can have this working the way it was before?

thanks!

Hugh

0 Likes 0 ·
nickdb avatar image nickdb ♦♦ scoraigwind commented ·
open the remote console.

Go to settings -> firmware -> stored backup firmware.

choose the option next to the older version to re-enable it.

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ scoraigwind commented ·
understood - I've accepted my own answer- hope that is ok, to have it come on top for anyone else looking for this information.



0 Likes 0 ·
kwunhung avatar image kwunhung scoraigwind commented ·

It's a painful decision

0 Likes 0 ·
raymiller avatar image raymiller commented ·
I use the BMV712 for the SOC as the Pylontech U2500 24V is very inaccurate even when I have adjusted (overtime) the BMV charge efficiency to match the real-world battery performance.
0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·

Hi all, an update: we’re planning to fix this. I expect a fix to be available as a beta version in 1 to 3 weeks.

It would be great if you can help test that and confirm it works allright for such systems.

More information about the Venus OS beta testing program here: https://www.victronenergy.com/live/ccgx:beta-testing

Meanwhile, I recommend to use v2.73. Thats the best work around.

My apologies again - we underestimated how many systems are affected by this.

Matthijs

0 Likes 0 ·
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

Hi all, we have released beta versions v2.82~1, which addresses specifically this issue (CAN-bus connected battery such as BYD or Pylon + BMV or other battery monitor). It would be great if you could test the new beta version and confirm that it fixes the problem for you.

We tested it extensively, and works well. Both on systems with an extra BMV or other battery monitor, and systems without.

Results best posted here: https://community.victronenergy.com/questions/119272/venus-os-v2822-available-for-testing.html

Thank you! Matthijs

4 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

scoraigwind avatar image scoraigwind commented ·
hi thanks,

But I got confused and installed 2.81 instead. My inverter crashed and I got error 67 on the MPPTs again. So I went back to 2.73 for now to keep the lights on in the house. I am figuring out how to do the beta test version. But I am also struggling with another issue that I want to resolve first.

I cannot access the remote console in ethernet any more. I have a customer with the same problem. He is on 2.81 and he cannot get into his remote console. Even though I went back to 2.73 I still can't get to my console. I have to go via VRM. I wonder if you can help with this problem first and then maybe I will try beta testing the beta release on my system? How do I get to remote console on ethernet. I can see it is connected and I use the correct address but cannot find it.

cheers

Hugh

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ scoraigwind commented ·

Hi Hugh, see mail. Matthijs

0 Likes 0 ·
scoraigwind avatar image scoraigwind mvader (Victron Energy) ♦♦ commented ·
I got it working by rebooting my laptop. Sorry it took me a while to think of this solution. Maybe I will try the new beta version but not today. Enough excitement.

thanks

Hugh

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ scoraigwind commented ·
Yes, great!

Wrt testing v2.82~4, so far so good - besides ourselves also two other systems tested it and all good so far.

Matthijs

0 Likes 0 ·