question

xtratrestrial avatar image
xtratrestrial asked

VenusOS 2.80 / 2.81 + GuiMods - Issues on Multiplus II GX 48/5000/70-50

Equipment

  • MultiPlus II GX 48/5000/70-50
  • Autotransformer 120/240-100
  • SmartSolar MPPT 250/100 rev2

MultiPlus Software

  • Firmware Version 494
  • VenusOS 2.81 + GuiMods (!)
  • Product ID 2623

Issues

  • VRM warns "Installation too busy. Realtime data link disabled."
  • VRM realtime data never happens and doesn't update until refreshed (still refreshing to lagging data)
  • Local & remote console both show lagging data (local console approximately 15s behind)
  • Hum from inductive loads

The issues mentioned above began on 1/ 31, leading me to check firmware status and see if the MultiPlus took a firmware update. It had. I figured the VRM issues were caused by a firmware bug, but the hum is very strange. It's the same noise I've heard from the microwave and espresso machine when using our backup inverter (non-pure sine). I haven't looked at the sine wave yet, but console shows us at 60.1Hz still so, not sure what's going on there.


Thought I would just deal with the data issues until others complained and a firmware update was issued. I see that this already happened, but it was for a different issue related to internal modem using PIN locked SIM. Hence, 2.81 installed with auto update still on at the time and it did not fix my issues. In fact, a bunch of my settings were lost including my AC current input limit causing me to stall out the generator last night.


Still seeing all the above issues, even after attempting several things. Option to revert to 2.73 is taken away with the issue of 2.81 update and the nanopi only stores current and one old firmware. Attempted restore to 2.80 and, as you would expect, issues still exist. Attempted to download firmware to revert to 2.73 from the Victron Professsional portal, but only 494 firmware exists for Product ID 2623.


Two Requests:

  1. Please fix the issues within 2.81. I was excited to see this firmware as I've been waiting for some of the implementations, such as the tank sensors. Hopefully this information is helpful in working bugs out.
  2. Where can I get archived firmware to revert to for my device. I absolutely have to go back to 2.73 at this point. This firmware unfortunately is not release candidate worthy yet.
firmware update
1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·
Hi @xTratrestrial , as this is not standard Victron software, but self modified, I changed the title a bit to make it clear as well as moved it to the Modifications space.
0 Likes 0 ·
4 Answers
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

Hi both- thanks. Noted, and I sent @xTratrestrial an email with a request if we can login. We have several systems with MP-II GX and ES-II GX in our lab; all not having this issue.

Hence keen to login.

to let us log in, enable Settings -> General -> Remote Support.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Matthias Lange - DE avatar image
Matthias Lange - DE answered ·

Via Victron Professional you can find a text file with a link to a Venus OS archive.
https://professional.victronenergy.com

You are mixing up something here.
2.81 is the version of Venus OS running on the GX module inside the MultiPlus.
Venus OS updates only automatically if you set it up to do it. The default setting is to check for updates but NOT install them.

2623 is the hardware ID of the MultiPlus and 494 is the FW version on the MultiPlus.
The FW on the MultiPlus will never be updated automatically and there is also no setting to do that.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

xtratrestrial avatar image
xtratrestrial answered ·

@Matthias Lange - DE - You are correct, I was confusing the VenusOS version and device firmware. I thought the two were simply different references to the same image. I do see seperate images for device firmware and VenusOS installations looking at the Professional portal now. The thing that's misleading is that the VenusOS version shows under "Firmware" in remote console. Hence, I assumed that updating one contained the image for device firmware and installed both together. Looks like that's incorrect.

That being said, the interesting thing is that everything broke with the VenusOS update. It happened all on the date that it was released and I haven't played with anything in months. ...and yes, I know that I enabled auto update. I didn't set it to install latest release candidate though, so I figured that QA would have been thorough with any release that it auto updated to.

@mvader (Victron Energy) - Thank you. I will get remote support enabled and respond to the email. For now, I figured a few things. In an attempt to get things working right again, I reverted to 2.80 and allowed the GX to auto update to 2.81 again. When it completed this time, I noticed that it was running at 50Hz! As stated in my original post, I checked this yesterday when I noted that the microwave and espresso machine sounded like they were using a dirty sine wave (a space heater we have wasn't working either). It said it was at 60Hz at that time, but I think the software was lying to me after the update! As soon as I downloaded the configs today and rebuilt them from scratch, as using my old configs was not working in Remote VE Configure, everything has returned to normal with the exception of the VRM portal, which still shows "Too busy. Realtime link is disabled", and shows that whether connected via Ethernet or WiFi, and WiFi was tried with home network and WiFi hotspot on a phone.

tldr;

  • Reverted to 2.80 and allowed to take 2.81 again. Lost frequency setting and many others.
  • Original 2.81 update showed running @ 60Hz, but all inductive loads behaved like 50Hz.
  • After 2nd 2.81 update and rebuilding config from scratch, shows running at 60Hz and loads behave/sound like normal.
  • VRM is still not working right. Never goes to realtime. Always shows "Too busy. Realtime link is disabled." Never happened ever before prior to Jan 31 v2.80 update.


nanopi resources are healthy:

Mem: 309516K used, 200808K free, 1852K shrd, 12708K buff, 90644K cached

CPU: 1% usr 1% sys 0% nic 96% idle 0% io 0% irq 0% sirq

Load average: 0.09 0.26 0.37 2/305 6400


Haven't looked at logs yet. I'll enable remote support and respond to the email so you can.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

xtratrestrial avatar image
xtratrestrial answered ·

Update: mvader reached out to me and requested remote access. He logged into my MultiPlus II GX and saw that I had in fact installed GuiMods. I read that GuiMods doesn't survive update, but I didn't know that it could interfere with an installation during update. Turns out that this was the case. It caused the issues that were stated above, somehow....and caused the issues with the lag I was experiencing.

I haven't been using GuiMods, as I was never able to get the GeneratorConnector to work anyway, since I had to build a custom assistant for it and use two device's relays to get remote start to work with my generator (Predator 9500), since it doesn't use a control board and needs seperate start/stop circuits. When mvader looked into dbus on my install, he saw a ton of messages tossing errors around from non-Victron interfaces.

Later, I looked at the dbus output as well and saw the same. I attempted to uninstall GuiMods to fix this, but was unsuccessful with all attempts:

Tried the following:

  • Removed the hooks (/data/rc*.local) and downgraded/upgraded to 2.81. Still saw same dbus output.
  • Ran all installed GuiMods setup scripts manually and used their uninstall option. Downgraded/upgraded to 2.81. Still same output.
  • Ran "blind install" of SetupHelper and used it to upgrade/reinstall all scripts I originally installed, then uninstall and downgrade/upgrade to 2.81. Still same output.

Finally, I setted on factory reset, covered here:

https://www.victronenergy.com/media/pg/CCGX/en/factory-reset.html

After factory reset and reconfiguring, latency has returned to normal in VRM Portal.

Thanks for your help @mvader (Victron Energy) ! Class A customer service and support. Never had a customer service experience that was quicker and more involved. Very much appreciated.

1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Kevin Windrem avatar image Kevin Windrem commented ·
I am concerned that GuiMods or SetupHelper could cause such a system degradation.

My packages ARE designed to survive a Venus firmware update but there are cases where the currently installed package version isn't compatible with the most recent firmware. I have seen this cause PackageManager to attempt to reinstall the package over and over again. This is something I need to prevent in the future but I don't think that was the cause of dbus being overwhelmed. I could be wrong.

In any case, if they are still laying around, I'd like to review your logs. If you can zip them and post them to eight my SetupHelper or GuiMods repo at GitHub I'll take a look:

/data/log/SetupHelper (a single file)

/data/log/PackageManager (a directory containing up to 5 different log files)

Thanks.

0 Likes 0 ·