question

Jonny Sollid avatar image
Jonny Sollid asked

Detail instructions for installing Grafana and Docker.

Hi.

I need help to make open source Grafana work on my Mac mini. https://github.com/victronenergy/venus-docker-grafana

The problem is that I struggle to install this correctly. Unfortunately I am not so good at command lines, so hope somebody could give me some tips to get this up and going?

I have cerbo-gx and this is connected with ve.can to nmea2000 cable and mqtt is activated so it should be correct. I have mac os catalina and installed Grafana and docker as described in the venus-docker-grafana project and so far I think this is ok.

-Step 2 with download of docker compose file I only get one page of text and here is my problem with command lines..

-The same goes for step 3, no idea how to place the file..

-Step 4 I do not get access to (http: // localhost: 8088 /) but docker is installed and works fine when I open the app.

-Step 5 and access to Grafane works fine.

Am I far off target or are there minor details missing? Very grateful if somebody can help solve this.

grafana
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

9 Answers
Peter Buijs - NL avatar image
Peter Buijs - NL answered ·

My laptop is an old Dell E6500 with SSD which is not capable of running HyperV but still running W10-64bits and VirtualBox latest version.

In VirtualBox I have a Debian 10 image running with docker installed.

I run the docker Venus-Grafana containers in this configuration and it works fine for me.


2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

Have you already tried watching some youtube videos on how to get docker going on macOS?

https://www.youtube.com/results?search_query=docker+on+mac


Just installing the docker framerwork/app on your mac is not enough, you must get it to download and install the Victron-applications, aka venus-docker-grafana, aka docker containers.


I've never done that myself on mac. But I can image it can even be done without resorting to the commandline. Perhaps someone else around here can help, and for sure I recommend to go watch youtube videos around the topic.

12 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

kimmo-paukku avatar image kimmo-paukku commented ·

Hello Sirs,

I would also need a help with this. By the way, does this work with CCGX and VRM ?

Im using windows 10 and I managed to install Grafana and download venus-docker-grafana-master file... but then, what I should do next...database etc... ?

Steps 1 and 2 are OK now...

Is there maybe any step by step instructions how to get this running ?

Thank you very much.

Br, Kimmo

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ kimmo-paukku commented ·

Yes CCGX and VRM is sufficient.

the database will be created on your computer automatically: inside one of the docker images that will be downloaded and started up automatically once you managed to do what you need to with that docker compose file.

I would expect there is someone here on community that can help?

Or read & study a lot. There must be plenty docker youtube videos and such.

Personally I dont have too much experience with docker sorry.

0 Likes 0 ·
kimmo-paukku avatar image kimmo-paukku mvader (Victron Energy) ♦♦ commented ·

Hi Sir,

Thanks for your comment !!

Yeah..... for sure, its better to wait help...….. Im not an expert with this... :) :)

Br, Kimmo


0 Likes 0 ·
Jonny Sollid avatar image Jonny Sollid kimmo-paukku commented ·

Have you made this work? Maybe I can help if you describe how far you have come with programming. I have just succeeded in completing the installation.

Regards

Jonny

0 Likes 0 ·
Jonny Sollid avatar image Jonny Sollid commented ·

Hi and thanks for the tips. Have watched a lot of videos and read a lot, making progress.

I downloaded the file with thermal command "docker pull victronenergy / venus-docker-grafana" but when I try to run command "docker-compose up --detach --project-name victron_grafana" as described, this does not produce any results, no conterines running in Docker. When I use the command "docker run victronenergy/venus-docker-grafana" the conteiner (jovial_mayervictronenergy / venus-docker-grafana) runs in Docker and everything looks ok but can't access the admin interface @ http: // localhost: 8088.

So my question is if I have installed image correctly?

My second question is how should I get access to the admin interface (@ http: // localhost: 8088) and what is this interface related to?

To be precise, I stuck to points 3 and 4 https://github.com/victronenergy/venus-docker-grafana

Very grateful for help that can get me one step further.

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ Jonny Sollid commented ·

Hi, I tried it the other day on W10: dowloaded the compose file, and then ran the docker compose command.


note: no pulling by me!

0 Likes 0 ·
Jonny Sollid avatar image Jonny Sollid mvader (Victron Energy) ♦♦ commented ·

When I click on link to file in point 2, I only get text, no file. Should this text be used in comand lines?


0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ Jonny Sollid commented ·

Hi, In your browser, there is a save as function, use that when seeing the text.

0 Likes 0 ·
Jonny Sollid avatar image Jonny Sollid mvader (Victron Energy) ♦♦ commented ·

Thanks so much for the information. I didn't know that this text was a recipe (yaml) file. After several hours of reading I have learned something new and interesting and now understand what this is going to do. Currently it is not possible to get file to install on mac, process does not start. Only get a list of commands for docker compose.

0 Likes 0 ·
Jonny Sollid avatar image Jonny Sollid Jonny Sollid commented ·

Make progress. I'm unsure why, but when I run the "docker-compose up" command, the installation starts and completes. Command "docker-compose up --datach" also works, but when i add command for editing name "--project-name victron_grafana
." it does not start installing. Why this does not work I am unsure, anyone who knows?

I now only have one error message to succeed. port 3000 is already in use with grafana and it prevents installation. Anyone who knows how to solve this? (See the bottom of this text)



Jonnys-Mac-mini:~ jonny$ docker-compose up

WARNING: Found multiple config files with supported names: docker-compose.yml, docker-compose.yaml

WARNING: Using docker-compose.yml


Creating network "jonny_default" with the default driver

Creating volume "jonny_influxdb-storage" with default driver

Creating volume "jonny_grafana-storage" with default driver

Creating volume "jonny_config-storage" with default driver

Pulling server (victronenergy/venus-docker-server:latest)...

latest: Pulling from victronenergy/venus-docker-server

844c33c7e6ea: Pull complete

ada5d61ae65d: Pull complete

f8427fdf4292: Pull complete

f025bafc4ab8: Pull complete

7a9577c07934: Pull complete

9b4289f800f5: Pull complete

60d1e19e1116: Pull complete

2ec0fae87e1d: Pull complete

ac5856441d80: Pull complete

19ee55f85592: Pull complete

cf65c7d973d2: Pull complete

Digest: sha256:417bc68346f219247c7102d11e30c91b1a423b9bcacc2960201e3c4cbd32b7f3

Status: Downloaded newer image for victronenergy/venus-docker-server:latest

Pulling upnp (victronenergy/venus-docker-upnp:latest)...

latest: Pulling from victronenergy/venus-docker-upnp

844c33c7e6ea: Already exists

ada5d61ae65d: Already exists

f8427fdf4292: Already exists

f025bafc4ab8: Already exists

7a9577c07934: Already exists

9b4289f800f5: Already exists

60d1e19e1116: Already exists

2ec0fae87e1d: Already exists

ac5856441d80: Already exists

6685a8597c49: Pull complete

Digest: sha256:aed57731c2381e960e5c1a09e0ce35d0f17410ba55a04e12189a046a99e0892e

Status: Downloaded newer image for victronenergy/venus-docker-upnp:latest

Pulling influxdb (influxdb:1.7)...

1.7: Pulling from library/influxdb

99760bc62448: Pull complete

e3fa264a7a88: Pull complete

a222a2af289f: Pull complete

8765a3c7874a: Pull complete

c7b3245ee24b: Pull complete

cef6cc086bee: Pull complete

f2252b187abd: Pull complete

3fc2f5b8e2f1: Pull complete

Digest: sha256:fdf22099c7f705730b71574b9b277e3f2485b4d1d952d2a189e9d3b748ac1ab8

Status: Downloaded newer image for influxdb:1.7

Creating jonny_influxdb_1 ...

Creating jonny_server_1 ...

Creating jonny_upnp_1 ...

Creating jonny_influxdb_1 ... done

Creating jonny_server_1 ... done

Creating jonny_upnp_1 ... done

: bind: address already in use


ERROR: for graphing Cannot start service graphing: Ports are not available: listen tcp 0.0.0.0:3000: bind: address already in use

ERROR: Encountered errors while bringing up the project.

Jonnys-Mac-mini:~ jonny$


0 Likes 0 ·
Jonny Sollid avatar image Jonny Sollid mvader (Victron Energy) ♦♦ commented ·

Finally everything works great! The reason for error "port 3000" was because I had installed Homebrew as described on Grafana install instructions for mac, obviously this was wrong and after uninstall everything works perfectly. There is no need for Homebrew to run Grafana.

Is there a Docker container that imports all nmea2000 from the Yacht devices gateway?

Big thanks to "mvader" who got me on the right track. This is a powerful and highly configurable display of data. Happy weekend to you all :)

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ Jonny Sollid commented ·

Hi @jonnys,

well done!

The best solution which I’m familiar with to read all nmea2000 data into influxdb is SignalK. But be prepared, thats not a “turn-key” solution.

More info here:

https://github.com/victronenergy/venus/wiki/Signal-K

Thats about running SignalK on your GX Device, which not all GX Devices can do. You can also run SignalK differently, perhaps even on your Mac, thats beyond my experience.

0 Likes 0 ·
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

Hi @jonnys and others. We made a small video showing how to install venus docker grafana


Part 1 - Getting started - https://youtube.com/watch?v=IkNuadRbANA

Part 2 - How to use & configure your graphs once up & running - https://youtu.be/B-sGH0etieM


20 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

wkirby avatar image wkirby ♦♦ commented ·

Wonderful, thanks Matthijs.
I followed your video, working perfectly in Ubuntu 18.04 server.

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·

And now there is a part two video, on how to use & configure your graphs once up & running;

https://youtu.be/B-sGH0etieM

All the best, Matthijs

0 Likes 0 ·
Jonny Sollid avatar image Jonny Sollid mvader (Victron Energy) ♦♦ commented ·
Thank you very much for the video! Just what I needed to get this set up correctly. Are there more pre-configured dashboards available for download?

@mvalder Saw a pic of your dashboard in another thread and it contains exactly the same products I have installed and was very nicely set up. If you want to share this, it would have saved me some programming. Thank you so much for developing this great data display tool.

0 Likes 0 ·
Mark avatar image Mark ♦♦ Jonny Sollid commented ·

FYI the readme file has also been updated with more information about how to setup a dashboard.

https://github.com/victronenergy/venus-docker-grafana/blob/master/README.md

Which existing dashboard are you referring to exactly?

0 Likes 0 ·
Show more comments
Rob Mitchell avatar image Rob Mitchell commented ·

Raspberry PI 4B

I've installed docker on the RIP however I have no clue what the commands are to install the docker-compose.yaml file on the command line. I'm new to this.

I've also installed Grafana for Raspberry PI following some other instructions online and I can log into it using localhost:3000 no problem. I just don't know how to install the Venus docker grafana on Raspberry Pi.

Any help / instruction would be appreciated.


Thanks

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ Rob Mitchell commented ·

Hi Rob, I recommend to watch the windows video to understand what to do with the compose yml file.


it will be slightly different on the pi, but still involves dowloading that file, storing it somewhere on the pi, and then from the commandline (aka Terminal or Shell) run the command as per r


and note that you don’t have to install grafana yourself. It will be installed automatically, as part of those docker containers.


enjoy!

0 Likes 0 ·
dunnp avatar image dunnp Rob Mitchell commented ·

Hi Rob

I'm stuck at about the same place trying to get Docker installed on a RPi3B. I was able to pull the venus image files to the Pi and get the docker-compose yaml to run - but it doesn't seem to understand the commands within the file (see screenshot). I don't know if 'bash' is the correct way to execute the yaml file, but it's the only way I've got any response other than 'command not found'. I'm not great with Linux.

The process is certainly different to Windows - no Docker Desktop that I can find; everything loops back to command line Docker installs.

Like you, I have Grafana working fine and Docker seems to be running - but getting the Victron containers into Docker has me stumped. Some step(s) missing obviously.

Any guidance from this community greatly appreciated.

Cheers

Phil

0 Likes 0 ·
1591702096227.png (430.2 KiB)
Scott Bender avatar image Scott Bender dunnp commented ·

Please see the instructions for the correct docker-compose command to run.

0 Likes 0 ·
Show more comments
sureshskr avatar image sureshskr Rob Mitchell commented ·

Please look at my comment on the below link


https://community.victronenergy.com/articles/72317/docker-grafana-dashboards-on-raspberry-pi.html

0 Likes 0 ·
Gabriel Tanasescu avatar image
Gabriel Tanasescu answered ·

I ca not install ..I have W10 Home ....message ..Docker Desktop requires Windows 10 Pro/Enterprise (15063+) or Windows 10 Home (19018+).

6 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ commented ·

Hi, yes since I have that.


have you already used google to see if there is a solution for you?

(I’m sorry, but I can’t start using google for you.. I hope you understand. If I’d do that, there would be no time left to sleep in a day ;o). ).

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ mvader (Victron Energy) ♦♦ commented ·

Ps after some reading, it looks like its possible to install docker on windows home.


two options

1) a work around by install virtual box first.

2) installing a test version of docker: https://www.docker.com/blog/docker-desktop-for-windows-home-is-here/

0 Likes 0 ·
dunnp avatar image dunnp mvader (Victron Energy) ♦♦ commented ·

Hi Matthijs

That workaround for W10 Home worked - eventually! It's a long tedious Windows upgrade with a few traps along the way. I use a Surface Pro with a only a 128GB SSD and the upgrade needs a lot of space - eg 10-15GB. Luckily it allows you to plug in an external drive to complete the upgrade. Mine took about 4hrs to complete!!

That Windows Insider Program appears to have done an upgrade to W10 Pro (from my limited knowledge).

After that the Docker/Grafana installation succeeded - eventually. I struggled with the documentation provided; it's a bit 'lightweight' and could be improved with some expansion on some of the steps listed in github.

My suggestions:

Step 1 should mention that W10 Home could be problematic because the Docker installation needs either Hyper-V or WSL2 and direct the reader to the workaround.

Running Docker is covered in the video but should be included in the written instructions. Something like:

1a Note that Docker Desktop runs in the background and nothing will appear on the screen, Click on Show hidden items, click on the Docker icon and then Dashboard.

1b Wait until Docker is running - the icon in the bottom left says 'running'

Step 4 is also covered in the video but should be described in text..

4a Open Powershell and navigate to the folder where you saved docker-compose.yaml in Step 3.

4b Execute docker-compose up --detach in Powershell

4c The installation process takes a quite a few minutes. Check it completes without errors.

Step 5 is the one that held me up for ages. It should mention the default username and password is admin and admin respectively as per step 7.

I know this all seems simple and obvious once you've done it, but for a first-timer like me it wasn't very clear.

It looks like a great tool and I am keen to explore it's capabilities. Thanks for your efforts to provide it.


Cheers

Phil

0 Likes 0 ·
mvader (Victron Energy) avatar image mvader (Victron Energy) ♦♦ dunnp commented ·

Hi Phil,

Thats sounds like a crazy long process indeed!

I’ve integrated your comments to the documentation in the readme. Most of it. For the w10 home I’m staying a bit vague on purpose.

Thank you for the input. I can’t make such instructions perfect without input like yours

Best regards & enjoy (& share a pic of your dashboard!)

0 Likes 0 ·
dunnp avatar image dunnp mvader (Victron Energy) ♦♦ commented ·

Hi Matthijs

Re W10 Home, last night I tried to install Docker Desktop on another laptop running plain W10 Home and it refused, requiring W10 Pro or a recent(?) version of Home (build 190-something) as others have reported. So I think a caution on Home is warranted in the instructions. When I have a spare few hours (!) I'll run that Windows Insider upgrade and note the steps and traps.

Here is what I've done so far...

Still fiddling with the UI and learning the 'personality' of Docker and Grafana - and plenty more questions to ask!!

Thanks for your support.

Cheers

Phil

0 Likes 0 ·
1591652615886.png (194.2 KiB)
dunnp avatar image dunnp mvader (Victron Energy) ♦♦ commented ·

Hi again Matthijs

I know you don't want to get too deep into the W10 Home bit - it's not really a Victron thing after all - but some extra detail may help people like me stumbling around trying to get Grafana working.

I have now done 4 Docker-Grafana installs and noted some traps that caught me out. My notes..

For Windows 10 Home the minimum requirement is Version 2004 Build 19041. Run winver from a command prompt to check. For earlier versions, join the Windows Insider Program (Settings>Update & Security>Windows Insider Program) which (eventually) update to v2004 build 19041. This takes several hours and quite a few restarts to complete!!

The Docker installation requires a virtual machine – either Hyper-V or WSL2 (Windows Subsystem for Linux). For these to run, virtualisation must be enabled in BIOS. The Docker installation will prompt if this needs action. It may also prompt to upgrade from WSL to WSL2.

One other thing I'm watching is CPU load - I did a clean install on a low-spec Intel NUC running W10 Home and at a 5-sec refresh rate it runs at 100% CPU most of the time. At 10-sec refresh rate it fluctuates between 20 and 100%. Obviously this will vary with dashboard complexity, but something to be aware of if running on low end hardware.

Cheers

Phil

0 Likes 0 ·
Mark avatar image
Mark answered ·

@jonnys per your request, I just made up a generic but fairly decent dashboard that you can directly import to start off with, then modify as desired to suit your system and preferences.


The dashboard json file is attached below;

Marks Basic Dashboard-1590581387865.zip

(I had to zip it in order to upload as .json file extension was not accepted)


To Import:

Hover over the + symbol to expand the 'Create' list, then click on 'Import'.

In the Import screen click on 'Upload .json File' and locate the dashboard file you want to import.

Then you should get something like this;


1590582212611.png (492.3 KiB)
marks-basic-dashboard-1590581387865.zip (5.5 KiB)
1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Jonny Sollid avatar image Jonny Sollid commented ·
Great! I set up instruments this weekend based on a dashboard image, but with this "json file" I also have the ability to see settings and many tips for setup and viewing which is of great use! Thanks for taking the time to share the dashboard!
I have installed node red and signal-k on my cerbo gx and the next step is to integrate all nmea 2000 data in grafana, exciting! :)


0 Likes 0 ·
kimmo-paukku avatar image
kimmo-paukku answered ·

Hello Sirs,

Is there any change to get/calculate Total PV Lifetime kWh ?

I have 3pcs MPPTs, those "purple boxes" and I would like to get total lifetime kWh as well.

Is it possible somehow ? …… Im not a specialist... :)

Thanks for advance.





2020-05-27-2.png (244.2 KiB)
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

solarhack avatar image
solarhack answered ·

Hi @Kimmo Paukku I installed this a few weeks ago and have done awesome things. The answer to your question is it depends.

a) You can add totals across the full life or you data (30 days be default, you can change this), but it must sum metrics stored in the DB. You would have to do a InfluxDBQL. I suggest you duplicate on of the graphs and then edit it, once in edit click the pencil icon to get into the query builders source code. The table you should use is probably "solarcharger/Yield/Power"

The following links may be helpful: https://www.influxdata.com/blog/tldr-influxdb-tech-tips-january-26-2017/

https://www.influxdata.com/blog/tldr-influxdb-tech-tips-april-28-2016/


b) You cannot do complex computations or cross query aggregations In order to do these you would have to update to the new InfluxDB or add the Flux datasource. This isn't too hard to do if you are a familiar with the Grafana. I'm busy creating my own Docker file to do just that. I'll share it once I'm sure it works.


1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

kimmo-paukku avatar image kimmo-paukku commented ·

Hello Sir,

Thanks for your reply.

….but this goes way over my skills.... so Im good without it. :)

Im not a specialist with these things... ;)

Thanks

0 Likes 0 ·
solarhack avatar image
solarhack answered ·

Check the Docker network configuration, to see what ports is is using and allowing access to.

https://docs.docker.com/engine/reference/commandline/port/

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

edgard avatar image
edgard answered ·

hola, necesito ayuda para instalar Docker Desktop en mi PC , la cual tiene windows home 2019, ya habilite la opción Subsistema de Windows para Linux en mi pc y la reinicie, pero cuando intento instalar el Docker Desktop me dice que necesito poseer: Docker Desktop requires Windows 10 Pro/Enterprise (15063+) or Windows 10 Home (19018+).




2020-06-07-225402.png (27.1 KiB)
2020-06-07-225646.png (24.7 KiB)
2020-06-07-225705.png (34.1 KiB)
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.