question

What type of system are you using for the Python scripts, an independent GNU/Linux based system, Intel or ARM such as orange or apple or raspberry Pi ?

For all my Python development I always use systemd as the manager. I have never used rc.local as a way to do this.

Systemd can do everything I need, get programs running on boot and restart if needed. Plenty of online information on its use.

For systemd , you make a unit file (a text config file) that describes the actions.

But reading what you want to do, I would look at using a micro-controller, maybe even an Arduino type board and have it do all the analogue and digital processing. Then it can supply that to your system as a modbus slave, say modbus RTU or TCP.

Using a micro-controller board is more suited and in many ways easier for many situations where you need to deal with specific I/O that is often not so easy with an OS based system. For example, you say you want to read from temperature sensors, what type, analogue or a certain type of digital, and how would you get the GNU/Linux based board to read that (knowing that there are certain Python modules for such, but are quite technical and need a good amount of expertise to use) ?

Using a micro-controller, they are generally C/C++ or in some cases, micro-python.

Right, I understand better what you have now.

I would still look into using systemd, it is the modern way to handle processes, either root or non-root. You will have good control of the scripts then, check out the online documentation. Systemd is what is run on just about al modern GNU/Linux systems now, from everything to do with startup to managing all system processes, including user processes. It sticks, meaning it is there always, regardless of any form of updates to system or user parts.

If you want to use Python, make sure you use a virtual environment, where any specific modules can live independently from main system modules. That is the method I always use. But, I don't use or run Venus, I don't like it for many reasons, so I build my own system, all Python based.

If you really get into Python, then also look at helpers like Poetry as the package dependency manager, it makes life so much easier and you can just code away.

I am thinking you should really be looking at a micro-controller. How are you expecting to get temperature data from an external sensor into the system ? Do you have good knowledge of the electrical interface connection of such sensors and the electrical interface capabilities of your RPi ?

I'm not able to help much with RPi's, I choose not to use them, though I do have some in the parts box from a past job.

What type of sensor, analogue or digital ? These things are very simple with a micro-controller.

Sounds like you really do wish to use the local user rc file over using the systemd, so I did a quick online search, there is plenty of talk about it. The link that might help you understand some more is this one how-to-enable-etc-rc-local-for-running-commands-on-linux-boot.

You should be able to make a singe Python application that does all that you want, not necessarily lots of separate scripts. Look at using asyncio to essentially multiplex between the different tasks with the one Python application. Again, using asyncio is a preferred method to concurrently handle multiple I/O bound tasks, such as handling requests from gpio.

I'm still curious on the sensors and how you will connect them and access them, are they analogue or digital sensors ? Are they maybe using an I2C or SPI interface, would be good to know ?

You say "more about how the software is limiting my use of python", what software is that ?

In your first post you mention "Meaning I want to be able to have custom libraries, modules and scripts running at boot." Do you mean you would like to have a single Python application set to run at some point in the boot process, and to keep running for ever ? Does this Python application print to screen or save data to a file or database or a network connection, how is the data being stored or saved or displayed ?

Yep, I bet there are many people who have tinkered and worked hard to make programs do what they want and are happy to share, we call it fun :)

Okay, I am understanding more now, thanks for the write up to explain it better.

Python includes the time module, yes.

I have no real knowledge of the inner workings of VenusOS, and would assume it is all based on a conglomeration of shell scripts and Python, with, maybe a sprinkle of some compiled C/C++, I am not sure at all. I know they use the dbus as a method to communicate between processes. If you look you will find some info from Louisvdw who has developed some Python code for interfacing various BMS to VenusOS via dbus, that might provide you with some insight, by looking at the code on github.

But, if the VenusOS is specifically built it may well not have the standard Python modules that normally exist. Maybe someone else can answer that.

But, if you create a Python virtual environment, then you can bring in any module you want. The virtual environment will have pip, so it can pull in any module.

Using a virtual environment allows you full freedom and not interfere with the Python parts installed that might be used and specific for VenusOS.

Using a virtual environment allows you to easily delete the your project without effecting your main system. It provides very good isolation.

Again, look at using poetry to handle the complete project, from creating the virtual environment to other structures.

For asyncio, look at the Python documentation, spend a few minutes reading it together with some other online tutorials. But to get started, just leave out asyncio until you get a basic application going.

Make sure you are using a current version of python, such as 3.9 or greater.

Sounds like you have had experience with sensors and micro-controllers and that this is not an area that is going to be difficult for you.

After midnight here, sleep time :)

Hi @Jared Bovill . I think I'm in a similar case to yours. I have a raspberry pi zero 2w running venus v2.90, and I want to run there a python script that adquire the data from the devices connected to the raspberry pi and make a data processing. I have my script written in my pc perfectly, and the problem is that in Venus I can't install the libraries I need for my script, as in Venus I don't have pip and I dont know how to get it. If you could help me I would really appreciate it

Venus OS does not have systems but does have a services mechanism. Like systems, these services start at boot and will be restarted if the process terminates.

services can be any code with a wrapper placed in /service. Actually the service is placed in /opt/victronenergy/service due to the read-only root partition. Anything there is copied to /service (an overlay in RAM disk) at boot.

You can look at the stock services in /opt/victronenergy for ideas

There is much about setting up a development environment here:

https://www.victronenergy.com/live/ccgx:root_access

As of Venus OS v2.90, the root partition is mounted read-only but there is a shell script that remounts it read-write so you can add your scripts.

dbus provides access to pretty much all resources managed by Venus OS. Anything outside of it's control would of course need it's own "driver". Much on dbus here:

https://github.com/victronenergy/venus/wiki/dbus

There are libraries to access Victron stuff including dbus for python. Many stock services are python and have copies of the library.

As you probably already know, Venus OS replaces the root partition when updating firmware. This includes /service and /opt/v*/service. So you will need something to reinstall your service after a firmware update. This is where /data/rc.local usually comes into play since everything in /data survives a firmware update.

I have several "packages" that run python code as a service to extend Venus OS capability. You might take a look at these packages to give you some ideas on integrating your code with Venus OS.

SetupHelper includes some shell script utilities to help with installing code and reinstalling after a firmware update. It includes PackageManager which runs as a python service. So this might be a good reference for ideas.

https://github.com/kwindrem/SetupHelper