13

So I have this project with many dependencies that are being installed from pip and are documented in requirements.txt I need to add another dependency now that doesn't exist on pip and I have it as an RPM in some address. What is the most Pythonic way to install it as a requirement? Thanks! The code will run on RHEL and Fedora

Pavel Zagalsky
  • 1,620
  • 7
  • 22
  • 52
  • a quick dirty hack is run `yum` via `subprocess` from `setup.py` and raise an exception with appropriate error message if `yum` fails or not found. – Nizam Mohamed Dec 31 '16 at 11:36
  • @NizamMohamed - "setup.py" should know about Python dependencies, not RPMs. Your above suggestion would be extremely bad practice. – Alex Smith Dec 31 '16 at 19:40
  • @alex `setup.py` is a python script which can run any python code. `distutils` has been enhanced by non-standard-lib`setuptools`. Who defines what is a bad practice? – Nizam Mohamed Dec 31 '16 at 19:44
  • @NizamMohamed - By adding a subprocess to call yum to "setup.py", you will (1) make your Python project platform specific to Fedora/RHEL, (2) require root privilege to run any `setup.py` command, and (3) what you're suggesting adds considerable complexity to `setup.py` just to generate an error message. Remember that declaring a normal Python dependency in `setup.py` will result in an error message being generated if that dependency cannot be met, regardless of whether that dependency must be installed via PIP or RPM. – Alex Smith Dec 31 '16 at 20:34
  • @alex If the need arises (1) use `platform` module and act accordingly (2) use `sudo` to prompt (3) I said it's a hack! – Nizam Mohamed Dec 31 '16 at 21:54
  • @NizamMohamed - (1) The `platform` module isn't going to make a `yum` invocation any less platform specific. `yum` is specifically for Fedora/RHEL and derivative platforms. (2) `sudo` is only a mechanism to escalate to root privilege, which doesn't change the fact that you still need root privilege to install. (3) Yes, you did say it was a dirty hack, which I fully agree with, but I'm also pointing out that it has *no advantage* over declaring normal Python dependencies in `setup.py`. – Alex Smith Dec 31 '16 at 22:58

3 Answers3

7

In this case, the Pythonic thing to do would be to simply fail if a dependency cannot be met. It's okay, and your users will appreciate a helpful error if they haven't satisfied a prerequisite for the installation. Consider the numerous Python packages out there with C library dependencies in order to be built and installed correctly. In your project, still declare all of your Python dependencies in your "setup.py" and "requirements.txt" files, but there's nothing in the Python packaging toolchain which will install an RPM for you (nor should it!), so just stop there and let the install fail if an RPM wasn't installed.

Aside from that, you may want to consider packaging your Python application itself as an RPM. You have RPM dependencies, and your target platform is Fedora/RHEL. By packaging your application as an RPM, you can declare dependencies on other RPMs which automates installation of those required packages. Worry about being Pythonic within the build phase of your RPM, and use RPM magic to do the rest.

I recommend against the use of configuration management tools (Puppet, Ansible, etc.), as they will overly complicate your build process. Those tools are great for their intended uses, but here it would be like using a cannon to swat a fly.

Alex Smith
  • 1,495
  • 12
  • 13
5

The conventional way to manage such dependencies is using a Configuration Management system, such as SaltStack — I personally recommend it, since it is written in Python and can be extended with Python modules. Other choices include Puppet, Chef or Ansible.

Using a configuration management system, you declare the packages, config files and services to install and configure on target operation system. Then, you run the agent and it does all the heavy lifting: copying files, installing RPM packages and enabling/disabling services. The advantages are huge: your OS configuration is described as a code, you always get predictable results and save time on future installations.

Please note, that using CM introduces quite steep learning curve. However, when you wrap your head around it, you are never going back to doing OS configuration by hand.

The simpler approach could be wrapping the necessary commands in a bash script, called something like install_dependencies.sh. One is supposed to run this script as a part of application deployment, so it is a good idea to document the process somewhere.

dusty
  • 573
  • 4
  • 12
0

Since we're discussing options - I'll throw in one that hasn't been discussed.

Docker containers.

You can install a base image with the OS you want, and then the Docker file will install all the dependencies you need. It can also pip install the requirements.

This keeps the server clean of any installations it doesn't need, and versioning is easy since you'll just have a new container with new code/dependencies without overlapping with the old version.

Rcynic
  • 392
  • 3
  • 10