0

I'm writing a python application to manage a cluster of linux machines and I'm looking for an efficient way to send commands to all the hosts.

The general architecture of the application is:

  • Management interface
  • Command dispatcher
  • Nodes

I started to write the app using a simple python script on local nodes and then invoking it from SSH (thus using shell commands to dispatch commands) but I'm looking for a more efficient and native solution.

It would be great if I could execute python code remotely without using SSH and pass (pickled) python objects around.

Consider that it should be able to communicate to several hundred of hosts over the network and support SSL/TLS.

As a reference consider the VmWare vCenter architecture, as my goal is to create something very similar, so what kind of approach/technology you would use?

Amro
  • 123,847
  • 25
  • 243
  • 454
Martino Dino
  • 585
  • 6
  • 14
  • 1
    Sounds something like salt (http://saltstack.org/), I suspect a perusal of their code might be helpful :) – malangi Mar 03 '12 at 09:00
  • Wow fast looking at their docs this seems to be the solution, will check and adapt a sample to check if it's the right approach, thanks for pointing out – Martino Dino Mar 03 '12 at 09:10
  • @liwp thanks for the layout edit, next time will format questions better ;) – Martino Dino Mar 03 '12 at 10:38

1 Answers1

0

Like @malangi, I was going to suggest Salt as well. If you really want to roll your own solution rather than reuse Salt, you could use ZeroMQ as the comms backend (this is what Salt does). ZeroMQ is super fast and hides all the network comms from you behind a really nice and simple, yet powerful socket API.

It looks like ZeroMQ doesn't do SSL/TLS connections yet: see this SO question. But have a look at this page on the ZeroMQ wiki for alternatives.

Community
  • 1
  • 1
liwp
  • 6,746
  • 1
  • 27
  • 39
  • Apparently salt supports PKI encryption, so at least for me it's acceptable. As for the general architecture, salt seems to have all that i need, in the next days will try to deploy a small virtual test cluster and start the testing. – Martino Dino Mar 03 '12 at 10:07
  • @martino-dino seems you already found your solution, however if you somehow end up rolling your own you might also consider [Versile Python](https://versile.com/products/vpy/) for communication (full disclosure: I am a Versile developer). It enables object-level communication which may include proxies to [native python](https://versile.com/doc/vpy/current/recipes/native.html) objects, and it supports PKI secured connections. – Versile Mar 03 '12 at 11:47
  • @Versile Thanks for your response, I'll look into this new framework if it's suitable for my needs and report back. – Martino Dino Mar 09 '12 at 19:42