0

I have a highly threaded application running on amazon EC2. I would to convert this application to a cluster on EC2. I would like to use starcluster for this as its easy to manage the cluster.

However I am new to cluster/distributed computing. A after googling I found the following list of python libraries for cluster computing:

http://wiki.python.org/moin/ParallelProcessing (look at the cluster computing section)

I would like know if all the libraries will work with starcluster. Is there anything I need to keep in mind like a dependency when choosing a library since I want the application to work with starcluster?

Nithin
  • 2,223
  • 4
  • 21
  • 29

1 Answers1

1

Basically, StarCluster is a tool to help you manage your cluster. It can add/remove node, set them within a placement and security group, register them into Open Grid Scheduler and more. You can also easily create commands and plugins to help you in your work.

How were you intending to use StarCluster?

  1. If it's as a watcher to load balance your cluster then there shouldn't be any problems.
  2. If it's as an actor (making it directly do the computation by launching it with a command you would craft yourself and parallelizing its execution among the cluster) then I don't know. It might be possible, but StarCluster was not designed for it. We can read from the website:

    StarCluster has been designed to simplify the process of building, configuring, and managing clusters of virtual machines on Amazon’s EC2 cloud.

Finch_Powers
  • 2,938
  • 1
  • 24
  • 34
  • 1
    I was trying to use python-celery to distribute computations across nodes. I have not found any docs relating to star cluster and celery yet. – Nithin Mar 02 '13 at 07:37