Stack Overflow
Stack Overflow

Questions tagged [starcluster]

An open source cluster-computing toolkit for Amazon’s Elastic Compute Cloud (EC2).

StarCluster is an open source cluster-computing toolkit mainly developed in for Amazon’s Elastic Compute Cloud () released under the LGPL license.

StarCluster has been designed to automate and simplify the process of building, configuring, and managing clusters of virtual machines on Amazon’s EC2 cloud. StarCluster allows anyone to easily create a cluster computing environment in the cloud suited for distributed and parallel computing applications and systems.

52 questions
0
votes
1 answer

Compilation of c++ program in starcluster error

I'm new to Starcluster software and I'm currently trying to compile my first complex program in a 3 node cluster. I followed the instructions of cluster creation, placed the files in the sgeadmin folder and tried to compile. The following error pops…
Bernardo
  • 17
  • 5
0
votes
1 answer

starcluster / SGE launch EC2 (node / RAM management issue)

I launched several EC2 instances using starcluster and ran the same pipeline on each node. I used qsub to submit job without specifying the nodes. Why starcluster /SGE assign the job to node that already be occupied instead of to empty node? Can…
unicorn
  • 101
  • 2
0
votes
1 answer

Starcluster python3 support by 2to3 okay?

Is it okay to use 2to3.py to make Starcluster work under the environment of python3?
Lingze
  • 61
  • 2
0
votes
1 answer

AWS-Starcluster config unable to recognize region

I have StarCluster config file that looks like this: #################################### ## StarCluster Configuration File…
neversaint
  • 60,904
  • 137
  • 310
  • 477
0
votes
0 answers

mpicc and mpi_run not properly updating executables quickly

I am using Amazon EC2 nodes and running an MPI parallel program in C. I am using starcluster to manage the instances. The program compiles fine using mpicc. The executable is then on a mounted space shared by all nodes. However, when I run the…
HoosierPhysics
  • 133
  • 1
  • 8
0
votes
0 answers

For OpenMPI, I keep getting a world size of 1, even though I am utilizing 2 processors

I am using OpenMPI, starcluster, and Amazon EC2, while running through the tutorials at http://mpitutorial.com/tutorials/ . I set up a master and 2 nodes; node001 and node002. However, whenever I call MPI_Init and retrieve info from the Comm about…
HoosierPhysics
  • 133
  • 1
  • 8
0
votes
1 answer

Detect if I am in master from inside of starcluster plugin

When you try to add a node to an existing cluster of Starcluster, some plugins can fail depending if you are in master or secondary node. If your plugin has something like: for node in nodes: do something... when you add a node…
emanuele
  • 2,519
  • 8
  • 38
  • 56
0
votes
1 answer

Install Freeimage plugin in Starcluster

How can I install the Freeimage library in a Starcluster cluster so that it can be used with the scikit-image module? I set up a cluster on AWS using Starcluster and I want to run some script that requires loading .jp2 images with the scikit-image…
Maturin
  • 387
  • 1
  • 4
  • 14
0
votes
1 answer

MIT StarCluster and S3

I am trying to run a mapreduce job on spot instances. I launch my instances by using StarClusters and its hadoop plugin. I have no problem upload the data then put it into HDFS and then copy the result back from the HDFS. My question is that is…
The Zoo
  • 55
  • 10
0
votes
1 answer

Specify starcluster sshmaster port

When using starcluster, is it possible to specify the port number when SSHing into the master node? For example: starcluster sshmaster mycluster --port 22
mgig
  • 2,395
  • 4
  • 21
  • 36
0
votes
1 answer

new starcluster release broke setting up of nfs

I just upgraded starcluster to use modern machines and I now get the following traceback. Previous versions work fine with identical configs. What changes need to made to the config for it to work? It looks like it is exporting a file that does…
Michael WS
  • 2,450
  • 4
  • 24
  • 46
0
votes
1 answer

How to use IPython 2.3.1 using StarCluster in stead of 0.13.1?

StarCluster seems to use IPython 0.13.1 by default. Is there a way to upgrade this to IPython 2.3.1? Can it be done via the config file? Or manually after the cluster is started? Here is my config, with only minor security…
Dave
  • 177
  • 1
  • 7
0
votes
0 answers

Saving the system state of Amazon Ubuntu EC2 spot instances and restoring from a new instance?

I'm running different versions of an economic model (in C++) on Amazon's EC2* that take between 30 mins. and three weeks to run. I want to switch to spot instances to save money. Is there a way to periodically save the system state to a file stored…
JC4138782
  • 1
0
votes
1 answer

Monitoring StarCluster / Sun Grid Engine Cluster Performance

I am a bit new to using StarCluster and SGE. I was wondering what the best practice is for monitoring "Cluster Performance", that is, to determine how many of a certain job the cluster can run in some unit of time. I am familiar with qstat command…
user2439313
  • 1
0
votes
1 answer

Clustering in Amazon ec2 using Starcluster

Is it possible to deploy instances from AMI to multiple zones in Amazon EC2 using starcluster? Can anyone give me your feedback on this please? I need to deploy instances to various Zones in Amazon EC2.
user2470844
  • 1
  • 2
1 2
3
4