Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [distributed-computing]

41 questions
1
vote
2 answers

Server utilization and how to deal with datacenter failure

New technologies like Docker, Mesos, Kubernetes etc. allow much better server utilisation in an organisation. However, I'd like to know how utilisation can be maximised across two datacenters, taking into account the failure of an entire…
Donovan Muller
  • 135
  • 3
1
vote
2 answers

Distributed storage and computing

Dear Serverfault community, After researching a number of distributed file systems for deployment in a production environment with the main purpose of performing both batch and real-time distributed computing I've identified the following list as…
Tim van Elteren
  • 11
  • 1
1
vote
2 answers

When should an EC2 instance use "Amazon instance store-backed"?

What are the benefits over using Amazon instance store-backed over EBS block storage? The only thing I can see from the comparison of the two is that instance store-backed seems to incur less charges. I will be deploying multiple instances of the…
S-K'
  • 1,301
  • 3
  • 11
  • 15
1
vote
3 answers

Distributed computing through AWS

I know it's possible to balance traffic using ELB/EC2 instances but is there any way to distribute CPU calculations over a cluster of instances. I'm wanting to process lots of video/audio using ffmpeg, but if the server gets too slow, I'd like to be…
Jamie Taylor
  • 372
  • 1
  • 6
  • 15
1
vote
2 answers

Build parallel processing grid from 32-bit desktops

I am running simulation on my laptop(Core 2 Duo 1.8Ghz with 4GB RAM running windows 7) which is taking very long time. I have a couple of unused 32-bit (Core 2 Duo 1.8Ghz with 4GB RAM) desktops. I would like to connect them(as slaves) in a grid…
8A52
  • 113
  • 2
1
vote
1 answer

Can kubernetes cluster nodes be geologically spread out?

Me and a couple of friends are thinking of setting up a kubernetes cluster where our homeservers will act as the nodes. As our nodes will be spread out between our apartments I am worried this will create problems when it comes to: Exposing…
Hannes Knutsson
  • 37
  • 1
  • 3
0
votes
0 answers

Need to convert 1000 images in 5 seconds-- what are our options?

we're looking for a server architecture that will allow for converting 1000 large images in 5 seconds. As a test, we ran some benchmarks using a 16 core server, using GNU Parallel to run 1,000 image conversions. ls -1 *.pdf | parallel --eta convert…
user1661677
  • 111
  • 1
0
votes
1 answer

How to manage failover in ZooKeeper across datacenters using observers

I have an application running in 3 different datacenters that uses ZooKeeper for many tasks. Following the recommended practices, we have deployed three ZooKeeper ensembles where one datacenter contains common ZooKeeper instances and the other two…
Matheus Portela
  • 181
  • 1
  • 9
0
votes
1 answer

Nginx as GeoIP Global LoadBalancer and actual distance from app server

my client is using Magento for it's ecommerce global solution. Main country are US / EU / CN. For now only 1 VM in Italy, serving all traffic. I would like to create 3 VMs, one for each country, on AWS or Google Cloud, not important at this…
x86fantini
  • 302
  • 1
  • 3
  • 9
0
votes
1 answer

How to distribute multiple executions of an app across many machines

I've got a simulation app (64-bit windows) that runs without any user interaction. This app gathers information and pushes it to a remote MS SQL Server. What I'd like to do is execute this simulation as many times as I can on multiple machines…
Salec
  • 101
  • 1
0
votes
2 answers

Distribute the load all around the world

Let's say we're starting a project which is meant to be served all around the world How am I supposed to distribute the database and server load, and increase (optimize) the service for large audience? I know about reverse proxy, load balance, using…
Marek Sebera
  • 271
  • 3
  • 16
0
votes
2 answers

Calculating the power of a cluster vs a single workstation

For the same price we can get either Worktsation which is 2 x 6 core Xeons 3.4GHz, 128GB RAM, 600GB SAS RAID 0 = 12 Cores, 128GB RAM Or Cluster 8 x (4 Core i7 3.4GHz, 16GB RAM, 250GB SATA, Intel 82579LM), Gigabit ethernet Netgear switch = 32 Cores,…
Hax
  • 139
  • 6
0
votes
1 answer

Using Opennebula as an IaaS: Running benchmark accross all nodes, possibe?

Our current situation: We have one Opennebula cloud infrastructure consisting of a cluster controller and 6 cluster nodes configured and working. All Nodes run debian instances on top of a kvm hypervisor. We have done this as a part of a school…
user101012
  • 13
  • 1
  • 5
0
votes
2 answers

Condor Compute Cluster: quick information about Distributed Systems?

If you guys have never heard of the idea behind Condor Cluster: http://www.cs.wisc.edu/condor/ It's developed by the University of Wisconsin but used by many other universities since its birth. I was just wondering, in general though, how Cluster…
Dark Templar
  • 97
  • 4
0
votes
2 answers

Good books on engineering distributed systems

I am looking for a good book / blog / resources on Engineering Best Practices (developing, testing, automation, tools & deployment) for distributed systems. Most of what I see is principle, architecture and algorithm oriented book. Is there an…
Ravisankar S
  • 11
  • 2
1
2
3