Cloud Computing - Multiple Physical Computers, One Logical Computer

Question

I know that you can set up multiple virtual machines per physical computer. I'm wondering if it's possible to make multiple physical computers behave as one logical unit?

Fundamentally the way I imagine it working is that you can throw 10 computers into a facility one day. You've got one client that requires the equivalent of two computers worth, and 100 others that eat up the remaining 8. As demands change you're just reallocating logical resources, maybe the 2 computer client now requires a third physical system. You just add it to the cloud, and don't worry about sharding the database, or migrating data over to a new server.

Can it work this way?

If yes, why would anyone ever do things like hand partition their database servers anymore? Just add more computing resources. You scale horizontally with the hardware, but your server appears to scale vertically. There's no need to modify your application's supporting infrastructure to support multiple databases etc.

Answer 1

Sounds a LOT like grid-computing, try to have a look at this: http://en.wikipedia.org/wiki/Grid_computing

The cloud-services available will also let you do some sorts of this, eg. S3 Storage from Amazon, where you just upload your data to a large "bucket" and they will move the data around on their multiple machines - you still just use your "filepointer" (key) to access it.

You can even multiplex your data through something they call CloudFront (http://aws.amazon.com/cloudfront/) but its meant as a "content delivery network" providing you will loads of fast webservers sending data to millions of users. Many largescale websites use these already.

Regarding transparency with a program running which needs more power, you can either program something that will transfer the job/relay it - but you will some sort of distribution mechanism to handle it. Either as part of your operatingsystem or if its more application based, just applications working together in a "computergrid/farm".

Most large 3D rendering programs suchs as MAYA and 3D Studio MAX has "Renderfarm" software too. You place all your physical hardware machines in a network and "enable them as rendernodes" and on a "mainserver" you place your projects for rendering. Each node will then request a "job" from the master and they will render pieces of the animations and post the results back to the master/destination afterwards.

Same trick was/is used by the SETI project. http://www.seti.org/

They provide you will a small "screensaver" and this screensaver downloads a bit of data from their giant datacenter - then the programs starts to crunch on the data downloaded. Once done analyzing it uploads the results to the master and requests a new one. All happends without you do anything but install the "node".

This way the thousands of computers running this little client will help in a giant grid network.

If everyone had a "grid computer operating system" you would never know what or who or howmuch was running on your computer - that could bring up some interesting security issues + if your system got "hacked".. what then? is the whole network down or just your grid access?

For reference, please look up "THE BORGS" and "STAR TREK" too... LOL!

Answer 2

Great question. It’s coming, not out yet, but the promise of what I call multi-server-single-image is known by Intel, IBM, and TidalScale. TidalScale can deliver what you ask right now in software with 10G+ Ethernet interconnects. Intel promised the functionality in hardware 6 years ago via fiber QPI interconnects which never materialized. SGI had it long ago for supercomputers (multi-server NUMA bus) but they died. Intel is talking about it again with UPI over PCIe links. See https://www.nextplatform.com/2021/11/23/finally-a-coherent-interconnect-strategy-cxl-absorbs-gen-z/

Answer 3

It can be done, but not in a "throw them in there, hook up the hardware, and have it work automagically" kind of way.( at least, not as far as I know). You would want to configure a server to act as a proxy(on the server side of the internet) and the applications on each server. Thus, users log in to the proxy, which passes them off to an application server. Then the application server takes care of all the processor-intensive stuff(simply passing bits from one side to another isn't as processor-intensive). Then, the applications might talk to database server(s).

Citrix works this way.

Answer 4

A Beowulf cluster is approximately that, although it is specialized for scientific computing.

Also, OpenVMS has (yes, has) a clustering ability to do that, sharing disks and memory, and I think it was ported to Ultrix, which is now dead (maybe HP will port it to HPUX, who knows)