Apache Mesos is a cluster manager that provides an efficient, scalable, and robust way to share resources (CPUs, memory, disk, ports, et.) across frameworks or applications that are distributed across a cluster of machines. Mesos can be thought of as a kernel for the data center that provides a unified view of resources on all nodes and seamless access to these resources.
Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks.
Getting Started
- The Mesos project has a Getting Started with Apache Mesos guide
- A simple Mesos Hello World in Scala
- Go Bindings for interacting with the Mesos API
- Apache Mesos Community on Github
- Mesosphere organization on Github
How Does It Work?
Apache Mesos is based on a master-worker relationship, but there are other important components:
- Mesos workers "own" the resources (CPU, memory, network ports, etc.) being managed by Mesos.
- The Mesos master is responsible for scheduling and allocating the resources to various frameworks and managing the task life cycle. It is the resource broker.
- Frameworks - distributed applications that run using the resources Mesos provides (run on top of Mesos). Frameworks require tasks to be performed, and tasks consume resources.
- Communication - components of Mesos communicate with frameworks over HTTP or an HTTP-like protocol using APIs.
- Auxiliary resources - these are resources required by, but not managed by, Mesos (e.g., network file system, client connections to services running on Mesos, etc.)
Examples of What Mesos Can Do
Some examples of things that you can do with Mesos:
- Run a Hadoop cluster (or any frameworks built on top of Hadoop, e.g., Giraph, Hive, etc.) while sharing the resources of the cluster with another framework
- Run a Spark cluster while sharing the resources of the cluster with another framework
- Run a NoSQL database cluster (e.g., Hive, Shark, Spark SQL, Cassandra, Hypertable) while sharing the resources of the cluster with another framework
- Run a real-time data streaming service (e.g., Spark Streaming, Storm)
- Run elastic YARN clusters (see Myriad)
- Run a standalone HDFS (see Mesosphere HDFS)
Who Uses Mesos?
Mesos helps power a large number of companies; the Mesos Project maintains a long list of companies using Mesos.
