mesos: cluster management system
TRANSCRIPT
• to understand target problem domain.
• to know the operating system for the Datacenter.
• to gain an architectural overview of a Mesos cluster.
• to learn how to run applications in a Mesos cluster.
• to understand the Service Discovery & Load Balancing
• to get an overview on Mesos Frameworks.
http://mesos.apache.org Agenda
Your Expectations
Mesos is the name of the cluster management framework and the Apache Project.
Mesosphere (Mesosphere, Inc.) is the company which is commercializing the open source project Apache Mesos, and provides consulting services.
http://mesos.apache.org Mesos
Mesos vs Mesosphere
http://mesos.apache.org Problem Domain
Static partitioning Dynamic resource sharing
Resource Management
• Realignment of the partitions in the clusters means un-deploy/redeploy of the running software on the instances and requires error-prone and tedious manual effort.
• It is hard to predict the workload and fluctuations of the load makes an efficient resource utilisation impossible.
• Changes in infrastructure may cause changes in the applications so the coupling between Developers and Devops does remain strong.
http://mesos.apache.org Problem Domain
Problem Description
Datacenter as a Computer
• Process scheduling.
• Fairly resource (on the same CPU, Memory, Disk, etc) sharing.
• Package managers (aptitude, yum, etc.)
• Unified API between apps and hardware.
http://mesos.apache.org Datacenter OS
DC as a Computer
• Cluster-wide application scheduling.
• Fairly resource sharing in a distributed environment.
• Docker as package manager tool.
• Unified API between apps and hardware.
Operating Systems Mesos as a DC OS
• orchestrates the infrastructure so that it can be extended by adding new hardware or shrunk to downscale, dynamically.
• abstracts the underlying infrastructure while providing a unified API to the applications.
• employs the LXC (Linux Containers) interface to maximize resource utilisation via operating system level virtualization (like Docker or using Docker).
http://mesos.apache.org Problem Domain
Apache Mesos
http://mesos.apache.org Architecture
Architecture
Coordination over Zookeeper
Mesos CLI
Ports are ranged resources
http://mesos.apache.org Datacenter OS
How does it work ?
Mesos Framework = Scheduler + Executor
http://mesos.apache.org Datacenter OS
How does it work ?
Resource negotiation between components
• You can develop your own custom Frameworks for Mesos.
• or you can deploy open source Mesos Frameworks like Spark, Hadoop, etc.
• or, getting even better, you can deploy Docker containers.
• Mesos can orchestrate Docker containers in a Mesos cluster, as well.
http://mesos.apache.org Frameworks
Applications on Mesos
http://mesos.apache.org Frameworks
Mesos Frameworks
OSS Frameworks as Mesos Frameworks
http://mesos.apache.org Marathon
Marathon
• Marathon is the PaaS on Apache Mesos.
• Marathon provides REST API to automate your workflows.
• Marathon UI allows Dev/Ops to manage the frameworks by point-n-click.
http://mesos.apache.org Architecture
Load Balancing thru HAProxy
User Requests
http://mesos.apache.org Architecture
Service Discovery
User Requests
Mesos DNS provides its clients a
REST endpoint for Service Discovery
http://mesos.apache.org Hands-on
Hands-on
• Introduction: Mesos Dashboard
• Introduction: Marathon UI and REST API.
• Case Study: Elastic Jenkins on Mesos.
