Merkat
Merkat is a platform that allows users of an organization to automatically manage and scale their applications while maximizing the infrastructure’s utilization. Merkat is generic and extensible, allowing users to automate the application deployment and management process. Users have the flexibility to control how many resources are allocated to their applications and to define their own resource demand adaptation policies. Merkat applies an unique approach to multiplex the infrastructure capacity between the applications, by implementing a proportional-share market and allowing applications to adapt autonomously to resource price and their given performance objectives. The price of the acquired resources acts as a control mechanism to ensure that resources are distributed to applications according to the user’s value for them. Merkat was evaluated on Grid’5000 with several scientific applications.
The main features of Merkat are:
- controlled software environments: By virtualizing resources, Merkat gives control to users over their software requirements. Merkat provides per-application isolation: each application runs in an environment customized to its needs.
- autonomous SLO-driven application management: In Merkat applications can autonomously adapt their resource demands to meet their Service Level Objectives (SLOs). Each application runs in a controlled environment, called virtual platform. The resource management is decentralized: each virtual platform can take resource demand adaptation decisions independently from the others. Thus, Merkat becomes flexible and can support multiple application types and SLOs.
- market-based resource regulation: To regulate user’s access to resources Merkat uses a market mechanism. This market relies on a proportional-share policy, providing efficient resource allocation: resources in terms of CPU and memory are allocated in a fine-grained manner to the provisioned VMs. The market’s currency distribution policy ensures that users receive limited budgets and value truthfully the execution of their applications. The combination of currency distribution and dynamic resource pricing ensures fair resource utilization.
. On top of the implemented proportional-share market, we provide two policies to scale the resource demand of the applications running on the infrastructure:
- vertical scaling: a policy that adapts the application resource demand per VM to the current resource availability and resource price;
- horizontal scaling: a policy that adapts the application resource demand in terms of number of VMs to the current resource availability and resource price.
Using these mechanisms, users can scale their applications both horizontally and vertically in order to meet their SLOs (e.g., deadlines).
Project Members
Current Status
Merkat is implemented in Python and currently relies on three services: the application management system, the VM scheduler, and the virtual currency manager. To manage the infrastructure, these services interact with an IaaS Cloud Manager. The following figure gives a better overview of Merkat.
|
Supported applications: We currently provide plugins to scale MPI applications, Torque and Condor frameworks and we are evaluating Merkat at EDF R&D using EDF high-performance applications.
Dependencies: Merkat depends on the following technologies:
- OpenNebula (as an IaaS Cloud Manager)
- ZeroMq
- paramiko
- libvirt, KVM and cgroups
- MySql
- Apache Zookeeper (soon)
Publications
Journals
- Stefania Costache, Christine Morin; Nikos Parlavantzas. “Market-based Autonomous Resource and Application Management in the Cloud”. Under preparation.
International Conferences
- Stefania Victoria Costache; Nikos Parlavantzas; Christine Morin; Samuel Kortas. “Merkat: A Market-based SLO-driven Cloud Platform”, 5th IEEE International Conference on Cloud Computing Technology and Science (CloudCom 2013), December 2013, Bristol, United Kingdom.
- Stefania Victoria Costache; Nikos Parlavantzas; Christine Morin; Samuel Kortas. “On the Use of a Proportional-Share Market for Application SLO Support in Clouds”, 19th International European Conference on Parallel and Distributed Computing (EuroPar 2013), August 2013, Aachen, Germany.
- Stefania Victoria Costache; Nikos Parlavantzas; Christine Morin; Samuel Kortas.”Themis: Economy-Based Automatic Resource Scaling for Cloud Systems“, 14th IEEE International Conference on High Performance Computing and Communications (HPCC 2012), Jun 2012, Liverpool, United Kingdom.
Workshops
- Stefania Victoria Costache; Nikos Parlavantzas; Christine Morin; Samuel Kortas. “An Economic Approach for Application QoS Management in Clouds“, 6th Workshop on Virtualization in High-Performance Cloud Computing (VHPC), Aug 2011, Bordeaux, France.
Posters
- Stefania Victoria Costache; Nikos Parlavantzas; Christine Morin; Samuel Kortas. “Themis: A Spot-Market Based Automatic Resource Scaling Framework“, HPDC 2012 Poster Session, Jun 2012, Delft, Netherlands.