Priority based resource scheduling techniques for a resource constrained stream processing system
A multitenant Storm cluster runs multiple stream processing applications and uses the default Isolation Scheduler to schedule them. Isolation Scheduler assigns resources to topologies based on static resource configuration and does not provide any means for prioritizing topologies based on their varying business requirements. Thus, performance degradation, even complete starvation of topologies with high priority is possible when the cluster is resource constrained and comprises an inadequate number of resources. Two priority based resource scheduling techniques are proposed to overcome these problems. A performance analysis based on prototyping and measurements demonstrates the effectiveness of the proposed techniques.
|Keywords||Big data, Distributed systems, Middleware, Resource allocation, Resource management, Resource scheduling, Stream processing|
|Conference||4th IEEE/ACM International Conference on Big Data Computing, Applications and Technologies, BDCAT 2017|
Chakraborty, R. (Rudraneel), & Majumdar, S. (2017). Priority based resource scheduling techniques for a resource constrained stream processing system. In BDCAT 2017 - Proceedings of the 4th IEEE/ACM International Conference on Big Data Computing, Applications and Technologies (pp. 21–31). doi:10.1145/3148055.3148066