Effective management of resources on a cloud or cluster is crucial for achieving the quality of service requirements of users, which are typically captured in service level agreements (SLAs). This paper focuses on improving the robustness of resource allocation and scheduling techniques that process an open stream of MapReduce jobs with SLAs, by introducing techniques to handle errors/inaccuracies in user-estimated execution times that are submitted as part of the job's SLA. Inaccuracies in the estimates of task execution times can prevent the resource allocation and scheduling algorithm from making effective scheduling decisions, leading to a degradation in system performance. Techniques for handling error during runtime are presented to handle the situation where jobs have already started executing and their estimated execution times are inaccurate. A simulation-based performance evaluation of the error handling techniques is conducted, which demonstrates that the techniques are effective in improving system performance.

Additional Metadata
Keywords Handling error in user-estimated job/task execution times, MapReduce with SLAs, Resource allocation and scheduling
Persistent URL dx.doi.org/10.1109/FiCloud.2017.32
Conference 5th IEEE International Conference on Future Internet of Things and Cloud, FiCloud 2017
Citation
Lim, N. (Norman), Majumdar, S, & Ashwood-Smith, P. (Peter). (2017). A run time technique for handling error in user-estimated execution times on systems processing mapReduce jobs with deadlines. In Proceedings - 2017 IEEE 5th International Conference on Future Internet of Things and Cloud, FiCloud 2017 (pp. 1–9). doi:10.1109/FiCloud.2017.32