Statistical prediction of task execution times through analytic benchmarking for scheduling in a heterogeneous environment

Abstract

In this paper, a method for estimating task execution times is presented in order to facilitate dynamic scheduling in a heterogeneous metacomputing environment. Execution time is treated as a random variable and is statistically estimated from past observations. This method predicts the execution time as a function of several parameters of the input data and does not require any direct information about the algorithms used by the tasks or the architecture of the machines. Techniques based upon the concept of analytic benchmarking/code profiling are used to characterize the performance differences between machines, allowing observations from dissimilar machines to be used when making a prediction. Experimental results are presented which use actual execution time data gathered from 16 heterogeneous machines.