Predicting the performance of wide area data transfers
- 1 January 2002
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
As Data Grids become more commonplace, large data sets are being replicated and distributed to multiple sites, leading to the problem of determining which replica can be accessed most efficiently. The answer to this question can depend on many factors, including physical characteristics of the resources and the load behavior on the CPUs, networks, and storage devices that are part of the end-to-end path linking possible sources and sinks. We develop a predictive framework that combines (1) integrated instrumentation that collects information about the end-to-end performance of past transfers, (2) predictors to estimate future transfer times, and (3) a data delivery infrastructure that provides users with access to both the raw data and our predictions. We evaluate the performance of our predictors by applying them to log data collected from a wide area testbed. These preliminary results provide insights into the effectiveness of using predictors in this situation.Keywords
This publication has 20 references indexed in Scilit:
- A time series model of long-term NSFNET backbone trafficPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- The Globus project: a status reportPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Performance prediction in production environmentsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Predicting queue times on space-sharing parallel computersPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- A distributed multi-storage resource architecture and I/O performance prediction for scientific computingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Host load prediction using linear modelsCluster Computing, 2000
- Predicting application run times using historical informationLecture Notes in Computer Science, 1998
- Time series models for internet trafficPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1996
- Predicting performance of parallel computationsIEEE Transactions on Parallel and Distributed Systems, 1990
- Analytic Queueing Network Models for Parallel Processing of Task SystemsIEEE Transactions on Computers, 1986