SuperWeb: towards a global Web-based parallel computing infrastructure

Abstract

The Internet, best known by most users as the World-Wide- Web, continues to expand at an amazing pace. We propose a new infrastructure to harness the combined resources, such as CPU cycles or disk storage, and make them available to everyone interested. This infrastructure has the potential for solving parallel supercomputing applications involving thousands of cooperating components. Our approach is based on recent advances in Internet connectivity and the implementation of safe distributed computing embodied in languages such as Java. We developed a prototype of a global computing infrastructure, called SuperWeb, that consists of hosts, brokers and clients. Hosts register a fraction of their computing resources (CPU time, mem- ory, bandwidth, disk space) with resource brokers. Client com- putations are then mapped by the broker onto the registered re- sources. We examine an economic model for trading computing resources, and discuss several technical challenges associated with such a global computing environment.