Network-aware scheduling for real-time execution support in data-intensive optical Grids

High performance communication support is a critical aspect for modern Grid environments where the deployment of data-intensive applications often requires moving Petabytes of data between geographically distant data repositories in specific time ranges. The best-effort delivery system of the Internet, often used as the underlying transport network, imposes severe constraints on the transfer of massive amounts of data, and thus restricts the deployment of the above applications on wide-area scales. Besides the lack of bandwidth, the inability to provide dedicated links makes the current network technology not well suited for performance critical Grid computing. A solution is needed to provide dedicated end-to-end connections, dynamically allocable on-demand or by scheduled reservation to critical data-intensive applications. Wavelength routing is at the state of the art the most promising solution to realize such connections. Accordingly, we propose an evolutionary Grid approach built on inter-domain end-to-end lightpath provisioning, controlled by a meta-scheduling logic on advance reservation basis, offering the required connectivity services to the involved Grid nodes by influencing job scheduling decisions with network-related information. We discuss opportunities and challenges in the network control plane and Grid middleware design, together with the required interfaces, adapters and algorithms critical to the provision of network-assisted extensible resource scheduling services. The proposed solution will result in a flexible Grid architecture that supports cooperation between different scheduling entities, based on a scalable framework for dynamic configuration and interconnection of multiple types of resources for high performance Grid applications over globally distributed optical network systems.

[1]  Ram Dantu,et al.  Constraint-Based LSP Setup using LDP , 2002, RFC.

[2]  Peter Merz,et al.  Calana: a general-purpose agent-based grid scheduler , 2005, HPDC-14. Proceedings. 14th IEEE International Symposium on High Performance Distributed Computing, 2005..

[3]  Mark J. Clement,et al.  The Performance Impact of Advance Reservation Meta-scheduling , 2000, JSSPP.

[4]  Francesco Palmieri,et al.  GMPLS-based service differentiation for scalable QoS support in all-optical Grid applications , 2006, Future Gener. Comput. Syst..

[5]  Lou Berger,et al.  Generalized Multi-Protocol Label Switching (GMPLS) Signaling Resource ReserVation Protocol-Traffic Engineering (RSVP-TE) Extensions , 2003, RFC.

[6]  Kireeti Kompella,et al.  OSPF Extensions in Support of Generalized MPLS , 2001 .

[7]  Joel Mambretti,et al.  Optical Switching Middleware for the OptlPuter , 2003 .

[8]  Andrew A. Chien,et al.  The OptIPuter , 2003, CACM.

[9]  F. Travostino,et al.  DWDM-RAM: an architecture for data intensive services enabled by next generation dynamic optical networks , 2004, IEEE Global Telecommunications Conference Workshops, 2004. GlobeCom Workshops 2004..

[10]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[11]  Jarek Nabrzyski,et al.  Grid resource management: state of the art and future trends , 2004 .

[12]  Steven Tuecke,et al.  The Physiology of the Grid An Open Grid Services Architecture for Distributed Systems Integration , 2002 .

[13]  Michiaki Hayashi,et al.  G-lambda: Coordination of a Grid Scheduler and Lambda Path Service over GMPLS , 2006, 2006 European Conference on Optical Communications.

[14]  Volker Sander,et al.  GARA: a uniform quality of service architecture , 2004 .

[15]  Kireeti Kompella,et al.  OSPF Extensions in Support of Generalized Multi-Protocol Label Switching (GMPLS) , 2005, RFC.

[16]  Bruno Volckaert,et al.  Network aware scheduling in grids , 2004 .

[17]  Rajkumar Buyya,et al.  A taxonomy and survey of grid resource management systems for distributed computing , 2002, Softw. Pract. Exp..

[18]  Eduardo Huedo,et al.  A modular meta-scheduling architecture for interfacing with pre-WS and WS Grid resource management services , 2007, Future Gener. Comput. Syst..

[19]  Jon B. Weissman,et al.  Metascheduling: a scheduling model for metacomputing systems , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[20]  Eric Mannie,et al.  Generalized Multi-Protocol Label Switching (GMPLS) Architecture , 2004, RFC.

[21]  Jonathan P. Lang,et al.  Link Management Protocol (LMP) for Dense Wavelength Division Multiplexing (DWDM) Optical Line Systems , 2005, RFC.

[22]  Harry R. Lewis,et al.  Data Structures and Their Algorithms , 1991 .

[23]  Bill St. Arnaud,et al.  Optical BGP (OBGP): InterAS lightpath provisioning , 2001 .

[24]  Michael Thomas,et al.  Data Intensive and Network Aware (DIANA) Grid Scheduling , 2007, Journal of Grid Computing.

[25]  José M. Alonso,et al.  A service-oriented WSRF-based architecture for metascheduling on computational Grids , 2008, Future Gener. Comput. Syst..