BigdataNet is an associated team (“équipe associée”) between Zenith and the distributed systems team of Profs. Amr El Abbadi and Divy Agrawal at University of California, Santa Barbara, since january 2013. The associated team is headed by Esther Pacitti and Patrick Valduriez for Zenith, and Amr El Abbadi and Divy Agrawal for UCSB.
With the advent of the Internet and the World-wide-web, there is an emergent need to develop user applications that access data and resources stored in the network. In order to facilitate the development of network-centric applications, new computational paradigms are needed that are scalable, elastic, available, and fault-tolerant. During the past decades two dominant paradigms referred to as Peer-to-Peer (P2P) Computing and Cloud Computing have become widely prevalent as computational paradigms for distributed applications. Peer-to-peer computing is a highly decentralized computing paradigm that leverages computing resources at the user level for supporting decentralized user level applications such as wide-scale media file sharing, telecommunication services (e.g., Skype), and others. Cloud computing on the other hand relies on large data-centers consisting of thousands of server-class machines and all application processing and application data is centralized in the network core, i.e., data-centers. The two paradigms in many ways are complimentary and provide different trade-offs. For instance, the cost for computing and storage is almost free in P2P but it suffers from the challenges of churn and low reliability of user machines. Cloud computing, on the other hand significantly simplifies the task of system administration in the data-center but requires a very large investment in building large-scale data-centers.
We plan to develop a hybrid platform that combines the two paradigms and leverages computing, storage, and network resources both in the data-centers (i.e., the cloud) as well as at the edges of the network (i.e., the peer or user machines). During this project, we will explore the suitability of this hybrid model for big-data applications such as scientific applications, social networks, and massive data analytics. The research challenges that need to be addressed are the massive scale of both systems and data, complexity of the overall architecture, and heterogeneity of overlay network.