Sunday 17 November 2013

PREGEL : Graph Framework

PREGEL 


This paper talks about Pregel: a large graph processing framework designed by google.The paper first highlights the main problems in processing large graphs and then talks about how pregel works.One of the advantage with Prgel is that pregel programmes can be scaled automatically and programmer can concentrate on the algorithm they implement without worrying about fault tolerance and other mechanism. In short Pregel is a simple,flexible yet powerful framework that will find its usage in distributed social networks and other large graph processing applications.

Summery:
1)As the size of network is increasing, there representation is becoming more and more difficult with billions,trillions of nodes.This paper provides a computation model for processing these large graphs.
2)Pregel is inspired from Valiants Bulk Synchronous Programming model.
3)It focuses on message passing instead of remote reads.
4)Author provides a model of computation . Input to pregel is a large graph with each vertex identified by an ID.A sequence of supersterps are followed until algorithm terminates.
5)Paper provides C++ API and examples which can be written using pregel like PageRank,Clustering,Shortest Path.
6)The network is used only for sending messages and therefore communication overhead is significantly reduced.
7)Experiments that shows how graph scales with changes in number of vertices.

Cons :
1)Author describes fairly complex system and leaves out many details.
2)Not much experiments provided.
3)Talk about Map Reduce but nowhere they have provided evaluation on the two.
4)What will happen in case of Master failure was not answered.

No comments:

Post a Comment