ScaleG: A Distributed Disk-based System for Vertex-centric Graph Processing (Extended Abstract)

Wang, X; Wen, D; Qin, L; Chang, L; Zhang, W

ScaleG: A Distributed Disk-based System for Vertex-centric Graph Processing (Extended Abstract)

Wang, X Wen, D

Qin, L

Chang, L Zhang, W

Permalink

Publisher:: IEEE
Publication Type:: Conference Proceeding
Citation:: Proceedings - International Conference on Data Engineering, 2022, 2022-May, pp. 1511-1512
Issue Date:: 2022-01-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

The embargo period expires on 2 Aug 2024

Adobe PDF

Download Accepted versionAdobe PDF (292.38 kB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Wang, X
dc.contributor.author	Wen, D https://orcid.org/0000-0002-0903-1503
dc.contributor.author	Qin, L https://orcid.org/0000-0001-6068-5062
dc.contributor.author	Chang, L
dc.contributor.author	Zhang, W
dc.date	2022-05-09
dc.date.accessioned	2023-02-28T00:45:41Z
dc.date.available	2023-02-28T00:45:41Z
dc.date.issued	2022-01-01
dc.identifier.citation	Proceedings - International Conference on Data Engineering, 2022, 2022-May, pp. 1511-1512
dc.identifier.isbn	9781665408837
dc.identifier.issn	1084-4627
dc.identifier.uri	http://hdl.handle.net/10453/166486
dc.description.abstract	Designing disk-based distributed graph systems has drawn a lot of research due to the strong expressiveness of the graph model and rapidly increasing graph volume. However, several challenges still exist in achieving both high computational efficiency and low network communication under the limitation of memory. In this paper, we design a novel distributed disk-based graph processing system, ScaleG, with a series of user-friendly programming interfaces. We propose several techniques to reduce both disk I/Os in each machine and message I/Os via the network. We manage all messages in memory and bound the volume of all messages by the number of vertices. We also carefully design the data structure to support partial computation and automatic vertex activation. We conduct extensive experiments on real-world big graphs to show the high efficiency of our system.
dc.language	en
dc.publisher	IEEE
dc.relation	http://purl.org/au-research/grants/arc/FT170100128
dc.relation	http://purl.org/au-research/grants/arc/DP180103096
dc.relation	http://purl.org/au-research/grants/arc/FT200100787
dc.relation.ispartof	Proceedings - International Conference on Data Engineering
dc.relation.ispartof	2022 IEEE 38th International Conference on Data Engineering (ICDE)
dc.relation.isbasedon	10.1109/ICDE53745.2022.00132
dc.rights	info:eu-repo/semantics/embargoedAccess
dc.title	ScaleG: A Distributed Disk-based System for Vertex-centric Graph Processing (Extended Abstract)
dc.type	Conference Proceeding
utslib.citation.volume	2022-May
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	open_access	*
utslib.copyright.embargo	2024-08-02T00:00:00+1000Z
dc.date.updated	2023-02-28T00:45:40Z
pubs.finish-date	2022-05-12
pubs.publication-status	Published
pubs.start-date	2022-05-09
pubs.volume	2022-May

Abstract:

Designing disk-based distributed graph systems has drawn a lot of research due to the strong expressiveness of the graph model and rapidly increasing graph volume. However, several challenges still exist in achieving both high computational efficiency and low network communication under the limitation of memory. In this paper, we design a novel distributed disk-based graph processing system, ScaleG, with a series of user-friendly programming interfaces. We propose several techniques to reduce both disk I/Os in each machine and message I/Os via the network. We manage all messages in memory and bound the volume of all messages by the number of vertices. We also carefully design the data structure to support partial computation and automatic vertex activation. We conduct extensive experiments on real-world big graphs to show the high efficiency of our system.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/166486