Scaling-up item-based collaborative filtering recommendation algorithm based on Hadoop

Jiang, J; Lu, J; Zhang, G; Long, G

Scaling-up item-based collaborative filtering recommendation algorithm based on Hadoop

Jiang, J

Lu, J

Zhang, G

Long, G

Permalink

Publication Type:: Conference Proceeding
Citation:: Proceedings - 2011 IEEE World Congress on Services, SERVICES 2011, 2011, pp. 490 - 497
Issue Date:: 2011-10-06

Closed Access

	Filename	Description	Size
	2013003627OK.pdf		1.74 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Jiang, J https://orcid.org/0000-0001-5301-7779	en_US
dc.contributor.author	Lu, J https://orcid.org/0000-0003-0690-4732	en_US
dc.contributor.author	Zhang, G https://orcid.org/0000-0003-3960-0583	en_US
dc.contributor.author	Long, G https://orcid.org/0000-0003-3740-9515	en_US
dc.date.issued	2011-10-06	en_US
dc.identifier.citation	Proceedings - 2011 IEEE World Congress on Services, SERVICES 2011, 2011, pp. 490 - 497	en_US
dc.identifier.isbn	9780769544618	en_US
dc.identifier.uri	http://hdl.handle.net/10453/29419
dc.description.abstract	Collaborative filtering (CF) techniques have achieved widespread success in E-commerce nowadays. The tremendous growth of the number of customers and products in recent years poses some key challenges for recommender systems in which high quality recommendations are required and more recommendations per second for millions of customers and products need to be performed. Thus, the improvement of scalability and efficiency of collaborative filtering (CF) algorithms become increasingly important and difficult. In this paper, we developed and implemented a scaling-up item-based collaborative filtering algorithm on MapReduce, by splitting the three most costly computations in the proposed algorithm into four Map-Reduce phases, each of which can be independently executed on different nodes in parallel. We also proposed efficient partition strategies not only to enable the parallel computation in each Map-Reduce phase but also to maximize data locality to minimize the communication cost. Experimental results effectively showed the good performance in scalability and efficiency of the item-based CF algorithm on a Hadoop cluster. © 2011 IEEE.	en_US
dc.relation.ispartof	Proceedings - 2011 IEEE World Congress on Services, SERVICES 2011	en_US
dc.relation.isbasedon	10.1109/SERVICES.2011.66	en_US
dc.title	Scaling-up item-based collaborative filtering recommendation algorithm based on Hadoop	en_US
dc.type	Conference Proceeding
utslib.for	1099 Other Technology	en_US
utslib.for	0806 Information Systems	en_US
dc.location.activity	Washington, DC	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
pubs.organisational-group	/University of Technology Sydney/Strength - CAI - Centre for Artificial Intelligence
utslib.copyright.status	closed_access
pubs.publication-status	Published	en_US

Abstract:

Collaborative filtering (CF) techniques have achieved widespread success in E-commerce nowadays. The tremendous growth of the number of customers and products in recent years poses some key challenges for recommender systems in which high quality recommendations are required and more recommendations per second for millions of customers and products need to be performed. Thus, the improvement of scalability and efficiency of collaborative filtering (CF) algorithms become increasingly important and difficult. In this paper, we developed and implemented a scaling-up item-based collaborative filtering algorithm on MapReduce, by splitting the three most costly computations in the proposed algorithm into four Map-Reduce phases, each of which can be independently executed on different nodes in parallel. We also proposed efficient partition strategies not only to enable the parallel computation in each Map-Reduce phase but also to maximize data locality to minimize the communication cost. Experimental results effectively showed the good performance in scalability and efficiency of the item-based CF algorithm on a Hadoop cluster. © 2011 IEEE.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/29419