Linear Weighted Regression and Energy-Aware Greedy Scheduling for Heterogeneous Big Data

Kallam, S; Patan, R; Ramana, TV; Gandomi, AH

Linear Weighted Regression and Energy-Aware Greedy Scheduling for Heterogeneous Big Data

Kallam, S Patan, R Ramana, TV Gandomi, AH

Permalink

Publisher:: MDPI AG
Publication Type:: Journal Article
Citation:: Electronics, 2021, 10, (5), pp. 1-15
Issue Date:: 2021-02-26

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Published versionAdobe PDF (1.28 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Kallam, S
dc.contributor.author	Patan, R
dc.contributor.author	Ramana, TV
dc.contributor.author	Gandomi, AH
dc.date.accessioned	2022-03-23T02:30:41Z
dc.date.available	2022-03-23T02:30:41Z
dc.date.issued	2021-02-26
dc.identifier.citation	Electronics, 2021, 10, (5), pp. 1-15
dc.identifier.issn	2079-9292
dc.identifier.issn	2079-9292
dc.identifier.uri	http://hdl.handle.net/10453/155479
dc.description.abstract	Data are presently being produced at an increased speed in different formats, which complicates the design, processing, and evaluation of the data. The MapReduce algorithm is a distributed file system that is used for big data parallel processing. Current implementations of MapReduce assist in data locality along with robustness. In this study, a linear weighted regression and energy-aware greedy scheduling (LWR-EGS) method were combined to handle big data. The LWR-EGS method initially selects tasks for an assignment and then selects the best available machine to identify an optimal solution. With this objective, first, the problem was modeled as an integer linear weighted regression program to choose tasks for the assignment. Then, the best available machines were selected to find the optimal solution. In this manner, the optimization of resources is said to have taken place. Then, an energy efficiency-aware greedy scheduling algorithm was presented to select a position for each task to minimize the total energy consumption of the MapReduce job for big data applications in heterogeneous environments without a significant performance loss. To evaluate the performance, the LWR-EGS method was compared with two related approaches via MapReduce. The experimental results showed that the LWR-EGS method effectively reduced the total energy consumption without producing large scheduling overheads. Moreover, the method also reduced the execution time when compared to state-of-the-art methods. The LWR-EGS method reduced the energy consumption, average processing time, and scheduling overhead by 16%, 20%, and 22%, respectively, compared to existing methods.
dc.language	English
dc.publisher	MDPI AG
dc.relation.ispartof	Electronics
dc.relation.isbasedon	10.3390/electronics10050554
dc.rights	info:eu-repo/semantics/openAccess
dc.subject	0906 Electrical and Electronic Engineering
dc.title	Linear Weighted Regression and Energy-Aware Greedy Scheduling for Heterogeneous Big Data
dc.type	Journal Article
utslib.citation.volume	10
utslib.for	0906 Electrical and Electronic Engineering
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	open_access	*
pubs.consider-herdc	false
dc.date.updated	2022-03-23T02:30:39Z
pubs.issue	5
pubs.publication-status	Published
pubs.volume	10
utslib.citation.issue	5

Abstract:

Data are presently being produced at an increased speed in different formats, which complicates the design, processing, and evaluation of the data. The MapReduce algorithm is a distributed file system that is used for big data parallel processing. Current implementations of MapReduce assist in data locality along with robustness. In this study, a linear weighted regression and energy-aware greedy scheduling (LWR-EGS) method were combined to handle big data. The LWR-EGS method initially selects tasks for an assignment and then selects the best available machine to identify an optimal solution. With this objective, first, the problem was modeled as an integer linear weighted regression program to choose tasks for the assignment. Then, the best available machines were selected to find the optimal solution. In this manner, the optimization of resources is said to have taken place. Then, an energy efficiency-aware greedy scheduling algorithm was presented to select a position for each task to minimize the total energy consumption of the MapReduce job for big data applications in heterogeneous environments without a significant performance loss. To evaluate the performance, the LWR-EGS method was compared with two related approaches via MapReduce. The experimental results showed that the LWR-EGS method effectively reduced the total energy consumption without producing large scheduling overheads. Moreover, the method also reduced the execution time when compared to state-of-the-art methods. The LWR-EGS method reduced the energy consumption, average processing time, and scheduling overhead by 16%, 20%, and 22%, respectively, compared to existing methods.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/155479