Design of Spark SQL Based Framework for Advanced Analytics


KIPS Transactions on Software and Data Engineering, Vol. 5, No. 10, pp. 477-482, Oct. 2016
10.3745/KTSDE.2016.5.10.477,   PDF Download:
Keywords: Dvanced Analytics, Spark, Complex Query, Distributed Processing Platform, Hadoop, MapReduce
Abstract

As being the advanced analytics indispensable on big data for agile decision-making and tactical planning in enterprises, distributed processing platforms, such as Hadoop and Spark which distribute and handle the large volume of data on multiple nodes, receive great attention in the field. In Spark platform stack, Spark SQL unveiled recently to make Spark able to support distributed processing framework based on SQL. However, Spark SQL cannot effectively handle advanced analytics that involves machine learning and graph processing in terms of iterative tasks and task allocations. Motivated by these issues, this paper proposes the design of SQL-based big data optimal processing engine and processing framework to support advanced analytics in Spark environments. Big data optimal processing engines copes with complex SQL queries that involves multiple parameters and join, aggregation and sorting operations in distributed/parallel manner and the proposing framework optimizes machine learning process in terms of relational operations.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
J. Chung, "Design of Spark SQL Based Framework for Advanced Analytics," KIPS Transactions on Software and Data Engineering, vol. 5, no. 10, pp. 477-482, 2016. DOI: 10.3745/KTSDE.2016.5.10.477.

[ACM Style]
Jaehwa Chung. 2016. Design of Spark SQL Based Framework for Advanced Analytics. KIPS Transactions on Software and Data Engineering, 5, 10, (2016), 477-482. DOI: 10.3745/KTSDE.2016.5.10.477.