BDGS: A scalable big data generator suite in big data benchmarking

Zijian Ming; Chunjie Luo; Wanling Gao; Rui Han; Qiang Yang; Lei Wang; Jianfeng Zhan

doi:10.1007/978-3-319-10596-3_11

BDGS: A scalable big data generator suite in big data benchmarking

Zijian Ming, Chunjie Luo, Wanling Gao, Rui Han, Qiang Yang, Lei Wang, Jianfeng Zhan^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

52 Citations (Scopus)

Abstract

Data generation is a key issue in big data benchmarking that aims to generate application-specific data sets to meet the 4V requirements of big data. Specifically, big data generators need to generate scalable data (Volume) of different types (Variety) under controllable generation rates (Velocity) while keeping the important characteristics of raw data (Veracity). This gives rise to various new challenges about how we design generators efficiently and successfully. To date, most existing techniques can only generate limited types of data and support specific big data systems such as Hadoop. Hence we develop a tool, called Big Data Generator Suite (BDGS), to efficiently generate scalable big data while employing data models derived from real data to preserve data veracity. The effectiveness of BDGS is demonstrated by developing six data generators covering three representative data types (structured, semi-structured and unstructured) and three data sources (text, graph, and table data).

Original language	English
Title of host publication	Advancing Big Data Benchmarks - Proceedings of the 2013 Workshop Series on Big Data Benchmarking WBDB.cn and WBDB.us, Revised Selected Papers
Editors	Meikel Poess, Tilmann Rabl, Hans-Arno Jacobsen, Chaitanya Baru, Nambiar Raghunath, Milind Bhandarkar
Publisher	Springer Verlag
Pages	138-154
Number of pages	17
ISBN (Electronic)	9783319105956
DOIs	https://doi.org/10.1007/978-3-319-10596-3_11
Publication status	Published - 2014
Externally published	Yes
Event	2013 Workshop Series on Big Data Benchmarking, WBDB.cn and WBDB.us. - Boston, United States Duration: 14 Sept 2014 → 18 Sept 2014

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	8585
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	2013 Workshop Series on Big Data Benchmarking, WBDB.cn and WBDB.us.
Country/Territory	United States
City	Boston
Period	14/09/14 → 18/09/14

Keywords

Benchmark
Big data
Data generator
Scalable
Veracity

Access to Document

10.1007/978-3-319-10596-3_11

Cite this

Ming, Z., Luo, C., Gao, W., Han, R., Yang, Q., Wang, L., & Zhan, J. (2014). BDGS: A scalable big data generator suite in big data benchmarking. In M. Poess, T. Rabl, H.-A. Jacobsen, C. Baru, N. Raghunath, & M. Bhandarkar (Eds.), Advancing Big Data Benchmarks - Proceedings of the 2013 Workshop Series on Big Data Benchmarking WBDB.cn and WBDB.us, Revised Selected Papers (pp. 138-154). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8585). Springer Verlag. https://doi.org/10.1007/978-3-319-10596-3_11

Ming, Zijian ; Luo, Chunjie ; Gao, Wanling et al. / BDGS : A scalable big data generator suite in big data benchmarking. Advancing Big Data Benchmarks - Proceedings of the 2013 Workshop Series on Big Data Benchmarking WBDB.cn and WBDB.us, Revised Selected Papers. editor / Meikel Poess ; Tilmann Rabl ; Hans-Arno Jacobsen ; Chaitanya Baru ; Nambiar Raghunath ; Milind Bhandarkar. Springer Verlag, 2014. pp. 138-154 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{4fbe36a8053743ca8db5fb6db610b264,

title = "BDGS: A scalable big data generator suite in big data benchmarking",

abstract = "Data generation is a key issue in big data benchmarking that aims to generate application-specific data sets to meet the 4V requirements of big data. Specifically, big data generators need to generate scalable data (Volume) of different types (Variety) under controllable generation rates (Velocity) while keeping the important characteristics of raw data (Veracity). This gives rise to various new challenges about how we design generators efficiently and successfully. To date, most existing techniques can only generate limited types of data and support specific big data systems such as Hadoop. Hence we develop a tool, called Big Data Generator Suite (BDGS), to efficiently generate scalable big data while employing data models derived from real data to preserve data veracity. The effectiveness of BDGS is demonstrated by developing six data generators covering three representative data types (structured, semi-structured and unstructured) and three data sources (text, graph, and table data).",

keywords = "Benchmark, Big data, Data generator, Scalable, Veracity",

author = "Zijian Ming and Chunjie Luo and Wanling Gao and Rui Han and Qiang Yang and Lei Wang and Jianfeng Zhan",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2014.; 2013 Workshop Series on Big Data Benchmarking, WBDB.cn and WBDB.us. ; Conference date: 14-09-2014 Through 18-09-2014",

year = "2014",

doi = "10.1007/978-3-319-10596-3_11",

language = "English",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "138--154",

editor = "Meikel Poess and Tilmann Rabl and Hans-Arno Jacobsen and Chaitanya Baru and Nambiar Raghunath and Milind Bhandarkar",

booktitle = "Advancing Big Data Benchmarks - Proceedings of the 2013 Workshop Series on Big Data Benchmarking WBDB.cn and WBDB.us, Revised Selected Papers",

address = "Germany",

}

Ming, Z, Luo, C, Gao, W, Han, R, Yang, Q, Wang, L & Zhan, J 2014, BDGS: A scalable big data generator suite in big data benchmarking. in M Poess, T Rabl, H-A Jacobsen, C Baru, N Raghunath & M Bhandarkar (eds), Advancing Big Data Benchmarks - Proceedings of the 2013 Workshop Series on Big Data Benchmarking WBDB.cn and WBDB.us, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8585, Springer Verlag, pp. 138-154, 2013 Workshop Series on Big Data Benchmarking, WBDB.cn and WBDB.us., Boston, United States, 14/09/14. https://doi.org/10.1007/978-3-319-10596-3_11

BDGS: A scalable big data generator suite in big data benchmarking. / Ming, Zijian; Luo, Chunjie; Gao, Wanling et al.
Advancing Big Data Benchmarks - Proceedings of the 2013 Workshop Series on Big Data Benchmarking WBDB.cn and WBDB.us, Revised Selected Papers. ed. / Meikel Poess; Tilmann Rabl; Hans-Arno Jacobsen; Chaitanya Baru; Nambiar Raghunath; Milind Bhandarkar. Springer Verlag, 2014. p. 138-154 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8585).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - BDGS

T2 - 2013 Workshop Series on Big Data Benchmarking, WBDB.cn and WBDB.us.

AU - Ming, Zijian

AU - Luo, Chunjie

AU - Gao, Wanling

AU - Han, Rui

AU - Yang, Qiang

AU - Wang, Lei

AU - Zhan, Jianfeng

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2014.

PY - 2014

Y1 - 2014

N2 - Data generation is a key issue in big data benchmarking that aims to generate application-specific data sets to meet the 4V requirements of big data. Specifically, big data generators need to generate scalable data (Volume) of different types (Variety) under controllable generation rates (Velocity) while keeping the important characteristics of raw data (Veracity). This gives rise to various new challenges about how we design generators efficiently and successfully. To date, most existing techniques can only generate limited types of data and support specific big data systems such as Hadoop. Hence we develop a tool, called Big Data Generator Suite (BDGS), to efficiently generate scalable big data while employing data models derived from real data to preserve data veracity. The effectiveness of BDGS is demonstrated by developing six data generators covering three representative data types (structured, semi-structured and unstructured) and three data sources (text, graph, and table data).

AB - Data generation is a key issue in big data benchmarking that aims to generate application-specific data sets to meet the 4V requirements of big data. Specifically, big data generators need to generate scalable data (Volume) of different types (Variety) under controllable generation rates (Velocity) while keeping the important characteristics of raw data (Veracity). This gives rise to various new challenges about how we design generators efficiently and successfully. To date, most existing techniques can only generate limited types of data and support specific big data systems such as Hadoop. Hence we develop a tool, called Big Data Generator Suite (BDGS), to efficiently generate scalable big data while employing data models derived from real data to preserve data veracity. The effectiveness of BDGS is demonstrated by developing six data generators covering three representative data types (structured, semi-structured and unstructured) and three data sources (text, graph, and table data).

KW - Benchmark

KW - Big data

KW - Data generator

KW - Scalable

KW - Veracity

UR - http://www.scopus.com/inward/record.url?scp=84910067732&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-10596-3_11

DO - 10.1007/978-3-319-10596-3_11

M3 - Conference contribution

AN - SCOPUS:84910067732

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 138

EP - 154

BT - Advancing Big Data Benchmarks - Proceedings of the 2013 Workshop Series on Big Data Benchmarking WBDB.cn and WBDB.us, Revised Selected Papers

A2 - Poess, Meikel

A2 - Rabl, Tilmann

A2 - Jacobsen, Hans-Arno

A2 - Baru, Chaitanya

A2 - Raghunath, Nambiar

A2 - Bhandarkar, Milind

PB - Springer Verlag

Y2 - 14 September 2014 through 18 September 2014

ER -

Ming Z, Luo C, Gao W, Han R, Yang Q, Wang L et al. BDGS: A scalable big data generator suite in big data benchmarking. In Poess M, Rabl T, Jacobsen HA, Baru C, Raghunath N, Bhandarkar M, editors, Advancing Big Data Benchmarks - Proceedings of the 2013 Workshop Series on Big Data Benchmarking WBDB.cn and WBDB.us, Revised Selected Papers. Springer Verlag. 2014. p. 138-154. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-10596-3_11

BDGS: A scalable big data generator suite in big data benchmarking

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this