The week's pick
Reseach Article
Big Data: Does it Call for Distributed File System
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 131 - Number 13 |
| Year of Publication: 2015 |
| Authors: Komal Verma, Rajiv Pandey, Arpit Gupta |
10.5120/ijca2015907505
|
Komal Verma, Rajiv Pandey, Arpit Gupta . Big Data: Does it Call for Distributed File System. International Journal of Computer Applications. 131, 13 ( December 2015), 12-17. DOI=10.5120/ijca2015907505
Abstract
Today, in order to support decision for strategic advantages alignment, companies’ have started to realize the importance of using large data. It is being observed through different study cases that “Large data usually demands for faster processing”. As a result, companies are now investing more in processing larger sets of data rather than investing in expensive algorithms. A larger amount of data gives a better inference for decision making but also working with it can create challenge due to processing limitations. In order to easily manage and use this large amount of content in a proper systematic manner, Big Data, HDFS & other file systems were being introduced. Big data is used for ‘larger data sets having more varied and complex structure, having problems in analyzing, visualizing and storing for further processing’. The process of examining such large amounts of data inorderto reveal hidden patterns and secret correlations is named as Big Data Analytics. The useful information for companies or organizations will help them in gaining richer and deeper insights and getting advantages over the competition. Implementation of this Big Data needs to be analyzed and executed as accurately as possible. This term paper give an overview about what Big Data is, its classification, challenges it faces, need for Distributed File System, Hadoop and its components i.e. Hadoop Distributed file System and Map Reduce, and application of HDFS in Cloud Computing
References
- http://hpccsystems.com/ , last success 11.03.2013
- J. Manyika, M. chui, B. Brown, J. Bughin, R Dobbs,C. Roxburgh and A. H. Byers, “Big Data: The next frontier for innovation, competition, and productivity”, McKinsey Global Institute, 2011, http://www.mckinsey.com/~/media/McKinsey/dotcom/Insight%20and%20pubs/MGI/Research/Technology%20and%20Innovation/Big%20Data/MGI big data full report.ashx
- Xindong wu, Xingquan Zhu, Gong-Qing and Wel Ding, “Data Mining ith Big Data”, IEEE Transaction on knowledge and data engineering, Vol 26, No. 1, January 2014.
- Navint, “Why is BIG Data Important?”, a Navint Partners White Paper, May 2012 http://www.navint.comimages/Big.Data.pdf
- J, Manyika, M. Chui, B. Brown, Jacques Bughin, Richard Dobbs, Charles Roxburgh, A.H. Byers, “Big Data: The next frontier of innovation, competition and productivity”, McKinsey Global Institute, May, 2011 http://www.mckinsey.com/~/media/McKinsey/dotcom/Insight%20and%20pubs/MGI/Research/Technology%20and%20Innovation/Big%20Data/MGI big data full report.ashx
- http://en.wikipedia.org/wikiApache_Hadoop
- E. Geanina ULARU, F. Camelia PUICAN, A. APOSTU, M. VELICANU, “Perspective on Big Data and Bid Data Analytics”, page no.- 9; April, 2012
- Prem Jain, Stewart Tate, “Big Data Networked Storage Solution for Hadoop”, IBM ,Redpaper, http://www.ibm.com/redbooks
- Geert, “ Big Data Too Big to Ignore”, datacrunches,http://www.people.cs.kuleuven.be~joost.vennekens/DN/bigdata
- Hasan Mir, “Addressing Limitations of Distributed File System”
- DhrubaBorthaKur, “HDFS Architecture Guide”
- page no.- 4
- R. M. Kharode, A.R. Deshmukh, “International Journal of Advanced Research in Computer Science and Software Engineering”, “Study of Hadoop Distributed File System in Cloud Computing”, Vol. 5, Issue 1, January,2015 http://www.ijarcsse.com
- http://www.informationweek.com/software/businessintelligance/sas-gets-hip-tp-hadoop-for-big-dta/240009035?pgno=2
- Apache Hadoop, http://Hadoop.apache.org/
- Introduction to Cloud Computing by Shang Juh Kao
- K. Bakshi, “Considerations for Big Data: Architecture and Approach”, Aerospace Conference IEEE, Big Sky Montana, March 2012
- Bernice Purcell, “Th emergence of big data technology and analytics”, journal of Technology Research, http://www.aabri.commanuscript/121219
- C. Eaton, D. Deroos, T. Deutsch, G. Lapis, P. Zikopolos, “Understanding Big Data”, “Analytics for Enterprise class Hadoop and Streaming Data”
- S. Blazhievsky, N. Systems, “Introduction to Hadoop, Map Reduce and HDFS for Big Data Application, http://www.snia.org/sites/defaulteducation/tutorials/2013/fall/BigData/SergeBazhievsky_Introduction_to_Hadoop_MapReduce_v2.pdf
- Rajiv Pandey, Manoj Dhoundiyal, “Quantitative Evaluation of Big Data Categorical Variables Through R”, http://www.sciencedirect.com
- ExplainingBigData.https://www.youtube.com/watchv=7D1CQ_LOizA
- Five big data challengesAnd how to overcome them with visual analyticshttp://www.sas.com/resources/asset/five-big-data- challenges-article.pdf
- Dr. R. Pandey, N. Srivastava, Dr. S. Fatima, Extending R Boxplot analysis to Big Data in Education
- J. Nandimath, A. Patil, E. Banerjee, P. Kakade, S. Vaidya, “Big Data Analysis Using Apache Hadoop”
- Leskovec, Rajarman and Ullman, “ Mining of Massive Datasets, Map-Reduce”, Stanford University.
Index Terms
Keywords