Big data refers to large datasets that are challenging to store, search, share, visualize, and analyze and so the Testing. Information is emerging at volatile rate, coming into organization from diverse areas and in numerous formats. Traditional DW testing approach is inadequate due to Technology Changes, Infrastructure (DB/ETL on Cloud) and Big Data. Big Data validation is not only around validation of just what is different; it's also about validation of new integrated components to what you already have. There is unique testing prospects exists as poor data quality is still a major and exponentially growing problem. It's a digital world, which is causing massive increases in the volume (amount of data), velocity (speed of data in and out), and variety (range of data types and sources) of data. As a result, concern for realistic data sets, data accuracy, consistency and data quality is now a critical issue. The paper tries to explore testing challenges in Big Data adoption and outline a testing strategy to validate high volume, velocity and variety of information.

ETL Hadoop big data validation big data testing