Concept of Big data

Best Laptop for students

Concept of BIG DATA

The concept of big data arrived in executive suites via IT and engineering departments, and those technical roots have led to some disagreement of what the term means. At its core, big data refers to high volumes of information from multiple sources being fed into data stores on a time-sensitive basis. The shorthand: lots of data coming rapidly and from many places. Big data is an important foundation of quality business intelligence, providing enough information to detect meaningful trends.

Big Data Everywhere! 

•Lots of data is being collected
and warehoused

•Web data, e-commerce

•purchases at department/
grocery stores

•Bank/Credit Card
transactions

•Social Network

Type of Data

•Relational Data (Tables/Transaction/Legacy Data)

•Text Data (Web)

•Semi-structured Data (XML)

•Graph Data

•Social Network, Semantic Web (RDF), …

•Streaming Data

•You can only scan the data once

How much data?

•Google processes 20 PB a day (2008)

•Wayback Machine has 3 PB + 100 TB/month (3/2009)

•Facebook has 2.5 PB of user data + 15 TB/day (4/2009)

•eBay has 6.5 PB of user data + 50 TB/day (5/2009)

•CERN’s Large Hydron Collider (LHC) generates 15 PB a year

Technologies use for

Big Data Processing

HADOOP DISTRIBUTIONS

Hadoop is an open-source software framework for storing and processing big data in a distributed fashion on large clusters of commodity hardware. Essentially, it accomplishes two tasks: massive data storage and faster processing.

NOSQL DATABASES

A NoSQL database provides a way to store and retrieve data that is modelled in means other than the tabular relations used in relational databases. The data structures used by NoSQL databases (ex: key-value, graph, or document) differ from those used in relational databases, making some operations faster in NoSQL and others faster in relational databases. NoSQL databases are increasingly used in big data and real-time web applications.