SCSB1231 – DATA AND INFORMATION SCIENCE
UNIT 5 – PRINCIPLES OF BIG DATA
Introduction to Big Data - Challenges of processing Big Data (Volume, Velocity and
Variety perspective) - Use Cases.
What is Big Data?
Big data is a collection of large datasets that cannot be processed using traditional
computing techniques. It is not a single technique or a tool, rather it has become a complete
subject, which involves various tools, techniques and frameworks.
What Comes Under Big Data?
Big data involves the data produced by different devices and applications. Given below are
some of the fields that come under the umbrella of Big Data.
• Black Box Data − It is a component of helicopter, airplanes, and jets, etc. It captures
voices of the flight crew, recordings of microphones and earphones, and the
performance information of the aircraft.
• Social Media Data − social media such as Facebook and Twitter hold information
and the views posted by millions of people across the globe.
• Stock Exchange Data − The stock exchange data holds information about the ‘buy’
and ‘sell’ decisions made on a share of different companies made by the customers.
• Power Grid Data − The power grid data holds information consumed by a particular
node with respect to a base station.
• Transport Data − Transport data includes model, capacity, distance and availability of
a vehicle.
• Search Engine Data − Search engines retrieve lots of data from different databases.
, Thus, Big Data includes huge volume, high velocity, and extensible variety of data. The data
in it will be of three types.
• Structured data − Relational data.
• Semi Structured data − XML data.
• Unstructured data − Word, PDF, Text, Media Logs.
Benefits of Big Data
• Using the information kept in the social network like Facebook, the marketing
agencies are learning about the response for their campaigns, promotions, and other
advertising mediums.
• Using the information in the social media like preferences and product perception of
their consumers, product companies and retail organizations are planning their
production.
• Using the data regarding the previous medical history of patients, hospitals are
providing better and quick service.
Big Data Technologies
Big data technologies are important in providing more accurate analysis, which may lead to
more concrete decision-making resulting in greater operational efficiencies, cost reductions,
and reduced risks for the business.
To harness the power of big data, you would require an infrastructure that can manage and
process huge volumes of structured and unstructured data in real-time and can protect data
privacy and security.
There are various technologies in the market from different vendors including Amazon, IBM,
Microsoft, etc., to handle big data. While looking into the technologies that handle big data,
we examine the following two classes of technology −
Operational Big Data
This include systems like MongoDB that provide operational capabilities for real-time,
interactive workloads where data is primarily captured and stored.
UNIT 5 – PRINCIPLES OF BIG DATA
Introduction to Big Data - Challenges of processing Big Data (Volume, Velocity and
Variety perspective) - Use Cases.
What is Big Data?
Big data is a collection of large datasets that cannot be processed using traditional
computing techniques. It is not a single technique or a tool, rather it has become a complete
subject, which involves various tools, techniques and frameworks.
What Comes Under Big Data?
Big data involves the data produced by different devices and applications. Given below are
some of the fields that come under the umbrella of Big Data.
• Black Box Data − It is a component of helicopter, airplanes, and jets, etc. It captures
voices of the flight crew, recordings of microphones and earphones, and the
performance information of the aircraft.
• Social Media Data − social media such as Facebook and Twitter hold information
and the views posted by millions of people across the globe.
• Stock Exchange Data − The stock exchange data holds information about the ‘buy’
and ‘sell’ decisions made on a share of different companies made by the customers.
• Power Grid Data − The power grid data holds information consumed by a particular
node with respect to a base station.
• Transport Data − Transport data includes model, capacity, distance and availability of
a vehicle.
• Search Engine Data − Search engines retrieve lots of data from different databases.
, Thus, Big Data includes huge volume, high velocity, and extensible variety of data. The data
in it will be of three types.
• Structured data − Relational data.
• Semi Structured data − XML data.
• Unstructured data − Word, PDF, Text, Media Logs.
Benefits of Big Data
• Using the information kept in the social network like Facebook, the marketing
agencies are learning about the response for their campaigns, promotions, and other
advertising mediums.
• Using the information in the social media like preferences and product perception of
their consumers, product companies and retail organizations are planning their
production.
• Using the data regarding the previous medical history of patients, hospitals are
providing better and quick service.
Big Data Technologies
Big data technologies are important in providing more accurate analysis, which may lead to
more concrete decision-making resulting in greater operational efficiencies, cost reductions,
and reduced risks for the business.
To harness the power of big data, you would require an infrastructure that can manage and
process huge volumes of structured and unstructured data in real-time and can protect data
privacy and security.
There are various technologies in the market from different vendors including Amazon, IBM,
Microsoft, etc., to handle big data. While looking into the technologies that handle big data,
we examine the following two classes of technology −
Operational Big Data
This include systems like MongoDB that provide operational capabilities for real-time,
interactive workloads where data is primarily captured and stored.
