Lecture 1: Data science foundation
- Known historical origin of data science
- Data Science lifecycle
- Data science explanation in buzzwords
The way everything is interconnected with each other: The internet of things (IoT).
The internet of things (IoT)= the network of physical objects—“things”—that are embedded
with sensors, software, and other technologies for the purpose of connecting and
exchanging data with other devices and systems over the internet.
e.g. amazon alexa, google home, apple watch, control climate with smartphone in home. All
these things are connected via the IoT.
Smart cities: Brugge and Helmond (Brainport smart district) → 3D replica of the city that
allows them to measure the impact of measures (for e.g. in the field of mobility)
The IoT will generate a lot of data, = big data.
Big data is defined by:
- volume (size data)
- velocity (speed at which data is generated)
- variety (type of data)
- veracity (accuracy of data)
- value (useful data)
- - validity (data quality, governance, master data management on massive)
- variability (dynamic, evolving behavior in data source)
- venue (distributed heterogeneous data from multiple platforms)
- vocabulary (data models, semantics that describe data structure)
- vagueness (confusion over meaning of BigDat and tools used)
To analyze all data (big data), human-machine interaction is crucial. All the data cannot be
analyzed by just excel anymore.
In the 17th century people already performed data science/ data collection. They
systematically collected demographic and economic data by states, this was for tax
purposes. In the Netherlands was well showed in the holocaust, because Dutch collected
data well → misuse of population registry by the nazis.
Data science originates from computer science, statistics, mathematics and visualization.
First data analysis, until the PC happened, by IBM (1981).
1
,Data science is an interdisciplinary field that uses scientific methods, processes, algorithms
and systems to extract knowledge and insights from data.
Originates from:
- computer science
- machine learning
- data science
- traditional software
- maths & stats
- machine learning
- data science
- data analysis
- business/ domain expertise
- traditional software
- data science
- data analysis
Data literacy: understanding what happens with data and telling a story behind it, right clarity
and interpretation.
Data representation:
- Relational Databases are represented in a tabular format and are queried using SQL.
Relational databases check the structure of the information when storing it (schema
on write
- Systems dealing with big data typically use No-SQL databases. These represent
and store the data differently. The data structure is not checked on write (schema on
write) )but rather structure is re-constructed on read (schema on read) . Platforms
like Facebook, Twitter and Linked-in use No-SQL technology to manage their data.
2
,Big data and data analytics are part of data science. The AI field is where a machine
performs cognitive functions. AI is a broad field and machine-learning is only a part of that.
Machine-learning uses algorithms to make AI learn without programming it explicitly. Deep
learning is a type of machine-learning (using neural networks, using different types of
algorithms).
How to conduct data science? → data science lifecycle
Step 1: Frame the problem and formulate the research question
Step 2: Obtain right set of data
Step 3: Scrub your data, clean data (can be noise in there)
Step 4: Explore your data (what kind of data do I have, what does it reflect, any missing
values
Step 5-6: Model your data (make model based on data)
Step 7: Communicate and interpret results of the analysis.
Questions related to these different steps;
We typically use data science to answer 5 types of questions:
● How much or how many? (regression)
● Which category? (classification)
● Which group? (clustering)
● Is this weird? (anomaly detection)
● Which option should be taken? (recommendation)
3
, Data science applications in different domains;
E-commerce: adds that come up when google something
Data science for medical imaging; determine if someone had covid based on ct scans (lung
pics).
Some data science concepts.
4
- Known historical origin of data science
- Data Science lifecycle
- Data science explanation in buzzwords
The way everything is interconnected with each other: The internet of things (IoT).
The internet of things (IoT)= the network of physical objects—“things”—that are embedded
with sensors, software, and other technologies for the purpose of connecting and
exchanging data with other devices and systems over the internet.
e.g. amazon alexa, google home, apple watch, control climate with smartphone in home. All
these things are connected via the IoT.
Smart cities: Brugge and Helmond (Brainport smart district) → 3D replica of the city that
allows them to measure the impact of measures (for e.g. in the field of mobility)
The IoT will generate a lot of data, = big data.
Big data is defined by:
- volume (size data)
- velocity (speed at which data is generated)
- variety (type of data)
- veracity (accuracy of data)
- value (useful data)
- - validity (data quality, governance, master data management on massive)
- variability (dynamic, evolving behavior in data source)
- venue (distributed heterogeneous data from multiple platforms)
- vocabulary (data models, semantics that describe data structure)
- vagueness (confusion over meaning of BigDat and tools used)
To analyze all data (big data), human-machine interaction is crucial. All the data cannot be
analyzed by just excel anymore.
In the 17th century people already performed data science/ data collection. They
systematically collected demographic and economic data by states, this was for tax
purposes. In the Netherlands was well showed in the holocaust, because Dutch collected
data well → misuse of population registry by the nazis.
Data science originates from computer science, statistics, mathematics and visualization.
First data analysis, until the PC happened, by IBM (1981).
1
,Data science is an interdisciplinary field that uses scientific methods, processes, algorithms
and systems to extract knowledge and insights from data.
Originates from:
- computer science
- machine learning
- data science
- traditional software
- maths & stats
- machine learning
- data science
- data analysis
- business/ domain expertise
- traditional software
- data science
- data analysis
Data literacy: understanding what happens with data and telling a story behind it, right clarity
and interpretation.
Data representation:
- Relational Databases are represented in a tabular format and are queried using SQL.
Relational databases check the structure of the information when storing it (schema
on write
- Systems dealing with big data typically use No-SQL databases. These represent
and store the data differently. The data structure is not checked on write (schema on
write) )but rather structure is re-constructed on read (schema on read) . Platforms
like Facebook, Twitter and Linked-in use No-SQL technology to manage their data.
2
,Big data and data analytics are part of data science. The AI field is where a machine
performs cognitive functions. AI is a broad field and machine-learning is only a part of that.
Machine-learning uses algorithms to make AI learn without programming it explicitly. Deep
learning is a type of machine-learning (using neural networks, using different types of
algorithms).
How to conduct data science? → data science lifecycle
Step 1: Frame the problem and formulate the research question
Step 2: Obtain right set of data
Step 3: Scrub your data, clean data (can be noise in there)
Step 4: Explore your data (what kind of data do I have, what does it reflect, any missing
values
Step 5-6: Model your data (make model based on data)
Step 7: Communicate and interpret results of the analysis.
Questions related to these different steps;
We typically use data science to answer 5 types of questions:
● How much or how many? (regression)
● Which category? (classification)
● Which group? (clustering)
● Is this weird? (anomaly detection)
● Which option should be taken? (recommendation)
3
, Data science applications in different domains;
E-commerce: adds that come up when google something
Data science for medical imaging; determine if someone had covid based on ct scans (lung
pics).
Some data science concepts.
4
