Chapter
Data Handling using
3 Pandas - II
“We owe a lot to the Indians, who
taught us how to count, without
which no worthwhile scientific
discovery could have been made.”
— Albert Einstein
In this chapter
»» Introduction
»» Descriptive Statistics
3.1 Introduction
»» Data Aggregations
As discussed in the previous chapter, Pandas »» Sorting a DataFrame
is a well established Python Library used for
»» GROUP BY Functions
manipulation, processing and analysis of
data. We have already discussed the basic »» Altering the Index
operations on Series and DataFrame like »» Other DataFrame
creating them and then accessing data from Operations
them. Pandas provides more powerful and »» Handling Missing
useful functions for data analysis. Values
In this chapter, we will be working with »» Import and Export
more advanced features of DataFrame like of Data between
sorting data, answering analytical questions Pandas and MySQL
using the data, cleaning data and applying
different useful functions on the data. Below
is the example data on which we will be
applying the advanced features of Pandas.
2024-25
Chapter 3.indd 63 11/26/2020 12:46:0
, 64 Informatics Practices
Case Study
Let us consider the data of marks scored in unit tests
held in school. For each unit test, the marks scored by
all students of the class is recorded. Maximum marks
are 25 in each subject. The subjects are Maths, Science.
Social Studies (S.St.), Hindi, and English. For simplicity,
we assume there are 4 students in the class and the
table below shows their marks in Unit Test 1, Unit Test
2 and Unit Test 3. Table 3.1 shows this data.
Table 3.1 Case Study
Result
Name/ Unit Maths Science S.St. Hindi Eng
Subjects Test
Raman 1 22 21 18 20 21
Raman 2 21 20 17 22 24
Raman 3 14 19 15 24 23
Zuhaire 1 20 17 22 24 19
Zuhaire 2 23 15 21 25 15
Zuhaire 3 22 18 19 23 13
Aashravy 1 23 19 20 15 22
Aashravy 2 24 22 24 17 21
Aashravy 3 12 25 19 21 23
Mishti 1 15 22 25 22 22
Mishti 2 18 21 25 24 23
Mishti 3 17 18 20 25 20
Let us store the data in a DataFrame, as shown in
Program 3.1:
Program 3-1 Store the Result data in a DataFrame called marksUT.
>>> import pandas as pd
>>> marksUT= {'Name':['Raman','Raman','Raman','Zuhaire','Zuhaire','Zu
haire', 'Ashravy','Ashravy','Ashravy','Mishti','Mishti','Mishti'],
'UT':[1,2,3,1,2,3,1,2,3,1,2,3],
'Maths':[22,21,14,20,23,22,23,24,12,15,18,17],
'Science':[21,20,19,17,15,18,19,22,25,22,21,18],
'S.St':[18,17,15,22,21,19,20,24,19,25,25,20],
'Hindi':[20,22,24,24,25,23,15,17,21,22,24,25],
'Eng':[21,24,23,19,15,13,22,21,23,22,23,20]
}
>>> df=pd.DataFrame(marksUT)
>>> print(df)
2024-25
Chapter 3.indd 64 11/26/2020 12:46:0
, Data Handling using Pandas - II 65
Name UT Maths Science S.St Hindi Eng
0 Raman 1 22 21 18 20 21
1 Raman 2 21 20 17 22 24
2 Raman 3 14 19 15 24 23
3 Zuhaire 1 20 17 22 24 19
4 Zuhaire 2 23 15 21 25 15
5 Zuhaire 3 22 18 19 23 13
6 Ashravy 1 23 19 20 15 22
7 Ashravy 2 24 22 24 17 21
8 Ashravy 3 12 25 19 21 23
9 Mishti 1 15 22 25 22 22
10 Mishti 2 18 21 25 24 23
11 Mishti 3 17 18 20 25 20
3.2 Descriptive Statistics
Descriptive Statistics are used to summarise the given
data. In other words, they refer to the methods which
are used to get some basic idea about the data.
In this section, we will be discussing descriptive
statistical methods that can be applied to a DataFrame.
These are max, min, count, sum, mean, median, mode,
quartiles, variance. In each case, we will consider the
above created DataFrame df.
3.2.1 Calculating Maximum Values
DataFrame.max() is used to calculate the maximum
values from the DataFrame, regardless of its data types.
The following statement outputs the maximum value of
each column of the DataFrame:
>>> print(df.max())
Name Zuhaire #Maximum value in name column
#(alphabetically)
UT 3 #Maximum value in column UT
Maths 24 #Maximum value in column Maths
Science 25 #Maximum value in column Science
S.St 25 #Maximum value in column S.St
Hindi 25 #Maximum value in column Hindi
Eng 24 #Maximum value in column Eng
dtype: object
If we want to output maximum value for the columns
having only numeric values, then we can set the
parameter numeric_only=True in the max() method, as
shown below:
2024-25
Chapter 3.indd 65 11/26/2020 12:46:0
, 66 Informatics Practices
>>> print(df.max(numeric_only=True))
UT 3
Maths 24
Science 25
S.St 25
Hindi 25
Eng 24
dtype: int64
Program 3-2 Write the statements to output the
maximum marks obtained in each subject
in Unit Test 2.
>>> dfUT2 = df[df.UT == 2]
>>> print('\nResult of Unit Test 2:
\n\n',dfUT2)
Result of Unit Test 2:
Name UT Maths Science S.St Hindi Eng
1 Raman 2 21 20 17 22 24
4 Zuhaire 2 23 15 21 25 15
7 Ashravy 2 24 22 24 17 21
10 Mishti 2 18 21 25 24 23
The output of Program
3.2 can also be
achieved using the >>> print('\nMaximum Mark obtained in
following statements Each Subject in Unit Test 2: \n\n',dfUT2.
max(numeric_only=True))
>>> dfUT2=df[df
['UT']==2].max
(numeric_only=True) Maximum Mark obtained in Each Subject in Unit
>>> print(dfUT2) Test 2:
UT 2
Maths 24
Science 22
S.St 25
Hindi 25
Eng 24
dtype: int64
By default, the max() method finds the maximum
value of each column (which means, axis=0). However,
to find the maximum value of each row, we have to
specify axis = 1 as its argument.
#maximum marks for each student in each unit
test among all the subjects
2024-25
Chapter 3.indd 66 11/26/2020 12:46:0
Data Handling using
3 Pandas - II
“We owe a lot to the Indians, who
taught us how to count, without
which no worthwhile scientific
discovery could have been made.”
— Albert Einstein
In this chapter
»» Introduction
»» Descriptive Statistics
3.1 Introduction
»» Data Aggregations
As discussed in the previous chapter, Pandas »» Sorting a DataFrame
is a well established Python Library used for
»» GROUP BY Functions
manipulation, processing and analysis of
data. We have already discussed the basic »» Altering the Index
operations on Series and DataFrame like »» Other DataFrame
creating them and then accessing data from Operations
them. Pandas provides more powerful and »» Handling Missing
useful functions for data analysis. Values
In this chapter, we will be working with »» Import and Export
more advanced features of DataFrame like of Data between
sorting data, answering analytical questions Pandas and MySQL
using the data, cleaning data and applying
different useful functions on the data. Below
is the example data on which we will be
applying the advanced features of Pandas.
2024-25
Chapter 3.indd 63 11/26/2020 12:46:0
, 64 Informatics Practices
Case Study
Let us consider the data of marks scored in unit tests
held in school. For each unit test, the marks scored by
all students of the class is recorded. Maximum marks
are 25 in each subject. The subjects are Maths, Science.
Social Studies (S.St.), Hindi, and English. For simplicity,
we assume there are 4 students in the class and the
table below shows their marks in Unit Test 1, Unit Test
2 and Unit Test 3. Table 3.1 shows this data.
Table 3.1 Case Study
Result
Name/ Unit Maths Science S.St. Hindi Eng
Subjects Test
Raman 1 22 21 18 20 21
Raman 2 21 20 17 22 24
Raman 3 14 19 15 24 23
Zuhaire 1 20 17 22 24 19
Zuhaire 2 23 15 21 25 15
Zuhaire 3 22 18 19 23 13
Aashravy 1 23 19 20 15 22
Aashravy 2 24 22 24 17 21
Aashravy 3 12 25 19 21 23
Mishti 1 15 22 25 22 22
Mishti 2 18 21 25 24 23
Mishti 3 17 18 20 25 20
Let us store the data in a DataFrame, as shown in
Program 3.1:
Program 3-1 Store the Result data in a DataFrame called marksUT.
>>> import pandas as pd
>>> marksUT= {'Name':['Raman','Raman','Raman','Zuhaire','Zuhaire','Zu
haire', 'Ashravy','Ashravy','Ashravy','Mishti','Mishti','Mishti'],
'UT':[1,2,3,1,2,3,1,2,3,1,2,3],
'Maths':[22,21,14,20,23,22,23,24,12,15,18,17],
'Science':[21,20,19,17,15,18,19,22,25,22,21,18],
'S.St':[18,17,15,22,21,19,20,24,19,25,25,20],
'Hindi':[20,22,24,24,25,23,15,17,21,22,24,25],
'Eng':[21,24,23,19,15,13,22,21,23,22,23,20]
}
>>> df=pd.DataFrame(marksUT)
>>> print(df)
2024-25
Chapter 3.indd 64 11/26/2020 12:46:0
, Data Handling using Pandas - II 65
Name UT Maths Science S.St Hindi Eng
0 Raman 1 22 21 18 20 21
1 Raman 2 21 20 17 22 24
2 Raman 3 14 19 15 24 23
3 Zuhaire 1 20 17 22 24 19
4 Zuhaire 2 23 15 21 25 15
5 Zuhaire 3 22 18 19 23 13
6 Ashravy 1 23 19 20 15 22
7 Ashravy 2 24 22 24 17 21
8 Ashravy 3 12 25 19 21 23
9 Mishti 1 15 22 25 22 22
10 Mishti 2 18 21 25 24 23
11 Mishti 3 17 18 20 25 20
3.2 Descriptive Statistics
Descriptive Statistics are used to summarise the given
data. In other words, they refer to the methods which
are used to get some basic idea about the data.
In this section, we will be discussing descriptive
statistical methods that can be applied to a DataFrame.
These are max, min, count, sum, mean, median, mode,
quartiles, variance. In each case, we will consider the
above created DataFrame df.
3.2.1 Calculating Maximum Values
DataFrame.max() is used to calculate the maximum
values from the DataFrame, regardless of its data types.
The following statement outputs the maximum value of
each column of the DataFrame:
>>> print(df.max())
Name Zuhaire #Maximum value in name column
#(alphabetically)
UT 3 #Maximum value in column UT
Maths 24 #Maximum value in column Maths
Science 25 #Maximum value in column Science
S.St 25 #Maximum value in column S.St
Hindi 25 #Maximum value in column Hindi
Eng 24 #Maximum value in column Eng
dtype: object
If we want to output maximum value for the columns
having only numeric values, then we can set the
parameter numeric_only=True in the max() method, as
shown below:
2024-25
Chapter 3.indd 65 11/26/2020 12:46:0
, 66 Informatics Practices
>>> print(df.max(numeric_only=True))
UT 3
Maths 24
Science 25
S.St 25
Hindi 25
Eng 24
dtype: int64
Program 3-2 Write the statements to output the
maximum marks obtained in each subject
in Unit Test 2.
>>> dfUT2 = df[df.UT == 2]
>>> print('\nResult of Unit Test 2:
\n\n',dfUT2)
Result of Unit Test 2:
Name UT Maths Science S.St Hindi Eng
1 Raman 2 21 20 17 22 24
4 Zuhaire 2 23 15 21 25 15
7 Ashravy 2 24 22 24 17 21
10 Mishti 2 18 21 25 24 23
The output of Program
3.2 can also be
achieved using the >>> print('\nMaximum Mark obtained in
following statements Each Subject in Unit Test 2: \n\n',dfUT2.
max(numeric_only=True))
>>> dfUT2=df[df
['UT']==2].max
(numeric_only=True) Maximum Mark obtained in Each Subject in Unit
>>> print(dfUT2) Test 2:
UT 2
Maths 24
Science 22
S.St 25
Hindi 25
Eng 24
dtype: int64
By default, the max() method finds the maximum
value of each column (which means, axis=0). However,
to find the maximum value of each row, we have to
specify axis = 1 as its argument.
#maximum marks for each student in each unit
test among all the subjects
2024-25
Chapter 3.indd 66 11/26/2020 12:46:0
