<h5 id="31-introduction">3.1 INTRODUCTION</h5> <ul> <li>The text provides data on test scores of 4 students - Raman, Zuhaire, Ashravy, and Mishti.</li> <li>Each student took 3 tests. Raman’s total score is 63, Zuhaire’s is 71, Ashravy’s is 76, and Mishti’s is 67.</li> <li>The range of Raman’s scores is 10, Zuhaire’s is 8, Ashravy’s is 13, and Mishti’s is 10.</li> <li>The average of Raman’s scores is 21, Zuhaire’s is 23.67, Ashravy’s is 25.33, and Mishti’s is 22.33.</li> <li>The median of Raman’s scores is 21, Zuhaire’s is 22, Ashravy’s is 24, and Mishti’s is 22.</li> </ul> <p>======</p> <h5 id="32-descriptive-statistics">3.2 Descriptive Statistics</h5> <ul> <li>Descriptive Statistics are methods used to summarize and get basic ideas about data.</li> <li>Applicable methods include max, min, count, sum, mean, median, mode, quartiles, and variance.</li> <li>The ‘max’ function returns the highest value in a DataFrame column.</li> <li>The ‘min’ function returns the lowest value in a DataFrame column.</li> <li>The ‘count’ function returns the number of non-null values in a DataFrame column.</li> </ul> <p>======</p> <h5 id="321-calculating-maximum-values">3.2.1 Calculating Maximum Values</h5> <ul> <li>The text provides an example of using the max() function in Python to find the maximum marks obtained by each student in each subject for Unit Test 2.</li> <li>The max() function can also be used to find the maximum value of each row (i.e., each student’s total marks) by specifying axis = 1.</li> <li>By default, the max() function finds the maximum value of each column (i.e., the maximum marks obtained in each subject) with axis = 0.</li> <li>The concept of axis is important in statistical operations in Python, where axis = 0 typically refers to row-wise operations and axis = 1 refers to column-wise operations.</li> <li>However, in the max() function, axis = 1 gives row-wise output and axis = 0 gives column-wise output, which is different from most other Python functions.</li> </ul> <p>======</p> <h5 id="322-calculating-minimum-values">3.2.2 Calculating Minimum Values</h5> <ul> <li>The text provides a summary of Mishti’s minimum marks obtained in each subject across unit tests.</li> <li>The minimum marks obtained by Mishti in Maths, Science, S.St, Hindi, and Eng are 15, 18, 20, 22, and 20, respectively.</li> <li>The minimum marks were obtained in the following unit tests: Maths - 15, Science - 18, S.St - 20, Hindi - 22, Eng - 20.</li> <li>The data was obtained by filtering the dataframe for Mishti’s name and then calculating the minimum values for each subject.</li> <li>The output of the program can also be achieved using pandas functions dfMishti=df[[‘Maths’,‘Science’,‘S.St’,‘Hindi’,‘Eng’]][df.Name==‘Mishti’].min() and print(dfMishti).</li> </ul> <p>======</p> <h5 id="323-calculating-sum-of-values">3.2.3 Calculating Sum of Values</h5> <ul> <li>The text provides information about a data frame <code>dfRaman</code> containing Raman’s marks in different subjects in multiple units.</li> <li>The sum of marks obtained by Raman in each subject across all units is: Maths - 57, Science - 60, S.St - 50, Hindi - 66, Eng - 68.</li> <li>There is an activity to write Python statements for printing the sum of English marks scored by Mishti, but the actual code or solution is not provided.</li> <li>Another activity is to write Python statements to print the sum of marks obtained by Raman in all subjects in each unit test, for which the solution is provided as <code>dfRaman[[‘Maths’,‘Science’,‘S.St’,‘Hindi’,‘Eng’]].sum(axis=1)</code>.</li> </ul> <p>Note: I have summarized the text as per the instructions, but there are no equations or formulae in the text.</p> <p>======</p> <h5 id="324-calculating-number-of-values">3.2.4 Calculating Number of Values</h5> <ul> <li>DataFrame.count() is used to display the total number of values for each column or row of a DataFrame.</li> <li>To count the rows, the argument axis=1 is used with DataFrame.count().</li> <li>Program 3.5 displays the use of DataFrame.count() to count the number of values in a row.</li> <li>The output of Program 3-5 shows the count of values in each row of the DataFrame.</li> <li>The count of rows can be used to analyze the data and perform further operations.</li> </ul> <p>======</p> <h5 id="325-calculating-mean">3.2.5 Calculating Mean</h5> <ul> <li>The average of marks obtained by Zuhaire in Unit Test 1 is 20.4.</li> <li>The average of marks obtained by Zuhaire in Unit Test 2 is 19.85.</li> <li>The average of marks obtained by Zuhaire in Unit Test 3 is 19.0.</li> <li>The data type of the averages is float64.</li> <li>The code to get this output would involve calculating the mean of the marks obtained by Zuhaire in each unit test.</li> </ul> <p>======</p> <h5 id="326-calculating-median">3.2.6 Calculating Median</h5> <ul> <li>The dataframe ‘dfMathsUT1’ contains the marks scored in Mathematics in UT1.</li> <li>The median marks for Mathematics in UT1 can be found by creating a new dataframe ‘dfMaths’ containing only the ‘Maths’ column of ‘df’ and then using the median() function.</li> <li>The median of the marks in Mathematics for UT1 is 21.0.</li> <li>The activity 3.2 demonstrates finding the median of the values in a dataframe with an even number of rows, where the median is calculated as the average of the two middle values.</li> <li>The text does not provide any equations or formulae for the calculation of the median.</li> </ul> <p>======</p> <h5 id="327-calculating-mode">3.2.7 Calculating Mode</h5> <ul> <li>The mode of a dataset is the value that appears most frequently.</li> <li>The mode can be calculated using the DataFrame.mode() function.</li> <li>For example, to find the mode of the ‘Hindi’ column in a DataFrame named df, you would use the command df[‘Hindi’].mode().</li> <li>In the given dataset, the mode of the ‘Hindi’ column is 24.</li> <li>The mode for the ‘Maths’ column would be calculated similarly, but using the command df[‘Maths’].mode(). The text does not provide the ‘Maths’ column data, so the mode cannot be calculated from the information given.</li> </ul> <p>======</p> <h5 id="328-calculating-quartile">3.2.8 Calculating Quartile</h5> <ul> <li>The text provided contains a list of numerical values without any accompanying context or explanation.</li> <li>There are no equations or formulae present in the text.</li> <li>It is not possible to summarize this data further, as it is already in its simplest form.</li> <li>The values range from 0.25 to 24.00, with no clear pattern or correlation between them.</li> <li>In its current state, the text does not convey any meaningful information.</li> </ul> <p>======</p> <h5 id="329-calculating-variance">3.2.9 Calculating Variance</h5> <ul> <li>The function used to calculate variance in a DataFrame is <code>DataFrame.var()</code>.</li> <li>It displays the variance of the specified columns in the DataFrame.</li> <li>The formula for variance is the average of the squared differences from the mean.</li> <li>For example, to find the variance of ‘Maths’, ‘Science’, ‘S.St’, ‘Hindi’, and ‘Eng’ columns, use <code>df[['Maths','Science','S.St','Hindi','Eng']].var()</code>.</li> <li>The result will be displayed in a new DataFrame with the column names and their variances.</li> </ul> <p>======</p> <h5 id="3210-calculating-standard-deviation">3.2.10 Calculating Standard Deviation</h5> <ul> <li>The text provides statistical data.</li> <li>There are 6 sets of data with 12 data points in each set.</li> <li>The mean, standard deviation, minimum, lower quartile, median, upper quartile, and maximum are calculated for each set.</li> <li>The mean ranges from 2.000000 to 21.833333.</li> <li>The standard deviation ranges from 0.852803 to 3.370999.</li> <li>The minimum value ranges from 1.000000 to 3.000000. The maximum value ranges from 13.000000 to 25.000000.</li> </ul> <p>======</p> <h5 id="33-data-aggregations">3.3 Data Aggregations</h5> <ul> <li>The text provides an example of using the aggregate function in pandas dataframe.</li> <li>The maximum and minimum values for the ‘Maths’ column are calculated using the aggregate function with ‘max’ and ‘min’ as arguments.</li> <li>The same calculation is repeated with the axis parameter set to 0, which gives the same result.</li> <li>The sum of marks obtained by each student in Maths and Science is calculated using the aggregate function with ‘sum’ as argument and axis=1 for row-wise summation.</li> <li>The use of the sum function with axis=1 results in the total marks obtained by each student in Maths and Science.</li> </ul> <p>======</p> <h5 id="34-sorting-a-dataframe">3.4 Sorting a DataFrame</h5> <ul> <li>The text provides the scores of several students in a unit test.</li> <li>The students are Ashravy, Mishti, Raman, and Zuhaire.</li> <li>The unit test is for the science subject.</li> <li>The scores of each student in the three units are given.</li> <li>To get a sorted list of marks scored by all students in Unit Test 2, the given code can be used. It filters the data for Unit Test 2.</li> </ul> <p>======</p> <h5 id="sort-according-to-ascending-order-of-marks-in-science">Sort according to ascending order of marks in Science</h5> <ul> <li>The task is to sort the marks in English in the DataFrame df based on Unit Test 3 in descending order.</li> <li>This can be achieved by creating a new DataFrame, dfUT3, that filters the original DataFrame for rows where the UT column equals 3.</li> <li>The sorted DataFrame can then be obtained by using the sort_values() function on the dfUT3 DataFrame, with the ‘English’ column as the argument and ascending set to False.</li> <li>No equations or formulae are provided in the text.</li> <li>Examples and their solutions are also not included in the text.</li> </ul> <p>======</p> <h5 id="sort-according-to-descending-order-of-marks-in-science">Sort according to descending order of marks in Science</h5> <ul> <li>In a DataFrame, sorting can be performed based on multiple columns.</li> <li>The given text provides an example of sorting a DataFrame <code>df</code> based on marks in Science in Unit Test 3 in ascending order.</li> <li>If marks in Science are the same, sorting will be done on the basis of marks in Hindi.</li> <li>The code for this sorting is provided as <code>dfUT3 = df[df.UT == 3]</code> to get the data corresponding to Unit Test 3.</li> <li>Sorting based on multiple columns is not demonstrated directly in the provided code, but the concept is explained in the text.</li> </ul> <p>======</p> <h5 id="sort-the-data-according-to-science-and-then-according-to-hindi">Sort the data according to Science and then according to Hindi</h5> <ul> <li>The list is sorted based on students’ scores in Science.</li> <li>In case of a tie in Science marks, the students’ Hindi marks are considered for sorting.</li> <li>Three students, Zuhaire and Mishti, as well as Raman and Ashravy, have equal marks in Science.</li> <li>Zuhaire’s and Mishti’s ranks are determined by their Hindi marks (17 and 20, respectively).</li> <li>Raman’s and Ashravy’s ranks are determined by their Hindi marks (14 and 12, respectively).</li> </ul> <p>======</p> <h5 id="35-group-by-functions">3.5 GROUP BY Functions</h5> <ul> <li>The text provided appears to be a table of data related to individuals with names: Ashravy, Mishti, Raman, and Zuhaire.</li> <li>The parameters for each individual are: a first value (ranging from 19.67 to 21.67), a second value (ranging from 44.33 to 2.33), a third value (similar for all, approximately 1.53), and a final value (ranging from 23.0 to 22.0).</li> <li>It is not clear from the text what these values represent.</li> <li>There are no equations or formulae provided in the text.</li> <li>No examples or solutions are included in the text.</li> </ul> <p>======</p> <h5 id="36-altering-the-index">3.6 Altering The InDeX</h5> <ul> <li>The text provides data on the performance of four students in different subjects.</li> <li>Each student is identified by a name: Raman, Zuhaire, Ashravy, and Mishti.</li> <li>The numbers following the student’s name represent their performance in different subjects, possibly represented by the numbers 1, 2, 3, and so on.</li> <li>The first number for each student is their score in Subject 1, the second number is their score in Subject 2, and so on.</li> <li>For example, Raman scored 22, 21, 18, 20, 21 in Subjects 1 to 5 respectively. Similarly, Zuhaire scored 20, 17, 22, 24, 19 in Subjects 1 to 5.</li> </ul> <p>======</p> <h5 id="37-other-dataframe-operations">3.7 Other DataFrame Operations</h5> <ul> <li>The <code>groupby()</code> function is used to group DataFrame rows based on some criteria. It is similar to the SQL’s GROUP BY statement.</li> <li>The <code>pivot_table()</code> function is used to create a spreadsheet-style pivot table as a DataFrame. It is a higher level function and provides more options than <code>groupby()</code>.</li> <li>The <code>sort_values()</code> function is used to sort DataFrame values by one or more columns. It has an <code>ascending</code> parameter to set the sorting order.</li> <li>The <code>merge()</code> function is used to merge two or more DataFrames. It is similar to the SQL’s JOIN statement.</li> <li>The <code>apply()</code> function is used to apply a function along an axis of the DataFrame; either column-wise or row-wise.</li> </ul> <p>======</p> <h5 id="371-reshaping-data">3.7.1 Reshaping Data</h5> <ul> <li>The text provides information about the prices of certain items that come in different colors.</li> <li>The items in question are pens.</li> <li>There are two available colors for these pens: blue and red.</li> <li>The price of a blue pen is 50 Rs.</li> <li>The price of a red pen is 25 Rs.</li> </ul> <p>======</p> <h5 id="38-handuing-missing-values">3.8 Handuing Missing Values</h5> <ul> <li>The text provides information about students’ scores in different subjects and units.</li> <li>Raman’s Maths teacher decides to calculate his score by considering the average of his scores in Unit Tests 1, 2, and 3, while his English teacher decides to consider his score in Unit Test 3 for Unit Test 4.</li> <li>The Science teacher decides to give Raman zero marks in Unit Test 4.</li> <li>The text also mentions a code for checking missing values and replacing them with appropriate values.</li> <li>No equations or formulae are provided in the text.</li> </ul> <p>======</p> <h5 id="381-checking-missing-values">3.8.1 Checking Missing Values</h5> <ul> <li>The text presents the data of a student named Raman’s performance in different subjects.</li> <li>The subjects tested are: UT (Unit Test), Maths, Science, Social Studies (S.St), Hindi, and English.</li> <li>The scores obtained by Raman in each subject are: UT - 22, 21, 14, NaN; Maths - 21, 20, 19, 15; Science - 18, 17, NaN, 19; S.St - 20, 22, 24, 18; Hindi - 21.01, 22, 14, 23.03; English - 24.02, NaN, NaN, NaN.</li> <li>The marks scored in Hindi by Raman have been separately stored in a pandas series, dfHindi.</li> <li>The percentage of marks scored by Raman in Hindi is calculated by summing the marks obtained in Hindi and dividing by 25 (total possible marks) multiplied by 100. The percentage is (66.01/100)*100% which is approximately 66.01%.</li> </ul> <p>======</p> <h5 id="denominator-in-the-above-formula-represents-the-aggregate-of-marks-of-all">denominator in the above formula represents the aggregate of marks of all</h5> <ul> <li>The text provides a formula for calculating the percentage of marks scored by a student, with the denominator being the aggregate of marks of all tests.</li> <li>A Python program (Program 3-13) is presented to find the percentage of marks obtained by a student named Raman in the Maths subject.</li> <li>The program uses a DataFrame (<code>dfRaman</code>) to store the marks scored by Raman in the Maths subject.</li> <li>The number of tests (<code>row</code>) is determined by finding the length of the Maths DataFrame column.</li> <li>The percentage of marks scored by Raman in Maths is calculated and printed by considering the total marks for the tests in which Raman was present and the maximum marks for each test. In the case of his absence in a test, the marks for that test are considered as 0.</li> </ul> <p>======</p> <h5 id="382-dropping-missing-values">3.8.2 Dropping Missing Values</h5> <ul> <li>The text provided is a table of scores for four students: Raman, Zuhaire, Ashravy, and Mishti.</li> <li>The columns represent the student’s name, unit test number, score on the first test, score on the second test, lowest score, highest score, and average score.</li> <li>The equation for calculating the percentage of marks scored by Raman in Maths is: (sum of Maths scores / (total possible score * number of unit tests - number of NA values in Maths scores)) * 100%.</li> <li>Based on this equation, Raman scored 76.0% in Maths.</li> <li>The table does not provide enough information to calculate the percentage scores for the other students.</li> </ul> <p>======</p> <h5 id="383-estimating-missing-values">3.8.3 Estimating Missing Values</h5> <ul> <li>Missing values can be estimated using various methods such as replacing with a specific number (0 or 1), using the value before or after the missing value, or using the average/minimum/maximum value of that attribute.</li> <li>The <code>fillna(num)</code> function can be used to replace missing values with a specified value. For example, <code>fillna(0)</code> replaces missing values with 0.</li> <li>The <code>fillna(method='pad')</code> function replaces the missing value with the value before it, while <code>fillna(method='bfill')</code> replaces the missing value with the value after it.</li> <li>The filled missing values can be used to compute percentages or other statistical measures.</li> <li>Replacing missing values with estimations will change the dataset and the results of data analysis will be approximations of actual results.</li> </ul> <p>======</p> <h5 id="39-import-and-export-of-data-between-pandas-and-mysql">3.9 Import and Export of Data between Pandas AND MySQL</h5> <ul> <li>The text contains information about 4 cars (Car1, Car2, Car3, Car4), with details about their model, version, year, and fuel type.</li> <li>Car1 was available in LXI version in 2017 with Petrol fuel and its cost was 582613.00. In 2018, Car1 was available in VXI version with Petrol fuel and its cost was 673112.00.</li> <li>Car2 was available in Sigma1.2 version in 2019 with Petrol fuel and its cost was 567031.00. In 2018, Car2 was available in Delta1.2 version with Petrol fuel and its cost was 647858.00.</li> <li>Car3 was available in 5 STR STD version in 2017 with CNG fuel and its cost was 355205.00. In 2018, Car3 was available in CARE version with CNG fuel and its cost was 654914.00.</li> <li>Car4 was available in LXI version in 2017 with Petrol fuel and its cost was 514000.00. In 2018, Car4 was available in VXI version with Petrol fuel and its cost was 614000.00.</li> </ul> <p>======</p> <h5 id="391-importing-data-from-mysql-to-pandas">3.9.1 Importing Data from MySQL to Pandas</h5> <ul> <li>The text provides information about the details of four cars, labeled Car1, Car2, Car3, and Car4.</li> <li>Each car has different models, with varying features and engine types (Petrol or CNG) across the years.</li> <li>Cars’ model codes are: D001, D002, B001, B002, E001, E002, S001, and S002.</li> <li>Engine types include Petrol1, Petrol2, Petrol3, Petrol4, CNG5, and CNG6.</li> <li>Car models were introduced in the following years: 2017 (LXI, 5STR STD, CNG5, Petrol7), 2018 (VXI, Delta1.2, CARE, Petrol4, S002, Petrol), and 2019 (Sigma1.2).</li> </ul> <p>======</p> <h5 id="392-exporting-data-from-pandas-to-mysql">3.9.2 Exporting Data from Pandas to MySQL</h5> <ul> <li>The process of writing a pandas DataFrame to a MySQL database is called exporting data from Pandas to MySQL.</li> <li>The function used for this purpose is _pandas.DataFrame.to_sql(table, sql_conn, if_exists=“fail”, index=False/True).</li> <li>The ’table’ parameter specifies the name of the table in which the DataFrame values are to be written.</li> <li>The ‘if_exists’ parameter can have three values: ‘fail’, ‘replace’, or ‘append’, determining how the DataFrame data should be entered in the table.</li> <li>The ‘index’ parameter determines whether the DataFrame index will be copied to the MySQL table (True) or not (False).</li> <li>An example of writing DataFrame ‘df’ to a MySQL database is provided, which creates a table ‘showroom_info’ in the database.</li> </ul> <p>======</p>