<h5 id="21-introduction-to-python-libraries">2.1 Introduction to Python Libraries</h5> <ul> <li>Python libraries are collections of built-in modules that perform various actions without writing detailed programs.</li> <li>NumPy (Numerical Python) is a package for numerical data analysis and scientific computing, using multidimensional arrays.</li> <li>Pandas is a high-level data manipulation tool used for analyzing data, with three important data structures: Series, DataFrame, and Panel.</li> <li>Pandas is built on NumPy and Matplotlib, providing a convenient place for most data analysis and visualization work.</li> <li>Matplotlib is used for plotting graphs and visualization, built on Numpy and designed to work well with Numpy and Pandas.</li> <li>Pandas and Numpy differences: Numpy arrays require homogeneous data, while Pandas DataFrame can have different data types; Pandas have a simpler interface for operations like file loading, plotting, selection, joining, GROUP BY; Pandas DataFrames make it easy to keep track of data, used for tabular format, and Numpy for numeric array-based data manipulation.</li> </ul> <p>======</p> <h5 id="211-installing-pandas">2.1.1. Installing Pandas</h5> <ul> <li>Installing Pandas is similar to installing NumPy.</li> <li>Pandas can be installed from the command line.</li> <li>The command to install Pandas is: “pip install pandas”.</li> <li>Both Pandas and NumPy require Python to be already installed on the system.</li> <li>This is true for all Python libraries.</li> <li>No examples or solutions are provided in this text.</li> </ul> <p>======</p> <h5 id="212-data-structure-in-pandas">2.1.2. Data Structure in Pandas</h5> <ul> <li>A data structure is a collection of data values and operations that can be applied to that data, enabling efficient storage, retrieval, and modification.</li> <li>In Pandas, two commonly used data structures are: <ul> <li>Series: It is a one-dimensional labeled array capable of holding any data type (integers, strings, floating point numbers, Python objects, etc.).</li> <li>DataFrame: It is a two-dimensional labeled data structure with columns potentially of different types. You can think of it like a spreadsheet or SQL table, or a dictionary of Series objects.</li> </ul> </li> <li>These data structures are built on top of NumPy arrays for performance.</li> <li>They provide a lot of functionalities for data manipulation, analysis, and visualization, making them ideal for working with structured data.</li> <li>The primary benefit of using Pandas data structures is the ability to store and manipulate large amounts of data efficiently.</li> </ul> <p>======</p> <h5 id="22-series">2.2 Series</h5> <ul> <li>A Series is a one-dimensional array in Pandas.</li> <li>It contains a sequence of values of any data type.</li> <li>It has default numeric data labels starting from zero, which are called indices.</li> <li>It is possible to assign values of other data types as indices.</li> <li>A Series can be imagined as a column in a spreadsheet.</li> </ul> <p>======</p> <h5 id="221-creation-of-series">2.2.1 Creation of Series</h5> <ul> <li>Scalar values can be used to create a Series in Pandas, with the default index ranging from 0 to N-1, where N is the number of data elements.</li> <li>It is also possible to assign user-defined labels to the index and use them to access elements of a Series.</li> <li>A Series can be created from a one-dimensional NumPy array, with the option to use letters or strings as indices.</li> <li>If the length of the index and array are not the same, a ValueError will be displayed.</li> <li>A Series can be created from a dictionary, with the keys of the dictionary becoming indices in the Series.</li> </ul> <p>======</p> <h5 id="222-accessing-elements-of-a-series">2.2.2 Accessing Elements of a Series</h5> <ul> <li>Accessing elements in a series can be done using indexing, which is similar to that for NumPy arrays. Indexes can be of two types: positional index (integer value) and labeled index (user-defined label).</li> <li>The positional index takes an integer value that corresponds to its position in the series starting from 0. For example, seriesNum[2] will display the value 30 for the positional index 2.</li> <li>Labeled index takes any user-defined label as index. For example, seriesMnths[“Mar”] will display the value 3 for the labelled index Mar.</li> <li>Slicing is used to extract a part of a series. This can be done by specifying the start and end parameters [start :end] with the series name. When labeled indexes are used for slicing, then value at the end index label is also included in the output.</li> <li>More than one element of a series can be accessed using a list of positional integers or a list of index labels. The index values associated with the series can be altered by assigning new index values.</li> </ul> <p>======</p> <h5 id="223-attributes-of-series">2.2.3 Attributes of Series</h5> <ul> <li>The given text is about working with a Series object in Python, specifically named ‘seriesCapCntry’.</li> <li>Names can be assigned to the Series and its index for easier understanding and manipulation. In this case, the Series is named ‘Capitals’ and its index is named ‘Countries’.</li> <li>The ‘values’ attribute of the Series object can be used to print a list of the values in the Series. Here, the values are [‘NewDelhi’, ‘WashingtonDC’, ‘London’, ‘Paris’].</li> <li>The ‘size’ attribute of the Series object can be used to print the number of values in the Series. Here, the size is 4.</li> <li>The ’empty’ attribute of the Series object can be used to check if the Series is empty. Here, the Series is not empty, so the output is False.</li> </ul> <p>======</p> <h5 id="create-an-empty-series">Create an empty series</h5> <ul> <li>The given code creates a pandas Series with three elements: 12, NaN, and 10.</li> <li>The output of the given code <code>print(s2)</code> will be: <code>12.0, nan, 10.0</code>.</li> <li>To count and display only non-null values in the series, you can use the <code>size</code> and <code>notna()</code> functions as follows: <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-python" data-lang="python"><span style="display:flex;"><span>s2<span style="color:#f92672">.</span>notna()<span style="color:#f92672">.</span>sum() </span></span></code></pre></div></li> <li>This code first uses the <code>notna()</code> function to create a Boolean series where True represents non-null values.</li> <li>Then, it uses the <code>sum()</code> function to count the number of True values, which gives the count of non-null values in the original series.</li> </ul> <p>======</p> <h5 id="224-methods-of-series">2.2.4 Methods of Series</h5> <ul> <li>The <code>head(n)</code> function returns the first <code>n</code> members of a series, with <code>n</code> being 5 by default.</li> <li>The <code>tail(n)</code> function returns the last <code>n</code> members of a series, with <code>n</code> being 5 by default.</li> <li>The <code>count()</code> function returns the number of non-NaN values in a series.</li> <li>In the given example, <code>seriesTenTwenty.head(2)</code> returns the first two numbers of the series, while <code>seriesTenTwenty.tail(2)</code> returns the second last and the last number of the series.</li> <li>The data type of the series is <code>int32</code>.</li> </ul> <p>======</p> <h5 id="225-mathematical-operations-on-series">2.2.5 Mathematical Operations on Series</h5> <ul> <li>The text presents examples of mathematical operations on two series with missing values.</li> <li>Two tables, Tables 2.2 and 2.3, show the changes in series elements and corresponding output for multiplication without and with replacing missing values by 0.</li> <li>Subtraction of two series can be done using the subtraction operator or the explicit subtraction method sub().</li> <li>In the example, the missing values in seriesB are replaced with 1000 before subtracting seriesB from seriesA using the explicit subtraction method sub().</li> <li>No equations or formulae are provided in the text, only examples of operations on series with missing values.</li> </ul> <p>======</p> <h5 id="using-fill-value-1000-while-making-explicit">using fill value 1000 while making explicit</h5> <ul> <li>The text discusses handling missing values in Series data using the fill value 1000 in an explicit call of a method.</li> <li>It explains how to multiply two Series using the multiplication operator and the resultant Series.</li> <li>An activity is given to draw two tables for division similar to existing ones, showing changes in Series elements and corresponding output without replacing missing values and after replacing them with 0.</li> <li>The text further explains how to replace missing values with 0 before multiplying SeriesB with SeriesA using the explicit multiplication method mul().</li> </ul> <p>======</p> <h5 id="using-fill-value-0-while-making">using fill value 0 while making</h5> <ul> <li>In data analysis and processing, a fill value is a default value used to replace missing or null data.</li> <li>The choice of fill value can affect the results of statistical analyses.</li> <li>The fill value of 0 is commonly used, especially when working with numerical data.</li> <li>However, using 0 as a fill value can introduce bias or inaccuracies, especially if 0 is a valid data point.</li> <li>Alternative fill values include mean, median, or mode of the non-null data, or using algorithms to estimate missing values.</li> </ul> <p>======</p> <h5 id="explicit-call-of-the-method">explicit call of the method</h5> <ul> <li>Explicit call of a method is preferred when series may have missing values and we want to replace it by a specific value to have a concrete output in place of NaN.</li> <li>Division of two series can be done in two ways: using the division operator or the explicit division method.</li> <li>Using the division operator may result in NaN values if there are missing values in the series.</li> <li>The explicit division method can replace missing values with a specific fill value before dividing the series.</li> <li>In the example, missing values are replaced with 0 before dividing seriesA by seriesB using the explicit division method.</li> </ul> <p>======</p> <h5 id="call-of-the-method">call of the method</h5> <ul> <li>The text provides a call of a method with the following parameters: a, b, infc, d, infe, y, z, and dtype, all of which are float64.</li> <li>The parameters have the following values: a = -0.10, b = infc = d = infe = 0.00, and y = z = 0.05.</li> <li>The parameter dtype is also defined but its value is not specified in the text.</li> <li>The parameters infc and infe appear to be abbreviations, but their full meanings are not provided in the text.</li> <li>No equations or formulae are presented in the text, and no examples or solutions are included.</li> </ul> <p>======</p> <h5 id="23-dataframe">2.3 DataFrame</h5> <ul> <li>The text provides population and density data for four regions.</li> <li>Assam has a population of 78438 and density of 27972.</li> <li>Delhi has a population of 1483 and density of 6.723.</li> <li>Kerala has a population of 38852 and density of 1663.</li> <li>The data for Assam and Kerala include population and density, while the data for Delhi only includes population.</li> <li>Density is given in the unit of people per square kilometer.</li> </ul> <p>======</p> <h5 id="231-creation-of-dataframe">2.3.1 Creation of DataFrame</h5> <ul> <li>The text provides information on three mathematical series.</li> <li>Series 1 consists of terms with the values 0.2, 0.3, 0.4, 0.5, and 0.9.</li> <li>Series 2 and Series 3 both have terms with the values -10, -50, 100, 20, 10, and 0.</li> <li>The specific patterns or rules for calculating the terms in each series are not provided.</li> <li>There are missing values (NaNs) in Series 2 and Series 3, which may need to be addressed before performing calculations with these series.</li> <li>No equations or formulae are provided in the text.</li> </ul> <p>======</p> <h5 id="232-operations-on-rows-and-columns-in-dataframes">2.3.2 Operations on rows and columns in DataFrames</h5> <ul> <li>The text provides the scores of a student in different subjects across various exams.</li> <li>The subject with the highest average score is English (94.6).</li> <li>The subject with the lowest average score is Hindi (71.2).</li> <li>The student’s average score in Maths is 90.4.</li> <li>The student’s average score in Science is 86.4.</li> <li>The student’s average score in English is 96.2.</li> <li>The student’s average score in Hindi is 81.2.</li> </ul> <p>======</p> <h5 id="233-accessing-dataframes-element-through-indexing">2.3.3 Accessing DataFrames Element through Indexing</h5> <ul> <li>Boolean indexing is a technique to select subsets of data based on actual values in a DataFrame.</li> <li>It uses conditions on column names to filter data values, for example: ResultDF.loc[‘Maths’] > 90.</li> <li>The above example displays True or False based on the condition, for the ‘Maths’ column in the DataFrame.</li> <li>To check in which subjects ‘Arnab’ has scored more than 90, we can write: ResultDF.loc[:, ‘Arnab’] > 90.</li> <li>This will return True for the subjects where the condition is satisfied, for example: Maths and Hindi for ‘Arnab’.</li> </ul> <p>======</p> <h5 id="234-accessing-dataframes-element-through-slicing">2.3.4 Accessing DataFrames Element through Slicing</h5> <ul> <li>The text provided contains the scores of a student in different subjects.</li> <li>The student has scored 90, 92, 89, and 81 in Maths.</li> <li>In Hindi, the student has achieved 97, 96, 88, 67, and 99.</li> <li>No equations or formulae are provided in the text.</li> <li>Examples and their solutions are also not included.</li> <li>The student’s performance varies, with higher scores in Maths and Hindi.</li> </ul> <p>======</p> <h5 id="235-joining-merging-and-concatenation-of-dataframes">2.3.5 Joining, Merging and Concatenation of DataFrames</h5> <ul> <li>The append() method can be used to add a Series or a Dictionary to a DataFrame in Python.</li> <li>This method can be used to add data at the end of the DataFrame.</li> <li>No equations or formulae are provided in the text.</li> <li>Examples and their solutions are not provided in the text.</li> <li>The summarization is clear and short, following the user’s instructions.</li> </ul> <p>======</p> <h5 id="236-attributes-of-dataframes">2.3.6 Attributes of DataFrames</h5> <ul> <li>The <code>.index</code> property of a DataFrame displays the row labels. For the DataFrame <code>ForestAreaDF</code>, it is set to <code>Index([‘GeoArea’, ‘VeryDense’,‘ModeratelyDense’, ‘OpenForest’], dtype =’object’)</code>.</li> <li>The <code>.columns</code> property of a DataFrame displays the column labels. For the DataFrame <code>ForestAreaDF</code>, it is set to <code>Index([‘Assam’, ‘Kerala’, ‘Delhi’], dtype=’object’)</code>.</li> <li>The <code>.dtypes</code> property of a DataFrame displays the data type of each column. For the DataFrame <code>ForestAreaDF</code>, it shows that ‘Assam’ and ‘Kerala’ are of type <code>int64</code> and ‘Delhi’ is of type <code>float64</code>.</li> <li>The <code>.values</code> property of a DataFrame displays a NumPy ndarray having all the values in the DataFrame, without the axes labels.</li> <li>The <code>.shape</code> property of a DataFrame returns a tuple representing the dimensionality of the DataFrame. For <code>ForestAreaDF</code>, it is <code>(4, 3)</code>, indicating 4 rows and 3 columns.</li> </ul> <p>======</p> <h5 id="24-importing-and-exporting-data-between-csv-files-and-dataframes">2.4 Importing and Exporting Data between CSV Files and DataFrames</h5> <ul> <li>DataFrames can be created by importing data from CSV files.</li> <li>The importing process assumes that values in the CSV file are separated by commas.</li> <li>Conversely, data from a DataFrame can be stored or exported as a .csv file.</li> <li>This process allows for easy data exchange between different software applications.</li> <li>The data importing and exporting process is a fundamental feature of DataFrames.</li> </ul> <p>======</p> <h5 id="241-importing-a-csv-file-to-a-dataframe">2.4.1 Importing a CSV file to a DataFrame</h5> <ul> <li>The text provides information about five individuals.</li> <li>Each individual is identified by a name.</li> <li>The corresponding age and a unique identifier (ID) are also given for each individual.</li> <li>The ages of the individuals range from 18 to 51.</li> <li>The IDs range from 1 to 604, indicating possibly a sequence or classification for each individual.</li> </ul> <p>======</p> <h5 id="242-exporting-a-dataframe-to-a-csv-file">2.4.2 Exporting a DataFrame to a CSV file</h5> <ul> <li> <p>The text provided is about creating and manipulating CSV files using Python’s pandas library.</p> </li> <li> <p>A DataFrame’s <code>to_csv()</code> function is used to create a CSV file, with each row representing a record and each record consisting of one or more fields (columns).</p> </li> <li> <p>The separator between values can be specified using the <code>sep</code> parameter. For example, <code>sep='@'</code> will use the ‘@’ character as a separator.</p> </li> <li> <p>The <code>header</code> and <code>index</code> parameters can be set to <code>False</code> to prevent the column names and row labels from being saved to the file.</p> </li> <li> <p>The allowed characters for use as a separator include comma, as well as any other character. This can be useful when dealing with data that contains commas.</p> </li> </ul> <p>======</p> <h5 id="25-pandas-series-vs-numpy-ndarray">2.5 Pandas Series Vs NumPy ndarray</h5> <ul> <li>In series, it is possible to define a labeled index for array element access, which can consist of numbers or letters.</li> <li>NumPy arrays are accessed by their integer position, using numbers only, and can be indexed in descending order.</li> <li>The indexing of series starts at zero for the first element, and the index is fixed. If two series are not aligned, NaN or missing values are generated.</li> <li>In contrast, NumPy arrays do not have a concept of NaN values, and if there are no matching values in arrays, alignment fails.</li> <li>Series require more memory for storage, while NumPy arrays occupy less memory.</li> </ul> <p>======</p>