Topic 5: What is pandas?

-

 What is Pandas?


Pandas is an open-source Python library designed for data manipulation, analysis, and cleaning.
It offers powerful data structures such as Series for 1-dimensional data and DataFrame for 2-dimensional tabular data.
Built on top of NumPy, Pandas ensures fast and efficient performance for large datasets. It simplifies tasks like data filtering, transformation, aggregation, and visualization. Pandas supports reading and writing data from multiple sources, including CSV, Excel, and SQL. Its intuitive syntax allows users to handle missing data and perform complex operations easily. Overall, Pandas is an essential tool for data science, analytics, and machine learning workflows.

Diagram: Pandas Workflow

Need of Pandas?

Without Pandas, working with large datasets in plain Python is difficult.

Pandas helps by providing: 1. Efficient data handling Load, modify, and analyze data easily.

2. Flexible data structures – Handle missing values, duplicates, etc.

3. Integration – Works well with NumPy, Matplotlib, Scikit-learn, etc.

4. Data Import/Export– Easily read/write data in CSV, Excel, JSON, SQL, etc.

5. Faster data processing– Optimized for performance.


Core Data Structures in Pandas: 

There are two type of data structure in pandas such as Series for 1-dimensional data and DataFrame for 2-dimensional tabular data. 
Core Data Structure in Pandas


Common Pandas Inbuilt Functions with Examples


1️⃣Creating Series and DataFrame

python

import pandas as pd

Series

s = pd.Series([10, 20, 30, 40])

print(s)


 DataFrame

data = {'Name': ['Alice', 'Bob', 'Charlie'],

        'Age': [25, 30, 35],

        'City': ['Delhi', 'Mumbai', 'Chennai']}

df = pd.DataFrame(data)

print(df)

2️⃣ head() and tail()

 Display first rows and last few rows.

python

print(df.head(2))   # First 2 rows

print(df.tail(1))   # Last 1 row

3️⃣ info()

 Shows basic information about the DataFrame.

python

df.info()

Output:

<class 'pandas.core.frame.DataFrame'>

RangeIndex: 3 entries, 0 to 2

Data columns (total 3 columns):

4️⃣ describe()

 Provides statistical summary of numeric columns.

python

print(df.describe())

Output:

|            | Age  |

| -----    | ---- |

| count  | 3.0  |

| mean  | 30.0 |

| min    | 25.0 |

| max    | 35.0 |


5️⃣ shape

 Returns (rows, columns) of DataFrame.

python

print(df.shape)

 Output: (3, 3)

6️⃣columns and index

Get column names and row index.

python

print(df.columns)

print(df.index)

7️⃣ sort_values()

 Sort data based on column values.

python

print(df.sort_values(by='Age', ascending=False))

8️⃣ iloc[] and loc[]

 Access specific rows and columns.

python

print(df.iloc[0])       # First row (by index position)

print(df.loc[1, 'City'])  # Value at row 1 and column ‘City’

9️⃣ isnull() and dropna()

Handle missing data.

python

df.isnull()     # Check for missing values

df.dropna()     # Remove rows with missing values

🔟fillna()

 Fill missing values with specified data.

python

df.fillna(value=0)


1️⃣1️⃣ groupby()

Group data by certain columns and apply functions.

python

group = df.groupby('City')['Age'].mean()

print(group)

1️⃣2️⃣ merge(), concat(), join()

 Combine multiple DataFrames.

python

df1 = pd.DataFrame({'ID':[1,2], 'Name':['A','B']})

df2 = pd.DataFrame({'ID':[1,2], 'Salary':[50000,60000]})

result = pd.merge(df1, df2, on='ID')

print(result)

1️⃣3️⃣ read_csv() & to_csv()

 Import/export data easily.

python

df = pd.read_csv('data.csv')

df.to_csv('output.csv', index=False)

1️⃣4️⃣ value_counts()

 Count frequency of unique values in a column.

python

print(df['City'].value_counts())

Pandas Quiz

Summary

| Function                                         | Purpose                 |

| ----------------------------------          | ----------------------- |

| `head()` / `tail()`                             | Show first/last rows    |

| `info()`                                            | Data info summary       |

| `describe()`                                     | Statistics summary      |

| `shape`                                            | Dimensions of DataFrame |

| `sort_values()`                                | Sort data               |

| `isnull()`, `dropna()`, `fillna()`       | Handle missing data     |

| `groupby()`                                     | Aggregate data          |

| `merge()`, `concat()`                      | Combine DataFrames      |

| `read_csv()`, `to_csv()`                  | Data import/export      |


Pandas Quiz

Comments

Post a Comment

Popular posts from this blog

Topic1:- Introduction of Machine Learning

-
Image
What is Machine Learning (ML)? Machine Learning is a branch of Artificial Intelligence (AI) that enables computers to learn from data and improve their performance automatically without being explicitly programmed for every task.  In simple terms: Machine Learning is about teaching computers to learn patterns from data and make decisions or predictions. Example: Imagine you want to make a program that predicts whether an email is spam or not spam . You collect thousands of emails (data). You label them as spam or not spam (training data). You train an ML model on these examples. Later, the model can predict new, unseen emails as spam or not spam automatically. Types of Machine Learning Machine Learning is generally divided into three main types . Types of Machine Learning 1. Supervised Learning In supervised learning , the model learns from labeled data — data that already has the correct output Goal: To make predictions based on example input...
Read more

Topic3:- python's Important Libraries for Machine Learning

-
Image
Important Python libraries for Machine Learning and Data Science : 1. NumPy (Numerical Python) NumPy is a library used for numerical computation in Python. It provides multidimensional arrays (called ndarray ) and supports fast mathematical operations. Key Features: Efficient array operations (faster than Python lists) Mathematical functions: mean, sum, std, etc. Linear algebra, Fourier transforms, random numbers Example: import numpy as np # Create a NumPy array arr = np.array([1, 2, 3, 4, 5]) # Perform operations print("Mean:", np.mean(arr)) print("Array * 2:", arr * 2) Output: Mean: 3.0 Array * 2: [ 2 4 6 8 10] 2. Pandas     Pandas is used for data manipulation and analysis . It provides two main data structures: Series (1D) DataFrame (2D table like Excel) Key Features: Handling tabular data easily Data cleaning, merging, reshaping, filtering Reading/writing CSV, Excel, SQL files Example: import pandas as pd...
Read more

Topic 2:- Machine Learning Workflow

-
Image
Workflow  The Machine Learning (ML) workflow is a step-by-step process used to build, train, test, and evaluate an ML model. It ensures that data is properly prepared, the model is correctly trained, and its performance is accurately measured.   Main Stages of the ML Workflow ML Overview(step-by-step) 1. Data Collection Gather data from various sources such as CSV files, databases, APIs, sensors, or online datasets. Example: Collecting house price data (area, location, price). 2. Data Preprocessing Clean and prepare data before training. Handle missing values, outliers, and categorical encoding. Apply feature scaling and normalization. 3. Train/Test Split Dataset Division Split the dataset into two parts: Training Set: Used to train the model (70–80% of data) Testing Set: Used to test the model (20–30% of data)   Purpose: To check how well the model performs on unseen da...
Read more