Python Notes
HubHistory of Python - Complete Guide 2026Python Applications - Complete Guide 2026Python Features - Complete Guide 2026What is Python? - Complete Guide 2026Why Learn Python? - Complete Guide 2026Your First Python Program – Hello World & Beyond 2026How to Install Python on Windows, Mac & Linux – Complete Guide 2026Install VS Code for Python – Complete Setup Guide with Extensions 2026Python Environment Setup – Virtual Environments, pip & Project Structure 2026Python IDLE – Complete Guide for Beginners 2026Python CommentsPython Data TypesPython Input and OutputPython OperatorsPython SyntaxPython Strings BasicsPython Type CastingPython VariablesBreak, Continue and Pass in PythonElif Statement in PythonFor Loop in PythonIf-Else Statement in PythonIf Statement in PythonNested If in PythonPattern Programs in PythonWhile Loop in PythonDecorators in PythonFunction Arguments in PythonFunctions Basics in PythonGenerators in PythonLambda Functions in PythonRecursive Functions in PythonReturn Statement in PythonScope of Variables in PythonAbstraction in PythonClasses and Objects in PythonConstructors in PythonEncapsulation in PythonInheritance in PythonMagic Methods (Dunder Methods) in PythonMethod Overriding in PythonPolymorphism in PythonPython Comprehensions – List, Dict, Set & GeneratorPython Dictionaries – Key-Value StorePython Dictionary Methods – Complete ReferencePython List Methods – All Built-in MethodsPython Lists – Complete GuidePython Set Methods – Complete ReferencePython Sets – Unordered Unique CollectionsPython Tuples – Immutable SequencesPython Asynchronous Programming with asyncioPython Context ManagersAdvanced Python DecoratorsAdvanced Python GeneratorsPython Iterators - Advanced GuidePython Memory ManagementPython MultiprocessingPython MultithreadingCRUD Operations in Python DatabasesMySQL with Python (mysql-connector-python)ORM Introduction with SQLAlchemyPostgreSQL with Python (psycopg2)SQLite3 with PythonIntroduction to DjangoDjango Models and ORMDjango REST API with DRFDjango ViewsIntroduction to FlaskFlask RoutingFlask Templates with Jinja2Email Automation with PythonPDF Automation with PythonSelenium Web Automation with PythonTask Scheduler with PythonWeb Scraping with PythonData Analysis with PythonData Cleaning with PythonData Visualization ProjectData Visualization with MatplotlibNumPy IntroductionPandas IntroductionStatistical Visualization with SeabornClassification AlgorithmsClustering AlgorithmsIntroduction to Machine LearningMachine Learning Project: Churn PredictionModel Evaluation and SelectionRegression AlgorithmsSave pipeline (includes preprocessing + model)AI Assistant ProjectCalculator ProjectChatbot ProjectExpense Tracker ProjectFace Detection ProjectPassword Generator ProjectWeather App ProjectPython Coding Interview QuestionsPython OOP Interview Questions and AnswersPython Master CheatsheetPython Interview Questions and AnswersPython Best PracticesCommon Python Errors and SolutionsPython Debugging TechniquesPython Learning Resources
Learn Pandas, the powerful Python data manipulation library. Master DataFrames, Series, data loading, selection, filtering, groupby, merging, and essential data operations for data analysis.
Pandas is the most powerful Python library for data manipulation and analysis. It provides high-performance, easy-to-use data structures — primarily the DataFrame and Series — for working with structured data.
Installation
bash example
pip install pandas openpyxl xlrdpython example
import pandas as pd
import numpy as np
print(pd.__version__) # 2.x.xSeries — 1D Labeled Array
Example
import pandas as pd
import numpy as np
# Create a Series
s1 = pd.Series([10, 20, 30, 40, 50])
print(s1)
# 0 10
# 1 20
# 2 30
# 3 40
# 4 50
# dtype: int64
# Custom index
s2 = pd.Series([100, 200, 300], index=['a', 'b', 'c'])
print(s2['b']) # 200
# From dictionary
data = {'Math': 95, 'Science': 88, 'English': 72, 'History': 85}
grades = pd.Series(data)
print(grades)
# Series properties
print(grades.index) # Index(['Math', 'Science', 'English', 'History'])
print(grades.values) # [95 88 72 85]
print(grades.dtype) # int64
print(grades.name) # None
# Operations on Series
print(grades + 5) # Add 5 to each
print(grades[grades > 80]) # Filter
print(grades.mean()) # 85.0
DataFrame — 2D Labeled Data Structure
Example
import pandas as pd
# From dictionary of lists
data = {
'Name': ['Alice', 'Bob', 'Carol', 'Dave', 'Eve'],
'Age': [25, 30, 35, 28, 22],
'Department': ['Engineering', 'Marketing', 'Engineering', 'HR', 'Marketing'],
'Salary': [85000, 72000, 95000, 68000, 78000],
'Years': [3, 7, 10, 5, 2],
'Active': [True, True, True, False, True]
}
df = pd.DataFrame(data)
print(df)
# Name Age Department Salary Years Active
# 0 Alice 25 Engineering 85000 3 True
# 1 Bob 30 Marketing 72000 7 True
# ...
# DataFrame properties
print(df.shape) # (5, 6) — rows, columns
print(df.dtypes) # Data types of each column
print(df.columns) # Column names
print(df.index) # Row index (0-4)
print(df.info()) # Concise summary
print(df.describe()) # Statistical summary of numeric columns
print(df.head(3)) # First 3 rows
print(df.tail(2)) # Last 2 rows
Loading Data
Example
import pandas as pd
# Read CSV
df = pd.read_csv('data.csv')
df = pd.read_csv('data.csv', sep=';', encoding='utf-8')
df = pd.read_csv('data.csv', index_col='id', parse_dates=['date'])
df = pd.read_csv('data.csv', usecols=['name', 'age', 'salary'])
df = pd.read_csv('data.csv', nrows=100) # Only first 100 rows
df = pd.read_csv('data.csv', skiprows=[1, 2]) # Skip rows
# Read Excel
df = pd.read_excel('data.xlsx', sheet_name='Sheet1')
df = pd.read_excel('data.xlsx', sheet_name=0)
# Read JSON
df = pd.read_json('data.json')
# From URL
url = 'https://raw.githubusercontent.com/mwaskom/seaborn-data/master/tips.csv'
tips = pd.read_csv(url)
# Read from string/StringIO (useful for testing)
from io import StringIO
csv_data = """name,age,city
Alice,25,NYC
Bob,30,LA"""
df = pd.read_csv(StringIO(csv_data))
# Create sample DataFrame for practice
from sklearn.datasets import load_iris
iris_data = load_iris()
iris_df = pd.DataFrame(iris_data.data, columns=iris_data.feature_names)
Selecting Data
Example
import pandas as pd
data = {
'Name': ['Alice', 'Bob', 'Carol', 'Dave', 'Eve'],
'Age': [25, 30, 35, 28, 22],
'Department': ['Engineering', 'Marketing', 'Engineering', 'HR', 'Marketing'],
'Salary': [85000, 72000, 95000, 68000, 78000],
}
df = pd.DataFrame(data)
# Select column(s)
print(df['Name']) # Series
print(df[['Name', 'Salary']]) # DataFrame (multiple columns)
# .loc — label-based selection
print(df.loc[0]) # Row at index 0
print(df.loc[0, 'Name']) # Single value
print(df.loc[1:3, 'Name':'Salary']) # Row slice, column slice
print(df.loc[df['Age'] > 25]) # Boolean selection
# .iloc — integer position-based selection
print(df.iloc[0]) # First row
print(df.iloc[0, 1]) # Row 0, column 1
print(df.iloc[1:3, 0:3]) # Rows 1-2, columns 0-2
print(df.iloc[-1]) # Last row
print(df.iloc[:, -1]) # Last column
# .at and .iat — scalar access (faster for single values)
print(df.at[0, 'Name']) # Label-based
print(df.iat[0, 0]) # Integer position-based
Filtering and Querying
Example
import pandas as pd
df = pd.DataFrame({
'Name': ['Alice', 'Bob', 'Carol', 'Dave', 'Eve'],
'Age': [25, 30, 35, 28, 22],
'Department': ['Engineering', 'Marketing', 'Engineering', 'HR', 'Marketing'],
'Salary': [85000, 72000, 95000, 68000, 78000],
})
# Boolean filtering
engineers = df[df['Department'] == 'Engineering']
high_salary = df[df['Salary'] > 75000]
senior = df[df['Age'] >= 30]
# Multiple conditions
eng_high_sal = df[(df['Department'] == 'Engineering') & (df['Salary'] > 80000)]
mktg_or_hr = df[(df['Department'] == 'Marketing') | (df['Department'] == 'HR')]
# isin — check membership
depts = ['Engineering', 'Marketing']
df_filtered = df[df['Department'].isin(depts)]
# between — range check
mid_salary = df[df['Salary'].between(70000, 90000)]
# String methods
df[df['Name'].str.startswith('A')]
df[df['Name'].str.contains('li', case=False)]
df[df['Department'].str.lower() == 'engineering']
# Query method — SQL-like syntax
result = df.query("Age > 25 and Salary > 75000")
result = df.query("Department in ['Engineering', 'Marketing']")
result = df.query("Age.between(25, 30)", engine='python')
# notna / isna
df[df['Salary'].notna()] # Rows where Salary is not NaN
df[df['Name'].isna()] # Rows where Name is NaN
Adding and Modifying Data
Example
import pandas as pd
df = pd.DataFrame({
'Name': ['Alice', 'Bob', 'Carol'],
'Salary': [85000, 72000, 95000],
'Years': [3, 7, 10],
})
# Add new column
df['Bonus'] = df['Salary'] * 0.1
df['Seniority'] = df['Years'].apply(lambda x: 'Senior' if x >= 5 else 'Junior')
df['Full_info'] = df['Name'] + ' - $' + df['Salary'].astype(str)
# Modify column
df['Salary'] = df['Salary'] * 1.05 # 5% raise for everyone
# Modify specific values with loc
df.loc[df['Name'] == 'Alice', 'Salary'] = 90000
df.loc[1, 'Years'] = 8
# Apply function to column
df['Name_length'] = df['Name'].apply(len)
df['Grade'] = df['Salary'].apply(lambda x: 'A' if x > 80000 else 'B')
# Apply function to entire DataFrame
df_numeric = df.select_dtypes(include='number')
df_normalized = df_numeric.apply(lambda col: (col - col.min()) / (col.max() - col.min()))
# Map values
dept_codes = {'Engineering': 'ENG', 'Marketing': 'MKT', 'HR': 'HUM'}
# df['Dept_code'] = df['Department'].map(dept_codes)
# Rename columns
df = df.rename(columns={'Name': 'Employee', 'Salary': 'Annual_Salary'})
df.columns = ['emp', 'sal', 'yrs', 'bonus', 'seniority', 'info', 'nlen', 'grade']
Handling Missing Data
Example
import pandas as pd
import numpy as np
# Create DataFrame with NaN values
df = pd.DataFrame({
'Name': ['Alice', 'Bob', None, 'Dave', 'Eve'],
'Age': [25, np.nan, 35, 28, np.nan],
'Salary': [85000, 72000, np.nan, 68000, 78000],
'Score': [95, np.nan, 88, np.nan, 92],
})
# Detect missing values
print(df.isnull()) # Boolean mask
print(df.isna()) # Same as isnull()
print(df.isnull().sum()) # Count nulls per column
print(df.isnull().sum().sum()) # Total null count
print(f"Missing: {df.isnull().mean() * 100:.1f}%") # Percentage
# Drop rows/columns with NaN
df_clean = df.dropna() # Drop rows with any NaN
df_clean = df.dropna(how='all') # Drop rows where ALL are NaN
df_clean = df.dropna(subset=['Name', 'Age']) # Drop if NaN in these cols
df_clean = df.dropna(thresh=3) # Keep rows with at least 3 non-NaN
# Fill missing values
df_filled = df.fillna(0) # Fill all with 0
df_filled = df.fillna({'Age': df['Age'].mean(), 'Salary': df['Salary'].median()})
df_filled = df.fillna(method='ffill') # Forward fill
df_filled = df.fillna(method='bfill') # Backward fill
# Interpolate
df['Age'] = df['Age'].interpolate() # Linear interpolation
# Replace specific values
df['Name'] = df['Name'].fillna('Unknown')
df = df.replace({np.nan: None})
df = df.replace({'Score': {np.nan: df['Score'].mean()}})
Sorting
Example
import pandas as pd
df = pd.DataFrame({
'Name': ['Alice', 'Bob', 'Carol', 'Dave'],
'Department': ['Eng', 'Mkt', 'Eng', 'HR'],
'Salary': [85000, 72000, 95000, 68000],
'Age': [25, 30, 35, 28],
})
# Sort by values
df_sorted = df.sort_values('Salary') # Ascending
df_sorted = df.sort_values('Salary', ascending=False) # Descending
df_sorted = df.sort_values(['Department', 'Salary'], ascending=[True, False])
# Sort by index
df_sorted = df.sort_index()
df_sorted = df.sort_index(ascending=False)
# Rank
df['Salary_rank'] = df['Salary'].rank(ascending=False)
df['Percentile'] = df['Salary'].rank(pct=True)
# Nlargest / nsmallest
top3 = df.nlargest(3, 'Salary')
bottom2 = df.nsmallest(2, 'Age')
GroupBy Operations
Example
import pandas as pd
df = pd.DataFrame({
'Name': ['Alice', 'Bob', 'Carol', 'Dave', 'Eve', 'Frank'],
'Department': ['Eng', 'Mkt', 'Eng', 'HR', 'Mkt', 'HR'],
'Salary': [85000, 72000, 95000, 68000, 78000, 71000],
'Years': [3, 7, 10, 5, 2, 8],
'Score': [92, 85, 97, 78, 88, 82],
})
# Basic groupby
dept_groups = df.groupby('Department')
# Aggregate
print(dept_groups['Salary'].mean())
print(dept_groups['Salary'].sum())
print(dept_groups['Salary'].max())
print(dept_groups.size()) # Count per group
# Multiple aggregations
agg_result = dept_groups['Salary'].agg(['mean', 'min', 'max', 'std', 'count'])
# Different agg for different columns
agg_result = dept_groups.agg({
'Salary': ['mean', 'max'],
'Years': 'mean',
'Score': ['mean', 'min']
})
# Named aggregations (pandas 0.25+)
result = dept_groups.agg(
avg_salary=('Salary', 'mean'),
max_salary=('Salary', 'max'),
avg_years=('Years', 'mean'),
emp_count=('Name', 'count')
)
# Transform — returns same shape
df['dept_avg_salary'] = dept_groups['Salary'].transform('mean')
df['salary_vs_dept'] = df['Salary'] - df['dept_avg_salary']
# Apply custom function
def dept_stats(group):
return pd.Series({
'count': len(group),
'avg_sal': group['Salary'].mean(),
'top_earner': group.loc[group['Salary'].idxmax(), 'Name']
})
custom_stats = dept_groups.apply(dept_stats)
print(custom_stats)
Merging and Joining
Example
import pandas as pd
employees = pd.DataFrame({
'emp_id': [1, 2, 3, 4, 5],
'name': ['Alice', 'Bob', 'Carol', 'Dave', 'Eve'],
'dept_id': [101, 102, 101, 103, 102],
})
departments = pd.DataFrame({
'dept_id': [101, 102, 103, 104],
'dept_name': ['Engineering', 'Marketing', 'HR', 'Finance'],
'budget': [500000, 200000, 100000, 300000],
})
salaries = pd.DataFrame({
'emp_id': [1, 2, 3, 5, 6],
'salary': [85000, 72000, 95000, 78000, 65000],
})
# Inner join — only matching records
inner = pd.merge(employees, departments, on='dept_id')
# Left join — all from left, matching from right
left = pd.merge(employees, salaries, on='emp_id', how='left')
# Right join
right = pd.merge(employees, departments, on='dept_id', how='right')
# Outer join — all records
outer = pd.merge(employees, salaries, on='emp_id', how='outer')
# Different column names
result = pd.merge(
employees, departments,
left_on='dept_id', right_on='dept_id',
suffixes=('_emp', '_dept')
)
# Concat — stack DataFrames
q1 = pd.DataFrame({'month': ['Jan', 'Feb', 'Mar'], 'sales': [100, 120, 115]})
q2 = pd.DataFrame({'month': ['Apr', 'May', 'Jun'], 'sales': [130, 125, 140]})
full_year = pd.concat([q1, q2], ignore_index=True)
Pivot Tables
Example
import pandas as pd
df = pd.DataFrame({
'Date': ['2026-01', '2026-01', '2026-02', '2026-02'] * 2,
'Region': ['North', 'South', 'North', 'South'] * 2,
'Product': ['A', 'A', 'B', 'B', 'A', 'B', 'A', 'B'],
'Sales': [100, 80, 120, 90, 110, 85, 95, 75],
'Units': [10, 8, 12, 9, 11, 8, 9, 7],
})
# Pivot table
pivot = pd.pivot_table(
df,
values='Sales',
index='Region',
columns='Date',
aggfunc='sum',
fill_value=0
)
# Multiple values
pivot2 = pd.pivot_table(
df,
values=['Sales', 'Units'],
index='Region',
columns='Product',
aggfunc={'Sales': 'sum', 'Units': 'mean'},
margins=True # Add row/column totals
)
# Crosstab
ct = pd.crosstab(df['Region'], df['Product'], values=df['Sales'], aggfunc='sum')
Saving Data
Example
import pandas as pd
df = pd.DataFrame({'name': ['Alice', 'Bob'], 'score': [95, 87]})
# Save to CSV
df.to_csv('output.csv', index=False)
df.to_csv('output.csv', index=False, encoding='utf-8-sig') # For Excel compatibility
# Save to Excel
df.to_excel('output.xlsx', index=False, sheet_name='Results')
# Multiple sheets
with pd.ExcelWriter('multi_sheet.xlsx') as writer:
df.to_excel(writer, sheet_name='Employees', index=False)
df.to_excel(writer, sheet_name='Summary', index=False)
# Save to JSON
df.to_json('output.json', orient='records', indent=2)
# Save to Parquet (efficient columnar format)
df.to_parquet('output.parquet', index=False)
Summary
In this lesson, you learned:
- ✅ Creating Series and DataFrames
- ✅ Loading data from CSV, Excel, JSON
- ✅ Selecting data with
[],.loc,.iloc - ✅ Filtering with boolean masks and
.query() - ✅ Adding, modifying, and applying functions to data
- ✅ Handling missing values (fillna, dropna, interpolate)
- ✅ Sorting and ranking
- ✅ GroupBy aggregations and transforms
- ✅ Merging, joining, and concatenating DataFrames
- ✅ Pivot tables for data summarization
📤 Output Examples
Yahan par kuch code examples ke outputs diye gaye hain jo upar ke sections mein shown nahi the. Ye aapko samajhne mein help karenge ki actual output kaisa dikhta hai. 👇
DataFrame Properties Output
Example
import pandas as pd
data = {
'Name': ['Alice', 'Bob', 'Carol', 'Dave', 'Eve'],
'Age': [25, 30, 35, 28, 22],
'Department': ['Engineering', 'Marketing', 'Engineering', 'HR', 'Marketing'],
'Salary': [85000, 72000, 95000, 68000, 78000],
}
df = pd.DataFrame(data)
print(df.shape)
print(df.columns.tolist())
print(df.describe())
Output
(5, 4)
['Name', 'Age', 'Department', 'Salary']
Age Salary
count 5.000000 5.000000
mean 28.000000 79600.000000
std 4.949747 10356.203202
min 22.000000 68000.000000
25% 25.000000 72000.000000
50% 28.000000 78000.000000
75% 30.000000 85000.000000
max 35.000000 95000.000000Filtering Output
Example
import pandas as pd
df = pd.DataFrame({
'Name': ['Alice', 'Bob', 'Carol', 'Dave', 'Eve'],
'Age': [25, 30, 35, 28, 22],
'Department': ['Engineering', 'Marketing', 'Engineering', 'HR', 'Marketing'],
'Salary': [85000, 72000, 95000, 68000, 78000],
})
# Filter: Engineers with salary > 80000
result = df[(df['Department'] == 'Engineering') & (df['Salary'] > 80000)]
print(result)
Output
Name Age Department Salary 0 Alice 25 Engineering 85000 2 Carol 35 Engineering 95000
Query Method Output
python example
result = df.query("Age > 25 and Salary > 75000")
print(result)Output
Name Age Department Salary 1 Bob 30 Marketing 72000 # ❌ Not included (Salary <= 75000) 0 Alice 25 Engineering 85000 # ❌ Not included (Age <= 25) 2 Carol 35 Engineering 95000
Output
Name Age Department Salary 2 Carol 35 Engineering 95000
Handling Missing Data Output
Example
import pandas as pd
import numpy as np
df = pd.DataFrame({
'Name': ['Alice', 'Bob', None, 'Dave', 'Eve'],
'Age': [25, np.nan, 35, 28, np.nan],
'Salary': [85000, 72000, np.nan, 68000, 78000],
})
print(df.isnull().sum())
print(f"\nTotal Missing: {df.isnull().sum().sum()}")
Output
Name 1 Age 2 Salary 1 dtype: int64 Total Missing: 4
GroupBy Output
Example
import pandas as pd
df = pd.DataFrame({
'Name': ['Alice', 'Bob', 'Carol', 'Dave', 'Eve', 'Frank'],
'Department': ['Eng', 'Mkt', 'Eng', 'HR', 'Mkt', 'HR'],
'Salary': [85000, 72000, 95000, 68000, 78000, 71000],
})
result = df.groupby('Department').agg(
avg_salary=('Salary', 'mean'),
max_salary=('Salary', 'max'),
emp_count=('Name', 'count')
)
print(result)
Output
avg_salary max_salary emp_count Department Eng 90000.0 95000 2 HR 69500.0 71000 2 Mkt 75000.0 78000 2
Merge Output
Example
import pandas as pd
employees = pd.DataFrame({
'emp_id': [1, 2, 3],
'name': ['Alice', 'Bob', 'Carol'],
'dept_id': [101, 102, 101],
})
departments = pd.DataFrame({
'dept_id': [101, 102, 103],
'dept_name': ['Engineering', 'Marketing', 'HR'],
})
result = pd.merge(employees, departments, on='dept_id')
print(result)
Output
emp_id name dept_id dept_name 0 1 Alice 101 Engineering 1 3 Carol 101 Engineering 2 2 Bob 102 Marketing
Sorting Output
Example
import pandas as pd
df = pd.DataFrame({
'Name': ['Alice', 'Bob', 'Carol', 'Dave'],
'Salary': [85000, 72000, 95000, 68000],
})
print(df.nlargest(3, 'Salary'))
Output
Name Salary 2 Carol 95000 0 Alice 85000 1 Bob 72000
Pivot Table Output
Example
import pandas as pd
df = pd.DataFrame({
'Region': ['North', 'South', 'North', 'South'],
'Product': ['A', 'A', 'B', 'B'],
'Sales': [100, 80, 120, 90],
})
pivot = pd.pivot_table(df, values='Sales', index='Region', columns='Product', aggfunc='sum')
print(pivot)
Output
Product A B Region North 100 120 South 80 90
⚠️ Common Mistakes
Pandas use karte waqt ye mistakes bahut commonly hoti hain. Inhe avoid karna seekho! 🚫
1. ❌ Chained Assignment Warning
Example
# ❌ WRONG — SettingWithCopyWarning
df[df['Age'] > 25]['Salary'] = 100000 # May not modify original df!
# ✅ CORRECT — Use .loc
df.loc[df['Age'] > 25, 'Salary'] = 100000
💡 Pandas mein chained indexing (df[...][...]) se assignment karna unreliable hai. Hamesha.locuse karo!
2. ❌ Single &/| vs and/or
Example
# ❌ WRONG — Python keywords don't work with Series
df[df['Age'] > 25 and df['Salary'] > 70000] # ValueError!
# ✅ CORRECT — Use bitwise operators with parentheses
df[(df['Age'] > 25) & (df['Salary'] > 70000)]
💡 Multiple conditions mein&(AND) aur|(OR) use karo, aur har condition ko parentheses()mein wrap karo.
3. ❌ inplace=True Confusion
python example
# ❌ WRONG — inplace returns None, assignment creates bug
df = df.dropna(inplace=True) # df becomes None!
# ✅ CORRECT — Either use inplace OR reassignment
df.dropna(inplace=True) # Modifies df directly
# OR
df = df.dropna() # Returns new df💡 inplace=True ke saath kabhi variable assignment mat karo. Dono mein se ek hi use karo!4. ❌ Forgetting axis Parameter
Example
# ❌ WRONG — Drops ROW with label 'Salary' (error if not found)
df.drop('Salary')
# ✅ CORRECT — Drop column
df.drop('Salary', axis=1)
# OR
df.drop(columns=['Salary'])
💡drop()default mein rows drop karta hai (axis=0). Columns ke liyeaxis=1yacolumns=use karo.
5. ❌ Not Resetting Index After Filtering
Example
# ❌ After filtering, index may be non-continuous
filtered = df[df['Age'] > 25]
print(filtered.index) # [1, 2, 3] — gaps in index
# ✅ Reset index for clean sequential index
filtered = df[df['Age'] > 25].reset_index(drop=True)
print(filtered.index) # [0, 1, 2]
💡 Filtering ke baad agar sequential index chahiye toh reset_index(drop=True) use karo.6. ❌ Modifying DataFrame During Iteration
Example
# ❌ WRONG — Slow and dangerous
for i, row in df.iterrows():
df.at[i, 'Salary'] = row['Salary'] * 1.1
# ✅ CORRECT — Vectorized operation (100x faster)
df['Salary'] = df['Salary'] * 1.1
💡 Pandas mein loops avoid karo! Vectorized operations ya .apply() use karo — ye bahut fast hain.7. ❌ Using == to Check NaN
Example
# ❌ WRONG — NaN == NaN is False in Python!
df[df['Age'] == np.nan] # Returns empty DataFrame
# ✅ CORRECT — Use isna() or isnull()
df[df['Age'].isna()]
💡 NaN comparison ke liye hamesha.isna()ya.isnull()use karo.==NaN ke saath kaam nahi karta!
✅ Key Takeaways
Yahan par is lesson ke sabse important points hain jo aapko yaad rakhne chahiye: 🧠
- ✅ Series = 1D labeled array, DataFrame = 2D labeled table — ye Pandas ke do fundamental data structures hain
- ✅
.loclabel-based selection ke liye hai,.ilocinteger position-based selection ke liye — dono ka purpose alag hai - ✅ Multiple conditions filtering mein
&/|use karo (notand/or) aur parentheses mandatory hain - ✅
.query()method SQL-like readable syntax deta hai — complex filters ke liye best hai - ✅ Missing data handle karna important hai —
fillna(),dropna(),interpolate()main tools hain - ✅ GroupBy = Split-Apply-Combine pattern —
agg(),transform(),apply()teen power tools hain - ✅ Merge SQL JOINs jaisa kaam karta hai —
inner,left,right,outerjoins available hain - ✅ Vectorized operations loops se 100x fast hain — hamesha prefer karo
- ✅
read_csv()ke parameters powerful hain —usecols,nrows,parse_datesse optimization hoti hai - ✅ Pivot tables data summarization ke liye best hain —
aggfunc,margins,fill_valueimportant parameters hain
❓ FAQ
Q1: Pandas aur NumPy mein kya difference hai? 🤔
Answer: NumPy homogeneous numerical arrays ke liye hai (same data type), jabki Pandas heterogeneous tabular data ke liye hai (mixed types like strings, numbers, dates). Pandas internally NumPy use karta hai but labels, indexes, aur rich operations add karta hai.
Example
import numpy as np
import pandas as pd
# NumPy — pure numerical
arr = np.array([1, 2, 3, 4])
# Pandas — labeled, mixed types
df = pd.DataFrame({'Name': ['Alice'], 'Age': [25], 'Score': [95.5]})
Q2: .loc aur .iloc mein kya difference hai? 📍
Answer: .loc label-based hai — column names aur index labels use karta hai. .iloc integer position-based hai — 0-based indexing use karta hai. Dono mein slicing ka behavior bhi different hai — .loc both ends inclusive hai, .iloc end exclusive hai.
Example
df.loc[0:3] # Rows with label 0 TO 3 (inclusive) — 4 rows
df.iloc[0:3] # Rows at position 0, 1, 2 (exclusive end) — 3 rows
Q3: apply() aur vectorized operations mein kya prefer karein? ⚡
Answer: Hamesha vectorized operations pehle try karo — ye C-level speed pe chalti hain. apply() tab use karo jab complex custom logic ho jo vectorize na ho sake. Performance: vectorized > apply() > iterrows().
Example
# ✅ Vectorized — FAST
df['bonus'] = df['salary'] * 0.1
# 🟡 Apply — Moderate
df['grade'] = df['salary'].apply(lambda x: 'A' if x > 80000 else 'B')
# ❌ iterrows — SLOW
for i, row in df.iterrows():
df.at[i, 'bonus'] = row['salary'] * 0.1
Q4: Large CSV file ko efficiently kaise load karein? 📂
Answer: Bade files ke liye ye techniques use karo:
Example
# 1. Sirf zaruri columns load karo
df = pd.read_csv('big.csv', usecols=['name', 'salary'])
# 2. Limited rows
df = pd.read_csv('big.csv', nrows=10000)
# 3. Chunked reading
for chunk in pd.read_csv('big.csv', chunksize=50000):
process(chunk)
# 4. Data types specify karo (memory saving)
df = pd.read_csv('big.csv', dtype={'id': 'int32', 'name': 'category'})
Q5: merge() aur concat() mein kya difference hai? 🔗
Answer: merge() = horizontal joining based on common column(s), like SQL JOIN. concat() = stacking DataFrames vertically (rows) ya horizontally (columns) without matching logic.
Example
# merge — join on common key
pd.merge(df1, df2, on='id')
# concat — stack rows
pd.concat([df1, df2], ignore_index=True)
Q6: groupby() ke baad transform() aur agg() mein kya difference hai? 📊
Answer: agg() reduced output deta hai (one row per group). transform() same-shape output deta hai (original DataFrame jaisi shape), har row ko apne group ka aggregated value milta hai.
Example
# agg — reduced (one row per department)
df.groupby('Dept')['Salary'].agg('mean')
# transform — same shape (each row gets its dept's mean)
df['dept_avg'] = df.groupby('Dept')['Salary'].transform('mean')
Q7: DataFrame ko memory mein optimize kaise karein? 💾
Answer: Ye techniques use karo:
categorydtype for low-cardinality string columns- Smaller numeric types (
int32instead ofint64) pd.to_numeric()withdowncastparameter
Example
# Before optimization
print(df.memory_usage(deep=True).sum()) # e.g., 5MB
# Optimize
df['Department'] = df['Department'].astype('category')
df['Age'] = pd.to_numeric(df['Age'], downcast='integer')
# After optimization
print(df.memory_usage(deep=True).sum()) # e.g., 1.2MB
Q8: SettingWithCopyWarning kaise fix karein? ⚠️
Answer: Ye warning tab aati hai jab aap chained indexing se DataFrame modify karte ho. Fix: hamesha .loc use karo ya explicit copy banao.
Example
# ❌ Warning trigger
subset = df[df['Age'] > 25]
subset['Salary'] = 100000 # Warning!
# ✅ Fix 1: Use .loc on original
df.loc[df['Age'] > 25, 'Salary'] = 100000
# ✅ Fix 2: Explicit copy (if you need separate df)
subset = df[df['Age'] > 25].copy()
subset['Salary'] = 100000 # No warning
🎯 Interview Questions
Q1: Pandas mein DataFrame aur Series mein kya difference hai?
Answer: Series ek 1-dimensional labeled array hai — ek single column of data ke equivalent. DataFrame ek 2-dimensional labeled data structure hai — ek table/spreadsheet ke equivalent jismein multiple Series (columns) hoti hain.
Example
import pandas as pd
# Series — 1D
s = pd.Series([10, 20, 30], name='scores')
# DataFrame — 2D (collection of Series)
df = pd.DataFrame({'Name': ['Alice', 'Bob'], 'Score': [95, 87]})
# DataFrame ka har column ek Series hai
print(type(df['Name'])) # <class 'pandas.core.series.Series'>
Q2: loc vs iloc explain karein with example.
Answer: loc label-based indexing use karta hai (actual row/column names), jabki iloc integer position-based indexing use karta hai (0-based positions). Important difference: loc slicing mein both endpoints inclusive hain, iloc mein end exclusive hai.
Example
df = pd.DataFrame({'A': [10, 20, 30]}, index=['x', 'y', 'z'])
print(df.loc['x':'y']) # Rows 'x' and 'y' (inclusive)
print(df.iloc[0:1]) # Only row at position 0 (exclusive end)
Q3: Pandas mein missing values kaise handle karte hain?
Answer: Missing values ko handle karne ke multiple approaches hain:
- Detection:
isna(),isnull(),notna() - Removal:
dropna()— rows/columns remove karta hai - Imputation:
fillna()— values replace karta hai (mean, median, mode, constant) - Interpolation:
interpolate()— mathematical estimation se fill karta hai
Example
# Detect
df.isnull().sum()
# Remove rows with any NaN
df.dropna()
# Fill with column mean
df['Age'].fillna(df['Age'].mean(), inplace=True)
# Forward fill (time series ke liye useful)
df.fillna(method='ffill')
Q4: groupby() internally kaise kaam karta hai?
Answer: GroupBy Split-Apply-Combine pattern follow karta hai:
- Split: Data ko groups mein divide karta hai based on key column
- Apply: Har group pe independently function apply karta hai
- Combine: Results ko wapas ek DataFrame mein combine karta hai
Example
# Split-Apply-Combine in action
df = pd.DataFrame({
'Dept': ['Eng', 'HR', 'Eng', 'HR'],
'Salary': [85000, 68000, 95000, 71000]
})
# Split → groups: Eng=[85000, 95000], HR=[68000, 71000]
# Apply → mean: Eng=90000, HR=69500
# Combine → result DataFrame
result = df.groupby('Dept')['Salary'].mean()
print(result)
# Dept
# Eng 90000.0
# HR 69500.0
Q5: merge() ke different types of joins explain karein.
Answer:
| Join Type | Description |
|---|---|
inner | Sirf matching records dono tables se (default) |
left | Left table ke saare records + matching from right |
right | Right table ke saare records + matching from left |
outer | Dono tables ke saare records (NaN for non-matches) |
Example
left = pd.DataFrame({'id': [1, 2, 3], 'name': ['A', 'B', 'C']})
right = pd.DataFrame({'id': [2, 3, 4], 'score': [85, 90, 75]})
pd.merge(left, right, on='id', how='inner') # id: 2, 3
pd.merge(left, right, on='id', how='left') # id: 1, 2, 3
pd.merge(left, right, on='id', how='outer') # id: 1, 2, 3, 4
Q6: apply() vs map() vs applymap() mein kya difference hai?
Answer:
apply()— Series ya DataFrame pe kaam karta hai (row/column level)map()— Sirf Series pe kaam karta hai (element-wise)applymap()— Sirf DataFrame pe kaam karta hai (element-wise, deprecated in pandas 2.1+, usemap()instead)
Example
# map — Series pe, element-wise
df['Name'].map(str.upper)
df['Dept'].map({'Eng': 'Engineering', 'HR': 'Human Resources'})
# apply — Series ya DataFrame pe
df['Salary'].apply(lambda x: x * 1.1) # Series pe
df.apply(lambda row: row['Salary'] / row['Years'], axis=1) # Row-wise
# applymap (pandas < 2.1) — DataFrame pe, element-wise
df.select_dtypes('number').applymap(lambda x: round(x, 2))
Q7: Pandas mein large dataset handle karne ke best practices kya hain?
Answer:
- Chunked reading:
pd.read_csv(chunksize=N)for processing in batches - Column selection:
usecolsse sirf zaruri columns load karo - Dtype optimization:
category,int32,float32use karo - Parquet format: CSV se 10x fast reading, 5x less storage
- Query before load: Database se filtered data lao
Example
# Memory-efficient loading
df = pd.read_csv('large.csv',
usecols=['id', 'name', 'amount'],
dtype={'id': 'int32', 'name': 'category', 'amount': 'float32'},
nrows=100000
)
# Parquet for repeated access
df.to_parquet('data.parquet')
df = pd.read_parquet('data.parquet') # 10x faster than CSV
Q8: pivot_table() aur groupby() mein kya difference hai?
Answer: Dono aggregation karte hain, lekin format different hai:
groupby()→ Long format (stacked rows)pivot_table()→ Wide format (spreadsheet-like, values as both rows AND columns)
Example
df = pd.DataFrame({
'Region': ['N', 'S', 'N', 'S'],
'Product': ['A', 'A', 'B', 'B'],
'Sales': [100, 80, 120, 90]
})
# groupby — long format
df.groupby(['Region', 'Product'])['Sales'].sum()
# Region Product
# N A 100
# B 120
# S A 80
# B 90
# pivot_table — wide format
pd.pivot_table(df, values='Sales', index='Region', columns='Product', aggfunc='sum')
# A B
# N 100 120
# S 80 90
Q9: copy() kab zaruri hai Pandas mein?
Answer: Jab aap ek DataFrame ka subset lete ho aur usse modify karna chahte ho bina original ko affect kiye, tab .copy() zaruri hai. Without copy, pandas may return a view (reference) that modifies the original.
Example
# ❌ Without copy — may modify original
subset = df[df['Age'] > 25]
subset['Salary'] = 0 # SettingWithCopyWarning!
# ✅ With copy — safe modification
subset = df[df['Age'] > 25].copy()
subset['Salary'] = 0 # Original df unchanged
Q10: Pandas mein MultiIndex kya hai aur kab use hota hai?
Answer: MultiIndex (Hierarchical Index) ek DataFrame ko multiple levels of indexing deta hai — ye complex groupings aur pivot operations ke results ko efficiently represent karta hai.
Example
# MultiIndex creation
arrays = [['Q1', 'Q1', 'Q2', 'Q2'], ['Jan', 'Feb', 'Mar', 'Apr']]
index = pd.MultiIndex.from_arrays(arrays, names=['Quarter', 'Month'])
df = pd.DataFrame({'Sales': [100, 120, 130, 110]}, index=index)
print(df)
# Sales
# Quarter Month
# Q1 Jan 100
# Feb 120
# Q2 Mar 130
# Apr 110
# Accessing levels
print(df.loc['Q1']) # All Q1 data
print(df.loc[('Q1', 'Jan')]) # Specific entry
# groupby automatically creates MultiIndex
result = df.groupby(['Region', 'Product']).sum()
# result.index is a MultiIndex
Exam Focus
Revise definitions, diagrams, examples, and short-answer points for Pandas Introduction.
Interview Use
Prepare one clear explanation, one practical example, and one common mistake for this Python Master Course topic.
Search Terms
python-master-course, python master course, python, master, course, data, science, pandas
Related Python Master Course Topics
Continue learning this concept
Data ScienceNumPy IntroductionLearn NumPy, the foundational Python library for numerical computing. Master arrays, array operations, broadcasting, indexing, reshaping, linear algebra, and random number generation with practical examples.Data ScienceData Analysis with PythonLearn exploratory data analysis (EDA) with Python, Pandas, and NumPy. Master descriptive statistics, correlation analysis, group comparisons, time series basics, and building insight-driven data reports.Data ScienceData Cleaning with PythonMaster data cleaning techniques using Python, Pandas, and NumPy. Learn to handle missing values, duplicates, outliers, data type conversions, string cleaning, and building a complete data cleaning pipeline.Data ScienceData Visualization ProjectBuild a complete data visualization project analyzing real-world COVID-19 and economic data. Step-by-step tutorial covering data loading, cleaning, EDA, and creating a professional multi-panel visualization report.Data ScienceData Visualization with MatplotlibMaster Matplotlib, PythonData ScienceStatistical Visualization with SeabornLearn Seaborn for beautiful statistical data visualization. Master distribution plots, categorical plots, regression plots, heatmaps, pair plots, and FacetGrid with practical data science examples.