Skip to main content
Editor

Pandas GroupBy: Split-Apply-Combine for Powerful Aggregations

Advanced30 min7 exercises130 XP
0/7 exercises

Imagine you have a giant pile of receipts from a store. You want to know how much each department earned last month. What do you do? You sort the receipts into piles by department, add up each pile, and write down the totals.

That is exactly what groupby() does in Pandas. It splits your data into groups based on some column, applies a calculation to each group, and combines the results back together. This three-step pattern is called split-apply-combine, and it is one of the most powerful ideas in data analysis.

In this tutorial, you'll learn how to group data, aggregate with multiple functions, use named aggregation, transform values within groups, filter entire groups, and work with multi-level grouping.

How Does GroupBy Work?

When you call df.groupby('column'), Pandas does not immediately compute anything. Instead, it creates a GroupBy object -- a lazy container that knows which rows belong to which group. The real work happens when you call an aggregation method like .sum(), .mean(), or .count().

Basic groupby and aggregation
import pandas as pd
sales = pd.DataFrame({
'region': ['East', 'West', 'East', 'West', 'East', 'West'],
'product': ['Widget', 'Widget', 'Gadget', 'Gadget', 'Widget', 'Gadget'],
'revenue': [200, 150, 300, 250, 180, 320]
})
# Step 1: Split into groups
grouped = sales.groupby('region')
print(type(grouped))
print()
# Step 2 & 3: Apply sum, combine results
print(grouped['revenue'].sum())
Output
<class 'pandas.core.groupby.generic.DataFrameGroupBy'>

region
East    680
West    720
Name: revenue, dtype: int64

The result is a Series with the group labels as the index. East had 200 + 300 + 180 = 680 in revenue, and West had 150 + 250 + 320 = 720.

How Do You Aggregate with Multiple Functions?

Often you need more than just a sum. You might want the mean, count, and max all at once. The .agg() method lets you pass a list of function names or even a dictionary that maps columns to different functions.

Using agg() with multiple functions
import pandas as pd
sales = pd.DataFrame({
'region': ['East', 'West', 'East', 'West', 'East', 'West'],
'revenue': [200, 150, 300, 250, 180, 320],
'units': [10, 8, 15, 12, 9, 16]
})
# Multiple aggregations on one column
result = sales.groupby('region')['revenue'].agg(['sum', 'mean', 'max'])
print(result)
print()
# Different aggregations for different columns
result2 = sales.groupby('region').agg({
'revenue': ['sum', 'mean'],
'units': 'sum'
})
print(result2)
Output
        sum        mean  max
region                      
East    680  226.666667  300
West    720  240.000000  320

       revenue             units
           sum        mean   sum
region                          
East       680  226.666667    34
West       720  240.000000    36
Named aggregation syntax
import pandas as pd
sales = pd.DataFrame({
'region': ['East', 'West', 'East', 'West', 'East', 'West'],
'revenue': [200, 150, 300, 250, 180, 320],
'units': [10, 8, 15, 12, 9, 16]
})
# Named aggregation - clean column names
result = sales.groupby('region').agg(
total_revenue=('revenue', 'sum'),
avg_revenue=('revenue', 'mean'),
total_units=('units', 'sum')
)
print(result)
Output
        total_revenue  avg_revenue  total_units
region                                         
East              680   226.666667           34
West              720   240.000000           36

What Is GroupBy Transform?

Sometimes you don't want a summary table. You want to add a column to the original DataFrame that contains the group-level result for every row. For example, you want each row to show the average revenue of its region. That is what .transform() does -- it returns a Series with the same length as the original data.

transform() broadcasts group results back
import pandas as pd
sales = pd.DataFrame({
'region': ['East', 'West', 'East', 'West', 'East', 'West'],
'revenue': [200, 150, 300, 250, 180, 320]
})
# Add group mean as a new column
sales['region_avg'] = sales.groupby('region')['revenue'].transform('mean')
print(sales)
print()
# Use transform to normalize within groups
sales['pct_of_group'] = sales['revenue'] / sales.groupby('region')['revenue'].transform('sum') * 100
print(sales[['region', 'revenue', 'pct_of_group']].to_string(index=False))
Output
  region  revenue  region_avg
0   East      200  226.666667
1   West      150  240.000000
2   East      300  226.666667
3   West      250  240.000000
4   East      180  226.666667
5   West      320  240.000000

region  revenue  pct_of_group
  East      200     29.411765
  West      150     20.833333
  East      300     44.117647
  West      250     34.722222
  East      180     26.470588
  West      320     44.444444

How Do You Filter Entire Groups?

The .filter() method lets you keep or drop entire groups based on a condition. For example, you might want only regions that had more than two sales, or departments where the average score was above 80.

Filtering groups with a condition
import pandas as pd
scores = pd.DataFrame({
'class': ['A', 'A', 'A', 'B', 'B', 'C', 'C', 'C', 'C'],
'student': ['s1', 's2', 's3', 's4', 's5', 's6', 's7', 's8', 's9'],
'grade': [90, 85, 78, 55, 60, 95, 88, 92, 87]
})
# Keep only classes where the average grade is above 75
filtered = scores.groupby('class').filter(lambda g: g['grade'].mean() > 75)
print(filtered)
Output
  class student  grade
0     A      s1     90
1     A      s2     85
2     A      s3     78
5     C      s6     95
6     C      s7     88
7     C      s8     92
8     C      s9     87

Class B had an average grade of 57.5, which is below 75, so every row from class B was removed. Classes A and C passed the filter and kept all their rows.

Grouping by Multiple Columns

You can pass a list of columns to groupby() to create subgroups. This is like sorting receipts first by department, then by payment method within each department.

Multi-column groupby
import pandas as pd
sales = pd.DataFrame({
'region': ['East', 'East', 'West', 'West', 'East', 'West'],
'product': ['Widget', 'Gadget', 'Widget', 'Gadget', 'Widget', 'Widget'],
'revenue': [200, 300, 150, 250, 180, 170]
})
result = sales.groupby(['region', 'product'])['revenue'].sum()
print(result)
print()
print(result.reset_index())
Output
region  product
East    Gadget     300
        Widget     380
West    Gadget     250
        Widget     320
Name: revenue, dtype: int64

  region product  revenue
0   East  Gadget      300
1   East  Widget      380
2   West  Gadget      250
3   West  Widget      320

Custom Group Logic with apply()

When built-in aggregations aren't enough, .apply() lets you run any function on each group. The function receives a DataFrame (the group) and can return a scalar, a Series, or a DataFrame.

Using apply() for custom group operations
import pandas as pd
sales = pd.DataFrame({
'region': ['East', 'East', 'East', 'West', 'West', 'West'],
'revenue': [200, 300, 180, 150, 250, 320]
})
# Custom function: return top 2 revenue rows per group
def top_n(group, n=2):
return group.nlargest(n, 'revenue')
result = sales.groupby('region').apply(top_n, n=2, include_groups=False)
print(result)
Output
          revenue
region           
East   1      300
       0      200
West   5      320
       4      250

Practice Exercises

Basic GroupBy Sum
Write Code

Given a DataFrame df with columns department and salary, group by department and print the total salary for each department. The DataFrame is already created for you.

Print the result of the groupby sum (it should be a Series).

import pandas as pd
df = pd.DataFrame({
'department': ['Sales', 'Engineering', 'Sales', 'Engineering', 'Sales'],
'salary': [50000, 80000, 55000, 90000, 60000]
})
# Group by department, sum the salaries, and print
Named Aggregation
Write Code

Use named aggregation to group df by region and compute:

  • total_revenue: sum of revenue
  • avg_units: mean of units
  • Print the resulting DataFrame.

    import pandas as pd
    df = pd.DataFrame({
    'region': ['North', 'South', 'North', 'South', 'North'],
    'revenue': [100, 200, 150, 250, 120],
    'units': [5, 10, 7, 12, 6]
    })
    # Use named aggregation
    Predict the GroupBy Output
    Predict Output

    What will this code print? Think about how many rows each group has.

    import pandas as pd
    df = pd.DataFrame({'team': ['A', 'B', 'A', 'B', 'A'], 'score': [10, 20, 30, 40, 50]})
    print(df.groupby('team')['score'].count().tolist())
    # Type the exact output
    Normalize with Transform
    Write Code

    Add a new column called pct_of_dept to df that shows each employee's salary as a percentage of their department's total salary. Round to 1 decimal place.

    Print df[['name', 'department', 'pct_of_dept']].to_string(index=False).

    import pandas as pd
    df = pd.DataFrame({
    'name': ['Alice', 'Bob', 'Carol', 'Dave'],
    'department': ['Eng', 'Eng', 'Sales', 'Sales'],
    'salary': [90000, 110000, 60000, 40000]
    })
    # Add pct_of_dept column using transform
    Filter Groups by Size
    Write Code

    Filter df to keep only teams that have more than 2 members. Print the name column of the filtered DataFrame as a list.

    Use groupby().filter() with a lambda.

    import pandas as pd
    df = pd.DataFrame({
    'name': ['Alice', 'Bob', 'Carol', 'Dave', 'Eve', 'Frank'],
    'team': ['Red', 'Blue', 'Red', 'Red', 'Blue', 'Green']
    })
    # Filter to keep teams with more than 2 members
    Multi-Column GroupBy
    Write Code

    Group df by both region and product, then compute the sum of revenue. Reset the index and print the resulting DataFrame using .to_string(index=False).

    import pandas as pd
    df = pd.DataFrame({
    'region': ['East', 'East', 'West', 'West', 'East', 'West'],
    'product': ['A', 'B', 'A', 'B', 'A', 'A'],
    'revenue': [100, 200, 150, 250, 120, 180]
    })
    # Group by region and product, sum revenue, reset index
    Fix the GroupBy Bug
    Fix the Bug

    This code tries to add a column showing each row's deviation from its group mean, but it has a bug. Fix it so it works correctly.

    The output should print the deviation column as a list, rounded to 1 decimal place each.

    import pandas as pd
    df = pd.DataFrame({
    'group': ['X', 'X', 'Y', 'Y'],
    'value': [10, 20, 100, 200]
    })
    # Bug: this line is wrong
    df['deviation'] = df['value'] - df.groupby('group')['value'].mean()
    print([round(x, 1) for x in df['deviation'].tolist()])

    Harder problems

    More problems on this topic: Working with data

    Related Tutorials

    Was this tutorial helpful?