diff --git a/lab-dw-aggregating.ipynb b/lab-dw-aggregating.ipynb
index fadd718..51f4ec2 100644
--- a/lab-dw-aggregating.ipynb
+++ b/lab-dw-aggregating.ipynb
@@ -1,165 +1,1341 @@
{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "31969215-2a90-4d8b-ac36-646a7ae13744",
- "metadata": {
- "id": "31969215-2a90-4d8b-ac36-646a7ae13744"
- },
- "source": [
- "# Lab | Data Aggregation and Filtering"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d",
- "metadata": {
- "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d"
- },
- "source": [
- "In this challenge, we will continue to work with customer data from an insurance company. We will use the dataset called marketing_customer_analysis.csv, which can be found at the following link:\n",
- "\n",
- "https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\n",
- "\n",
- "This dataset contains information such as customer demographics, policy details, vehicle information, and the customer's response to the last marketing campaign. Our goal is to explore and analyze this data by first performing data cleaning, formatting, and structuring."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50",
- "metadata": {
- "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50"
- },
- "source": [
- "1. Create a new DataFrame that only includes customers who:\n",
- " - have a **low total_claim_amount** (e.g., below $1,000),\n",
- " - have a response \"Yes\" to the last marketing campaign."
- ]
- },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "31969215-2a90-4d8b-ac36-646a7ae13744",
+ "metadata": {
+ "id": "31969215-2a90-4d8b-ac36-646a7ae13744"
+ },
+ "source": [
+ "# Lab | Data Aggregation and Filtering"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d",
+ "metadata": {
+ "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d"
+ },
+ "source": [
+ "In this challenge, we will continue to work with customer data from an insurance company. We will use the dataset called marketing_customer_analysis.csv, which can be found at the following link:\n",
+ "\n",
+ "https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\n",
+ "\n",
+ "This dataset contains information such as customer demographics, policy details, vehicle information, and the customer's response to the last marketing campaign. Our goal is to explore and analyze this data by first performing data cleaning, formatting, and structuring."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50",
+ "metadata": {
+ "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50"
+ },
+ "source": [
+ "1. Create a new DataFrame that only includes customers who:\n",
+ " - have a **low total_claim_amount** (e.g., below $1,000),\n",
+ " - have a response \"Yes\" to the last marketing campaign."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b9be383e-5165-436e-80c8-57d4c757c8c3",
+ "metadata": {
+ "id": "b9be383e-5165-436e-80c8-57d4c757c8c3"
+ },
+ "source": [
+ "2. Using the original Dataframe, analyze:\n",
+ " - the average `monthly_premium` and/or customer lifetime value by `policy_type` and `gender` for customers who responded \"Yes\", and\n",
+ " - compare these insights to `total_claim_amount` patterns, and discuss which segments appear most profitable or low-risk for the company."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0",
+ "metadata": {
+ "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0"
+ },
+ "source": [
+ "3. Analyze the total number of customers who have policies in each state, and then filter the results to only include states where there are more than 500 customers."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d",
+ "metadata": {
+ "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d"
+ },
+ "source": [
+ "4. Find the maximum, minimum, and median customer lifetime value by education level and gender. Write your conclusions."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b42999f9-311f-481e-ae63-40a5577072c5",
+ "metadata": {
+ "id": "b42999f9-311f-481e-ae63-40a5577072c5"
+ },
+ "source": [
+ "## Bonus"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "81ff02c5-6584-4f21-a358-b918697c6432",
+ "metadata": {
+ "id": "81ff02c5-6584-4f21-a358-b918697c6432"
+ },
+ "source": [
+ "5. The marketing team wants to analyze the number of policies sold by state and month. Present the data in a table where the months are arranged as columns and the states are arranged as rows."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b6aec097-c633-4017-a125-e77a97259cda",
+ "metadata": {
+ "id": "b6aec097-c633-4017-a125-e77a97259cda"
+ },
+ "source": [
+ "6. Display a new DataFrame that contains the number of policies sold by month, by state, for the top 3 states with the highest number of policies sold.\n",
+ "\n",
+ "*Hint:*\n",
+ "- *To accomplish this, you will first need to group the data by state and month, then count the number of policies sold for each group. Afterwards, you will need to sort the data by the count of policies sold in descending order.*\n",
+ "- *Next, you will select the top 3 states with the highest number of policies sold.*\n",
+ "- *Finally, you will create a new DataFrame that contains the number of policies sold by month for each of the top 3 states.*"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009",
+ "metadata": {
+ "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009"
+ },
+ "source": [
+ "7. The marketing team wants to analyze the effect of different marketing channels on the customer response rate.\n",
+ "\n",
+ "Hint: You can use melt to unpivot the data and create a table that shows the customer response rate (those who responded \"Yes\") by marketing channel."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d",
+ "metadata": {
+ "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d"
+ },
+ "source": [
+ "External Resources for Data Filtering: https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "id": "449513f4-0459-46a0-a18d-9398d974c9ad",
+ "metadata": {
+ "id": "449513f4-0459-46a0-a18d-9398d974c9ad"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "b9be383e-5165-436e-80c8-57d4c757c8c3",
- "metadata": {
- "id": "b9be383e-5165-436e-80c8-57d4c757c8c3"
- },
- "source": [
- "2. Using the original Dataframe, analyze:\n",
- " - the average `monthly_premium` and/or customer lifetime value by `policy_type` and `gender` for customers who responded \"Yes\", and\n",
- " - compare these insights to `total_claim_amount` patterns, and discuss which segments appear most profitable or low-risk for the company."
+ "data": {
+ "text/html": [
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " unnamed:_0 | \n",
+ " customer | \n",
+ " state | \n",
+ " customer_lifetime_value | \n",
+ " response | \n",
+ " coverage | \n",
+ " education | \n",
+ " effective_to_date | \n",
+ " employmentstatus | \n",
+ " gender | \n",
+ " ... | \n",
+ " number_of_open_complaints | \n",
+ " number_of_policies | \n",
+ " policy_type | \n",
+ " policy | \n",
+ " renew_offer_type | \n",
+ " sales_channel | \n",
+ " total_claim_amount | \n",
+ " vehicle_class | \n",
+ " vehicle_size | \n",
+ " vehicle_type | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 | \n",
+ " 0 | \n",
+ " DK49336 | \n",
+ " Arizona | \n",
+ " 4809.216960 | \n",
+ " No | \n",
+ " Basic | \n",
+ " College | \n",
+ " 2/18/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 9 | \n",
+ " Corporate Auto | \n",
+ " Corporate L3 | \n",
+ " Offer3 | \n",
+ " Agent | \n",
+ " 292.800000 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 1 | \n",
+ " 1 | \n",
+ " KX64629 | \n",
+ " California | \n",
+ " 2228.525238 | \n",
+ " No | \n",
+ " Basic | \n",
+ " College | \n",
+ " 1/18/11 | \n",
+ " Unemployed | \n",
+ " F | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer4 | \n",
+ " Call Center | \n",
+ " 744.924331 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 2 | \n",
+ " 2 | \n",
+ " LZ68649 | \n",
+ " Washington | \n",
+ " 14947.917300 | \n",
+ " No | \n",
+ " Basic | \n",
+ " Bachelor | \n",
+ " 2/10/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 2 | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer3 | \n",
+ " Call Center | \n",
+ " 480.000000 | \n",
+ " SUV | \n",
+ " Medsize | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " 3 | \n",
+ " 3 | \n",
+ " XL78013 | \n",
+ " Oregon | \n",
+ " 22332.439460 | \n",
+ " Yes | \n",
+ " Extended | \n",
+ " College | \n",
+ " 1/11/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 2 | \n",
+ " Corporate Auto | \n",
+ " Corporate L3 | \n",
+ " Offer2 | \n",
+ " Branch | \n",
+ " 484.013411 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " 4 | \n",
+ " 4 | \n",
+ " QA50777 | \n",
+ " Oregon | \n",
+ " 9025.067525 | \n",
+ " No | \n",
+ " Premium | \n",
+ " Bachelor | \n",
+ " 1/17/11 | \n",
+ " Medical Leave | \n",
+ " F | \n",
+ " ... | \n",
+ " NaN | \n",
+ " 7 | \n",
+ " Personal Auto | \n",
+ " Personal L2 | \n",
+ " Offer1 | \n",
+ " Branch | \n",
+ " 707.925645 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
5 rows × 26 columns
\n",
+ "
"
+ ],
+ "text/plain": [
+ " unnamed:_0 customer state customer_lifetime_value response \\\n",
+ "0 0 DK49336 Arizona 4809.216960 No \n",
+ "1 1 KX64629 California 2228.525238 No \n",
+ "2 2 LZ68649 Washington 14947.917300 No \n",
+ "3 3 XL78013 Oregon 22332.439460 Yes \n",
+ "4 4 QA50777 Oregon 9025.067525 No \n",
+ "\n",
+ " coverage education effective_to_date employmentstatus gender ... \\\n",
+ "0 Basic College 2/18/11 Employed M ... \n",
+ "1 Basic College 1/18/11 Unemployed F ... \n",
+ "2 Basic Bachelor 2/10/11 Employed M ... \n",
+ "3 Extended College 1/11/11 Employed M ... \n",
+ "4 Premium Bachelor 1/17/11 Medical Leave F ... \n",
+ "\n",
+ " number_of_open_complaints number_of_policies policy_type policy \\\n",
+ "0 0.0 9 Corporate Auto Corporate L3 \n",
+ "1 0.0 1 Personal Auto Personal L3 \n",
+ "2 0.0 2 Personal Auto Personal L3 \n",
+ "3 0.0 2 Corporate Auto Corporate L3 \n",
+ "4 NaN 7 Personal Auto Personal L2 \n",
+ "\n",
+ " renew_offer_type sales_channel total_claim_amount vehicle_class \\\n",
+ "0 Offer3 Agent 292.800000 Four-Door Car \n",
+ "1 Offer4 Call Center 744.924331 Four-Door Car \n",
+ "2 Offer3 Call Center 480.000000 SUV \n",
+ "3 Offer2 Branch 484.013411 Four-Door Car \n",
+ "4 Offer1 Branch 707.925645 Four-Door Car \n",
+ "\n",
+ " vehicle_size vehicle_type \n",
+ "0 Medsize NaN \n",
+ "1 Medsize NaN \n",
+ "2 Medsize A \n",
+ "3 Medsize A \n",
+ "4 Medsize NaN \n",
+ "\n",
+ "[5 rows x 26 columns]"
]
- },
+ },
+ "execution_count": 11,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "import pandas as pd\n",
+ "import numpy as np\n",
+ "\n",
+ "df = pd.read_csv(\"https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\")\n",
+ "\n",
+ "df.columns = df.columns.str.lower().str.replace(\" \", \"_\")\n",
+ "\n",
+ "df.head()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "id": "f8f41ff1-60bf-4b89-89b3-468bd4b53f9d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "df['effective_to_date'] = pd.to_datetime(\n",
+ " df['effective_to_date'],\n",
+ " format='%d/%m/%y',\n",
+ " errors='coerce'\n",
+ ")\n",
+ "\n",
+ "df['month'] = df['effective_to_date'].dt.month_name()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "id": "0edb099f-d393-431e-a32d-124056b58d47",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0",
- "metadata": {
- "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0"
- },
- "source": [
- "3. Analyze the total number of customers who have policies in each state, and then filter the results to only include states where there are more than 500 customers."
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " unnamed:_0 | \n",
+ " customer | \n",
+ " state | \n",
+ " customer_lifetime_value | \n",
+ " response | \n",
+ " coverage | \n",
+ " education | \n",
+ " effective_to_date | \n",
+ " employmentstatus | \n",
+ " gender | \n",
+ " ... | \n",
+ " number_of_policies | \n",
+ " policy_type | \n",
+ " policy | \n",
+ " renew_offer_type | \n",
+ " sales_channel | \n",
+ " total_claim_amount | \n",
+ " vehicle_class | \n",
+ " vehicle_size | \n",
+ " vehicle_type | \n",
+ " month | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 3 | \n",
+ " 3 | \n",
+ " XL78013 | \n",
+ " Oregon | \n",
+ " 22332.439460 | \n",
+ " Yes | \n",
+ " Extended | \n",
+ " College | \n",
+ " 2011-01-11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 2 | \n",
+ " Corporate Auto | \n",
+ " Corporate L3 | \n",
+ " Offer2 | \n",
+ " Branch | \n",
+ " 484.013411 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " A | \n",
+ " January | \n",
+ "
\n",
+ " \n",
+ " 8 | \n",
+ " 8 | \n",
+ " FM55990 | \n",
+ " California | \n",
+ " 5989.773931 | \n",
+ " Yes | \n",
+ " Premium | \n",
+ " College | \n",
+ " 2011-01-19 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 1 | \n",
+ " Personal Auto | \n",
+ " Personal L1 | \n",
+ " Offer2 | \n",
+ " Branch | \n",
+ " 739.200000 | \n",
+ " Sports Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ " January | \n",
+ "
\n",
+ " \n",
+ " 15 | \n",
+ " 15 | \n",
+ " CW49887 | \n",
+ " California | \n",
+ " 4626.801093 | \n",
+ " Yes | \n",
+ " Basic | \n",
+ " Master | \n",
+ " 2011-01-16 | \n",
+ " Employed | \n",
+ " F | \n",
+ " ... | \n",
+ " 1 | \n",
+ " Special Auto | \n",
+ " Special L1 | \n",
+ " Offer2 | \n",
+ " Branch | \n",
+ " 547.200000 | \n",
+ " SUV | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ " January | \n",
+ "
\n",
+ " \n",
+ " 19 | \n",
+ " 19 | \n",
+ " NJ54277 | \n",
+ " California | \n",
+ " 3746.751625 | \n",
+ " Yes | \n",
+ " Extended | \n",
+ " College | \n",
+ " 2011-02-26 | \n",
+ " Employed | \n",
+ " F | \n",
+ " ... | \n",
+ " 1 | \n",
+ " Personal Auto | \n",
+ " Personal L2 | \n",
+ " Offer2 | \n",
+ " Call Center | \n",
+ " 19.575683 | \n",
+ " Two-Door Car | \n",
+ " Large | \n",
+ " A | \n",
+ " February | \n",
+ "
\n",
+ " \n",
+ " 27 | \n",
+ " 27 | \n",
+ " MQ68407 | \n",
+ " Oregon | \n",
+ " 4376.363592 | \n",
+ " Yes | \n",
+ " Premium | \n",
+ " Bachelor | \n",
+ " 2011-02-28 | \n",
+ " Employed | \n",
+ " F | \n",
+ " ... | \n",
+ " 1 | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer2 | \n",
+ " Agent | \n",
+ " 60.036683 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ " February | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
5 rows × 27 columns
\n",
+ "
"
+ ],
+ "text/plain": [
+ " unnamed:_0 customer state customer_lifetime_value response \\\n",
+ "3 3 XL78013 Oregon 22332.439460 Yes \n",
+ "8 8 FM55990 California 5989.773931 Yes \n",
+ "15 15 CW49887 California 4626.801093 Yes \n",
+ "19 19 NJ54277 California 3746.751625 Yes \n",
+ "27 27 MQ68407 Oregon 4376.363592 Yes \n",
+ "\n",
+ " coverage education effective_to_date employmentstatus gender ... \\\n",
+ "3 Extended College 2011-01-11 Employed M ... \n",
+ "8 Premium College 2011-01-19 Employed M ... \n",
+ "15 Basic Master 2011-01-16 Employed F ... \n",
+ "19 Extended College 2011-02-26 Employed F ... \n",
+ "27 Premium Bachelor 2011-02-28 Employed F ... \n",
+ "\n",
+ " number_of_policies policy_type policy renew_offer_type \\\n",
+ "3 2 Corporate Auto Corporate L3 Offer2 \n",
+ "8 1 Personal Auto Personal L1 Offer2 \n",
+ "15 1 Special Auto Special L1 Offer2 \n",
+ "19 1 Personal Auto Personal L2 Offer2 \n",
+ "27 1 Personal Auto Personal L3 Offer2 \n",
+ "\n",
+ " sales_channel total_claim_amount vehicle_class vehicle_size \\\n",
+ "3 Branch 484.013411 Four-Door Car Medsize \n",
+ "8 Branch 739.200000 Sports Car Medsize \n",
+ "15 Branch 547.200000 SUV Medsize \n",
+ "19 Call Center 19.575683 Two-Door Car Large \n",
+ "27 Agent 60.036683 Four-Door Car Medsize \n",
+ "\n",
+ " vehicle_type month \n",
+ "3 A January \n",
+ "8 NaN January \n",
+ "15 NaN January \n",
+ "19 A February \n",
+ "27 NaN February \n",
+ "\n",
+ "[5 rows x 27 columns]"
]
- },
+ },
+ "execution_count": 14,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "low_claim_yes = df[(df['total_claim_amount'] < 1000) & (df['response'].str.lower() == 'yes')].copy()\n",
+ "low_claim_yes.shape\n",
+ "low_claim_yes.head()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "id": "e2bad9dd-9780-4fc8-ba5a-f3ef923e39c3",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d",
- "metadata": {
- "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d"
- },
- "source": [
- "4. Find the maximum, minimum, and median customer lifetime value by education level and gender. Write your conclusions."
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " | \n",
+ " customer_lifetime_value | \n",
+ " monthly_premium_auto | \n",
+ "
\n",
+ " \n",
+ " policy_type | \n",
+ " gender | \n",
+ " | \n",
+ " | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " Corporate Auto | \n",
+ " F | \n",
+ " 7712.63 | \n",
+ " 94.30 | \n",
+ "
\n",
+ " \n",
+ " M | \n",
+ " 7944.47 | \n",
+ " 92.19 | \n",
+ "
\n",
+ " \n",
+ " Personal Auto | \n",
+ " F | \n",
+ " 8339.79 | \n",
+ " 99.00 | \n",
+ "
\n",
+ " \n",
+ " M | \n",
+ " 7448.38 | \n",
+ " 91.09 | \n",
+ "
\n",
+ " \n",
+ " Special Auto | \n",
+ " F | \n",
+ " 7691.58 | \n",
+ " 92.31 | \n",
+ "
\n",
+ " \n",
+ " M | \n",
+ " 8247.09 | \n",
+ " 86.34 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " customer_lifetime_value monthly_premium_auto\n",
+ "policy_type gender \n",
+ "Corporate Auto F 7712.63 94.30\n",
+ " M 7944.47 92.19\n",
+ "Personal Auto F 8339.79 99.00\n",
+ " M 7448.38 91.09\n",
+ "Special Auto F 7691.58 92.31\n",
+ " M 8247.09 86.34"
]
- },
+ },
+ "execution_count": 16,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "responders = df[df['response'].str.lower() == 'yes'].copy()\n",
+ "\n",
+ "pivot_avg = pd.pivot_table(\n",
+ " responders,\n",
+ " values=['monthly_premium_auto', 'customer_lifetime_value'],\n",
+ " index=['policy_type', 'gender'],\n",
+ " aggfunc='mean'\n",
+ ").round(2)\n",
+ "\n",
+ "pivot_avg"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "id": "39e37279-226f-46c0-a051-7fe53f3bd4ad",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "b42999f9-311f-481e-ae63-40a5577072c5",
- "metadata": {
- "id": "b42999f9-311f-481e-ae63-40a5577072c5"
- },
- "source": [
- "## Bonus"
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " policy_type | \n",
+ " gender | \n",
+ " avg_monthly_premium | \n",
+ " avg_clv | \n",
+ " avg_total_claim | \n",
+ " median_total_claim | \n",
+ " count | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 | \n",
+ " Corporate Auto | \n",
+ " F | \n",
+ " 94.30 | \n",
+ " 7712.63 | \n",
+ " 433.74 | \n",
+ " 420.04 | \n",
+ " 169 | \n",
+ "
\n",
+ " \n",
+ " 1 | \n",
+ " Corporate Auto | \n",
+ " M | \n",
+ " 92.19 | \n",
+ " 7944.47 | \n",
+ " 408.58 | \n",
+ " 364.80 | \n",
+ " 154 | \n",
+ "
\n",
+ " \n",
+ " 2 | \n",
+ " Personal Auto | \n",
+ " F | \n",
+ " 99.00 | \n",
+ " 8339.79 | \n",
+ " 452.97 | \n",
+ " 424.33 | \n",
+ " 540 | \n",
+ "
\n",
+ " \n",
+ " 3 | \n",
+ " Personal Auto | \n",
+ " M | \n",
+ " 91.09 | \n",
+ " 7448.38 | \n",
+ " 457.01 | \n",
+ " 412.80 | \n",
+ " 536 | \n",
+ "
\n",
+ " \n",
+ " 4 | \n",
+ " Special Auto | \n",
+ " F | \n",
+ " 92.31 | \n",
+ " 7691.58 | \n",
+ " 453.28 | \n",
+ " 420.04 | \n",
+ " 35 | \n",
+ "
\n",
+ " \n",
+ " 5 | \n",
+ " Special Auto | \n",
+ " M | \n",
+ " 86.34 | \n",
+ " 8247.09 | \n",
+ " 429.53 | \n",
+ " 345.60 | \n",
+ " 32 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " policy_type gender avg_monthly_premium avg_clv avg_total_claim \\\n",
+ "0 Corporate Auto F 94.30 7712.63 433.74 \n",
+ "1 Corporate Auto M 92.19 7944.47 408.58 \n",
+ "2 Personal Auto F 99.00 8339.79 452.97 \n",
+ "3 Personal Auto M 91.09 7448.38 457.01 \n",
+ "4 Special Auto F 92.31 7691.58 453.28 \n",
+ "5 Special Auto M 86.34 8247.09 429.53 \n",
+ "\n",
+ " median_total_claim count \n",
+ "0 420.04 169 \n",
+ "1 364.80 154 \n",
+ "2 424.33 540 \n",
+ "3 412.80 536 \n",
+ "4 420.04 35 \n",
+ "5 345.60 32 "
]
- },
+ },
+ "execution_count": 18,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "comparison = (\n",
+ " responders\n",
+ " .groupby(['policy_type', 'gender'])\n",
+ " .agg(\n",
+ " avg_monthly_premium = ('monthly_premium_auto', 'mean'),\n",
+ " avg_clv = ('customer_lifetime_value', 'mean'),\n",
+ " avg_total_claim = ('total_claim_amount', 'mean'),\n",
+ " median_total_claim = ('total_claim_amount', 'median'),\n",
+ " count = ('customer', 'count')\n",
+ " )\n",
+ " .round(2)\n",
+ " .reset_index()\n",
+ ")\n",
+ "\n",
+ "comparison"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "id": "eeef8e0b-30ac-45d8-8f86-9d1809eee4d3",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "81ff02c5-6584-4f21-a358-b918697c6432",
- "metadata": {
- "id": "81ff02c5-6584-4f21-a358-b918697c6432"
- },
- "source": [
- "5. The marketing team wants to analyze the number of policies sold by state and month. Present the data in a table where the months are arranged as columns and the states are arranged as rows."
+ "data": {
+ "text/plain": [
+ "( state n_customers\n",
+ " 1 California 3150\n",
+ " 3 Oregon 2601\n",
+ " 0 Arizona 1703\n",
+ " 2 Nevada 882\n",
+ " 4 Washington 798,\n",
+ " state n_customers\n",
+ " 1 California 3150\n",
+ " 3 Oregon 2601\n",
+ " 0 Arizona 1703\n",
+ " 2 Nevada 882\n",
+ " 4 Washington 798)"
]
- },
+ },
+ "execution_count": 19,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "customers_by_state = (\n",
+ " df.groupby('state')['customer'].nunique()\n",
+ " .reset_index(name='n_customers')\n",
+ " .sort_values('n_customers', ascending=False)\n",
+ ")\n",
+ "\n",
+ "customers_by_state_top = customers_by_state[customers_by_state['n_customers'] > 500]\n",
+ "\n",
+ "customers_by_state, customers_by_state_top"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "id": "052f3fee-7614-4ec7-bf4e-6f748d1b2b5c",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "b6aec097-c633-4017-a125-e77a97259cda",
- "metadata": {
- "id": "b6aec097-c633-4017-a125-e77a97259cda"
- },
- "source": [
- "6. Display a new DataFrame that contains the number of policies sold by month, by state, for the top 3 states with the highest number of policies sold.\n",
- "\n",
- "*Hint:*\n",
- "- *To accomplish this, you will first need to group the data by state and month, then count the number of policies sold for each group. Afterwards, you will need to sort the data by the count of policies sold in descending order.*\n",
- "- *Next, you will select the top 3 states with the highest number of policies sold.*\n",
- "- *Finally, you will create a new DataFrame that contains the number of policies sold by month for each of the top 3 states.*"
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " education | \n",
+ " gender | \n",
+ " max | \n",
+ " min | \n",
+ " median | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 | \n",
+ " Bachelor | \n",
+ " F | \n",
+ " 73225.96 | \n",
+ " 1904.00 | \n",
+ " 5640.51 | \n",
+ "
\n",
+ " \n",
+ " 1 | \n",
+ " Bachelor | \n",
+ " M | \n",
+ " 67907.27 | \n",
+ " 1898.01 | \n",
+ " 5548.03 | \n",
+ "
\n",
+ " \n",
+ " 2 | \n",
+ " College | \n",
+ " F | \n",
+ " 61850.19 | \n",
+ " 1898.68 | \n",
+ " 5623.61 | \n",
+ "
\n",
+ " \n",
+ " 3 | \n",
+ " College | \n",
+ " M | \n",
+ " 61134.68 | \n",
+ " 1918.12 | \n",
+ " 6005.85 | \n",
+ "
\n",
+ " \n",
+ " 4 | \n",
+ " Doctor | \n",
+ " F | \n",
+ " 44856.11 | \n",
+ " 2395.57 | \n",
+ " 5332.46 | \n",
+ "
\n",
+ " \n",
+ " 5 | \n",
+ " Doctor | \n",
+ " M | \n",
+ " 32677.34 | \n",
+ " 2267.60 | \n",
+ " 5577.67 | \n",
+ "
\n",
+ " \n",
+ " 6 | \n",
+ " High School or Below | \n",
+ " F | \n",
+ " 55277.45 | \n",
+ " 2144.92 | \n",
+ " 6039.55 | \n",
+ "
\n",
+ " \n",
+ " 7 | \n",
+ " High School or Below | \n",
+ " M | \n",
+ " 83325.38 | \n",
+ " 1940.98 | \n",
+ " 6286.73 | \n",
+ "
\n",
+ " \n",
+ " 8 | \n",
+ " Master | \n",
+ " F | \n",
+ " 51016.07 | \n",
+ " 2417.78 | \n",
+ " 5729.86 | \n",
+ "
\n",
+ " \n",
+ " 9 | \n",
+ " Master | \n",
+ " M | \n",
+ " 50568.26 | \n",
+ " 2272.31 | \n",
+ " 5579.10 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " education gender max min median\n",
+ "0 Bachelor F 73225.96 1904.00 5640.51\n",
+ "1 Bachelor M 67907.27 1898.01 5548.03\n",
+ "2 College F 61850.19 1898.68 5623.61\n",
+ "3 College M 61134.68 1918.12 6005.85\n",
+ "4 Doctor F 44856.11 2395.57 5332.46\n",
+ "5 Doctor M 32677.34 2267.60 5577.67\n",
+ "6 High School or Below F 55277.45 2144.92 6039.55\n",
+ "7 High School or Below M 83325.38 1940.98 6286.73\n",
+ "8 Master F 51016.07 2417.78 5729.86\n",
+ "9 Master M 50568.26 2272.31 5579.10"
]
- },
+ },
+ "execution_count": 20,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "clv_stats = (\n",
+ " df.groupby(['education', 'gender'])['customer_lifetime_value']\n",
+ " .agg(['max', 'min', 'median'])\n",
+ " .round(2)\n",
+ " .reset_index()\n",
+ ")\n",
+ "\n",
+ "clv_stats"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "id": "a143d941-c908-4151-be82-2b5f671c2ec0",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009",
- "metadata": {
- "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009"
- },
- "source": [
- "7. The marketing team wants to analyze the effect of different marketing channels on the customer response rate.\n",
- "\n",
- "Hint: You can use melt to unpivot the data and create a table that shows the customer response rate (those who responded \"Yes\") by marketing channel."
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " month | \n",
+ " January | \n",
+ " February | \n",
+ "
\n",
+ " \n",
+ " state | \n",
+ " | \n",
+ " | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " Arizona | \n",
+ " 1008.0 | \n",
+ " 929.0 | \n",
+ "
\n",
+ " \n",
+ " California | \n",
+ " 1918.0 | \n",
+ " 1634.0 | \n",
+ "
\n",
+ " \n",
+ " Nevada | \n",
+ " 551.0 | \n",
+ " 442.0 | \n",
+ "
\n",
+ " \n",
+ " Oregon | \n",
+ " 1565.0 | \n",
+ " 1344.0 | \n",
+ "
\n",
+ " \n",
+ " Washington | \n",
+ " 463.0 | \n",
+ " 425.0 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ "month January February\n",
+ "state \n",
+ "Arizona 1008.0 929.0\n",
+ "California 1918.0 1634.0\n",
+ "Nevada 551.0 442.0\n",
+ "Oregon 1565.0 1344.0\n",
+ "Washington 463.0 425.0"
]
- },
+ },
+ "execution_count": 21,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "policies_state_month = (\n",
+ " df.groupby(['state', 'month'])['policy']\n",
+ " .count()\n",
+ " .reset_index(name='n_policies')\n",
+ ")\n",
+ "\n",
+ "policies_pivot = policies_state_month.pivot_table(\n",
+ " index='state', columns='month', values='n_policies', fill_value=0\n",
+ ")\n",
+ "\n",
+ "month_order = [\n",
+ " 'January', 'February', 'March', 'April', 'May', 'June',\n",
+ " 'July', 'August', 'September', 'October', 'November', 'December'\n",
+ "]\n",
+ "policies_pivot = policies_pivot.reindex(columns=[m for m in month_order if m in policies_pivot.columns])\n",
+ "\n",
+ "policies_pivot"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "id": "a712e6a0-b0e3-450d-8220-f758d4da3b9b",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d",
- "metadata": {
- "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d"
- },
- "source": [
- "External Resources for Data Filtering: https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9"
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " month | \n",
+ " January | \n",
+ " February | \n",
+ "
\n",
+ " \n",
+ " state | \n",
+ " | \n",
+ " | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " Arizona | \n",
+ " 1008.0 | \n",
+ " 929.0 | \n",
+ "
\n",
+ " \n",
+ " California | \n",
+ " 1918.0 | \n",
+ " 1634.0 | \n",
+ "
\n",
+ " \n",
+ " Oregon | \n",
+ " 1565.0 | \n",
+ " 1344.0 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ "month January February\n",
+ "state \n",
+ "Arizona 1008.0 929.0\n",
+ "California 1918.0 1634.0\n",
+ "Oregon 1565.0 1344.0"
]
- },
+ },
+ "execution_count": 22,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "state_totals = (\n",
+ " policies_state_month.groupby('state')['n_policies']\n",
+ " .sum()\n",
+ " .reset_index()\n",
+ " .sort_values('n_policies', ascending=False)\n",
+ ")\n",
+ "\n",
+ "top3_states = state_totals.head(3)['state'].tolist()\n",
+ "\n",
+ "top3_pivot = policies_state_month[policies_state_month['state'].isin(top3_states)].pivot_table(\n",
+ " index='state', columns='month', values='n_policies', fill_value=0\n",
+ ")\n",
+ "\n",
+ "top3_pivot = top3_pivot.reindex(columns=[m for m in month_order if m in top3_pivot.columns])\n",
+ "\n",
+ "top3_pivot"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "id": "4df122a6-d195-4177-9843-5905786b889c",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "id": "449513f4-0459-46a0-a18d-9398d974c9ad",
- "metadata": {
- "id": "449513f4-0459-46a0-a18d-9398d974c9ad"
- },
- "outputs": [],
- "source": [
- "# your code goes here"
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " sales_channel | \n",
+ " response_rate | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 | \n",
+ " Agent | \n",
+ " 19.07 | \n",
+ "
\n",
+ " \n",
+ " 3 | \n",
+ " Web | \n",
+ " 11.71 | \n",
+ "
\n",
+ " \n",
+ " 1 | \n",
+ " Branch | \n",
+ " 11.38 | \n",
+ "
\n",
+ " \n",
+ " 2 | \n",
+ " Call Center | \n",
+ " 10.98 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " sales_channel response_rate\n",
+ "0 Agent 19.07\n",
+ "3 Web 11.71\n",
+ "1 Branch 11.38\n",
+ "2 Call Center 10.98"
]
+ },
+ "execution_count": 23,
+ "metadata": {},
+ "output_type": "execute_result"
}
- ],
- "metadata": {
- "colab": {
- "provenance": []
- },
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.13"
- }
+ ],
+ "source": [
+ "channel_response = (\n",
+ " df.groupby('sales_channel')['response']\n",
+ " .apply(lambda s: (s.str.lower() == 'yes').sum() / s.count())\n",
+ " .reset_index(name='response_rate')\n",
+ ")\n",
+ "\n",
+ "channel_response['response_rate'] = (channel_response['response_rate']*100).round(2)\n",
+ "channel_response.sort_values('response_rate', ascending=False)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "bbd3029e-60e4-49f0-9140-fe935de8a153",
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
},
- "nbformat": 4,
- "nbformat_minor": 5
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.7"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
}