diff --git a/lab-dw-aggregating.ipynb b/lab-dw-aggregating.ipynb
index fadd718..2552fcc 100644
--- a/lab-dw-aggregating.ipynb
+++ b/lab-dw-aggregating.ipynb
@@ -1,165 +1,2226 @@
{
- "cells": [
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "31969215-2a90-4d8b-ac36-646a7ae13744",
+ "metadata": {
+ "id": "31969215-2a90-4d8b-ac36-646a7ae13744"
+ },
+ "source": [
+ "# Lab | Data Aggregation and Filtering"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d",
+ "metadata": {
+ "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d"
+ },
+ "source": [
+ "In this challenge, we will continue to work with customer data from an insurance company. We will use the dataset called marketing_customer_analysis.csv, which can be found at the following link:\n",
+ "\n",
+ "https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\n",
+ "\n",
+ "This dataset contains information such as customer demographics, policy details, vehicle information, and the customer's response to the last marketing campaign. Our goal is to explore and analyze this data by first performing data cleaning, formatting, and structuring."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50",
+ "metadata": {
+ "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50"
+ },
+ "source": [
+ "1. Create a new DataFrame that only includes customers who:\n",
+ " - have a **low total_claim_amount** (e.g., below $1,000),\n",
+ " - have a response \"Yes\" to the last marketing campaign."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b9be383e-5165-436e-80c8-57d4c757c8c3",
+ "metadata": {
+ "id": "b9be383e-5165-436e-80c8-57d4c757c8c3"
+ },
+ "source": [
+ "2. Using the original Dataframe, analyze:\n",
+ " - the average `monthly_premium` and/or customer lifetime value by `policy_type` and `gender` for customers who responded \"Yes\", and\n",
+ " - compare these insights to `total_claim_amount` patterns, and discuss which segments appear most profitable or low-risk for the company."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0",
+ "metadata": {
+ "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0"
+ },
+ "source": [
+ "3. Analyze the total number of customers who have policies in each state, and then filter the results to only include states where there are more than 500 customers."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d",
+ "metadata": {
+ "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d"
+ },
+ "source": [
+ "4. Find the maximum, minimum, and median customer lifetime value by education level and gender. Write your conclusions."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b42999f9-311f-481e-ae63-40a5577072c5",
+ "metadata": {
+ "id": "b42999f9-311f-481e-ae63-40a5577072c5"
+ },
+ "source": [
+ "## Bonus"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "81ff02c5-6584-4f21-a358-b918697c6432",
+ "metadata": {
+ "id": "81ff02c5-6584-4f21-a358-b918697c6432"
+ },
+ "source": [
+ "5. The marketing team wants to analyze the number of policies sold by state and month. Present the data in a table where the months are arranged as columns and the states are arranged as rows."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b6aec097-c633-4017-a125-e77a97259cda",
+ "metadata": {
+ "id": "b6aec097-c633-4017-a125-e77a97259cda"
+ },
+ "source": [
+ "6. Display a new DataFrame that contains the number of policies sold by month, by state, for the top 3 states with the highest number of policies sold.\n",
+ "\n",
+ "*Hint:*\n",
+ "- *To accomplish this, you will first need to group the data by state and month, then count the number of policies sold for each group. Afterwards, you will need to sort the data by the count of policies sold in descending order.*\n",
+ "- *Next, you will select the top 3 states with the highest number of policies sold.*\n",
+ "- *Finally, you will create a new DataFrame that contains the number of policies sold by month for each of the top 3 states.*"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009",
+ "metadata": {
+ "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009"
+ },
+ "source": [
+ "7. The marketing team wants to analyze the effect of different marketing channels on the customer response rate.\n",
+ "\n",
+ "Hint: You can use melt to unpivot the data and create a table that shows the customer response rate (those who responded \"Yes\") by marketing channel."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d",
+ "metadata": {
+ "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d"
+ },
+ "source": [
+ "External Resources for Data Filtering: https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "449513f4-0459-46a0-a18d-9398d974c9ad",
+ "metadata": {
+ "id": "449513f4-0459-46a0-a18d-9398d974c9ad"
+ },
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "31969215-2a90-4d8b-ac36-646a7ae13744",
- "metadata": {
- "id": "31969215-2a90-4d8b-ac36-646a7ae13744"
- },
- "source": [
- "# Lab | Data Aggregation and Filtering"
+ "data": {
+ "text/html": [
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " Unnamed: 0 | \n",
+ " Customer | \n",
+ " State | \n",
+ " Customer Lifetime Value | \n",
+ " Response | \n",
+ " Coverage | \n",
+ " Education | \n",
+ " Effective To Date | \n",
+ " EmploymentStatus | \n",
+ " Gender | \n",
+ " ... | \n",
+ " Number of Open Complaints | \n",
+ " Number of Policies | \n",
+ " Policy Type | \n",
+ " Policy | \n",
+ " Renew Offer Type | \n",
+ " Sales Channel | \n",
+ " Total Claim Amount | \n",
+ " Vehicle Class | \n",
+ " Vehicle Size | \n",
+ " Vehicle Type | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 | \n",
+ " 0 | \n",
+ " DK49336 | \n",
+ " Arizona | \n",
+ " 4809.216960 | \n",
+ " No | \n",
+ " Basic | \n",
+ " College | \n",
+ " 2/18/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 9 | \n",
+ " Corporate Auto | \n",
+ " Corporate L3 | \n",
+ " Offer3 | \n",
+ " Agent | \n",
+ " 292.800000 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 1 | \n",
+ " 1 | \n",
+ " KX64629 | \n",
+ " California | \n",
+ " 2228.525238 | \n",
+ " No | \n",
+ " Basic | \n",
+ " College | \n",
+ " 1/18/11 | \n",
+ " Unemployed | \n",
+ " F | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer4 | \n",
+ " Call Center | \n",
+ " 744.924331 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 2 | \n",
+ " 2 | \n",
+ " LZ68649 | \n",
+ " Washington | \n",
+ " 14947.917300 | \n",
+ " No | \n",
+ " Basic | \n",
+ " Bachelor | \n",
+ " 2/10/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 2 | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer3 | \n",
+ " Call Center | \n",
+ " 480.000000 | \n",
+ " SUV | \n",
+ " Medsize | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " 3 | \n",
+ " 3 | \n",
+ " XL78013 | \n",
+ " Oregon | \n",
+ " 22332.439460 | \n",
+ " Yes | \n",
+ " Extended | \n",
+ " College | \n",
+ " 1/11/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 2 | \n",
+ " Corporate Auto | \n",
+ " Corporate L3 | \n",
+ " Offer2 | \n",
+ " Branch | \n",
+ " 484.013411 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " 4 | \n",
+ " 4 | \n",
+ " QA50777 | \n",
+ " Oregon | \n",
+ " 9025.067525 | \n",
+ " No | \n",
+ " Premium | \n",
+ " Bachelor | \n",
+ " 1/17/11 | \n",
+ " Medical Leave | \n",
+ " F | \n",
+ " ... | \n",
+ " NaN | \n",
+ " 7 | \n",
+ " Personal Auto | \n",
+ " Personal L2 | \n",
+ " Offer1 | \n",
+ " Branch | \n",
+ " 707.925645 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
5 rows × 26 columns
\n",
+ "
"
+ ],
+ "text/plain": [
+ " Unnamed: 0 Customer State Customer Lifetime Value Response \\\n",
+ "0 0 DK49336 Arizona 4809.216960 No \n",
+ "1 1 KX64629 California 2228.525238 No \n",
+ "2 2 LZ68649 Washington 14947.917300 No \n",
+ "3 3 XL78013 Oregon 22332.439460 Yes \n",
+ "4 4 QA50777 Oregon 9025.067525 No \n",
+ "\n",
+ " Coverage Education Effective To Date EmploymentStatus Gender ... \\\n",
+ "0 Basic College 2/18/11 Employed M ... \n",
+ "1 Basic College 1/18/11 Unemployed F ... \n",
+ "2 Basic Bachelor 2/10/11 Employed M ... \n",
+ "3 Extended College 1/11/11 Employed M ... \n",
+ "4 Premium Bachelor 1/17/11 Medical Leave F ... \n",
+ "\n",
+ " Number of Open Complaints Number of Policies Policy Type Policy \\\n",
+ "0 0.0 9 Corporate Auto Corporate L3 \n",
+ "1 0.0 1 Personal Auto Personal L3 \n",
+ "2 0.0 2 Personal Auto Personal L3 \n",
+ "3 0.0 2 Corporate Auto Corporate L3 \n",
+ "4 NaN 7 Personal Auto Personal L2 \n",
+ "\n",
+ " Renew Offer Type Sales Channel Total Claim Amount Vehicle Class \\\n",
+ "0 Offer3 Agent 292.800000 Four-Door Car \n",
+ "1 Offer4 Call Center 744.924331 Four-Door Car \n",
+ "2 Offer3 Call Center 480.000000 SUV \n",
+ "3 Offer2 Branch 484.013411 Four-Door Car \n",
+ "4 Offer1 Branch 707.925645 Four-Door Car \n",
+ "\n",
+ " Vehicle Size Vehicle Type \n",
+ "0 Medsize NaN \n",
+ "1 Medsize NaN \n",
+ "2 Medsize A \n",
+ "3 Medsize A \n",
+ "4 Medsize NaN \n",
+ "\n",
+ "[5 rows x 26 columns]"
]
- },
+ },
+ "execution_count": 2,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "import pandas as pd\n",
+ "import numpy as np\n",
+ "\n",
+ "url = \"https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\"\n",
+ "df = pd.read_csv(url)\n",
+ "\n",
+ "df.head()\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "91b79d04-5182-442f-a379-70822d77299c",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d",
- "metadata": {
- "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d"
- },
- "source": [
- "In this challenge, we will continue to work with customer data from an insurance company. We will use the dataset called marketing_customer_analysis.csv, which can be found at the following link:\n",
- "\n",
- "https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\n",
- "\n",
- "This dataset contains information such as customer demographics, policy details, vehicle information, and the customer's response to the last marketing campaign. Our goal is to explore and analyze this data by first performing data cleaning, formatting, and structuring."
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\n",
+ "RangeIndex: 10910 entries, 0 to 10909\n",
+ "Data columns (total 26 columns):\n",
+ " # Column Non-Null Count Dtype \n",
+ "--- ------ -------------- ----- \n",
+ " 0 Unnamed: 0 10910 non-null int64 \n",
+ " 1 Customer 10910 non-null object \n",
+ " 2 State 10279 non-null object \n",
+ " 3 Customer Lifetime Value 10910 non-null float64\n",
+ " 4 Response 10279 non-null object \n",
+ " 5 Coverage 10910 non-null object \n",
+ " 6 Education 10910 non-null object \n",
+ " 7 Effective To Date 10910 non-null object \n",
+ " 8 EmploymentStatus 10910 non-null object \n",
+ " 9 Gender 10910 non-null object \n",
+ " 10 Income 10910 non-null int64 \n",
+ " 11 Location Code 10910 non-null object \n",
+ " 12 Marital Status 10910 non-null object \n",
+ " 13 Monthly Premium Auto 10910 non-null int64 \n",
+ " 14 Months Since Last Claim 10277 non-null float64\n",
+ " 15 Months Since Policy Inception 10910 non-null int64 \n",
+ " 16 Number of Open Complaints 10277 non-null float64\n",
+ " 17 Number of Policies 10910 non-null int64 \n",
+ " 18 Policy Type 10910 non-null object \n",
+ " 19 Policy 10910 non-null object \n",
+ " 20 Renew Offer Type 10910 non-null object \n",
+ " 21 Sales Channel 10910 non-null object \n",
+ " 22 Total Claim Amount 10910 non-null float64\n",
+ " 23 Vehicle Class 10288 non-null object \n",
+ " 24 Vehicle Size 10288 non-null object \n",
+ " 25 Vehicle Type 5428 non-null object \n",
+ "dtypes: float64(4), int64(5), object(17)\n",
+ "memory usage: 2.2+ MB\n"
+ ]
},
{
- "cell_type": "markdown",
- "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50",
- "metadata": {
- "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50"
- },
- "source": [
- "1. Create a new DataFrame that only includes customers who:\n",
- " - have a **low total_claim_amount** (e.g., below $1,000),\n",
- " - have a response \"Yes\" to the last marketing campaign."
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " Unnamed: 0 | \n",
+ " Customer | \n",
+ " State | \n",
+ " Customer Lifetime Value | \n",
+ " Response | \n",
+ " Coverage | \n",
+ " Education | \n",
+ " Effective To Date | \n",
+ " EmploymentStatus | \n",
+ " Gender | \n",
+ " ... | \n",
+ " Number of Open Complaints | \n",
+ " Number of Policies | \n",
+ " Policy Type | \n",
+ " Policy | \n",
+ " Renew Offer Type | \n",
+ " Sales Channel | \n",
+ " Total Claim Amount | \n",
+ " Vehicle Class | \n",
+ " Vehicle Size | \n",
+ " Vehicle Type | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " count | \n",
+ " 10910.000000 | \n",
+ " 10910 | \n",
+ " 10279 | \n",
+ " 10910.000000 | \n",
+ " 10279 | \n",
+ " 10910 | \n",
+ " 10910 | \n",
+ " 10910 | \n",
+ " 10910 | \n",
+ " 10910 | \n",
+ " ... | \n",
+ " 10277.000000 | \n",
+ " 10910.000000 | \n",
+ " 10910 | \n",
+ " 10910 | \n",
+ " 10910 | \n",
+ " 10910 | \n",
+ " 10910.000000 | \n",
+ " 10288 | \n",
+ " 10288 | \n",
+ " 5428 | \n",
+ "
\n",
+ " \n",
+ " unique | \n",
+ " NaN | \n",
+ " 9134 | \n",
+ " 5 | \n",
+ " NaN | \n",
+ " 2 | \n",
+ " 3 | \n",
+ " 5 | \n",
+ " 59 | \n",
+ " 5 | \n",
+ " 2 | \n",
+ " ... | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 3 | \n",
+ " 9 | \n",
+ " 4 | \n",
+ " 4 | \n",
+ " NaN | \n",
+ " 6 | \n",
+ " 3 | \n",
+ " 1 | \n",
+ "
\n",
+ " \n",
+ " top | \n",
+ " NaN | \n",
+ " ID89933 | \n",
+ " California | \n",
+ " NaN | \n",
+ " No | \n",
+ " Basic | \n",
+ " Bachelor | \n",
+ " 1/27/11 | \n",
+ " Employed | \n",
+ " F | \n",
+ " ... | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer1 | \n",
+ " Agent | \n",
+ " NaN | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " freq | \n",
+ " NaN | \n",
+ " 7 | \n",
+ " 3552 | \n",
+ " NaN | \n",
+ " 8813 | \n",
+ " 6660 | \n",
+ " 3272 | \n",
+ " 239 | \n",
+ " 6789 | \n",
+ " 5573 | \n",
+ " ... | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 8128 | \n",
+ " 4118 | \n",
+ " 4483 | \n",
+ " 4121 | \n",
+ " NaN | \n",
+ " 5212 | \n",
+ " 7251 | \n",
+ " 5428 | \n",
+ "
\n",
+ " \n",
+ " mean | \n",
+ " 5454.500000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 8018.241094 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " ... | \n",
+ " 0.384256 | \n",
+ " 2.979193 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 434.888330 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " std | \n",
+ " 3149.590053 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 6885.081434 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " ... | \n",
+ " 0.912457 | \n",
+ " 2.399359 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 292.180556 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " min | \n",
+ " 0.000000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 1898.007675 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " ... | \n",
+ " 0.000000 | \n",
+ " 1.000000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 0.099007 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 25% | \n",
+ " 2727.250000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 4014.453113 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " ... | \n",
+ " 0.000000 | \n",
+ " 1.000000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 271.082527 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 50% | \n",
+ " 5454.500000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 5771.147235 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " ... | \n",
+ " 0.000000 | \n",
+ " 2.000000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 382.564630 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 75% | \n",
+ " 8181.750000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 8992.779137 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " ... | \n",
+ " 0.000000 | \n",
+ " 4.000000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 547.200000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " max | \n",
+ " 10909.000000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 83325.381190 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " ... | \n",
+ " 5.000000 | \n",
+ " 9.000000 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " 2893.239678 | \n",
+ " NaN | \n",
+ " NaN | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
11 rows × 26 columns
\n",
+ "
"
+ ],
+ "text/plain": [
+ " Unnamed: 0 Customer State Customer Lifetime Value Response \\\n",
+ "count 10910.000000 10910 10279 10910.000000 10279 \n",
+ "unique NaN 9134 5 NaN 2 \n",
+ "top NaN ID89933 California NaN No \n",
+ "freq NaN 7 3552 NaN 8813 \n",
+ "mean 5454.500000 NaN NaN 8018.241094 NaN \n",
+ "std 3149.590053 NaN NaN 6885.081434 NaN \n",
+ "min 0.000000 NaN NaN 1898.007675 NaN \n",
+ "25% 2727.250000 NaN NaN 4014.453113 NaN \n",
+ "50% 5454.500000 NaN NaN 5771.147235 NaN \n",
+ "75% 8181.750000 NaN NaN 8992.779137 NaN \n",
+ "max 10909.000000 NaN NaN 83325.381190 NaN \n",
+ "\n",
+ " Coverage Education Effective To Date EmploymentStatus Gender ... \\\n",
+ "count 10910 10910 10910 10910 10910 ... \n",
+ "unique 3 5 59 5 2 ... \n",
+ "top Basic Bachelor 1/27/11 Employed F ... \n",
+ "freq 6660 3272 239 6789 5573 ... \n",
+ "mean NaN NaN NaN NaN NaN ... \n",
+ "std NaN NaN NaN NaN NaN ... \n",
+ "min NaN NaN NaN NaN NaN ... \n",
+ "25% NaN NaN NaN NaN NaN ... \n",
+ "50% NaN NaN NaN NaN NaN ... \n",
+ "75% NaN NaN NaN NaN NaN ... \n",
+ "max NaN NaN NaN NaN NaN ... \n",
+ "\n",
+ " Number of Open Complaints Number of Policies Policy Type \\\n",
+ "count 10277.000000 10910.000000 10910 \n",
+ "unique NaN NaN 3 \n",
+ "top NaN NaN Personal Auto \n",
+ "freq NaN NaN 8128 \n",
+ "mean 0.384256 2.979193 NaN \n",
+ "std 0.912457 2.399359 NaN \n",
+ "min 0.000000 1.000000 NaN \n",
+ "25% 0.000000 1.000000 NaN \n",
+ "50% 0.000000 2.000000 NaN \n",
+ "75% 0.000000 4.000000 NaN \n",
+ "max 5.000000 9.000000 NaN \n",
+ "\n",
+ " Policy Renew Offer Type Sales Channel Total Claim Amount \\\n",
+ "count 10910 10910 10910 10910.000000 \n",
+ "unique 9 4 4 NaN \n",
+ "top Personal L3 Offer1 Agent NaN \n",
+ "freq 4118 4483 4121 NaN \n",
+ "mean NaN NaN NaN 434.888330 \n",
+ "std NaN NaN NaN 292.180556 \n",
+ "min NaN NaN NaN 0.099007 \n",
+ "25% NaN NaN NaN 271.082527 \n",
+ "50% NaN NaN NaN 382.564630 \n",
+ "75% NaN NaN NaN 547.200000 \n",
+ "max NaN NaN NaN 2893.239678 \n",
+ "\n",
+ " Vehicle Class Vehicle Size Vehicle Type \n",
+ "count 10288 10288 5428 \n",
+ "unique 6 3 1 \n",
+ "top Four-Door Car Medsize A \n",
+ "freq 5212 7251 5428 \n",
+ "mean NaN NaN NaN \n",
+ "std NaN NaN NaN \n",
+ "min NaN NaN NaN \n",
+ "25% NaN NaN NaN \n",
+ "50% NaN NaN NaN \n",
+ "75% NaN NaN NaN \n",
+ "max NaN NaN NaN \n",
+ "\n",
+ "[11 rows x 26 columns]"
]
- },
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "df.info()\n",
+ "df.describe(include=\"all\")\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "67831c16-d110-4d1c-a489-b75943283da0",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "b9be383e-5165-436e-80c8-57d4c757c8c3",
- "metadata": {
- "id": "b9be383e-5165-436e-80c8-57d4c757c8c3"
- },
- "source": [
- "2. Using the original Dataframe, analyze:\n",
- " - the average `monthly_premium` and/or customer lifetime value by `policy_type` and `gender` for customers who responded \"Yes\", and\n",
- " - compare these insights to `total_claim_amount` patterns, and discuss which segments appear most profitable or low-risk for the company."
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " unnamed:_0 | \n",
+ " customer | \n",
+ " state | \n",
+ " customer_lifetime_value | \n",
+ " response | \n",
+ " coverage | \n",
+ " education | \n",
+ " effective_to_date | \n",
+ " employmentstatus | \n",
+ " gender | \n",
+ " ... | \n",
+ " number_of_open_complaints | \n",
+ " number_of_policies | \n",
+ " policy_type | \n",
+ " policy | \n",
+ " renew_offer_type | \n",
+ " sales_channel | \n",
+ " total_claim_amount | \n",
+ " vehicle_class | \n",
+ " vehicle_size | \n",
+ " vehicle_type | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 | \n",
+ " 0 | \n",
+ " DK49336 | \n",
+ " Arizona | \n",
+ " 4809.216960 | \n",
+ " No | \n",
+ " Basic | \n",
+ " College | \n",
+ " 2/18/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 9 | \n",
+ " Corporate Auto | \n",
+ " Corporate L3 | \n",
+ " Offer3 | \n",
+ " Agent | \n",
+ " 292.800000 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 1 | \n",
+ " 1 | \n",
+ " KX64629 | \n",
+ " California | \n",
+ " 2228.525238 | \n",
+ " No | \n",
+ " Basic | \n",
+ " College | \n",
+ " 1/18/11 | \n",
+ " Unemployed | \n",
+ " F | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer4 | \n",
+ " Call Center | \n",
+ " 744.924331 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 2 | \n",
+ " 2 | \n",
+ " LZ68649 | \n",
+ " Washington | \n",
+ " 14947.917300 | \n",
+ " No | \n",
+ " Basic | \n",
+ " Bachelor | \n",
+ " 2/10/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 2 | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer3 | \n",
+ " Call Center | \n",
+ " 480.000000 | \n",
+ " SUV | \n",
+ " Medsize | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " 3 | \n",
+ " 3 | \n",
+ " XL78013 | \n",
+ " Oregon | \n",
+ " 22332.439460 | \n",
+ " Yes | \n",
+ " Extended | \n",
+ " College | \n",
+ " 1/11/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 2 | \n",
+ " Corporate Auto | \n",
+ " Corporate L3 | \n",
+ " Offer2 | \n",
+ " Branch | \n",
+ " 484.013411 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " 4 | \n",
+ " 4 | \n",
+ " QA50777 | \n",
+ " Oregon | \n",
+ " 9025.067525 | \n",
+ " No | \n",
+ " Premium | \n",
+ " Bachelor | \n",
+ " 1/17/11 | \n",
+ " Medical Leave | \n",
+ " F | \n",
+ " ... | \n",
+ " NaN | \n",
+ " 7 | \n",
+ " Personal Auto | \n",
+ " Personal L2 | \n",
+ " Offer1 | \n",
+ " Branch | \n",
+ " 707.925645 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
5 rows × 26 columns
\n",
+ "
"
+ ],
+ "text/plain": [
+ " unnamed:_0 customer state customer_lifetime_value response \\\n",
+ "0 0 DK49336 Arizona 4809.216960 No \n",
+ "1 1 KX64629 California 2228.525238 No \n",
+ "2 2 LZ68649 Washington 14947.917300 No \n",
+ "3 3 XL78013 Oregon 22332.439460 Yes \n",
+ "4 4 QA50777 Oregon 9025.067525 No \n",
+ "\n",
+ " coverage education effective_to_date employmentstatus gender ... \\\n",
+ "0 Basic College 2/18/11 Employed M ... \n",
+ "1 Basic College 1/18/11 Unemployed F ... \n",
+ "2 Basic Bachelor 2/10/11 Employed M ... \n",
+ "3 Extended College 1/11/11 Employed M ... \n",
+ "4 Premium Bachelor 1/17/11 Medical Leave F ... \n",
+ "\n",
+ " number_of_open_complaints number_of_policies policy_type policy \\\n",
+ "0 0.0 9 Corporate Auto Corporate L3 \n",
+ "1 0.0 1 Personal Auto Personal L3 \n",
+ "2 0.0 2 Personal Auto Personal L3 \n",
+ "3 0.0 2 Corporate Auto Corporate L3 \n",
+ "4 NaN 7 Personal Auto Personal L2 \n",
+ "\n",
+ " renew_offer_type sales_channel total_claim_amount vehicle_class \\\n",
+ "0 Offer3 Agent 292.800000 Four-Door Car \n",
+ "1 Offer4 Call Center 744.924331 Four-Door Car \n",
+ "2 Offer3 Call Center 480.000000 SUV \n",
+ "3 Offer2 Branch 484.013411 Four-Door Car \n",
+ "4 Offer1 Branch 707.925645 Four-Door Car \n",
+ "\n",
+ " vehicle_size vehicle_type \n",
+ "0 Medsize NaN \n",
+ "1 Medsize NaN \n",
+ "2 Medsize A \n",
+ "3 Medsize A \n",
+ "4 Medsize NaN \n",
+ "\n",
+ "[5 rows x 26 columns]"
]
- },
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "df.columns = df.columns.str.lower().str.replace(' ', '_')\n",
+ "df.head()\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "id": "761bd0ac-4996-489d-94b3-0b7535408b7d",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0",
- "metadata": {
- "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0"
- },
- "source": [
- "3. Analyze the total number of customers who have policies in each state, and then filter the results to only include states where there are more than 500 customers."
+ "data": {
+ "text/plain": [
+ "unnamed:_0 0\n",
+ "customer 0\n",
+ "state 631\n",
+ "customer_lifetime_value 0\n",
+ "response 631\n",
+ "coverage 0\n",
+ "education 0\n",
+ "effective_to_date 0\n",
+ "employmentstatus 0\n",
+ "gender 0\n",
+ "income 0\n",
+ "location_code 0\n",
+ "marital_status 0\n",
+ "monthly_premium_auto 0\n",
+ "months_since_last_claim 633\n",
+ "months_since_policy_inception 0\n",
+ "number_of_open_complaints 633\n",
+ "number_of_policies 0\n",
+ "policy_type 0\n",
+ "policy 0\n",
+ "renew_offer_type 0\n",
+ "sales_channel 0\n",
+ "total_claim_amount 0\n",
+ "vehicle_class 622\n",
+ "vehicle_size 622\n",
+ "vehicle_type 5482\n",
+ "dtype: int64"
]
- },
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "df.isnull().sum()\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "id": "91338749-4da3-4bca-8833-ee1212c1b027",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Delete rows with too many null values\n",
+ "df = df.dropna(subset=['customer_lifetime_value', 'total_claim_amount'])\n",
+ "\n",
+ "# fill null values in categorical columns with \"Unknown\"\n",
+ "df['state'] = df['state'].fillna(\"Unknown\")\n",
+ "df['gender'] = df['gender'].fillna(\"Unknown\")\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "id": "943c5800-9838-4cb8-93b3-0db3e7158061",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#Convert numerical values in (float/int)\n",
+ "df['customer_lifetime_value'] = df['customer_lifetime_value'].astype(float)\n",
+ "df['total_claim_amount'] = df['total_claim_amount'].astype(float)\n",
+ "df['monthly_premium_auto'] = df['monthly_premium_auto'].astype(float)\n",
+ "#Convert categorical values in string\n",
+ "df['gender'] = df['gender'].astype('string')\n",
+ "df['policy_type'] = df['policy_type'].astype('string')\n",
+ "df['education'] = df['education'].astype('string')\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "id": "b946d969-357a-4279-ade2-82186192cb5b",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#delete duplicates\n",
+ "df = df.drop_duplicates()\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "id": "38a6c272-3c52-4486-a9ae-247725496925",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#Reset Index\n",
+ "df = df.reset_index(drop=True)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "id": "4fd7e472-bded-4e15-af53-ec0a41995978",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d",
- "metadata": {
- "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d"
- },
- "source": [
- "4. Find the maximum, minimum, and median customer lifetime value by education level and gender. Write your conclusions."
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " unnamed:_0 | \n",
+ " customer | \n",
+ " state | \n",
+ " customer_lifetime_value | \n",
+ " response | \n",
+ " coverage | \n",
+ " education | \n",
+ " effective_to_date | \n",
+ " employmentstatus | \n",
+ " gender | \n",
+ " ... | \n",
+ " number_of_open_complaints | \n",
+ " number_of_policies | \n",
+ " policy_type | \n",
+ " policy | \n",
+ " renew_offer_type | \n",
+ " sales_channel | \n",
+ " total_claim_amount | \n",
+ " vehicle_class | \n",
+ " vehicle_size | \n",
+ " vehicle_type | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 | \n",
+ " 0 | \n",
+ " DK49336 | \n",
+ " Arizona | \n",
+ " 4809.216960 | \n",
+ " No | \n",
+ " Basic | \n",
+ " College | \n",
+ " 2/18/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 9 | \n",
+ " Corporate Auto | \n",
+ " Corporate L3 | \n",
+ " Offer3 | \n",
+ " Agent | \n",
+ " 292.800000 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 1 | \n",
+ " 1 | \n",
+ " KX64629 | \n",
+ " California | \n",
+ " 2228.525238 | \n",
+ " No | \n",
+ " Basic | \n",
+ " College | \n",
+ " 1/18/11 | \n",
+ " Unemployed | \n",
+ " F | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer4 | \n",
+ " Call Center | \n",
+ " 744.924331 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 2 | \n",
+ " 2 | \n",
+ " LZ68649 | \n",
+ " Washington | \n",
+ " 14947.917300 | \n",
+ " No | \n",
+ " Basic | \n",
+ " Bachelor | \n",
+ " 2/10/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 2 | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer3 | \n",
+ " Call Center | \n",
+ " 480.000000 | \n",
+ " SUV | \n",
+ " Medsize | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " 3 | \n",
+ " 3 | \n",
+ " XL78013 | \n",
+ " Oregon | \n",
+ " 22332.439460 | \n",
+ " Yes | \n",
+ " Extended | \n",
+ " College | \n",
+ " 1/11/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 2 | \n",
+ " Corporate Auto | \n",
+ " Corporate L3 | \n",
+ " Offer2 | \n",
+ " Branch | \n",
+ " 484.013411 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " 4 | \n",
+ " 4 | \n",
+ " QA50777 | \n",
+ " Oregon | \n",
+ " 9025.067525 | \n",
+ " No | \n",
+ " Premium | \n",
+ " Bachelor | \n",
+ " 1/17/11 | \n",
+ " Medical Leave | \n",
+ " F | \n",
+ " ... | \n",
+ " NaN | \n",
+ " 7 | \n",
+ " Personal Auto | \n",
+ " Personal L2 | \n",
+ " Offer1 | \n",
+ " Branch | \n",
+ " 707.925645 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
5 rows × 26 columns
\n",
+ "
"
+ ],
+ "text/plain": [
+ " unnamed:_0 customer state customer_lifetime_value response \\\n",
+ "0 0 DK49336 Arizona 4809.216960 No \n",
+ "1 1 KX64629 California 2228.525238 No \n",
+ "2 2 LZ68649 Washington 14947.917300 No \n",
+ "3 3 XL78013 Oregon 22332.439460 Yes \n",
+ "4 4 QA50777 Oregon 9025.067525 No \n",
+ "\n",
+ " coverage education effective_to_date employmentstatus gender ... \\\n",
+ "0 Basic College 2/18/11 Employed M ... \n",
+ "1 Basic College 1/18/11 Unemployed F ... \n",
+ "2 Basic Bachelor 2/10/11 Employed M ... \n",
+ "3 Extended College 1/11/11 Employed M ... \n",
+ "4 Premium Bachelor 1/17/11 Medical Leave F ... \n",
+ "\n",
+ " number_of_open_complaints number_of_policies policy_type policy \\\n",
+ "0 0.0 9 Corporate Auto Corporate L3 \n",
+ "1 0.0 1 Personal Auto Personal L3 \n",
+ "2 0.0 2 Personal Auto Personal L3 \n",
+ "3 0.0 2 Corporate Auto Corporate L3 \n",
+ "4 NaN 7 Personal Auto Personal L2 \n",
+ "\n",
+ " renew_offer_type sales_channel total_claim_amount vehicle_class \\\n",
+ "0 Offer3 Agent 292.800000 Four-Door Car \n",
+ "1 Offer4 Call Center 744.924331 Four-Door Car \n",
+ "2 Offer3 Call Center 480.000000 SUV \n",
+ "3 Offer2 Branch 484.013411 Four-Door Car \n",
+ "4 Offer1 Branch 707.925645 Four-Door Car \n",
+ "\n",
+ " vehicle_size vehicle_type \n",
+ "0 Medsize NaN \n",
+ "1 Medsize NaN \n",
+ "2 Medsize A \n",
+ "3 Medsize A \n",
+ "4 Medsize NaN \n",
+ "\n",
+ "[5 rows x 26 columns]"
]
+ },
+ "execution_count": 10,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "df.head()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "id": "75b88d80-bfba-4f45-8907-19f3fd154738",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Shape: (1399, 26)\n"
+ ]
},
{
- "cell_type": "markdown",
- "id": "b42999f9-311f-481e-ae63-40a5577072c5",
- "metadata": {
- "id": "b42999f9-311f-481e-ae63-40a5577072c5"
- },
- "source": [
- "## Bonus"
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " unnamed:_0 | \n",
+ " customer | \n",
+ " state | \n",
+ " customer_lifetime_value | \n",
+ " response | \n",
+ " coverage | \n",
+ " education | \n",
+ " effective_to_date | \n",
+ " employmentstatus | \n",
+ " gender | \n",
+ " ... | \n",
+ " number_of_open_complaints | \n",
+ " number_of_policies | \n",
+ " policy_type | \n",
+ " policy | \n",
+ " renew_offer_type | \n",
+ " sales_channel | \n",
+ " total_claim_amount | \n",
+ " vehicle_class | \n",
+ " vehicle_size | \n",
+ " vehicle_type | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 3 | \n",
+ " 3 | \n",
+ " XL78013 | \n",
+ " Oregon | \n",
+ " 22332.439460 | \n",
+ " Yes | \n",
+ " Extended | \n",
+ " College | \n",
+ " 1/11/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 2 | \n",
+ " Corporate Auto | \n",
+ " Corporate L3 | \n",
+ " Offer2 | \n",
+ " Branch | \n",
+ " 484.013411 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " 8 | \n",
+ " 8 | \n",
+ " FM55990 | \n",
+ " California | \n",
+ " 5989.773931 | \n",
+ " Yes | \n",
+ " Premium | \n",
+ " College | \n",
+ " 1/19/11 | \n",
+ " Employed | \n",
+ " M | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ " Personal Auto | \n",
+ " Personal L1 | \n",
+ " Offer2 | \n",
+ " Branch | \n",
+ " 739.200000 | \n",
+ " Sports Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 15 | \n",
+ " 15 | \n",
+ " CW49887 | \n",
+ " California | \n",
+ " 4626.801093 | \n",
+ " Yes | \n",
+ " Basic | \n",
+ " Master | \n",
+ " 1/16/11 | \n",
+ " Employed | \n",
+ " F | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ " Special Auto | \n",
+ " Special L1 | \n",
+ " Offer2 | \n",
+ " Branch | \n",
+ " 547.200000 | \n",
+ " SUV | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ " 19 | \n",
+ " 19 | \n",
+ " NJ54277 | \n",
+ " California | \n",
+ " 3746.751625 | \n",
+ " Yes | \n",
+ " Extended | \n",
+ " College | \n",
+ " 2/26/11 | \n",
+ " Employed | \n",
+ " F | \n",
+ " ... | \n",
+ " 1.0 | \n",
+ " 1 | \n",
+ " Personal Auto | \n",
+ " Personal L2 | \n",
+ " Offer2 | \n",
+ " Call Center | \n",
+ " 19.575683 | \n",
+ " Two-Door Car | \n",
+ " Large | \n",
+ " A | \n",
+ "
\n",
+ " \n",
+ " 27 | \n",
+ " 27 | \n",
+ " MQ68407 | \n",
+ " Oregon | \n",
+ " 4376.363592 | \n",
+ " Yes | \n",
+ " Premium | \n",
+ " Bachelor | \n",
+ " 2/28/11 | \n",
+ " Employed | \n",
+ " F | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 1 | \n",
+ " Personal Auto | \n",
+ " Personal L3 | \n",
+ " Offer2 | \n",
+ " Agent | \n",
+ " 60.036683 | \n",
+ " Four-Door Car | \n",
+ " Medsize | \n",
+ " NaN | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
5 rows × 26 columns
\n",
+ "
"
+ ],
+ "text/plain": [
+ " unnamed:_0 customer state customer_lifetime_value response \\\n",
+ "3 3 XL78013 Oregon 22332.439460 Yes \n",
+ "8 8 FM55990 California 5989.773931 Yes \n",
+ "15 15 CW49887 California 4626.801093 Yes \n",
+ "19 19 NJ54277 California 3746.751625 Yes \n",
+ "27 27 MQ68407 Oregon 4376.363592 Yes \n",
+ "\n",
+ " coverage education effective_to_date employmentstatus gender ... \\\n",
+ "3 Extended College 1/11/11 Employed M ... \n",
+ "8 Premium College 1/19/11 Employed M ... \n",
+ "15 Basic Master 1/16/11 Employed F ... \n",
+ "19 Extended College 2/26/11 Employed F ... \n",
+ "27 Premium Bachelor 2/28/11 Employed F ... \n",
+ "\n",
+ " number_of_open_complaints number_of_policies policy_type \\\n",
+ "3 0.0 2 Corporate Auto \n",
+ "8 0.0 1 Personal Auto \n",
+ "15 0.0 1 Special Auto \n",
+ "19 1.0 1 Personal Auto \n",
+ "27 0.0 1 Personal Auto \n",
+ "\n",
+ " policy renew_offer_type sales_channel total_claim_amount \\\n",
+ "3 Corporate L3 Offer2 Branch 484.013411 \n",
+ "8 Personal L1 Offer2 Branch 739.200000 \n",
+ "15 Special L1 Offer2 Branch 547.200000 \n",
+ "19 Personal L2 Offer2 Call Center 19.575683 \n",
+ "27 Personal L3 Offer2 Agent 60.036683 \n",
+ "\n",
+ " vehicle_class vehicle_size vehicle_type \n",
+ "3 Four-Door Car Medsize A \n",
+ "8 Sports Car Medsize NaN \n",
+ "15 SUV Medsize NaN \n",
+ "19 Two-Door Car Large A \n",
+ "27 Four-Door Car Medsize NaN \n",
+ "\n",
+ "[5 rows x 26 columns]"
]
- },
+ },
+ "execution_count": 11,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# 1 Display a new DataFrame\n",
+ "# total_claim_amount < 1000\n",
+ "# response == \"Yes\"\n",
+ "\n",
+ "df_low_claim_yes = df[(df['total_claim_amount'] < 1000) & (df['response'] == 'Yes')]\n",
+ "\n",
+ "print(\"Shape:\", df_low_claim_yes.shape)\n",
+ "\n",
+ "df_low_claim_yes.head()\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "id": "243a20d7-f056-4921-afdb-5c208a9f22e0",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "81ff02c5-6584-4f21-a358-b918697c6432",
- "metadata": {
- "id": "81ff02c5-6584-4f21-a358-b918697c6432"
- },
- "source": [
- "5. The marketing team wants to analyze the number of policies sold by state and month. Present the data in a table where the months are arranged as columns and the states are arranged as rows."
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " policy_type | \n",
+ " gender | \n",
+ " monthly_premium_auto | \n",
+ " customer_lifetime_value | \n",
+ " total_claim_amount | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 | \n",
+ " Corporate Auto | \n",
+ " F | \n",
+ " 94.301775 | \n",
+ " 7712.628736 | \n",
+ " 433.738499 | \n",
+ "
\n",
+ " \n",
+ " 1 | \n",
+ " Corporate Auto | \n",
+ " M | \n",
+ " 92.188312 | \n",
+ " 7944.465414 | \n",
+ " 408.582459 | \n",
+ "
\n",
+ " \n",
+ " 2 | \n",
+ " Personal Auto | \n",
+ " F | \n",
+ " 98.998148 | \n",
+ " 8339.791842 | \n",
+ " 452.965929 | \n",
+ "
\n",
+ " \n",
+ " 3 | \n",
+ " Personal Auto | \n",
+ " M | \n",
+ " 91.085821 | \n",
+ " 7448.383281 | \n",
+ " 457.010178 | \n",
+ "
\n",
+ " \n",
+ " 4 | \n",
+ " Special Auto | \n",
+ " F | \n",
+ " 92.314286 | \n",
+ " 7691.584111 | \n",
+ " 453.280164 | \n",
+ "
\n",
+ " \n",
+ " 5 | \n",
+ " Special Auto | \n",
+ " M | \n",
+ " 86.343750 | \n",
+ " 8247.088702 | \n",
+ " 429.527942 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " policy_type gender monthly_premium_auto customer_lifetime_value \\\n",
+ "0 Corporate Auto F 94.301775 7712.628736 \n",
+ "1 Corporate Auto M 92.188312 7944.465414 \n",
+ "2 Personal Auto F 98.998148 8339.791842 \n",
+ "3 Personal Auto M 91.085821 7448.383281 \n",
+ "4 Special Auto F 92.314286 7691.584111 \n",
+ "5 Special Auto M 86.343750 8247.088702 \n",
+ "\n",
+ " total_claim_amount \n",
+ "0 433.738499 \n",
+ "1 408.582459 \n",
+ "2 452.965929 \n",
+ "3 457.010178 \n",
+ "4 453.280164 \n",
+ "5 429.527942 "
]
- },
+ },
+ "execution_count": 12,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# 2 average monthly_premium and/or customer lifetime value by policy_type and gender for customers who responded \"Yes\"\n",
+ "df_yes = df[df['response'] == 'Yes']\n",
+ "\n",
+ "agg_results = df_yes.groupby(['policy_type', 'gender']).agg({\n",
+ " 'monthly_premium_auto': 'mean',\n",
+ " 'customer_lifetime_value': 'mean',\n",
+ " 'total_claim_amount': 'mean'}).reset_index()\n",
+ "\n",
+ "agg_results\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "id": "2c32e5a7-72e9-4d23-981d-d1b71ddc1dcd",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "b6aec097-c633-4017-a125-e77a97259cda",
- "metadata": {
- "id": "b6aec097-c633-4017-a125-e77a97259cda"
- },
- "source": [
- "6. Display a new DataFrame that contains the number of policies sold by month, by state, for the top 3 states with the highest number of policies sold.\n",
- "\n",
- "*Hint:*\n",
- "- *To accomplish this, you will first need to group the data by state and month, then count the number of policies sold for each group. Afterwards, you will need to sort the data by the count of policies sold in descending order.*\n",
- "- *Next, you will select the top 3 states with the highest number of policies sold.*\n",
- "- *Finally, you will create a new DataFrame that contains the number of policies sold by month for each of the top 3 states.*"
+ "data": {
+ "text/plain": [
+ "state\n",
+ "California 3552\n",
+ "Oregon 2909\n",
+ "Arizona 1937\n",
+ "Nevada 993\n",
+ "Washington 888\n",
+ "Unknown 631\n",
+ "Name: count, dtype: int64"
]
- },
+ },
+ "execution_count": 14,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# 3 total number of customers in each state, and only include states where there are more than 500 customers.\n",
+ "\n",
+ "state_counts = df['state'].value_counts()\n",
+ "big_states = state_counts[state_counts > 500]\n",
+ "\n",
+ "big_states\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "id": "b45316d1-0e30-4c68-892c-4d8e0ab5b6c1",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "markdown",
- "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009",
- "metadata": {
- "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009"
- },
- "source": [
- "7. The marketing team wants to analyze the effect of different marketing channels on the customer response rate.\n",
- "\n",
- "Hint: You can use melt to unpivot the data and create a table that shows the customer response rate (those who responded \"Yes\") by marketing channel."
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " education | \n",
+ " gender | \n",
+ " max | \n",
+ " min | \n",
+ " median | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 | \n",
+ " Bachelor | \n",
+ " F | \n",
+ " 73225.95652 | \n",
+ " 1904.000852 | \n",
+ " 5640.505303 | \n",
+ "
\n",
+ " \n",
+ " 1 | \n",
+ " Bachelor | \n",
+ " M | \n",
+ " 67907.27050 | \n",
+ " 1898.007675 | \n",
+ " 5548.031892 | \n",
+ "
\n",
+ " \n",
+ " 2 | \n",
+ " College | \n",
+ " F | \n",
+ " 61850.18803 | \n",
+ " 1898.683686 | \n",
+ " 5623.611187 | \n",
+ "
\n",
+ " \n",
+ " 3 | \n",
+ " College | \n",
+ " M | \n",
+ " 61134.68307 | \n",
+ " 1918.119700 | \n",
+ " 6005.847375 | \n",
+ "
\n",
+ " \n",
+ " 4 | \n",
+ " Doctor | \n",
+ " F | \n",
+ " 44856.11397 | \n",
+ " 2395.570000 | \n",
+ " 5332.462694 | \n",
+ "
\n",
+ " \n",
+ " 5 | \n",
+ " Doctor | \n",
+ " M | \n",
+ " 32677.34284 | \n",
+ " 2267.604038 | \n",
+ " 5577.669457 | \n",
+ "
\n",
+ " \n",
+ " 6 | \n",
+ " High School or Below | \n",
+ " F | \n",
+ " 55277.44589 | \n",
+ " 2144.921535 | \n",
+ " 6039.553187 | \n",
+ "
\n",
+ " \n",
+ " 7 | \n",
+ " High School or Below | \n",
+ " M | \n",
+ " 83325.38119 | \n",
+ " 1940.981221 | \n",
+ " 6286.731006 | \n",
+ "
\n",
+ " \n",
+ " 8 | \n",
+ " Master | \n",
+ " F | \n",
+ " 51016.06704 | \n",
+ " 2417.777032 | \n",
+ " 5729.855012 | \n",
+ "
\n",
+ " \n",
+ " 9 | \n",
+ " Master | \n",
+ " M | \n",
+ " 50568.25912 | \n",
+ " 2272.307310 | \n",
+ " 5579.099207 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " education gender max min median\n",
+ "0 Bachelor F 73225.95652 1904.000852 5640.505303\n",
+ "1 Bachelor M 67907.27050 1898.007675 5548.031892\n",
+ "2 College F 61850.18803 1898.683686 5623.611187\n",
+ "3 College M 61134.68307 1918.119700 6005.847375\n",
+ "4 Doctor F 44856.11397 2395.570000 5332.462694\n",
+ "5 Doctor M 32677.34284 2267.604038 5577.669457\n",
+ "6 High School or Below F 55277.44589 2144.921535 6039.553187\n",
+ "7 High School or Below M 83325.38119 1940.981221 6286.731006\n",
+ "8 Master F 51016.06704 2417.777032 5729.855012\n",
+ "9 Master M 50568.25912 2272.307310 5579.099207"
]
+ },
+ "execution_count": 15,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Find the maximum, minimum, and median customer lifetime value by education level and gender\n",
+ "clv_stats = df.groupby(['education', 'gender'])['customer_lifetime_value'].agg(['max', 'min', 'median']).reset_index()\n",
+ "\n",
+ "clv_stats\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "240a775a-0193-45ff-a372-5d854b5aa638",
+ "metadata": {},
+ "source": [
+ "# Conclusions\n",
+ "\n",
+ "Median CLV is similar across genders (~5.3k–6.2k).\n",
+ "\n",
+ "“High School or Below” shows the highest max CLV (~83k).\n",
+ "\n",
+ "Doctoral customers have lower max CLV compared to other groups.\n",
+ "\n",
+ "Education level seems more influential on CLV than gender."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "id": "3254ce84-7c84-45a9-b937-a2ddcc01bbca",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "C:\\Users\\Gustavo\\AppData\\Local\\Temp\\ipykernel_6112\\345119802.py:6: UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.\n",
+ " df['effective_to_date'] = pd.to_datetime(df['effective_to_date'])\n"
+ ]
},
{
- "cell_type": "markdown",
- "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d",
- "metadata": {
- "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d"
- },
- "source": [
- "External Resources for Data Filtering: https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9"
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " month | \n",
+ " 1 | \n",
+ " 2 | \n",
+ "
\n",
+ " \n",
+ " state | \n",
+ " | \n",
+ " | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " Arizona | \n",
+ " 1008 | \n",
+ " 929 | \n",
+ "
\n",
+ " \n",
+ " California | \n",
+ " 1918 | \n",
+ " 1634 | \n",
+ "
\n",
+ " \n",
+ " Nevada | \n",
+ " 551 | \n",
+ " 442 | \n",
+ "
\n",
+ " \n",
+ " Oregon | \n",
+ " 1565 | \n",
+ " 1344 | \n",
+ "
\n",
+ " \n",
+ " Unknown | \n",
+ " 313 | \n",
+ " 318 | \n",
+ "
\n",
+ " \n",
+ " Washington | \n",
+ " 463 | \n",
+ " 425 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ "month 1 2\n",
+ "state \n",
+ "Arizona 1008 929\n",
+ "California 1918 1634\n",
+ "Nevada 551 442\n",
+ "Oregon 1565 1344\n",
+ "Unknown 313 318\n",
+ "Washington 463 425"
]
- },
+ },
+ "execution_count": 16,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# 5 number of policies sold by state and month.\n",
+ "# Rows = state\n",
+ "# Columns = month\n",
+ "# Values = quantity policy_number (or clients)\n",
+ "# Make sure that 'effective_to_date' is Date type\n",
+ "df['effective_to_date'] = pd.to_datetime(df['effective_to_date'])\n",
+ "\n",
+ "# Take the month\n",
+ "df['month'] = df['effective_to_date'].dt.month\n",
+ "\n",
+ "# Create the DF\n",
+ "policies_state_month = pd.crosstab(df['state'], df['month'])\n",
+ "\n",
+ "policies_state_month\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "id": "cf5c3866-ee9e-4e99-b77c-7cc6ea5a9c83",
+ "metadata": {},
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "id": "449513f4-0459-46a0-a18d-9398d974c9ad",
- "metadata": {
- "id": "449513f4-0459-46a0-a18d-9398d974c9ad"
- },
- "outputs": [],
- "source": [
- "# your code goes here"
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " month | \n",
+ " 1 | \n",
+ " 2 | \n",
+ "
\n",
+ " \n",
+ " state | \n",
+ " | \n",
+ " | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " Arizona | \n",
+ " 1008 | \n",
+ " 929 | \n",
+ "
\n",
+ " \n",
+ " California | \n",
+ " 1918 | \n",
+ " 1634 | \n",
+ "
\n",
+ " \n",
+ " Oregon | \n",
+ " 1565 | \n",
+ " 1344 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ "month 1 2\n",
+ "state \n",
+ "Arizona 1008 929\n",
+ "California 1918 1634\n",
+ "Oregon 1565 1344"
]
+ },
+ "execution_count": 17,
+ "metadata": {},
+ "output_type": "execute_result"
}
- ],
- "metadata": {
- "colab": {
- "provenance": []
- },
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.13"
+ ],
+ "source": [
+ "# 6. number of policies sold by month, by state, for the top 3 states\n",
+ "# Count total of policies por state\n",
+ "state_counts = df['state'].value_counts().head(3)\n",
+ "\n",
+ "# Filter dataset \n",
+ "top3_states = df[df['state'].isin(state_counts.index)]\n",
+ "\n",
+ "# Count policies sold per state and month\n",
+ "top3_policies = top3_states.groupby(['state','month']).size().unstack()\n",
+ "\n",
+ "top3_policies\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "id": "36c4c8d6-d0cb-4ee7-837f-09a0a95e9383",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " response | \n",
+ " No | \n",
+ " Yes | \n",
+ " response_rate_yes | \n",
+ "
\n",
+ " \n",
+ " sales_channel | \n",
+ " | \n",
+ " | \n",
+ " | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " Agent | \n",
+ " 3148 | \n",
+ " 742 | \n",
+ " 0.190746 | \n",
+ "
\n",
+ " \n",
+ " Branch | \n",
+ " 2539 | \n",
+ " 326 | \n",
+ " 0.113787 | \n",
+ "
\n",
+ " \n",
+ " Call Center | \n",
+ " 1792 | \n",
+ " 221 | \n",
+ " 0.109786 | \n",
+ "
\n",
+ " \n",
+ " Web | \n",
+ " 1334 | \n",
+ " 177 | \n",
+ " 0.117141 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ "response No Yes response_rate_yes\n",
+ "sales_channel \n",
+ "Agent 3148 742 0.190746\n",
+ "Branch 2539 326 0.113787\n",
+ "Call Center 1792 221 0.109786\n",
+ "Web 1334 177 0.117141"
+ ]
+ },
+ "execution_count": 19,
+ "metadata": {},
+ "output_type": "execute_result"
}
+ ],
+ "source": [
+ "# 7 effect of different marketing channels on the customer response rate.\n",
+ "# Calculate answers per channel\n",
+ "channel_response = df.groupby(['sales_channel','response']).size().unstack(fill_value=0)\n",
+ "\n",
+ "# Calculte rate of \"Yes\"\n",
+ "channel_response['response_rate_yes'] = channel_response['Yes'] / channel_response.sum(axis=1)\n",
+ "\n",
+ "channel_response\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "17bef4c4-392c-430a-9919-d1294bdbc695",
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
},
- "nbformat": 4,
- "nbformat_minor": 5
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.13.5"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
}