From 6c30458073701db25854735d75bf39d70bdc7abf Mon Sep 17 00:00:00 2001 From: Ricardo Castanheira Date: Fri, 12 Sep 2025 14:37:39 +0100 Subject: [PATCH] Solved lab v2 --- lab-dw-aggregating.ipynb | 1326 ++++++++++++++++++++++++++++++++++---- 1 file changed, 1183 insertions(+), 143 deletions(-) diff --git a/lab-dw-aggregating.ipynb b/lab-dw-aggregating.ipynb index fadd718..3406622 100644 --- a/lab-dw-aggregating.ipynb +++ b/lab-dw-aggregating.ipynb @@ -1,165 +1,1205 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "31969215-2a90-4d8b-ac36-646a7ae13744", - "metadata": { - "id": "31969215-2a90-4d8b-ac36-646a7ae13744" - }, - "source": [ - "# Lab | Data Aggregation and Filtering" - ] - }, - { - "cell_type": "markdown", - "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d", - "metadata": { - "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d" - }, - "source": [ - "In this challenge, we will continue to work with customer data from an insurance company. We will use the dataset called marketing_customer_analysis.csv, which can be found at the following link:\n", - "\n", - "https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\n", - "\n", - "This dataset contains information such as customer demographics, policy details, vehicle information, and the customer's response to the last marketing campaign. Our goal is to explore and analyze this data by first performing data cleaning, formatting, and structuring." - ] - }, + "cells": [ + { + "cell_type": "markdown", + "id": "31969215-2a90-4d8b-ac36-646a7ae13744", + "metadata": { + "id": "31969215-2a90-4d8b-ac36-646a7ae13744" + }, + "source": [ + "# Lab | Data Aggregation and Filtering" + ] + }, + { + "cell_type": "markdown", + "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d", + "metadata": { + "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d" + }, + "source": [ + "In this challenge, we will continue to work with customer data from an insurance company. We will use the dataset called marketing_customer_analysis.csv, which can be found at the following link:\n", + "\n", + "https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\n", + "\n", + "This dataset contains information such as customer demographics, policy details, vehicle information, and the customer's response to the last marketing campaign. Our goal is to explore and analyze this data by first performing data cleaning, formatting, and structuring." + ] + }, + { + "cell_type": "markdown", + "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50", + "metadata": { + "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50" + }, + "source": [ + "1. Create a new DataFrame that only includes customers who:\n", + " - have a **low total_claim_amount** (e.g., below $1,000),\n", + " - have a response \"Yes\" to the last marketing campaign." + ] + }, + { + "cell_type": "markdown", + "id": "b9be383e-5165-436e-80c8-57d4c757c8c3", + "metadata": { + "id": "b9be383e-5165-436e-80c8-57d4c757c8c3" + }, + "source": [ + "2. Using the original Dataframe, analyze:\n", + " - the average `monthly_premium` and/or customer lifetime value by `policy_type` and `gender` for customers who responded \"Yes\", and\n", + " - compare these insights to `total_claim_amount` patterns, and discuss which segments appear most profitable or low-risk for the company." + ] + }, + { + "cell_type": "markdown", + "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0", + "metadata": { + "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0" + }, + "source": [ + "3. Analyze the total number of customers who have policies in each state, and then filter the results to only include states where there are more than 500 customers." + ] + }, + { + "cell_type": "markdown", + "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d", + "metadata": { + "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d" + }, + "source": [ + "4. Find the maximum, minimum, and median customer lifetime value by education level and gender. Write your conclusions." + ] + }, + { + "cell_type": "markdown", + "id": "b42999f9-311f-481e-ae63-40a5577072c5", + "metadata": { + "id": "b42999f9-311f-481e-ae63-40a5577072c5" + }, + "source": [ + "## Bonus" + ] + }, + { + "cell_type": "markdown", + "id": "81ff02c5-6584-4f21-a358-b918697c6432", + "metadata": { + "id": "81ff02c5-6584-4f21-a358-b918697c6432" + }, + "source": [ + "5. The marketing team wants to analyze the number of policies sold by state and month. Present the data in a table where the months are arranged as columns and the states are arranged as rows." + ] + }, + { + "cell_type": "markdown", + "id": "b6aec097-c633-4017-a125-e77a97259cda", + "metadata": { + "id": "b6aec097-c633-4017-a125-e77a97259cda" + }, + "source": [ + "6. Display a new DataFrame that contains the number of policies sold by month, by state, for the top 3 states with the highest number of policies sold.\n", + "\n", + "*Hint:*\n", + "- *To accomplish this, you will first need to group the data by state and month, then count the number of policies sold for each group. Afterwards, you will need to sort the data by the count of policies sold in descending order.*\n", + "- *Next, you will select the top 3 states with the highest number of policies sold.*\n", + "- *Finally, you will create a new DataFrame that contains the number of policies sold by month for each of the top 3 states.*" + ] + }, + { + "cell_type": "markdown", + "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009", + "metadata": { + "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009" + }, + "source": [ + "7. The marketing team wants to analyze the effect of different marketing channels on the customer response rate.\n", + "\n", + "Hint: You can use melt to unpivot the data and create a table that shows the customer response rate (those who responded \"Yes\") by marketing channel." + ] + }, + { + "cell_type": "markdown", + "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d", + "metadata": { + "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d" + }, + "source": [ + "External Resources for Data Filtering: https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "449513f4-0459-46a0-a18d-9398d974c9ad", + "metadata": { + "id": "449513f4-0459-46a0-a18d-9398d974c9ad" + }, + "outputs": [], + "source": [ + "import pandas as pd\n", + "\n", + "url = \"https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\"\n", + "\n", + "data = pd.read_csv(url)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "1494426d-6b48-4a65-9bb9-663bd38fbbb8", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50", - "metadata": { - "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50" - }, - "source": [ - "1. Create a new DataFrame that only includes customers who:\n", - " - have a **low total_claim_amount** (e.g., below $1,000),\n", - " - have a response \"Yes\" to the last marketing campaign." + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Unnamed: 0CustomerStateCustomer Lifetime ValueResponseCoverageEducationEffective To DateEmploymentStatusGender...Number of Open ComplaintsNumber of PoliciesPolicy TypePolicyRenew Offer TypeSales ChannelTotal Claim AmountVehicle ClassVehicle SizeVehicle Type
00DK49336Arizona4809.216960NoBasicCollege2/18/11EmployedM...0.09Corporate AutoCorporate L3Offer3Agent292.800000Four-Door CarMedsizeNaN
11KX64629California2228.525238NoBasicCollege1/18/11UnemployedF...0.01Personal AutoPersonal L3Offer4Call Center744.924331Four-Door CarMedsizeNaN
22LZ68649Washington14947.917300NoBasicBachelor2/10/11EmployedM...0.02Personal AutoPersonal L3Offer3Call Center480.000000SUVMedsizeA
33XL78013Oregon22332.439460YesExtendedCollege1/11/11EmployedM...0.02Corporate AutoCorporate L3Offer2Branch484.013411Four-Door CarMedsizeA
44QA50777Oregon9025.067525NoPremiumBachelor1/17/11Medical LeaveF...NaN7Personal AutoPersonal L2Offer1Branch707.925645Four-Door CarMedsizeNaN
\n", + "

5 rows × 26 columns

\n", + "
" + ], + "text/plain": [ + " Unnamed: 0 Customer State Customer Lifetime Value Response \\\n", + "0 0 DK49336 Arizona 4809.216960 No \n", + "1 1 KX64629 California 2228.525238 No \n", + "2 2 LZ68649 Washington 14947.917300 No \n", + "3 3 XL78013 Oregon 22332.439460 Yes \n", + "4 4 QA50777 Oregon 9025.067525 No \n", + "\n", + " Coverage Education Effective To Date EmploymentStatus Gender ... \\\n", + "0 Basic College 2/18/11 Employed M ... \n", + "1 Basic College 1/18/11 Unemployed F ... \n", + "2 Basic Bachelor 2/10/11 Employed M ... \n", + "3 Extended College 1/11/11 Employed M ... \n", + "4 Premium Bachelor 1/17/11 Medical Leave F ... \n", + "\n", + " Number of Open Complaints Number of Policies Policy Type Policy \\\n", + "0 0.0 9 Corporate Auto Corporate L3 \n", + "1 0.0 1 Personal Auto Personal L3 \n", + "2 0.0 2 Personal Auto Personal L3 \n", + "3 0.0 2 Corporate Auto Corporate L3 \n", + "4 NaN 7 Personal Auto Personal L2 \n", + "\n", + " Renew Offer Type Sales Channel Total Claim Amount Vehicle Class \\\n", + "0 Offer3 Agent 292.800000 Four-Door Car \n", + "1 Offer4 Call Center 744.924331 Four-Door Car \n", + "2 Offer3 Call Center 480.000000 SUV \n", + "3 Offer2 Branch 484.013411 Four-Door Car \n", + "4 Offer1 Branch 707.925645 Four-Door Car \n", + "\n", + " Vehicle Size Vehicle Type \n", + "0 Medsize NaN \n", + "1 Medsize NaN \n", + "2 Medsize A \n", + "3 Medsize A \n", + "4 Medsize NaN \n", + "\n", + "[5 rows x 26 columns]" ] - }, + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "d5b53445-3707-468a-8a85-2eae107e14bd", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "b9be383e-5165-436e-80c8-57d4c757c8c3", - "metadata": { - "id": "b9be383e-5165-436e-80c8-57d4c757c8c3" - }, - "source": [ - "2. Using the original Dataframe, analyze:\n", - " - the average `monthly_premium` and/or customer lifetime value by `policy_type` and `gender` for customers who responded \"Yes\", and\n", - " - compare these insights to `total_claim_amount` patterns, and discuss which segments appear most profitable or low-risk for the company." - ] - }, + "name": "stdout", + "output_type": "stream", + "text": [ + "Unnamed: 0 False\n", + "Customer False\n", + "State True\n", + "Customer Lifetime Value False\n", + "Response True\n", + "Coverage False\n", + "Education False\n", + "Effective To Date False\n", + "EmploymentStatus False\n", + "Gender False\n", + "Income False\n", + "Location Code False\n", + "Marital Status False\n", + "Monthly Premium Auto False\n", + "Months Since Last Claim True\n", + "Months Since Policy Inception False\n", + "Number of Open Complaints True\n", + "Number of Policies False\n", + "Policy Type False\n", + "Policy False\n", + "Renew Offer Type False\n", + "Sales Channel False\n", + "Total Claim Amount False\n", + "Vehicle Class True\n", + "Vehicle Size True\n", + "Vehicle Type True\n", + "dtype: bool\n" + ] + } + ], + "source": [ + "null_columns = data.isnull().any()\n", + "print(null_columns)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "518a02ce-9964-4264-90ed-4e83a57409de", + "metadata": {}, + "outputs": [], + "source": [ + "null_columns = [\"Response\", \"Months Since Last Claim\", \"Number of Open Complaints\", \"Vehicle Class\", \"Vehicle Size\", \"Vehicle Type\"]\n", + "\n", + "data[null_columns] = data[null_columns].fillna(\"Unknown\")" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "1b260489-6ab7-4b1f-bc1b-455e357b02f6", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0", - "metadata": { - "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0" - }, - "source": [ - "3. Analyze the total number of customers who have policies in each state, and then filter the results to only include states where there are more than 500 customers." + "data": { + "text/plain": [ + "np.False_" ] - }, + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data.duplicated().any()" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "8408c5b6-5a86-42da-951f-4b6d4f984e76", + "metadata": {}, + "outputs": [], + "source": [ + "data[\"Customer Lifetime Value\"] = data[\"Customer Lifetime Value\"].round(1)\n", + "data[\"Total Claim Amount\"] = data[\"Customer Lifetime Value\"].round(1)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "560f622f-bc6f-461b-ad93-72f4f220e701", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d", - "metadata": { - "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d" - }, - "source": [ - "4. Find the maximum, minimum, and median customer lifetime value by education level and gender. Write your conclusions." + "data": { + "text/plain": [ + "Index(['Unnamed: 0', 'Customer', 'State', 'Customer Lifetime Value',\n", + " 'Response', 'Coverage', 'Education', 'Effective To Date',\n", + " 'EmploymentStatus', 'Gender', 'Income', 'Location Code',\n", + " 'Marital Status', 'Monthly Premium Auto', 'Months Since Last Claim',\n", + " 'Months Since Policy Inception', 'Number of Open Complaints',\n", + " 'Number of Policies', 'Policy Type', 'Policy', 'Renew Offer Type',\n", + " 'Sales Channel', 'Total Claim Amount', 'Vehicle Class', 'Vehicle Size',\n", + " 'Vehicle Type'],\n", + " dtype='object')" ] - }, + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data.columns" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "71faccc1-bdcf-4c09-b78a-728e8de59308", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "b42999f9-311f-481e-ae63-40a5577072c5", - "metadata": { - "id": "b42999f9-311f-481e-ae63-40a5577072c5" - }, - "source": [ - "## Bonus" + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Total Claim AmountResponse
652300.7Yes
692867.3Yes
1022471.5Yes
1712471.5Yes
1832674.7Yes
.........
107792580.8Yes
108002674.7Yes
108042725.4Yes
108182300.7Yes
108402453.6Yes
\n", + "

232 rows × 2 columns

\n", + "
" + ], + "text/plain": [ + " Total Claim Amount Response\n", + "65 2300.7 Yes\n", + "69 2867.3 Yes\n", + "102 2471.5 Yes\n", + "171 2471.5 Yes\n", + "183 2674.7 Yes\n", + "... ... ...\n", + "10779 2580.8 Yes\n", + "10800 2674.7 Yes\n", + "10804 2725.4 Yes\n", + "10818 2300.7 Yes\n", + "10840 2453.6 Yes\n", + "\n", + "[232 rows x 2 columns]" ] - }, + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Create a new DataFrame that only includes customers who:\n", + "# have a low total_claim_amount (e.g., below $1,000),\n", + "# have a response \"Yes\" to the last marketing campaign.\n", + "\n", + "same_columns = [\"Total Claim Amount\", \"Response\"]\n", + "\n", + "new_data = data[same_columns].copy()\n", + "\n", + "new_data = new_data[(new_data[\"Total Claim Amount\"] < 3000) & (new_data[\"Response\"] == \"Yes\")]\n", + "new_data" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "d58829f2-c9ef-414b-ae9c-3f544826515a", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "81ff02c5-6584-4f21-a358-b918697c6432", - "metadata": { - "id": "81ff02c5-6584-4f21-a358-b918697c6432" - }, - "source": [ - "5. The marketing team wants to analyze the number of policies sold by state and month. Present the data in a table where the months are arranged as columns and the states are arranged as rows." + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Customer Lifetime ValueMonthly Premium AutoResponseGenderPolicy Type
04809.261NoMCorporate Auto
12228.564NoFPersonal Auto
214947.9100NoMPersonal Auto
322332.497YesMCorporate Auto
49025.1117NoFPersonal Auto
..................
1090515563.4253NoFPersonal Auto
109065259.465NoFPersonal Auto
1090723893.3201NoFCorporate Auto
1090811972.0158NoFPersonal Auto
109096857.5101UnknownMPersonal Auto
\n", + "

10910 rows × 5 columns

\n", + "
" + ], + "text/plain": [ + " Customer Lifetime Value Monthly Premium Auto Response Gender \\\n", + "0 4809.2 61 No M \n", + "1 2228.5 64 No F \n", + "2 14947.9 100 No M \n", + "3 22332.4 97 Yes M \n", + "4 9025.1 117 No F \n", + "... ... ... ... ... \n", + "10905 15563.4 253 No F \n", + "10906 5259.4 65 No F \n", + "10907 23893.3 201 No F \n", + "10908 11972.0 158 No F \n", + "10909 6857.5 101 Unknown M \n", + "\n", + " Policy Type \n", + "0 Corporate Auto \n", + "1 Personal Auto \n", + "2 Personal Auto \n", + "3 Corporate Auto \n", + "4 Personal Auto \n", + "... ... \n", + "10905 Personal Auto \n", + "10906 Personal Auto \n", + "10907 Corporate Auto \n", + "10908 Personal Auto \n", + "10909 Personal Auto \n", + "\n", + "[10910 rows x 5 columns]" ] - }, + }, + "execution_count": 26, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Using the original Dataframe, analyze:\n", + "# the average monthly_premium and/or customer lifetime value by policy_type and gender for customers who responded \"Yes\"\n", + "\n", + "to_analyze = [\"Customer Lifetime Value\", \"Monthly Premium Auto\", \"Response\", \"Gender\", \"Policy Type\"]\n", + "\n", + "new_data2 = data[to_analyze].copy()\n", + "new_data2" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "id": "d27a45c1-102d-49ff-9bcf-c31f3c59929f", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "b6aec097-c633-4017-a125-e77a97259cda", - "metadata": { - "id": "b6aec097-c633-4017-a125-e77a97259cda" - }, - "source": [ - "6. Display a new DataFrame that contains the number of policies sold by month, by state, for the top 3 states with the highest number of policies sold.\n", - "\n", - "*Hint:*\n", - "- *To accomplish this, you will first need to group the data by state and month, then count the number of policies sold for each group. Afterwards, you will need to sort the data by the count of policies sold in descending order.*\n", - "- *Next, you will select the top 3 states with the highest number of policies sold.*\n", - "- *Finally, you will create a new DataFrame that contains the number of policies sold by month for each of the top 3 states.*" + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Policy TypeGenderMonthly Premium AutoCustomer Lifetime Value
0Corporate AutoF94.3017757712.625444
1Corporate AutoM92.1883127944.459091
2Personal AutoF98.9981488339.794444
3Personal AutoM91.0858217448.378731
4Special AutoF92.3142867691.591429
5Special AutoM86.3437508247.081250
\n", + "
" + ], + "text/plain": [ + " Policy Type Gender Monthly Premium Auto Customer Lifetime Value\n", + "0 Corporate Auto F 94.301775 7712.625444\n", + "1 Corporate Auto M 92.188312 7944.459091\n", + "2 Personal Auto F 98.998148 8339.794444\n", + "3 Personal Auto M 91.085821 7448.378731\n", + "4 Special Auto F 92.314286 7691.591429\n", + "5 Special Auto M 86.343750 8247.081250" ] - }, + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "new_data2_yes = new_data2[new_data2[\"Response\"] == \"Yes\"]\n", + "\n", + "result = (new_data2_yes.groupby([\"Policy Type\", \"Gender\"])[[\"Monthly Premium Auto\", \"Customer Lifetime Value\"]].mean().reset_index())\n", + "display(result)" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "id": "f6cb28fb-21ca-4fbe-9d29-95c11a3e7d1c", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009", - "metadata": { - "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009" - }, - "source": [ - "7. The marketing team wants to analyze the effect of different marketing channels on the customer response rate.\n", - "\n", - "Hint: You can use melt to unpivot the data and create a table that shows the customer response rate (those who responded \"Yes\") by marketing channel." + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Total Claim Amount
count1466.00000
mean7882.54618
std6314.25410
min2004.40000
25%3954.30000
50%5597.10000
75%9075.80000
max41787.90000
\n", + "
" + ], + "text/plain": [ + " Total Claim Amount\n", + "count 1466.00000\n", + "mean 7882.54618\n", + "std 6314.25410\n", + "min 2004.40000\n", + "25% 3954.30000\n", + "50% 5597.10000\n", + "75% 9075.80000\n", + "max 41787.90000" ] - }, + }, + "execution_count": 49, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# compare these insights to total_claim_amount patterns, and discuss which segments appear most profitable or low-risk for the company.\n", + "\n", + "comparison = [\"Total Claim Amount\", \"Response\", \"Gender\", \"Policy Type\"]\n", + "\n", + "data_comparison = data[comparison].copy()\n", + "comparison_yes = data_comparison[data_comparison[\"Response\"] == \"Yes\"]\n", + "comparison_yes.describe() # When comparing results, we can see the average claim amount for clients that match the criteria is 7882.\n", + "# From this information, the most profitable policy type is Special Auto but it is almost identical to Personal Auto." + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "id": "da707b55-c68f-46e1-ae81-5baf717892ac", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d", - "metadata": { - "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d" - }, - "source": [ - "External Resources for Data Filtering: https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9" - ] - }, + "name": "stdout", + "output_type": "stream", + "text": [ + "State\n", + "California 3552\n", + "Oregon 2909\n", + "Arizona 1937\n", + "Nevada 993\n", + "Washington 888\n", + "Name: count, dtype: int64\n" + ] + } + ], + "source": [ + "# Analyze the total number of customers who have policies in each state, \n", + "# and then filter the results to only include states where there are more than 500 customers.\n", + "\n", + "state_data = [\"State\"]\n", + "new_state_data = data[state_data].copy()\n", + "\n", + "state_counts = new_state_data[\"State\"].value_counts()\n", + "print(state_counts)\n", + "\n", + "#All states have over 500 customers" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "id": "e5df0345-5dd3-4bc9-a9ef-f037aeef5dcb", + "metadata": {}, + "outputs": [ { - "cell_type": "code", - "execution_count": null, - "id": "449513f4-0459-46a0-a18d-9398d974c9ad", - "metadata": { - "id": "449513f4-0459-46a0-a18d-9398d974c9ad" - }, - "outputs": [], - "source": [ - "# your code goes here" + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
EducationGenderminmaxmedian
0BachelorF1904.073226.05640.50
1BachelorM1898.067907.35548.00
2CollegeF1898.761850.25623.60
3CollegeM1918.161134.76005.80
4DoctorF2395.644856.15332.50
5DoctorM2267.632677.35577.70
6High School or BelowF2144.955277.46039.55
7High School or BelowM1941.083325.46286.75
8MasterF2417.851016.15729.90
9MasterM2272.350568.35579.10
\n", + "
" + ], + "text/plain": [ + " Education Gender min max median\n", + "0 Bachelor F 1904.0 73226.0 5640.50\n", + "1 Bachelor M 1898.0 67907.3 5548.00\n", + "2 College F 1898.7 61850.2 5623.60\n", + "3 College M 1918.1 61134.7 6005.80\n", + "4 Doctor F 2395.6 44856.1 5332.50\n", + "5 Doctor M 2267.6 32677.3 5577.70\n", + "6 High School or Below F 2144.9 55277.4 6039.55\n", + "7 High School or Below M 1941.0 83325.4 6286.75\n", + "8 Master F 2417.8 51016.1 5729.90\n", + "9 Master M 2272.3 50568.3 5579.10" ] + }, + "metadata": {}, + "output_type": "display_data" } - ], - "metadata": { - "colab": { - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.13" - } + ], + "source": [ + "# Find the maximum, minimum, and median customer lifetime value by education level and gender. Write your conclusions.\n", + "\n", + "for_statistics = [\"Customer Lifetime Value\", \"Education\", \"Gender\"]\n", + "\n", + "statistics_data = data[for_statistics].copy()\n", + "statistics_data\n", + "\n", + "summary = (\n", + " statistics_data.groupby([\"Education\", \"Gender\"])[\"Customer Lifetime Value\"]\n", + " .agg([\"min\", \"max\", \"median\"])\n", + " .reset_index()\n", + ")\n", + "\n", + "display(summary)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "68d3946a-6416-48cc-b940-0086f6821334", + "metadata": {}, + "outputs": [], + "source": [ + "# Clients with Masters or Doctor degrees normally pay higher minimum premiums, this may be because their income is higher therefore the car\n", + "# category higher and more expensive." + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python [conda env:base] *", + "language": "python", + "name": "conda-base-py" }, - "nbformat": 4, - "nbformat_minor": 5 + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 }