From 449b1dc7ce98beb56bb18f610c31d611b47c5272 Mon Sep 17 00:00:00 2001 From: Patricia Viladomiu Date: Mon, 1 Dec 2025 11:18:16 +0100 Subject: [PATCH] Lab Solved --- lab-dw-aggregating.ipynb | 1704 ++++++++++++++++++++++++++++++++++---- 1 file changed, 1541 insertions(+), 163 deletions(-) diff --git a/lab-dw-aggregating.ipynb b/lab-dw-aggregating.ipynb index fadd718..5a896c2 100644 --- a/lab-dw-aggregating.ipynb +++ b/lab-dw-aggregating.ipynb @@ -1,165 +1,1543 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "31969215-2a90-4d8b-ac36-646a7ae13744", - "metadata": { - "id": "31969215-2a90-4d8b-ac36-646a7ae13744" - }, - "source": [ - "# Lab | Data Aggregation and Filtering" - ] - }, - { - "cell_type": "markdown", - "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d", - "metadata": { - "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d" - }, - "source": [ - "In this challenge, we will continue to work with customer data from an insurance company. We will use the dataset called marketing_customer_analysis.csv, which can be found at the following link:\n", - "\n", - "https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\n", - "\n", - "This dataset contains information such as customer demographics, policy details, vehicle information, and the customer's response to the last marketing campaign. Our goal is to explore and analyze this data by first performing data cleaning, formatting, and structuring." - ] - }, - { - "cell_type": "markdown", - "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50", - "metadata": { - "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50" - }, - "source": [ - "1. Create a new DataFrame that only includes customers who:\n", - " - have a **low total_claim_amount** (e.g., below $1,000),\n", - " - have a response \"Yes\" to the last marketing campaign." - ] - }, - { - "cell_type": "markdown", - "id": "b9be383e-5165-436e-80c8-57d4c757c8c3", - "metadata": { - "id": "b9be383e-5165-436e-80c8-57d4c757c8c3" - }, - "source": [ - "2. Using the original Dataframe, analyze:\n", - " - the average `monthly_premium` and/or customer lifetime value by `policy_type` and `gender` for customers who responded \"Yes\", and\n", - " - compare these insights to `total_claim_amount` patterns, and discuss which segments appear most profitable or low-risk for the company." - ] - }, - { - "cell_type": "markdown", - "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0", - "metadata": { - "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0" - }, - "source": [ - "3. Analyze the total number of customers who have policies in each state, and then filter the results to only include states where there are more than 500 customers." - ] - }, - { - "cell_type": "markdown", - "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d", - "metadata": { - "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d" - }, - "source": [ - "4. Find the maximum, minimum, and median customer lifetime value by education level and gender. Write your conclusions." - ] - }, - { - "cell_type": "markdown", - "id": "b42999f9-311f-481e-ae63-40a5577072c5", - "metadata": { - "id": "b42999f9-311f-481e-ae63-40a5577072c5" - }, - "source": [ - "## Bonus" - ] - }, - { - "cell_type": "markdown", - "id": "81ff02c5-6584-4f21-a358-b918697c6432", - "metadata": { - "id": "81ff02c5-6584-4f21-a358-b918697c6432" - }, - "source": [ - "5. The marketing team wants to analyze the number of policies sold by state and month. Present the data in a table where the months are arranged as columns and the states are arranged as rows." - ] - }, - { - "cell_type": "markdown", - "id": "b6aec097-c633-4017-a125-e77a97259cda", - "metadata": { - "id": "b6aec097-c633-4017-a125-e77a97259cda" - }, - "source": [ - "6. Display a new DataFrame that contains the number of policies sold by month, by state, for the top 3 states with the highest number of policies sold.\n", - "\n", - "*Hint:*\n", - "- *To accomplish this, you will first need to group the data by state and month, then count the number of policies sold for each group. Afterwards, you will need to sort the data by the count of policies sold in descending order.*\n", - "- *Next, you will select the top 3 states with the highest number of policies sold.*\n", - "- *Finally, you will create a new DataFrame that contains the number of policies sold by month for each of the top 3 states.*" - ] - }, - { - "cell_type": "markdown", - "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009", - "metadata": { - "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009" - }, - "source": [ - "7. The marketing team wants to analyze the effect of different marketing channels on the customer response rate.\n", - "\n", - "Hint: You can use melt to unpivot the data and create a table that shows the customer response rate (those who responded \"Yes\") by marketing channel." - ] - }, - { - "cell_type": "markdown", - "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d", - "metadata": { - "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d" - }, - "source": [ - "External Resources for Data Filtering: https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "449513f4-0459-46a0-a18d-9398d974c9ad", - "metadata": { - "id": "449513f4-0459-46a0-a18d-9398d974c9ad" - }, - "outputs": [], - "source": [ - "# your code goes here" - ] - } - ], - "metadata": { - "colab": { - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.13" - } - }, - "nbformat": 4, - "nbformat_minor": 5 + "cells": [ + { + "cell_type": "markdown", + "id": "31969215-2a90-4d8b-ac36-646a7ae13744", + "metadata": { + "id": "31969215-2a90-4d8b-ac36-646a7ae13744" + }, + "source": [ + "# Lab | Data Aggregation and Filtering" + ] + }, + { + "cell_type": "markdown", + "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d", + "metadata": { + "id": "a8f08a52-bec0-439b-99cc-11d3809d8b5d" + }, + "source": [ + "In this challenge, we will continue to work with customer data from an insurance company. We will use the dataset called marketing_customer_analysis.csv, which can be found at the following link:\n", + "\n", + "https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\n", + "\n", + "This dataset contains information such as customer demographics, policy details, vehicle information, and the customer's response to the last marketing campaign. Our goal is to explore and analyze this data by first performing data cleaning, formatting, and structuring." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "e0d28023-f4ec-437f-8d8f-e5916abc4b04", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Unnamed: 0CustomerStateCustomer Lifetime ValueResponseCoverageEducationEffective To DateEmploymentStatusGender...Number of Open ComplaintsNumber of PoliciesPolicy TypePolicyRenew Offer TypeSales ChannelTotal Claim AmountVehicle ClassVehicle SizeVehicle Type
00DK49336Arizona4809.216960NoBasicCollege2/18/11EmployedM...0.09Corporate AutoCorporate L3Offer3Agent292.800000Four-Door CarMedsizeNaN
11KX64629California2228.525238NoBasicCollege1/18/11UnemployedF...0.01Personal AutoPersonal L3Offer4Call Center744.924331Four-Door CarMedsizeNaN
22LZ68649Washington14947.917300NoBasicBachelor2/10/11EmployedM...0.02Personal AutoPersonal L3Offer3Call Center480.000000SUVMedsizeA
\n", + "

3 rows × 26 columns

\n", + "
" + ], + "text/plain": [ + " Unnamed: 0 Customer State Customer Lifetime Value Response Coverage \\\n", + "0 0 DK49336 Arizona 4809.216960 No Basic \n", + "1 1 KX64629 California 2228.525238 No Basic \n", + "2 2 LZ68649 Washington 14947.917300 No Basic \n", + "\n", + " Education Effective To Date EmploymentStatus Gender ... \\\n", + "0 College 2/18/11 Employed M ... \n", + "1 College 1/18/11 Unemployed F ... \n", + "2 Bachelor 2/10/11 Employed M ... \n", + "\n", + " Number of Open Complaints Number of Policies Policy Type Policy \\\n", + "0 0.0 9 Corporate Auto Corporate L3 \n", + "1 0.0 1 Personal Auto Personal L3 \n", + "2 0.0 2 Personal Auto Personal L3 \n", + "\n", + " Renew Offer Type Sales Channel Total Claim Amount Vehicle Class \\\n", + "0 Offer3 Agent 292.800000 Four-Door Car \n", + "1 Offer4 Call Center 744.924331 Four-Door Car \n", + "2 Offer3 Call Center 480.000000 SUV \n", + "\n", + " Vehicle Size Vehicle Type \n", + "0 Medsize NaN \n", + "1 Medsize NaN \n", + "2 Medsize A \n", + "\n", + "[3 rows x 26 columns]" + ] + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# importar dades\n", + "\n", + "import pandas as pd\n", + "\n", + "url = \"https://raw.githubusercontent.com/data-bootcamp-v4/data/main/marketing_customer_analysis.csv\"\n", + "\n", + "df = pd.read_csv(url)\n", + "df.head(3)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "a9c24896-768a-45ce-95e8-2b2b7a1c4811", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
unnamed:_0customerstatecustomer_lifetime_valueresponsecoverageeducationeffective_to_dateemploymentstatusgender...number_of_open_complaintsnumber_of_policiespolicy_typepolicyrenew_offer_typesales_channeltotal_claim_amountvehicle_classvehicle_sizevehicle_type
00DK49336Arizona4809.216960NoBasicCollege2/18/11EmployedM...0.09Corporate AutoCorporate L3Offer3Agent292.800000Four-Door CarMedsizeNaN
11KX64629California2228.525238NoBasicCollege1/18/11UnemployedF...0.01Personal AutoPersonal L3Offer4Call Center744.924331Four-Door CarMedsizeNaN
22LZ68649Washington14947.917300NoBasicBachelor2/10/11EmployedM...0.02Personal AutoPersonal L3Offer3Call Center480.000000SUVMedsizeA
\n", + "

3 rows × 26 columns

\n", + "
" + ], + "text/plain": [ + " unnamed:_0 customer state customer_lifetime_value response coverage \\\n", + "0 0 DK49336 Arizona 4809.216960 No Basic \n", + "1 1 KX64629 California 2228.525238 No Basic \n", + "2 2 LZ68649 Washington 14947.917300 No Basic \n", + "\n", + " education effective_to_date employmentstatus gender ... \\\n", + "0 College 2/18/11 Employed M ... \n", + "1 College 1/18/11 Unemployed F ... \n", + "2 Bachelor 2/10/11 Employed M ... \n", + "\n", + " number_of_open_complaints number_of_policies policy_type policy \\\n", + "0 0.0 9 Corporate Auto Corporate L3 \n", + "1 0.0 1 Personal Auto Personal L3 \n", + "2 0.0 2 Personal Auto Personal L3 \n", + "\n", + " renew_offer_type sales_channel total_claim_amount vehicle_class \\\n", + "0 Offer3 Agent 292.800000 Four-Door Car \n", + "1 Offer4 Call Center 744.924331 Four-Door Car \n", + "2 Offer3 Call Center 480.000000 SUV \n", + "\n", + " vehicle_size vehicle_type \n", + "0 Medsize NaN \n", + "1 Medsize NaN \n", + "2 Medsize A \n", + "\n", + "[3 rows x 26 columns]" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# netejar columnes\n", + "\n", + "df.columns = (\n", + " df.columns\n", + " .str.lower()\n", + " .str.normalize('NFKD') # quita acentos\n", + " .str.encode('ascii', errors='ignore')\n", + " .str.decode('utf-8')\n", + " .str.replace(' ', '_')\n", + " .str.replace('[^0-9a-zA-Z_]', '')\n", + ")\n", + "df.head(3)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "b321c390-98bb-4905-a506-408a597f8b08", + "metadata": {}, + "outputs": [], + "source": [ + "# eliminar columnas que no ens donen informació\n", + "\n", + "df = df.drop('unnamed:_0', axis=1)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "8f18548e-28af-4231-8017-5ff90f6d5377", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "vehicle_type 5482\n", + "number_of_open_complaints 633\n", + "months_since_last_claim 633\n", + "response 631\n", + "state 631\n", + "vehicle_class 622\n", + "vehicle_size 622\n", + "customer_lifetime_value 0\n", + "customer 0\n", + "gender 0\n", + "employmentstatus 0\n", + "effective_to_date 0\n", + "education 0\n", + "coverage 0\n", + "monthly_premium_auto 0\n", + "income 0\n", + "location_code 0\n", + "number_of_policies 0\n", + "months_since_policy_inception 0\n", + "marital_status 0\n", + "policy_type 0\n", + "sales_channel 0\n", + "renew_offer_type 0\n", + "policy 0\n", + "total_claim_amount 0\n", + "dtype: int64" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# mirar quines columnes tenen null value\n", + "\n", + "df.isnull().sum().sort_values(ascending=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "500b45d3-0cbb-4b1a-b08e-3cf9c73625f8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([ 0., nan, 3., 1., 2., 4., 5.])" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# neteja de columna: number_of_open_complaints\n", + "\n", + "df[\"number_of_open_complaints\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "8b5d61e2-4692-4237-8cff-6c038974ae8d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0 0.0\n", + "Name: number_of_open_complaints, dtype: float64" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "mode_vale_complaints = df['number_of_open_complaints'].mode()\n", + "mode_vale_complaints" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "221e7903-0b53-4b1f-9590-d7770f408886", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "number_of_open_complaints\n", + "0.0 8160\n", + "1.0 1145\n", + "2.0 414\n", + "3.0 324\n", + "4.0 166\n", + "5.0 68\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['number_of_open_complaints'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "d684a217-7214-4034-895a-e3da4503a104", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "number_of_open_complaints\n", + "0.0 8793\n", + "1.0 1145\n", + "2.0 414\n", + "3.0 324\n", + "4.0 166\n", + "5.0 68\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['number_of_open_complaints'] = df['number_of_open_complaints'].fillna(df['number_of_open_complaints'].mode()[0])\n", + "\n", + "# comprovant\n", + "\n", + "df['number_of_open_complaints'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "c694042a-a567-441b-ba33-63b304f32b82", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([ 7., 3., 34., 10., nan, 2., 8., 35., 33., 19., 13., 5., 24.,\n", + " 25., 6., 20., 26., 14., 9., 29., 11., 1., 18., 16., 30., 12.,\n", + " 4., 22., 31., 21., 28., 17., 15., 32., 23., 27., 0.])" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# neteja de columna: months_since_last_claim\n", + "\n", + "df[\"months_since_last_claim\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "a82fa582-70c2-4445-9ced-5fe139bbcf48", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "months_since_last_claim\n", + "14.0 932\n", + "3.0 426\n", + "6.0 394\n", + "1.0 386\n", + "7.0 378\n", + "4.0 373\n", + "2.0 368\n", + "5.0 355\n", + "0.0 354\n", + "10.0 346\n", + "16.0 333\n", + "11.0 331\n", + "15.0 323\n", + "13.0 320\n", + "8.0 315\n", + "12.0 307\n", + "17.0 299\n", + "9.0 277\n", + "23.0 276\n", + "19.0 276\n", + "25.0 269\n", + "21.0 267\n", + "18.0 257\n", + "20.0 254\n", + "22.0 246\n", + "24.0 237\n", + "29.0 234\n", + "28.0 227\n", + "31.0 221\n", + "26.0 211\n", + "27.0 208\n", + "30.0 201\n", + "34.0 199\n", + "33.0 198\n", + "35.0 157\n", + "32.0 155\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['months_since_last_claim'] = df['months_since_last_claim'].fillna(df['months_since_last_claim'].median())\n", + "df['months_since_last_claim'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "fb43551e-93ef-4878-8052-70e39b42f753", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['No', 'Yes', nan], dtype=object)" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# neteja de columna: response\n", + "\n", + "df[\"response\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "fbc589c7-3f92-4331-8b4f-e7dbe0bf70d2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "0 No\n", + "Name: response, dtype: object" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "mode_vale_response = df['response'].mode()\n", + "mode_vale_response" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "192c2589-6df2-4333-ae3c-df9da28bf5b6", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "response\n", + "No 8813\n", + "Yes 1466\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['response'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "2603ea9f-ce48-413a-9dc3-135cbf39bff1", + "metadata": {}, + "outputs": [], + "source": [ + "df['response'] = df['response'].fillna(df['response'].mode()[0])" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "c89123e4-0ba4-4860-977e-0bc955190839", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "state\n", + "California 3552\n", + "Oregon 2909\n", + "Arizona 1937\n", + "Nevada 993\n", + "Washington 888\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# neteja de columna: state\n", + "\n", + "df['state'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "ad6f0dee-ef7b-42e3-b49c-855e1c4a71d2", + "metadata": {}, + "outputs": [], + "source": [ + "df[\"state\"] = df[\"state\"].fillna(df[\"state\"].mode()[0])" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "4ea954ca-8bb4-400b-90a0-38796d6c31ee", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "vehicle_class\n", + "Four-Door Car 5212\n", + "Two-Door Car 2118\n", + "SUV 2012\n", + "Sports Car 550\n", + "Luxury SUV 208\n", + "Luxury Car 188\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# neteja columna: vehicle class\n", + "\n", + "df['vehicle_class'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "2c0b7bd1-90b8-45ab-aef2-539d77b0f34f", + "metadata": {}, + "outputs": [], + "source": [ + "df[\"vehicle_class\"] = df[\"vehicle_class\"].fillna(df[\"vehicle_class\"].mode()[0])" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "f652b4b6-bb22-483c-bbb2-1984945093f2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "vehicle_size\n", + "Medsize 7251\n", + "Small 1966\n", + "Large 1071\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# neteja de columna: vehicle size\n", + "\n", + "df['vehicle_size'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "3da841d7-11cd-4ef9-a9d1-b38678010253", + "metadata": {}, + "outputs": [], + "source": [ + "df[\"vehicle_size\"] = df[\"vehicle_size\"].fillna(df[\"vehicle_size\"].mode()[0])" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "08cbea1d-874f-4cae-9c9f-c94c32c47252", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "vehicle_type 5482\n", + "state 0\n", + "customer_lifetime_value 0\n", + "response 0\n", + "coverage 0\n", + "education 0\n", + "effective_to_date 0\n", + "employmentstatus 0\n", + "customer 0\n", + "gender 0\n", + "income 0\n", + "marital_status 0\n", + "location_code 0\n", + "months_since_last_claim 0\n", + "months_since_policy_inception 0\n", + "number_of_open_complaints 0\n", + "monthly_premium_auto 0\n", + "number_of_policies 0\n", + "policy_type 0\n", + "renew_offer_type 0\n", + "policy 0\n", + "sales_channel 0\n", + "total_claim_amount 0\n", + "vehicle_class 0\n", + "vehicle_size 0\n", + "dtype: int64" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# comprovant\n", + "\n", + "df.isnull().sum().sort_values(ascending=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "id": "552b83e3-0150-4c38-ace3-cc3408362a9c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([nan, 'A'], dtype=object)" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# neteja columna: vehicle type\n", + "\n", + "df[\"vehicle_type\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "305db267-8f3d-4fa0-9b9f-6bb28dec71db", + "metadata": {}, + "outputs": [], + "source": [ + "# eliminar columna, degut a que no ens dona cap tipus d'informació i el 50% de les files son null values.\n", + "\n", + "df = df.drop('vehicle_type', axis=1)" + ] + }, + { + "cell_type": "markdown", + "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50", + "metadata": { + "id": "9c98ddc5-b041-4c94-ada1-4dfee5c98e50" + }, + "source": [ + "1. Create a new DataFrame that only includes customers who:\n", + " - have a **low total_claim_amount** (e.g., below $1,000),\n", + " - have a response \"Yes\" to the last marketing campaign." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "ba0da3e2-43f7-4091-9a42-648b0d66b869", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
customerstatecustomer_lifetime_valueresponsecoverageeducationeffective_to_dateemploymentstatusgenderincome...number_of_open_complaintsnumber_of_policiespolicy_typepolicyrenew_offer_typesales_channeltotal_claim_amountvehicle_classvehicle_sizevehicle_type
3XL78013Oregon22332.439460YesExtendedCollege1/11/11EmployedM49078...0.02Corporate AutoCorporate L3Offer2Branch484.013411Four-Door CarMedsizeA
8FM55990California5989.773931YesPremiumCollege1/19/11EmployedM66839...0.01Personal AutoPersonal L1Offer2Branch739.200000Sports CarMedsizeNaN
15CW49887California4626.801093YesBasicMaster1/16/11EmployedF79487...0.01Special AutoSpecial L1Offer2Branch547.200000SUVMedsizeNaN
\n", + "

3 rows × 25 columns

\n", + "
" + ], + "text/plain": [ + " customer state customer_lifetime_value response coverage education \\\n", + "3 XL78013 Oregon 22332.439460 Yes Extended College \n", + "8 FM55990 California 5989.773931 Yes Premium College \n", + "15 CW49887 California 4626.801093 Yes Basic Master \n", + "\n", + " effective_to_date employmentstatus gender income ... \\\n", + "3 1/11/11 Employed M 49078 ... \n", + "8 1/19/11 Employed M 66839 ... \n", + "15 1/16/11 Employed F 79487 ... \n", + "\n", + " number_of_open_complaints number_of_policies policy_type policy \\\n", + "3 0.0 2 Corporate Auto Corporate L3 \n", + "8 0.0 1 Personal Auto Personal L1 \n", + "15 0.0 1 Special Auto Special L1 \n", + "\n", + " renew_offer_type sales_channel total_claim_amount vehicle_class \\\n", + "3 Offer2 Branch 484.013411 Four-Door Car \n", + "8 Offer2 Branch 739.200000 Sports Car \n", + "15 Offer2 Branch 547.200000 SUV \n", + "\n", + " vehicle_size vehicle_type \n", + "3 Medsize A \n", + "8 Medsize NaN \n", + "15 Medsize NaN \n", + "\n", + "[3 rows x 25 columns]" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "filtered_customers = df[\n", + " (df['total_claim_amount'] < 1000) &\n", + " (df['response'] == \"Yes\")\n", + "]\n", + "\n", + "filtered_customers.head(3)" + ] + }, + { + "cell_type": "markdown", + "id": "b9be383e-5165-436e-80c8-57d4c757c8c3", + "metadata": { + "id": "b9be383e-5165-436e-80c8-57d4c757c8c3" + }, + "source": [ + "2. Using the original Dataframe, analyze:\n", + " - the average `monthly_premium` and/or customer lifetime value by `policy_type` and `gender` for customers who responded \"Yes\", and\n", + " - compare these insights to `total_claim_amount` patterns, and discuss which segments appear most profitable or low-risk for the company." + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "id": "04cc7742-2c87-4689-9246-f371b84559a8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
customer_lifetime_valuemonthly_premium_autototal_claim_amount
policy_typegender
Corporate AutoF7712.6394.30433.74
M7944.4792.19408.58
Personal AutoF8339.7999.00452.97
M7448.3891.09457.01
Special AutoF7691.5892.31453.28
M8247.0986.34429.53
\n", + "
" + ], + "text/plain": [ + " customer_lifetime_value monthly_premium_auto \\\n", + "policy_type gender \n", + "Corporate Auto F 7712.63 94.30 \n", + " M 7944.47 92.19 \n", + "Personal Auto F 8339.79 99.00 \n", + " M 7448.38 91.09 \n", + "Special Auto F 7691.58 92.31 \n", + " M 8247.09 86.34 \n", + "\n", + " total_claim_amount \n", + "policy_type gender \n", + "Corporate Auto F 433.74 \n", + " M 408.58 \n", + "Personal Auto F 452.97 \n", + " M 457.01 \n", + "Special Auto F 453.28 \n", + " M 429.53 " + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pivot = df[df[\"response\"]==\"Yes\"].pivot_table(\n", + " values=[\"monthly_premium_auto\",\"customer_lifetime_value\",\"total_claim_amount\"],\n", + " index=[\"policy_type\",\"gender\"],\n", + " aggfunc=\"mean\"\n", + ").round(2)\n", + "\n", + "pivot" + ] + }, + { + "cell_type": "markdown", + "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0", + "metadata": { + "id": "7050f4ac-53c5-4193-a3c0-8699b87196f0" + }, + "source": [ + "3. Analyze the total number of customers who have policies in each state, and then filter the results to only include states where there are more than 500 customers." + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "7e012ff8-0577-4b01-ac6a-d189b6f0984b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "state\n", + "California 4183\n", + "Oregon 2909\n", + "Arizona 1937\n", + "Nevada 993\n", + "Washington 888\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "state_counts = df[\"state\"].value_counts()\n", + "\n", + "states_over_500 = state_counts[state_counts > 500]\n", + "states_over_500" + ] + }, + { + "cell_type": "markdown", + "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d", + "metadata": { + "id": "b60a4443-a1a7-4bbf-b78e-9ccdf9895e0d" + }, + "source": [ + "4. Find the maximum, minimum, and median customer lifetime value by education level and gender. Write your conclusions." + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "id": "7171b8bc-d254-4887-91d6-1882a4fdac98", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
maxminmedian
customer_lifetime_valuecustomer_lifetime_valuecustomer_lifetime_value
educationgender
BachelorF73225.961904.005640.51
M67907.271898.015548.03
CollegeF61850.191898.685623.61
M61134.681918.126005.85
DoctorF44856.112395.575332.46
M32677.342267.605577.67
High School or BelowF55277.452144.926039.55
M83325.381940.986286.73
MasterF51016.072417.785729.86
M50568.262272.315579.10
\n", + "
" + ], + "text/plain": [ + " max min \\\n", + " customer_lifetime_value customer_lifetime_value \n", + "education gender \n", + "Bachelor F 73225.96 1904.00 \n", + " M 67907.27 1898.01 \n", + "College F 61850.19 1898.68 \n", + " M 61134.68 1918.12 \n", + "Doctor F 44856.11 2395.57 \n", + " M 32677.34 2267.60 \n", + "High School or Below F 55277.45 2144.92 \n", + " M 83325.38 1940.98 \n", + "Master F 51016.07 2417.78 \n", + " M 50568.26 2272.31 \n", + "\n", + " median \n", + " customer_lifetime_value \n", + "education gender \n", + "Bachelor F 5640.51 \n", + " M 5548.03 \n", + "College F 5623.61 \n", + " M 6005.85 \n", + "Doctor F 5332.46 \n", + " M 5577.67 \n", + "High School or Below F 6039.55 \n", + " M 6286.73 \n", + "Master F 5729.86 \n", + " M 5579.10 " + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "clv = df.pivot_table(\n", + " values=\"customer_lifetime_value\",\n", + " index=[\"education\", \"gender\"],\n", + " aggfunc=[\"max\", \"min\", \"median\"]\n", + ").round(2)\n", + "\n", + "clv" + ] + }, + { + "cell_type": "markdown", + "id": "23e916ad-1a89-4048-9b6a-632607c28b86", + "metadata": {}, + "source": [ + "Conclusions:\n", + "\n", + "Education level appears to influence customer lifetime value (CLV).\n", + "\n", + " - Bachelor and College groups show higher median CLV compared to the other education levels.\n", + " - Doctor degree holders consistently show lower maximum CLV values, suggesting they may be less profitable as customers in this dataset." + ] + }, + { + "cell_type": "markdown", + "id": "b42999f9-311f-481e-ae63-40a5577072c5", + "metadata": { + "id": "b42999f9-311f-481e-ae63-40a5577072c5" + }, + "source": [ + "## Bonus" + ] + }, + { + "cell_type": "markdown", + "id": "81ff02c5-6584-4f21-a358-b918697c6432", + "metadata": { + "id": "81ff02c5-6584-4f21-a358-b918697c6432" + }, + "source": [ + "5. The marketing team wants to analyze the number of policies sold by state and month. Present the data in a table where the months are arranged as columns and the states are arranged as rows." + ] + }, + { + "cell_type": "markdown", + "id": "b6aec097-c633-4017-a125-e77a97259cda", + "metadata": { + "id": "b6aec097-c633-4017-a125-e77a97259cda" + }, + "source": [ + "6. Display a new DataFrame that contains the number of policies sold by month, by state, for the top 3 states with the highest number of policies sold.\n", + "\n", + "*Hint:*\n", + "- *To accomplish this, you will first need to group the data by state and month, then count the number of policies sold for each group. Afterwards, you will need to sort the data by the count of policies sold in descending order.*\n", + "- *Next, you will select the top 3 states with the highest number of policies sold.*\n", + "- *Finally, you will create a new DataFrame that contains the number of policies sold by month for each of the top 3 states.*" + ] + }, + { + "cell_type": "markdown", + "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009", + "metadata": { + "id": "ba975b8a-a2cf-4fbf-9f59-ebc381767009" + }, + "source": [ + "7. The marketing team wants to analyze the effect of different marketing channels on the customer response rate.\n", + "\n", + "Hint: You can use melt to unpivot the data and create a table that shows the customer response rate (those who responded \"Yes\") by marketing channel." + ] + }, + { + "cell_type": "markdown", + "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d", + "metadata": { + "id": "e4378d94-48fb-4850-a802-b1bc8f427b2d" + }, + "source": [ + "External Resources for Data Filtering: https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "449513f4-0459-46a0-a18d-9398d974c9ad", + "metadata": { + "id": "449513f4-0459-46a0-a18d-9398d974c9ad" + }, + "outputs": [], + "source": [ + "# your code goes here" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python [conda env:base] *", + "language": "python", + "name": "conda-base-py" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 }