diff --git a/.ipynb_checkpoints/lab-dw-pandas-checkpoint.ipynb b/.ipynb_checkpoints/lab-dw-pandas-checkpoint.ipynb new file mode 100644 index 00000000..998d81cb --- /dev/null +++ b/.ipynb_checkpoints/lab-dw-pandas-checkpoint.ipynb @@ -0,0 +1,1613 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "25d7736c-ba17-4aff-b6bb-66eba20fbf4e", + "metadata": {}, + "source": [ + "# Lab | Pandas" + ] + }, + { + "cell_type": "markdown", + "id": "d1973e9e-8be6-4039-b70e-d73ee0d94c99", + "metadata": {}, + "source": [ + "In this lab, we will be working with the customer data from an insurance company, which can be found in the CSV file located at the following link: https://raw.githubusercontent.com/data-bootcamp-v4/data/main/file1.csv\n", + "\n", + "The data includes information such as customer ID, state, gender, education, income, and other variables that can be used to perform various analyses.\n", + "\n", + "Throughout the lab, we will be using the pandas library in Python to manipulate and analyze the data. Pandas is a powerful library that provides various data manipulation and analysis tools, including the ability to load and manipulate data from a variety of sources, including CSV files." + ] + }, + { + "cell_type": "markdown", + "id": "8045146f-f4f7-44d9-8cd9-130d6400c73a", + "metadata": {}, + "source": [ + "### Data Description\n", + "\n", + "- Customer - Customer ID\n", + "\n", + "- ST - State where customers live\n", + "\n", + "- Gender - Gender of the customer\n", + "\n", + "- Education - Background education of customers \n", + "\n", + "- Customer Lifetime Value - Customer lifetime value(CLV) is the total revenue the client will derive from their entire relationship with a customer. In other words, is the predicted or calculated value of a customer over their entire duration as a policyholder with the insurance company. It is an estimation of the net profit that the insurance company expects to generate from a customer throughout their relationship with the company. Customer Lifetime Value takes into account factors such as the duration of the customer's policy, premium payments, claim history, renewal likelihood, and potential additional services or products the customer may purchase. It helps insurers assess the long-term profitability and value associated with retaining a particular customer.\n", + "\n", + "- Income - Customers income\n", + "\n", + "- Monthly Premium Auto - Amount of money the customer pays on a monthly basis as a premium for their auto insurance coverage. It represents the recurring cost that the insured person must pay to maintain their insurance policy and receive coverage for potential damages, accidents, or other covered events related to their vehicle.\n", + "\n", + "- Number of Open Complaints - Number of complaints the customer opened\n", + "\n", + "- Policy Type - There are three type of policies in car insurance (Corporate Auto, Personal Auto, and Special Auto)\n", + "\n", + "- Vehicle Class - Type of vehicle classes that customers have Two-Door Car, Four-Door Car SUV, Luxury SUV, Sports Car, and Luxury Car\n", + "\n", + "- Total Claim Amount - the sum of all claims made by the customer. It represents the total monetary value of all approved claims for incidents such as accidents, theft, vandalism, or other covered events.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3a72419b-20fc-4905-817a-8c83abc59de6", + "metadata": {}, + "source": [ + "External Resources: https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9" + ] + }, + { + "cell_type": "markdown", + "id": "8f8ece17-e919-4e23-96c0-c7c59778436a", + "metadata": {}, + "source": [ + "## Challenge 1: Understanding the data\n", + "\n", + "In this challenge, you will use pandas to explore a given dataset. Your task is to gain a deep understanding of the data by analyzing its characteristics, dimensions, and statistical properties." + ] + }, + { + "cell_type": "markdown", + "id": "91437bd5-59a6-49c0-8150-ef0e6e6eb253", + "metadata": {}, + "source": [ + "- Identify the dimensions of the dataset by determining the number of rows and columns it contains.\n", + "- Determine the data types of each column and evaluate whether they are appropriate for the nature of the variable. You should also provide suggestions for fixing any incorrect data types.\n", + "- Identify the number of unique values for each column and determine which columns appear to be categorical. You should also describe the unique values of each categorical column and the range of values for numerical columns, and give your insights.\n", + "- Compute summary statistics such as mean, median, mode, standard deviation, and quartiles to understand the central tendency and distribution of the data for numerical columns. You should also provide your conclusions based on these summary statistics.\n", + "- Compute summary statistics for categorical columns and providing your conclusions based on these statistics." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "dd4e8cd8-a6f6-486c-a5c4-1745b0c035f4", + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "url = \"https://raw.githubusercontent.com/data-bootcamp-v4/data/main/file1.csv\"\n", + "df = pd.read_csv(url)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "33df41fb-04e2-4193-b715-e32810267eaf", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(4008, 11)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Identify the dimensions of the dataset by determining the number of rows and columns it contains.\n", + "\n", + "df.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "ff702199-66f9-4d45-bea7-9e051dd040e9", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "RangeIndex: 4008 entries, 0 to 4007\n", + "Data columns (total 11 columns):\n", + " # Column Non-Null Count Dtype \n", + "--- ------ -------------- ----- \n", + " 0 Customer 1071 non-null object \n", + " 1 ST 1071 non-null object \n", + " 2 GENDER 954 non-null object \n", + " 3 Education 1071 non-null object \n", + " 4 Customer Lifetime Value 1068 non-null object \n", + " 5 Income 1071 non-null float64\n", + " 6 Monthly Premium Auto 1071 non-null float64\n", + " 7 Number of Open Complaints 1071 non-null object \n", + " 8 Policy Type 1071 non-null object \n", + " 9 Vehicle Class 1071 non-null object \n", + " 10 Total Claim Amount 1071 non-null float64\n", + "dtypes: float64(3), object(8)\n", + "memory usage: 344.6+ KB\n" + ] + } + ], + "source": [ + "#Determine the data types of each column and evaluate whether they are appropriate for the nature of the variable. \n", + "#You should also provide suggestions for fixing any incorrect data types.\n", + "\n", + "df.info()" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "b342bedb-6843-4143-9955-f58ebc33dad6", + "metadata": {}, + "outputs": [], + "source": [ + "#Suggestions for data cleaning: \n", + "#Some of the columns have different formats like ST and GENDER\n", + "#Column \"Number of Open Complaints\" should be type integers\n", + "#Column \"Customer Lifetime Value\" should be type float" + ] + }, + { + "cell_type": "code", + "execution_count": 96, + "id": "ea91c5f7-659a-40b5-a3fb-36f4e9c983b4", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([0])" + ] + }, + "execution_count": 96, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Number of Open Complaints'].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "id": "3ee4592c-cc92-47e4-adf6-b92558a4b80a", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "int64\n" + ] + } + ], + "source": [ + "# this function cleans the column \"Number of Open Complaints\" removing the slashes and accessing the middle character (second)\n", + "# and categorizes it to integers\n", + "\n", + "def clean_complaints(value):\n", + " try:\n", + " return int(value.split('/')[1])\n", + " except:\n", + " return 0\n", + "\n", + "df['Number of Open Complaints'] = list(map(clean_complaints, df['Number of Open Complaints']))\n", + "\n", + "print(df['Number of Open Complaints'].dtype)" + ] + }, + { + "cell_type": "code", + "execution_count": 103, + "id": "33d52e4c-ad8b-4b27-b27e-a31386e005a1", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([ nan, 697953.59, 1288743.17, ..., 2031499.76, 323912.47,\n", + " 899704.02])" + ] + }, + "execution_count": 103, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Customer Lifetime Value'].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 106, + "id": "7c810335-49c0-4a44-b587-d8013649576f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "float64\n" + ] + } + ], + "source": [ + "def clean_clv(value):\n", + " try:\n", + " return float(value)\n", + " except:\n", + " value_cleaned = value.strip('%')\n", + " return float(value_cleaned)\n", + "\n", + "df['Customer Lifetime Value'] = list(map(clean_clv, df['Customer Lifetime Value']))\n", + "\n", + "print(df['Customer Lifetime Value'].dtype)" + ] + }, + { + "cell_type": "code", + "execution_count": 108, + "id": "bd658ef3-4bb7-4cf5-8ffb-45fd8dbb7141", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Customer 1071\n", + "State 5\n", + "Gender 2\n", + "Education 5\n", + "Customer Lifetime Value 1027\n", + "Income 774\n", + "Monthly Premium Auto 132\n", + "Number of Open Complaints 1\n", + "Policy Type 3\n", + "Vehicle Class 6\n", + "Total Claim Amount 761\n", + "dtype: int64" + ] + }, + "execution_count": 108, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Identify the number of unique values for each column and determine which columns appear to be categorical. \n", + "#You should also describe the unique values of each categorical column and the range of values for numerical columns, and give your insights.\n", + "df.nunique()" + ] + }, + { + "cell_type": "code", + "execution_count": 109, + "id": "abb686ce-25e4-49b8-bffc-e3065a40b788", + "metadata": {}, + "outputs": [], + "source": [ + "#rename the column Gender and State\n", + "\n", + "rename_map = {\n", + " 'ST': 'State',\n", + " 'GENDER': 'Gender'\n", + "}\n", + "\n", + "\n", + "df.columns = [rename_map.get(col, col) for col in df.columns]" + ] + }, + { + "cell_type": "code", + "execution_count": 110, + "id": "6a632747-e595-4b21-b2b8-bcd33739ccb7", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
CustomerStateGenderEducationCustomer Lifetime ValueIncomeMonthly Premium AutoNumber of Open ComplaintsPolicy TypeVehicle ClassTotal Claim Amount
0RB50392WashingtonNaNMasterNaN0.01000.00Personal AutoFour-Door Car2.704934
1QZ44356ArizonaFBachelor697953.590.094.00Personal AutoFour-Door Car1131.464935
2AI49188NevadaFBachelor1288743.1748767.0108.00Personal AutoTwo-Door Car566.472247
3WW63253CaliforniaMBachelor764586.180.0106.00Corporate AutoSUV529.881344
4GA49547WashingtonMHigh School or Below536307.6536357.068.00Personal AutoFour-Door Car17.269323
\n", + "
" + ], + "text/plain": [ + " Customer State Gender Education Customer Lifetime Value \\\n", + "0 RB50392 Washington NaN Master NaN \n", + "1 QZ44356 Arizona F Bachelor 697953.59 \n", + "2 AI49188 Nevada F Bachelor 1288743.17 \n", + "3 WW63253 California M Bachelor 764586.18 \n", + "4 GA49547 Washington M High School or Below 536307.65 \n", + "\n", + " Income Monthly Premium Auto Number of Open Complaints Policy Type \\\n", + "0 0.0 1000.0 0 Personal Auto \n", + "1 0.0 94.0 0 Personal Auto \n", + "2 48767.0 108.0 0 Personal Auto \n", + "3 0.0 106.0 0 Corporate Auto \n", + "4 36357.0 68.0 0 Personal Auto \n", + "\n", + " Vehicle Class Total Claim Amount \n", + "0 Four-Door Car 2.704934 \n", + "1 Four-Door Car 1131.464935 \n", + "2 Two-Door Car 566.472247 \n", + "3 SUV 529.881344 \n", + "4 Four-Door Car 17.269323 " + ] + }, + "execution_count": 110, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#check columns \n", + "\n", + "df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "id": "919ffee7-d5ea-4650-9128-e9b9dbd73d1a", + "metadata": {}, + "outputs": [], + "source": [ + "#Categorical columns are \"State, Gender, Education, Policy Type, Vehicle Class\"" + ] + }, + { + "cell_type": "code", + "execution_count": 111, + "id": "80e35e99-9abc-46f4-b11c-4b878e1c02c8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Washington', 'Arizona', 'Nevada', 'California', 'Oregon', nan],\n", + " dtype=object)" + ] + }, + "execution_count": 111, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#get unique values \n", + "\n", + "df.State.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 112, + "id": "31b3834e-d200-49d0-8b82-97e73968521c", + "metadata": {}, + "outputs": [], + "source": [ + "#cleanup data using a Dictionary\n", + "\n", + "state_map = {\n", + " 'AZ': 'Arizona',\n", + " 'Cali': 'California',\n", + " 'WA': 'Washington'\n", + "}\n", + "df['State'] = df['State'].map(state_map).fillna(df['State'])" + ] + }, + { + "cell_type": "code", + "execution_count": 113, + "id": "502f30ed-2dff-4901-b3d4-098389ffed8f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Washington', 'Arizona', 'Nevada', 'California', 'Oregon', nan],\n", + " dtype=object)" + ] + }, + "execution_count": 113, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.State.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 114, + "id": "29ca7495-89fa-40ea-a24f-60e7ac98feb5", + "metadata": {}, + "outputs": [], + "source": [ + "#The 5 different STATES present are Washington, Arizona, Nevada, California and Oregon" + ] + }, + { + "cell_type": "code", + "execution_count": 115, + "id": "9481c2c1-440e-426e-82b7-ce3b06897fed", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([nan, 'F', 'M'], dtype=object)" + ] + }, + "execution_count": 115, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.Gender.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 116, + "id": "18812464-7e70-47b5-b8ab-15a7e7877002", + "metadata": {}, + "outputs": [], + "source": [ + "gender_map = {\n", + " 'F': 'F',\n", + " 'M': 'M',\n", + " 'Femal': 'F',\n", + " 'Male': 'M',\n", + " 'female': 'F'\n", + "}\n", + "df['Gender'] = df['Gender'].map(gender_map)" + ] + }, + { + "cell_type": "code", + "execution_count": 117, + "id": "9293d6cb-0ea7-408c-bcb2-3611452d94e3", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([nan, 'F', 'M'], dtype=object)" + ] + }, + "execution_count": 117, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.Gender.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 118, + "id": "410bc527-1559-445d-a57a-ff2b83fd3767", + "metadata": {}, + "outputs": [], + "source": [ + "#There are two genders, Male and Female" + ] + }, + { + "cell_type": "code", + "execution_count": 119, + "id": "06fd0b16-b2db-45d1-88ef-1f4a66ee30da", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Master', 'Bachelor', 'High School or Below', 'College', 'Doctor',\n", + " nan], dtype=object)" + ] + }, + "execution_count": 119, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.Education.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 120, + "id": "348d7e9b-1d3a-4af6-925c-35215b7facd7", + "metadata": {}, + "outputs": [], + "source": [ + "education_map = {\n", + " 'Bachelors': 'Bachelor',\n", + " 'Bachelor': 'Bachelor',\n", + " 'Master': 'Master',\n", + " 'High School or Below': 'High School or Below',\n", + " 'College': 'College',\n", + " 'Doctor': 'Doctor'\n", + "}\n", + "df['Education'] = df['Education'].map(education_map)" + ] + }, + { + "cell_type": "code", + "execution_count": 121, + "id": "7e3f088d-77cb-4b4c-beaa-5bcffce4da90", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Master', 'Bachelor', 'High School or Below', 'College', 'Doctor',\n", + " nan], dtype=object)" + ] + }, + "execution_count": 121, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.Education.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 122, + "id": "9a169cd3-2a11-4f18-96ae-c1b6095f1c31", + "metadata": {}, + "outputs": [], + "source": [ + "#There are 5 Education levels, Master, Bachelor, High School or Below, College and Doctor" + ] + }, + { + "cell_type": "code", + "execution_count": 123, + "id": "8baa1659-b923-4f88-8142-c394a1c7233d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Personal Auto', 'Corporate Auto', 'Special Auto', nan],\n", + " dtype=object)" + ] + }, + "execution_count": 123, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Policy Type'].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 124, + "id": "863f9fe1-3d2e-480b-9e85-bf8f7e931fae", + "metadata": {}, + "outputs": [], + "source": [ + "#The three policy types are Personal, Corporate, and Special Auto" + ] + }, + { + "cell_type": "code", + "execution_count": 125, + "id": "908564f3-2aea-4d08-9ea2-af5379c9a7d8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Four-Door Car', 'Two-Door Car', 'SUV', 'Luxury SUV', 'Sports Car',\n", + " 'Luxury Car', nan], dtype=object)" + ] + }, + "execution_count": 125, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Vehicle Class'].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 126, + "id": "3b37da5e-6c6f-43fa-a95f-44bca0ff55f2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Customer Lifetime ValueIncomeMonthly Premium AutoNumber of Open ComplaintsTotal Claim Amount
count1.068000e+031071.0000001071.0000004008.01071.000000
mean7.936903e+0539295.701214193.2343600.0404.986909
std6.434784e+0530469.4270601601.1903690.0293.027260
min2.004351e+050.00000061.0000000.00.382107
25%4.034080e+0514072.00000068.0000000.0202.157702
50%5.881742e+0536234.00000083.0000000.0354.729129
75%8.962872e+0564631.000000109.5000000.0532.800000
max5.816655e+0699960.00000035354.0000000.02893.239678
\n", + "
" + ], + "text/plain": [ + " Customer Lifetime Value Income Monthly Premium Auto \\\n", + "count 1.068000e+03 1071.000000 1071.000000 \n", + "mean 7.936903e+05 39295.701214 193.234360 \n", + "std 6.434784e+05 30469.427060 1601.190369 \n", + "min 2.004351e+05 0.000000 61.000000 \n", + "25% 4.034080e+05 14072.000000 68.000000 \n", + "50% 5.881742e+05 36234.000000 83.000000 \n", + "75% 8.962872e+05 64631.000000 109.500000 \n", + "max 5.816655e+06 99960.000000 35354.000000 \n", + "\n", + " Number of Open Complaints Total Claim Amount \n", + "count 4008.0 1071.000000 \n", + "mean 0.0 404.986909 \n", + "std 0.0 293.027260 \n", + "min 0.0 0.382107 \n", + "25% 0.0 202.157702 \n", + "50% 0.0 354.729129 \n", + "75% 0.0 532.800000 \n", + "max 0.0 2893.239678 " + ] + }, + "execution_count": 126, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Compute summary statistics such as mean, median, mode, standard deviation, and quartiles to understand the central tendency and distribution of the data for numerical columns. \n", + "#You should also provide your conclusions based on these summary statistics.\n", + "\n", + "df.describe()" + ] + }, + { + "cell_type": "code", + "execution_count": 127, + "id": "bca423a5-9cfa-4192-8c9a-86883df6a6f0", + "metadata": {}, + "outputs": [], + "source": [ + "#For the Monthly Premium Auto column, I noticed the mean is 193.23, which is higher than the median at 83. \n", + "#This suggests that while most customers pay a premium around 83, a few customers pay an extremely high premium, which pulls the average up.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 128, + "id": "23d6541f-ef2f-4c5f-99ba-b5f8f45ea3e9", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Customer 1071\n", + "State 5\n", + "Gender 2\n", + "Education 5\n", + "Customer Lifetime Value 1027\n", + "Income 774\n", + "Monthly Premium Auto 132\n", + "Number of Open Complaints 1\n", + "Policy Type 3\n", + "Vehicle Class 6\n", + "Total Claim Amount 761\n", + "dtype: int64" + ] + }, + "execution_count": 128, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Compute summary statistics for categorical columns and providing your conclusions based on these statistics.\n", + "df.nunique()" + ] + }, + { + "cell_type": "code", + "execution_count": 129, + "id": "17e341d3-7783-4713-bf13-7abd52511906", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "State\n", + "California 331\n", + "Oregon 320\n", + "Arizona 211\n", + "Washington 111\n", + "Nevada 98\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 129, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['State'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 130, + "id": "91fe5a60-eb78-44c2-9606-4ca59793d69c", + "metadata": {}, + "outputs": [], + "source": [ + "#Most of customers are from California" + ] + }, + { + "cell_type": "code", + "execution_count": 131, + "id": "a485e070-846d-466a-974c-07b71f8b835d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Gender\n", + "F 502\n", + "M 452\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 131, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Gender'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 132, + "id": "1ac01890-dbdc-4b65-8791-01ca6c736dd8", + "metadata": {}, + "outputs": [], + "source": [ + "#The number of Female and Male customer is balanced even though there are more Female" + ] + }, + { + "cell_type": "code", + "execution_count": 133, + "id": "00c32bdc-17d8-4454-837d-c8526a437aff", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Education\n", + "Bachelor 331\n", + "College 313\n", + "High School or Below 296\n", + "Master 94\n", + "Doctor 37\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 133, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Education'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 134, + "id": "fe2c2235-317a-4896-8e13-7232889ba19a", + "metadata": {}, + "outputs": [], + "source": [ + "#Most of customers education level is either Bachelor or College " + ] + }, + { + "cell_type": "code", + "execution_count": 135, + "id": "2e5b82bb-e1d1-4b6b-9355-afb9b5a88f05", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Policy Type\n", + "Personal Auto 780\n", + "Corporate Auto 234\n", + "Special Auto 57\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 135, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Policy Type'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 136, + "id": "63c4bae3-e1c8-4933-acdf-e575b820d3bf", + "metadata": {}, + "outputs": [], + "source": [ + "#Personal Auto is the most common type among customers\"" + ] + }, + { + "cell_type": "code", + "execution_count": 137, + "id": "6bed82fc-88a4-4a15-821c-3fced12353ed", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Vehicle Class\n", + "Four-Door Car 576\n", + "Two-Door Car 205\n", + "SUV 199\n", + "Sports Car 57\n", + "Luxury SUV 20\n", + "Luxury Car 14\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 137, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Vehicle Class'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 138, + "id": "100c9448-b259-403e-8781-bfbf71774c6d", + "metadata": {}, + "outputs": [], + "source": [ + "#Four-Door Car is the most common Vehicle Class among customers\"" + ] + }, + { + "cell_type": "code", + "execution_count": 139, + "id": "07835d92-3a3f-45cf-9ab1-f87c8cff7b83", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "RangeIndex: 4008 entries, 0 to 4007\n", + "Data columns (total 11 columns):\n", + " # Column Non-Null Count Dtype \n", + "--- ------ -------------- ----- \n", + " 0 Customer 1071 non-null object \n", + " 1 State 1071 non-null object \n", + " 2 Gender 954 non-null object \n", + " 3 Education 1071 non-null object \n", + " 4 Customer Lifetime Value 1068 non-null float64\n", + " 5 Income 1071 non-null float64\n", + " 6 Monthly Premium Auto 1071 non-null float64\n", + " 7 Number of Open Complaints 4008 non-null int64 \n", + " 8 Policy Type 1071 non-null object \n", + " 9 Vehicle Class 1071 non-null object \n", + " 10 Total Claim Amount 1071 non-null float64\n", + "dtypes: float64(4), int64(1), object(6)\n", + "memory usage: 344.6+ KB\n" + ] + } + ], + "source": [ + "df.info()" + ] + }, + { + "cell_type": "markdown", + "id": "4a703890-63db-4944-b7ab-95a4f8185120", + "metadata": {}, + "source": [ + "## Challenge 2: analyzing the data" + ] + }, + { + "cell_type": "markdown", + "id": "0776a403-c56a-452f-ac33-5fd4fdb06fc7", + "metadata": {}, + "source": [ + "### Exercise 1" + ] + }, + { + "cell_type": "markdown", + "id": "eedbc484-da4d-4f9c-9343-e1d44311a87e", + "metadata": {}, + "source": [ + "The marketing team wants to know the top 5 less common customer locations. Create a pandas Series object that contains the customer locations and their frequencies, and then retrieve the top 5 less common locations in ascending order." + ] + }, + { + "cell_type": "code", + "execution_count": 158, + "id": "2dca5073-4520-4f42-9390-4b92733284ed", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "State\n", + "California 331\n", + "Oregon 320\n", + "Arizona 211\n", + "Washington 111\n", + "Nevada 98\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 158, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['State'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 159, + "id": "eda4c29a-2056-4942-b86b-ba0b319fe40d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "State\n", + "Nevada 98\n", + "Washington 111\n", + "Arizona 211\n", + "Oregon 320\n", + "California 331\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 159, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['State'].value_counts().sort_values(ascending=True)" + ] + }, + { + "cell_type": "markdown", + "id": "0ce80f43-4afa-43c7-a78a-c917444da4e0", + "metadata": {}, + "source": [ + "### Exercise 2\n", + "\n", + "The sales team wants to know the total number of policies sold for each type of policy. Create a pandas Series object that contains the policy types and their total number of policies sold, and then retrieve the policy type with the highest number of policies sold." + ] + }, + { + "cell_type": "markdown", + "id": "a9f13997-1555-4f98-aca6-970fda1d2c3f", + "metadata": {}, + "source": [ + "*Hint:*\n", + "- *Using value_counts() method simplifies this analysis.*\n", + "- *Futhermore, there is a method that returns the index of the maximum value in a column or row.*\n" + ] + }, + { + "cell_type": "code", + "execution_count": 157, + "id": "bcfad6c1-9af2-4b0b-9aa9-0dc5c17473c0", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Policy Type\n", + "Personal Auto 780\n", + "Corporate Auto 234\n", + "Special Auto 57\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 157, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Policy Type'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 160, + "id": "21077203-f396-4b32-bc70-107021b144f4", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Personal Auto'" + ] + }, + "execution_count": 160, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Policy Type'].value_counts().index[0]" + ] + }, + { + "cell_type": "markdown", + "id": "0b863fd3-bf91-4d5d-86eb-be29ed9f5b70", + "metadata": {}, + "source": [ + "### Exercise 3\n", + "\n", + "The sales team wants to know if customers with Personal Auto have a lower income than those with Corporate Auto. How does the average income compare between the two policy types?" + ] + }, + { + "cell_type": "markdown", + "id": "b1386d75-2810-4aa1-93e0-9485aa12d552", + "metadata": {}, + "source": [ + "- Use *loc* to create two dataframes: one containing only Personal Auto policies and one containing only Corporate Auto policies.\n", + "- Calculate the average income for each policy.\n", + "- Print the results." + ] + }, + { + "cell_type": "code", + "execution_count": 161, + "id": "0c0563cf-6f8b-463d-a321-651a972f82e5", + "metadata": {}, + "outputs": [], + "source": [ + "personal_auto_df = df.loc[df['Policy Type'] == 'Personal Auto']" + ] + }, + { + "cell_type": "code", + "execution_count": 162, + "id": "cf615079-eb4c-4d27-ac72-c80b4a4eea2c", + "metadata": {}, + "outputs": [], + "source": [ + "corporate_auto_df = df.loc[df['Policy Type'] == 'Corporate Auto']" + ] + }, + { + "cell_type": "code", + "execution_count": 165, + "id": "4de71be3-61f0-4fa5-abc1-55b142beca82", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "np.float64(38180.69871794872)" + ] + }, + "execution_count": 165, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "personal_auto_df['Income'].mean()" + ] + }, + { + "cell_type": "code", + "execution_count": 166, + "id": "35ce2448-f56b-40c3-8be3-8a698cf1ee0a", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "np.float64(41390.31196581197)" + ] + }, + "execution_count": 166, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "corporate_auto_df['Income'].mean()" + ] + }, + { + "cell_type": "markdown", + "id": "80b16c27-f4a5-4727-a229-1f88671cf4e2", + "metadata": {}, + "source": [ + "### Bonus: Exercise 4\n" + ] + }, + { + "cell_type": "markdown", + "id": "ac584986-299b-475f-ac2e-928c16c3f512", + "metadata": {}, + "source": [ + "Your goal is to identify customers with a high policy claim amount.\n", + "\n", + "Instructions:\n", + "\n", + "- Review again the statistics for total claim amount to gain an understanding of the data.\n", + "- To identify potential areas for improving customer retention and profitability, we want to focus on customers with a high policy claim amount. Consider customers with a high policy claim amount to be those in the top 25% of the total claim amount. Create a pandas DataFrame object that contains information about customers with a policy claim amount greater than the 75th percentile.\n", + "- Use DataFrame methods to calculate summary statistics about the high policy claim amount data. " + ] + }, + { + "cell_type": "markdown", + "id": "4e3af5f1-6023-4b05-9c01-d05392daa650", + "metadata": {}, + "source": [ + "*Note: When analyzing data, we often want to focus on certain groups of values to gain insights. Percentiles are a useful tool to help us define these groups. A percentile is a measure that tells us what percentage of values in a dataset are below a certain value. For example, the 75th percentile represents the value below which 75% of the data falls. Similarly, the 25th percentile represents the value below which 25% of the data falls. When we talk about the top 25%, we are referring to the values that fall above the 75th percentile, which represent the top quarter of the data. On the other hand, when we talk about the bottom 25%, we are referring to the values that fall below the 25th percentile, which represent the bottom quarter of the data. By focusing on these groups, we can identify patterns and trends that may be useful for making decisions and taking action.*\n", + "\n", + "*Hint: look for a method that gives you the percentile or quantile 0.75 and 0.25 for a Pandas Series.*" + ] + }, + { + "cell_type": "markdown", + "id": "2d234634-50bd-41e0-88f7-d5ba684455d1", + "metadata": {}, + "source": [ + "*Hint 2: check `Boolean selection according to the values of a single column` in https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9*" + ] + }, + { + "cell_type": "code", + "execution_count": 168, + "id": "b731bca6-a760-4860-a27b-a33efa712ce0", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "count 1071.000000\n", + "mean 404.986909\n", + "std 293.027260\n", + "min 0.382107\n", + "25% 202.157702\n", + "50% 354.729129\n", + "75% 532.800000\n", + "max 2893.239678\n", + "Name: Total Claim Amount, dtype: float64" + ] + }, + "execution_count": 168, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Total Claim Amount'].describe()" + ] + }, + { + "cell_type": "code", + "execution_count": 169, + "id": "010d67da-97ed-4f04-81e6-1a287e4893b6", + "metadata": {}, + "outputs": [], + "source": [ + "high_claim_df = df.loc[df['Total Claim Amount'] > 532.80]" + ] + }, + { + "cell_type": "code", + "execution_count": 170, + "id": "42957161-c840-4745-b7bb-298eee0234a0", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Customer Lifetime ValueIncomeMonthly Premium AutoNumber of Open ComplaintsTotal Claim Amount
count2.640000e+02264.000000264.000000264.0264.000000
mean9.086868e+0523677.344697165.1931820.0782.228263
std6.387678e+0527013.483721623.9309920.0292.751640
min2.287597e+050.00000063.0000000.0537.600000
25%4.723958e+050.00000099.0000000.0606.521741
50%7.781768e+0518807.000000114.0000000.0679.597985
75%1.054430e+0642423.750000133.2500000.0851.400000
max3.844586e+0699316.00000010202.0000000.02893.239678
\n", + "
" + ], + "text/plain": [ + " Customer Lifetime Value Income Monthly Premium Auto \\\n", + "count 2.640000e+02 264.000000 264.000000 \n", + "mean 9.086868e+05 23677.344697 165.193182 \n", + "std 6.387678e+05 27013.483721 623.930992 \n", + "min 2.287597e+05 0.000000 63.000000 \n", + "25% 4.723958e+05 0.000000 99.000000 \n", + "50% 7.781768e+05 18807.000000 114.000000 \n", + "75% 1.054430e+06 42423.750000 133.250000 \n", + "max 3.844586e+06 99316.000000 10202.000000 \n", + "\n", + " Number of Open Complaints Total Claim Amount \n", + "count 264.0 264.000000 \n", + "mean 0.0 782.228263 \n", + "std 0.0 292.751640 \n", + "min 0.0 537.600000 \n", + "25% 0.0 606.521741 \n", + "50% 0.0 679.597985 \n", + "75% 0.0 851.400000 \n", + "max 0.0 2893.239678 " + ] + }, + "execution_count": 170, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "high_claim_df.describe()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "be0910c7-ff19-4f71-ae32-c9d49155cb0f", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python [conda env:base] *", + "language": "python", + "name": "conda-base-py" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/lab-dw-pandas.ipynb b/lab-dw-pandas.ipynb index fbd46831..998d81cb 100644 --- a/lab-dw-pandas.ipynb +++ b/lab-dw-pandas.ipynb @@ -82,12 +82,1063 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "id": "dd4e8cd8-a6f6-486c-a5c4-1745b0c035f4", "metadata": {}, "outputs": [], "source": [ - "# Your code here" + "import pandas as pd\n", + "url = \"https://raw.githubusercontent.com/data-bootcamp-v4/data/main/file1.csv\"\n", + "df = pd.read_csv(url)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "33df41fb-04e2-4193-b715-e32810267eaf", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(4008, 11)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Identify the dimensions of the dataset by determining the number of rows and columns it contains.\n", + "\n", + "df.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "ff702199-66f9-4d45-bea7-9e051dd040e9", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "RangeIndex: 4008 entries, 0 to 4007\n", + "Data columns (total 11 columns):\n", + " # Column Non-Null Count Dtype \n", + "--- ------ -------------- ----- \n", + " 0 Customer 1071 non-null object \n", + " 1 ST 1071 non-null object \n", + " 2 GENDER 954 non-null object \n", + " 3 Education 1071 non-null object \n", + " 4 Customer Lifetime Value 1068 non-null object \n", + " 5 Income 1071 non-null float64\n", + " 6 Monthly Premium Auto 1071 non-null float64\n", + " 7 Number of Open Complaints 1071 non-null object \n", + " 8 Policy Type 1071 non-null object \n", + " 9 Vehicle Class 1071 non-null object \n", + " 10 Total Claim Amount 1071 non-null float64\n", + "dtypes: float64(3), object(8)\n", + "memory usage: 344.6+ KB\n" + ] + } + ], + "source": [ + "#Determine the data types of each column and evaluate whether they are appropriate for the nature of the variable. \n", + "#You should also provide suggestions for fixing any incorrect data types.\n", + "\n", + "df.info()" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "b342bedb-6843-4143-9955-f58ebc33dad6", + "metadata": {}, + "outputs": [], + "source": [ + "#Suggestions for data cleaning: \n", + "#Some of the columns have different formats like ST and GENDER\n", + "#Column \"Number of Open Complaints\" should be type integers\n", + "#Column \"Customer Lifetime Value\" should be type float" + ] + }, + { + "cell_type": "code", + "execution_count": 96, + "id": "ea91c5f7-659a-40b5-a3fb-36f4e9c983b4", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([0])" + ] + }, + "execution_count": 96, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Number of Open Complaints'].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "id": "3ee4592c-cc92-47e4-adf6-b92558a4b80a", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "int64\n" + ] + } + ], + "source": [ + "# this function cleans the column \"Number of Open Complaints\" removing the slashes and accessing the middle character (second)\n", + "# and categorizes it to integers\n", + "\n", + "def clean_complaints(value):\n", + " try:\n", + " return int(value.split('/')[1])\n", + " except:\n", + " return 0\n", + "\n", + "df['Number of Open Complaints'] = list(map(clean_complaints, df['Number of Open Complaints']))\n", + "\n", + "print(df['Number of Open Complaints'].dtype)" + ] + }, + { + "cell_type": "code", + "execution_count": 103, + "id": "33d52e4c-ad8b-4b27-b27e-a31386e005a1", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([ nan, 697953.59, 1288743.17, ..., 2031499.76, 323912.47,\n", + " 899704.02])" + ] + }, + "execution_count": 103, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Customer Lifetime Value'].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 106, + "id": "7c810335-49c0-4a44-b587-d8013649576f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "float64\n" + ] + } + ], + "source": [ + "def clean_clv(value):\n", + " try:\n", + " return float(value)\n", + " except:\n", + " value_cleaned = value.strip('%')\n", + " return float(value_cleaned)\n", + "\n", + "df['Customer Lifetime Value'] = list(map(clean_clv, df['Customer Lifetime Value']))\n", + "\n", + "print(df['Customer Lifetime Value'].dtype)" + ] + }, + { + "cell_type": "code", + "execution_count": 108, + "id": "bd658ef3-4bb7-4cf5-8ffb-45fd8dbb7141", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Customer 1071\n", + "State 5\n", + "Gender 2\n", + "Education 5\n", + "Customer Lifetime Value 1027\n", + "Income 774\n", + "Monthly Premium Auto 132\n", + "Number of Open Complaints 1\n", + "Policy Type 3\n", + "Vehicle Class 6\n", + "Total Claim Amount 761\n", + "dtype: int64" + ] + }, + "execution_count": 108, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Identify the number of unique values for each column and determine which columns appear to be categorical. \n", + "#You should also describe the unique values of each categorical column and the range of values for numerical columns, and give your insights.\n", + "df.nunique()" + ] + }, + { + "cell_type": "code", + "execution_count": 109, + "id": "abb686ce-25e4-49b8-bffc-e3065a40b788", + "metadata": {}, + "outputs": [], + "source": [ + "#rename the column Gender and State\n", + "\n", + "rename_map = {\n", + " 'ST': 'State',\n", + " 'GENDER': 'Gender'\n", + "}\n", + "\n", + "\n", + "df.columns = [rename_map.get(col, col) for col in df.columns]" + ] + }, + { + "cell_type": "code", + "execution_count": 110, + "id": "6a632747-e595-4b21-b2b8-bcd33739ccb7", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
CustomerStateGenderEducationCustomer Lifetime ValueIncomeMonthly Premium AutoNumber of Open ComplaintsPolicy TypeVehicle ClassTotal Claim Amount
0RB50392WashingtonNaNMasterNaN0.01000.00Personal AutoFour-Door Car2.704934
1QZ44356ArizonaFBachelor697953.590.094.00Personal AutoFour-Door Car1131.464935
2AI49188NevadaFBachelor1288743.1748767.0108.00Personal AutoTwo-Door Car566.472247
3WW63253CaliforniaMBachelor764586.180.0106.00Corporate AutoSUV529.881344
4GA49547WashingtonMHigh School or Below536307.6536357.068.00Personal AutoFour-Door Car17.269323
\n", + "
" + ], + "text/plain": [ + " Customer State Gender Education Customer Lifetime Value \\\n", + "0 RB50392 Washington NaN Master NaN \n", + "1 QZ44356 Arizona F Bachelor 697953.59 \n", + "2 AI49188 Nevada F Bachelor 1288743.17 \n", + "3 WW63253 California M Bachelor 764586.18 \n", + "4 GA49547 Washington M High School or Below 536307.65 \n", + "\n", + " Income Monthly Premium Auto Number of Open Complaints Policy Type \\\n", + "0 0.0 1000.0 0 Personal Auto \n", + "1 0.0 94.0 0 Personal Auto \n", + "2 48767.0 108.0 0 Personal Auto \n", + "3 0.0 106.0 0 Corporate Auto \n", + "4 36357.0 68.0 0 Personal Auto \n", + "\n", + " Vehicle Class Total Claim Amount \n", + "0 Four-Door Car 2.704934 \n", + "1 Four-Door Car 1131.464935 \n", + "2 Two-Door Car 566.472247 \n", + "3 SUV 529.881344 \n", + "4 Four-Door Car 17.269323 " + ] + }, + "execution_count": 110, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#check columns \n", + "\n", + "df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "id": "919ffee7-d5ea-4650-9128-e9b9dbd73d1a", + "metadata": {}, + "outputs": [], + "source": [ + "#Categorical columns are \"State, Gender, Education, Policy Type, Vehicle Class\"" + ] + }, + { + "cell_type": "code", + "execution_count": 111, + "id": "80e35e99-9abc-46f4-b11c-4b878e1c02c8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Washington', 'Arizona', 'Nevada', 'California', 'Oregon', nan],\n", + " dtype=object)" + ] + }, + "execution_count": 111, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#get unique values \n", + "\n", + "df.State.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 112, + "id": "31b3834e-d200-49d0-8b82-97e73968521c", + "metadata": {}, + "outputs": [], + "source": [ + "#cleanup data using a Dictionary\n", + "\n", + "state_map = {\n", + " 'AZ': 'Arizona',\n", + " 'Cali': 'California',\n", + " 'WA': 'Washington'\n", + "}\n", + "df['State'] = df['State'].map(state_map).fillna(df['State'])" + ] + }, + { + "cell_type": "code", + "execution_count": 113, + "id": "502f30ed-2dff-4901-b3d4-098389ffed8f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Washington', 'Arizona', 'Nevada', 'California', 'Oregon', nan],\n", + " dtype=object)" + ] + }, + "execution_count": 113, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.State.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 114, + "id": "29ca7495-89fa-40ea-a24f-60e7ac98feb5", + "metadata": {}, + "outputs": [], + "source": [ + "#The 5 different STATES present are Washington, Arizona, Nevada, California and Oregon" + ] + }, + { + "cell_type": "code", + "execution_count": 115, + "id": "9481c2c1-440e-426e-82b7-ce3b06897fed", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([nan, 'F', 'M'], dtype=object)" + ] + }, + "execution_count": 115, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.Gender.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 116, + "id": "18812464-7e70-47b5-b8ab-15a7e7877002", + "metadata": {}, + "outputs": [], + "source": [ + "gender_map = {\n", + " 'F': 'F',\n", + " 'M': 'M',\n", + " 'Femal': 'F',\n", + " 'Male': 'M',\n", + " 'female': 'F'\n", + "}\n", + "df['Gender'] = df['Gender'].map(gender_map)" + ] + }, + { + "cell_type": "code", + "execution_count": 117, + "id": "9293d6cb-0ea7-408c-bcb2-3611452d94e3", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([nan, 'F', 'M'], dtype=object)" + ] + }, + "execution_count": 117, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.Gender.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 118, + "id": "410bc527-1559-445d-a57a-ff2b83fd3767", + "metadata": {}, + "outputs": [], + "source": [ + "#There are two genders, Male and Female" + ] + }, + { + "cell_type": "code", + "execution_count": 119, + "id": "06fd0b16-b2db-45d1-88ef-1f4a66ee30da", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Master', 'Bachelor', 'High School or Below', 'College', 'Doctor',\n", + " nan], dtype=object)" + ] + }, + "execution_count": 119, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.Education.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 120, + "id": "348d7e9b-1d3a-4af6-925c-35215b7facd7", + "metadata": {}, + "outputs": [], + "source": [ + "education_map = {\n", + " 'Bachelors': 'Bachelor',\n", + " 'Bachelor': 'Bachelor',\n", + " 'Master': 'Master',\n", + " 'High School or Below': 'High School or Below',\n", + " 'College': 'College',\n", + " 'Doctor': 'Doctor'\n", + "}\n", + "df['Education'] = df['Education'].map(education_map)" + ] + }, + { + "cell_type": "code", + "execution_count": 121, + "id": "7e3f088d-77cb-4b4c-beaa-5bcffce4da90", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Master', 'Bachelor', 'High School or Below', 'College', 'Doctor',\n", + " nan], dtype=object)" + ] + }, + "execution_count": 121, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.Education.unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 122, + "id": "9a169cd3-2a11-4f18-96ae-c1b6095f1c31", + "metadata": {}, + "outputs": [], + "source": [ + "#There are 5 Education levels, Master, Bachelor, High School or Below, College and Doctor" + ] + }, + { + "cell_type": "code", + "execution_count": 123, + "id": "8baa1659-b923-4f88-8142-c394a1c7233d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Personal Auto', 'Corporate Auto', 'Special Auto', nan],\n", + " dtype=object)" + ] + }, + "execution_count": 123, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Policy Type'].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 124, + "id": "863f9fe1-3d2e-480b-9e85-bf8f7e931fae", + "metadata": {}, + "outputs": [], + "source": [ + "#The three policy types are Personal, Corporate, and Special Auto" + ] + }, + { + "cell_type": "code", + "execution_count": 125, + "id": "908564f3-2aea-4d08-9ea2-af5379c9a7d8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Four-Door Car', 'Two-Door Car', 'SUV', 'Luxury SUV', 'Sports Car',\n", + " 'Luxury Car', nan], dtype=object)" + ] + }, + "execution_count": 125, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Vehicle Class'].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 126, + "id": "3b37da5e-6c6f-43fa-a95f-44bca0ff55f2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Customer Lifetime ValueIncomeMonthly Premium AutoNumber of Open ComplaintsTotal Claim Amount
count1.068000e+031071.0000001071.0000004008.01071.000000
mean7.936903e+0539295.701214193.2343600.0404.986909
std6.434784e+0530469.4270601601.1903690.0293.027260
min2.004351e+050.00000061.0000000.00.382107
25%4.034080e+0514072.00000068.0000000.0202.157702
50%5.881742e+0536234.00000083.0000000.0354.729129
75%8.962872e+0564631.000000109.5000000.0532.800000
max5.816655e+0699960.00000035354.0000000.02893.239678
\n", + "
" + ], + "text/plain": [ + " Customer Lifetime Value Income Monthly Premium Auto \\\n", + "count 1.068000e+03 1071.000000 1071.000000 \n", + "mean 7.936903e+05 39295.701214 193.234360 \n", + "std 6.434784e+05 30469.427060 1601.190369 \n", + "min 2.004351e+05 0.000000 61.000000 \n", + "25% 4.034080e+05 14072.000000 68.000000 \n", + "50% 5.881742e+05 36234.000000 83.000000 \n", + "75% 8.962872e+05 64631.000000 109.500000 \n", + "max 5.816655e+06 99960.000000 35354.000000 \n", + "\n", + " Number of Open Complaints Total Claim Amount \n", + "count 4008.0 1071.000000 \n", + "mean 0.0 404.986909 \n", + "std 0.0 293.027260 \n", + "min 0.0 0.382107 \n", + "25% 0.0 202.157702 \n", + "50% 0.0 354.729129 \n", + "75% 0.0 532.800000 \n", + "max 0.0 2893.239678 " + ] + }, + "execution_count": 126, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Compute summary statistics such as mean, median, mode, standard deviation, and quartiles to understand the central tendency and distribution of the data for numerical columns. \n", + "#You should also provide your conclusions based on these summary statistics.\n", + "\n", + "df.describe()" + ] + }, + { + "cell_type": "code", + "execution_count": 127, + "id": "bca423a5-9cfa-4192-8c9a-86883df6a6f0", + "metadata": {}, + "outputs": [], + "source": [ + "#For the Monthly Premium Auto column, I noticed the mean is 193.23, which is higher than the median at 83. \n", + "#This suggests that while most customers pay a premium around 83, a few customers pay an extremely high premium, which pulls the average up.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 128, + "id": "23d6541f-ef2f-4c5f-99ba-b5f8f45ea3e9", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Customer 1071\n", + "State 5\n", + "Gender 2\n", + "Education 5\n", + "Customer Lifetime Value 1027\n", + "Income 774\n", + "Monthly Premium Auto 132\n", + "Number of Open Complaints 1\n", + "Policy Type 3\n", + "Vehicle Class 6\n", + "Total Claim Amount 761\n", + "dtype: int64" + ] + }, + "execution_count": 128, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Compute summary statistics for categorical columns and providing your conclusions based on these statistics.\n", + "df.nunique()" + ] + }, + { + "cell_type": "code", + "execution_count": 129, + "id": "17e341d3-7783-4713-bf13-7abd52511906", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "State\n", + "California 331\n", + "Oregon 320\n", + "Arizona 211\n", + "Washington 111\n", + "Nevada 98\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 129, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['State'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 130, + "id": "91fe5a60-eb78-44c2-9606-4ca59793d69c", + "metadata": {}, + "outputs": [], + "source": [ + "#Most of customers are from California" + ] + }, + { + "cell_type": "code", + "execution_count": 131, + "id": "a485e070-846d-466a-974c-07b71f8b835d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Gender\n", + "F 502\n", + "M 452\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 131, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Gender'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 132, + "id": "1ac01890-dbdc-4b65-8791-01ca6c736dd8", + "metadata": {}, + "outputs": [], + "source": [ + "#The number of Female and Male customer is balanced even though there are more Female" + ] + }, + { + "cell_type": "code", + "execution_count": 133, + "id": "00c32bdc-17d8-4454-837d-c8526a437aff", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Education\n", + "Bachelor 331\n", + "College 313\n", + "High School or Below 296\n", + "Master 94\n", + "Doctor 37\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 133, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Education'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 134, + "id": "fe2c2235-317a-4896-8e13-7232889ba19a", + "metadata": {}, + "outputs": [], + "source": [ + "#Most of customers education level is either Bachelor or College " + ] + }, + { + "cell_type": "code", + "execution_count": 135, + "id": "2e5b82bb-e1d1-4b6b-9355-afb9b5a88f05", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Policy Type\n", + "Personal Auto 780\n", + "Corporate Auto 234\n", + "Special Auto 57\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 135, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Policy Type'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 136, + "id": "63c4bae3-e1c8-4933-acdf-e575b820d3bf", + "metadata": {}, + "outputs": [], + "source": [ + "#Personal Auto is the most common type among customers\"" + ] + }, + { + "cell_type": "code", + "execution_count": 137, + "id": "6bed82fc-88a4-4a15-821c-3fced12353ed", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Vehicle Class\n", + "Four-Door Car 576\n", + "Two-Door Car 205\n", + "SUV 199\n", + "Sports Car 57\n", + "Luxury SUV 20\n", + "Luxury Car 14\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 137, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Vehicle Class'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 138, + "id": "100c9448-b259-403e-8781-bfbf71774c6d", + "metadata": {}, + "outputs": [], + "source": [ + "#Four-Door Car is the most common Vehicle Class among customers\"" + ] + }, + { + "cell_type": "code", + "execution_count": 139, + "id": "07835d92-3a3f-45cf-9ab1-f87c8cff7b83", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "RangeIndex: 4008 entries, 0 to 4007\n", + "Data columns (total 11 columns):\n", + " # Column Non-Null Count Dtype \n", + "--- ------ -------------- ----- \n", + " 0 Customer 1071 non-null object \n", + " 1 State 1071 non-null object \n", + " 2 Gender 954 non-null object \n", + " 3 Education 1071 non-null object \n", + " 4 Customer Lifetime Value 1068 non-null float64\n", + " 5 Income 1071 non-null float64\n", + " 6 Monthly Premium Auto 1071 non-null float64\n", + " 7 Number of Open Complaints 4008 non-null int64 \n", + " 8 Policy Type 1071 non-null object \n", + " 9 Vehicle Class 1071 non-null object \n", + " 10 Total Claim Amount 1071 non-null float64\n", + "dtypes: float64(4), int64(1), object(6)\n", + "memory usage: 344.6+ KB\n" + ] + } + ], + "source": [ + "df.info()" ] }, { @@ -116,12 +1167,56 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 158, "id": "2dca5073-4520-4f42-9390-4b92733284ed", "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "State\n", + "California 331\n", + "Oregon 320\n", + "Arizona 211\n", + "Washington 111\n", + "Nevada 98\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 158, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['State'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 159, + "id": "eda4c29a-2056-4942-b86b-ba0b319fe40d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "State\n", + "Nevada 98\n", + "Washington 111\n", + "Arizona 211\n", + "Oregon 320\n", + "California 331\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 159, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ - "# Your code here" + "df['State'].value_counts().sort_values(ascending=True)" ] }, { @@ -146,12 +1241,48 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 157, "id": "bcfad6c1-9af2-4b0b-9aa9-0dc5c17473c0", "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "Policy Type\n", + "Personal Auto 780\n", + "Corporate Auto 234\n", + "Special Auto 57\n", + "Name: count, dtype: int64" + ] + }, + "execution_count": 157, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ - "# Your code here" + "df['Policy Type'].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": 160, + "id": "21077203-f396-4b32-bc70-107021b144f4", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Personal Auto'" + ] + }, + "execution_count": 160, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Policy Type'].value_counts().index[0]" ] }, { @@ -176,12 +1307,64 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 161, "id": "0c0563cf-6f8b-463d-a321-651a972f82e5", "metadata": {}, "outputs": [], "source": [ - "# Your code here" + "personal_auto_df = df.loc[df['Policy Type'] == 'Personal Auto']" + ] + }, + { + "cell_type": "code", + "execution_count": 162, + "id": "cf615079-eb4c-4d27-ac72-c80b4a4eea2c", + "metadata": {}, + "outputs": [], + "source": [ + "corporate_auto_df = df.loc[df['Policy Type'] == 'Corporate Auto']" + ] + }, + { + "cell_type": "code", + "execution_count": 165, + "id": "4de71be3-61f0-4fa5-abc1-55b142beca82", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "np.float64(38180.69871794872)" + ] + }, + "execution_count": 165, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "personal_auto_df['Income'].mean()" + ] + }, + { + "cell_type": "code", + "execution_count": 166, + "id": "35ce2448-f56b-40c3-8be3-8a698cf1ee0a", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "np.float64(41390.31196581197)" + ] + }, + "execution_count": 166, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "corporate_auto_df['Income'].mean()" ] }, { @@ -226,20 +1409,191 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 168, "id": "b731bca6-a760-4860-a27b-a33efa712ce0", "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "count 1071.000000\n", + "mean 404.986909\n", + "std 293.027260\n", + "min 0.382107\n", + "25% 202.157702\n", + "50% 354.729129\n", + "75% 532.800000\n", + "max 2893.239678\n", + "Name: Total Claim Amount, dtype: float64" + ] + }, + "execution_count": 168, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Total Claim Amount'].describe()" + ] + }, + { + "cell_type": "code", + "execution_count": 169, + "id": "010d67da-97ed-4f04-81e6-1a287e4893b6", + "metadata": {}, "outputs": [], "source": [ - "# Your code here" + "high_claim_df = df.loc[df['Total Claim Amount'] > 532.80]" + ] + }, + { + "cell_type": "code", + "execution_count": 170, + "id": "42957161-c840-4745-b7bb-298eee0234a0", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Customer Lifetime ValueIncomeMonthly Premium AutoNumber of Open ComplaintsTotal Claim Amount
count2.640000e+02264.000000264.000000264.0264.000000
mean9.086868e+0523677.344697165.1931820.0782.228263
std6.387678e+0527013.483721623.9309920.0292.751640
min2.287597e+050.00000063.0000000.0537.600000
25%4.723958e+050.00000099.0000000.0606.521741
50%7.781768e+0518807.000000114.0000000.0679.597985
75%1.054430e+0642423.750000133.2500000.0851.400000
max3.844586e+0699316.00000010202.0000000.02893.239678
\n", + "
" + ], + "text/plain": [ + " Customer Lifetime Value Income Monthly Premium Auto \\\n", + "count 2.640000e+02 264.000000 264.000000 \n", + "mean 9.086868e+05 23677.344697 165.193182 \n", + "std 6.387678e+05 27013.483721 623.930992 \n", + "min 2.287597e+05 0.000000 63.000000 \n", + "25% 4.723958e+05 0.000000 99.000000 \n", + "50% 7.781768e+05 18807.000000 114.000000 \n", + "75% 1.054430e+06 42423.750000 133.250000 \n", + "max 3.844586e+06 99316.000000 10202.000000 \n", + "\n", + " Number of Open Complaints Total Claim Amount \n", + "count 264.0 264.000000 \n", + "mean 0.0 782.228263 \n", + "std 0.0 292.751640 \n", + "min 0.0 537.600000 \n", + "25% 0.0 606.521741 \n", + "50% 0.0 679.597985 \n", + "75% 0.0 851.400000 \n", + "max 0.0 2893.239678 " + ] + }, + "execution_count": 170, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "high_claim_df.describe()" ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "be0910c7-ff19-4f71-ae32-c9d49155cb0f", + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python [conda env:base] *", "language": "python", - "name": "python3" + "name": "conda-base-py" }, "language_info": { "codemirror_mode": { @@ -251,7 +1605,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.13" + "version": "3.13.5" } }, "nbformat": 4,