From 5e2b868e7d1dd89ca2f644edcaa3d4f890577fb7 Mon Sep 17 00:00:00 2001
From: Rui Braz <ruibraz@MacBook-Pro-de-Rui.local>
Date: Tue, 30 Sep 2025 17:39:29 +0100
Subject: [PATCH] Solved lab

---
 .../lab-web-scraping-checkpoint.ipynb         | 835 +++++++++++++++
 lab-web-scraping.ipynb                        | 957 +++++++++++++++---
 2 files changed, 1657 insertions(+), 135 deletions(-)
 create mode 100644 .ipynb_checkpoints/lab-web-scraping-checkpoint.ipynb

diff --git a/.ipynb_checkpoints/lab-web-scraping-checkpoint.ipynb b/.ipynb_checkpoints/lab-web-scraping-checkpoint.ipynb
new file mode 100644
index 0000000..e2013a6
--- /dev/null
+++ b/.ipynb_checkpoints/lab-web-scraping-checkpoint.ipynb
@@ -0,0 +1,835 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786",
+   "metadata": {
+    "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786"
+   },
+   "source": [
+    "# Lab | Web Scraping"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ce8882fc-4815-4567-92fa-b4816358ba7d",
+   "metadata": {
+    "id": "ce8882fc-4815-4567-92fa-b4816358ba7d"
+   },
+   "source": [
+    "Welcome to the \"Books to Scrape\" Web Scraping Adventure Lab!\n",
+    "\n",
+    "**Objective**\n",
+    "\n",
+    "In this lab, we will embark on a mission to unearth valuable insights from the data available on Books to Scrape, an online platform showcasing a wide variety of books. As data analyst, you have been tasked with scraping a specific subset of book data from Books to Scrape to assist publishing companies in understanding the landscape of highly-rated books across different genres. Your insights will help shape future book marketing strategies and publishing decisions.\n",
+    "\n",
+    "**Background**\n",
+    "\n",
+    "In a world where data has become the new currency, businesses are leveraging big data to make informed decisions that drive success and profitability. The publishing industry, much like others, utilizes data analytics to understand market trends, reader preferences, and the performance of books based on factors such as genre, author, and ratings. Books to Scrape serves as a rich source of such data, offering detailed information about a diverse range of books, making it an ideal platform for extracting insights to aid in informed decision-making within the literary world.\n",
+    "\n",
+    "**Task**\n",
+    "\n",
+    "Your task is to create a Python script using BeautifulSoup and pandas to scrape Books to Scrape book data, focusing on book ratings and genres. The script should be able to filter books with ratings above a certain threshold and in specific genres. Additionally, the script should structure the scraped data in a tabular format using pandas for further analysis.\n",
+    "\n",
+    "**Expected Outcome**\n",
+    "\n",
+    "A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`. The function should scrape book data from the \"Books to Scrape\" website and return a `pandas` DataFrame with the following columns:\n",
+    "\n",
+    "**Expected Outcome**\n",
+    "\n",
+    "- A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`.\n",
+    "- The function should return a DataFrame with the following columns:\n",
+    "  - **UPC**: The Universal Product Code (UPC) of the book.\n",
+    "  - **Title**: The title of the book.\n",
+    "  - **Price (£)**: The price of the book in pounds.\n",
+    "  - **Rating**: The rating of the book (1-5 stars).\n",
+    "  - **Genre**: The genre of the book.\n",
+    "  - **Availability**: Whether the book is in stock or not.\n",
+    "  - **Description**: A brief description or product description of the book (if available).\n",
+    "  \n",
+    "You will execute this script to scrape data for books with a minimum rating of `4.0 and above` and a maximum price of `£20`. \n",
+    "\n",
+    "Remember to experiment with different ratings and prices to ensure your code is versatile and can handle various searches effectively!\n",
+    "\n",
+    "**Resources**\n",
+    "\n",
+    "- [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)\n",
+    "- [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/index.html)\n",
+    "- [Books to Scrape](https://books.toscrape.com/)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3519921d-5890-445b-9a33-934ed8ee378c",
+   "metadata": {
+    "id": "3519921d-5890-445b-9a33-934ed8ee378c"
+   },
+   "source": [
+    "**Hint**\n",
+    "\n",
+    "Your first mission is to familiarize yourself with the **Books to Scrape** website. Navigate to [Books to Scrape](http://books.toscrape.com/) and explore the available books to understand their layout and structure. \n",
+    "\n",
+    "Next, think about how you can set parameters for your data extraction:\n",
+    "\n",
+    "- **Minimum Rating**: Focus on books with a rating of 4.0 and above.\n",
+    "- **Maximum Price**: Filter for books priced up to £20.\n",
+    "\n",
+    "After reviewing the site, you can construct a plan for scraping relevant data. Pay attention to the details displayed for each book, including the title, price, rating, and availability. This will help you identify the correct HTML elements to target with your scraping script.\n",
+    "\n",
+    "Make sure to build your scraping URL and logic based on the patterns you observe in the HTML structure of the book listings!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "25a83a0d-a742-49f6-985e-e27887cbf922",
+   "metadata": {
+    "id": "25a83a0d-a742-49f6-985e-e27887cbf922"
+   },
+   "source": [
+    "\n",
+    "---\n",
+    "\n",
+    "**Best of luck! Immerse yourself in the world of books, and may the data be with you!**"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0",
+   "metadata": {
+    "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0"
+   },
+   "source": [
+    "**Important Note**:\n",
+    "\n",
+    "In the fast-changing online world, websites often update and change their structures. When you try this lab, the **Books to Scrape** website might differ from what you expect.\n",
+    "\n",
+    "If you encounter issues due to these changes, like new rules or obstacles preventing data extraction, don’t worry! Get creative.\n",
+    "\n",
+    "You can choose another website that interests you and is suitable for scraping data. Options like Wikipedia, The New York Times, or even library databases are great alternatives. The main goal remains the same: extract useful data and enhance your web scraping skills while exploring a source of information you enjoy. This is your opportunity to practice and adapt to different web environments!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "40359eee-9cd7-4884-bfa4-83344c222305",
+   "metadata": {
+    "id": "40359eee-9cd7-4884-bfa4-83344c222305"
+   },
+   "outputs": [],
+   "source": [
+    "import requests\n",
+    "from bs4 import BeautifulSoup\n",
+    "import pandas as pd"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "4b44f27a-ebe5-491c-a41f-1173e75d235c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "url = \"https://books.toscrape.com/\"\n",
+    "response = requests.get(url)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "id": "ff20a80a-a350-4184-87ae-fde78db4371d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "200\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(response.status_code)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "id": "70159878-def0-4527-bc44-dba1609ff928",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "soup = BeautifulSoup(response.content, \"html.parser\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "id": "b038d411-fcfd-43dd-8863-5b8f6901b013",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "http://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html\n"
+     ]
+    }
+   ],
+   "source": [
+    "# The container for each book is an <article> tag with the class 'product_pod'\n",
+    "book_containers = soup.find_all('article', class_='product_pod')\n",
+    "\n",
+    "# Create a list to hold all the full links\n",
+    "book_links = []\n",
+    "base_url = \"http://books.toscrape.com/\"\n",
+    "\n",
+    "for book in book_containers:\n",
+    "    # Find the <a> tag inside the book's h3 tag\n",
+    "    a_tag = book.h3.a\n",
+    "    # Get the partial link from the href attribute\n",
+    "    partial_link = a_tag.get('href')\n",
+    "    # Create the full link and add it to our list\n",
+    "    full_link = base_url + partial_link\n",
+    "    book_links.append(full_link)\n",
+    "\n",
+    "print(book_links[0]) # see the first full link"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "id": "ffb0ff19-4b49-4d8c-b645-0eead4446016",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Successfully fetched: http://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html\n"
+     ]
+    }
+   ],
+   "source": [
+    "first_book_url = book_links[0]\n",
+    "\n",
+    "response = requests.get(first_book_url)\n",
+    "book_soup = BeautifulSoup(response.content, \"html.parser\")\n",
+    "\n",
+    "print(\"Successfully fetched:\", first_book_url)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 30,
+   "id": "7ac15680-ae41-4810-96db-57128155ac55",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Found UPC: a897fe39b1053632\n",
+      "Found Availability: In stock (22 available)\n"
+     ]
+    }
+   ],
+   "source": [
+    "product_table = book_soup.find('table', class_='table-striped')\n",
+    "\n",
+    "for row in product_table.find_all('tr'):\n",
+    "    header = row.find('th').get_text()\n",
+    "    if header == 'UPC':\n",
+    "        upc = row.find('td').get_text()\n",
+    "        print(f\"Found UPC: {upc}\")\n",
+    "    elif header == 'Availability':\n",
+    "        availability = row.find('td').get_text()\n",
+    "        print(f\"Found Availability: {availability}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "id": "faebbee1-5fb4-4f1e-b97a-3d94778936e4",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. Silverstein's humorous and creative verse can amuse the dowdiest of readers. Lemon-faced adults and fidgety kids sit still and read these rhythmic words and laugh and smile and love th It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. Silverstein's humorous and creative verse can amuse the dowdiest of readers. Lemon-faced adults and fidgety kids sit still and read these rhythmic words and laugh and smile and love that Silverstein. Need proof of his genius? RockabyeRockabye baby, in the treetopDon't you know a treetopIs no safe place to rock?And who put you up there,And your cradle, too?Baby, I think someone down here'sGot it in for you. Shel, you never sounded so good. ...more\n"
+     ]
+    }
+   ],
+   "source": [
+    "description_div = book_soup.find('div', id='product_description')\n",
+    "\n",
+    "description_p = description_div.find_next_sibling('p')\n",
+    "\n",
+    "description = description_p.get_text(strip = True)\n",
+    "\n",
+    "print(description)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 31,
+   "id": "3b8b74c5-70ca-4aa7-9d82-49f4f49edd86",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "base_url = \"http://books.toscrape.com/\"\n",
+    "catalogue_url = \"http://books.toscrape.com/catalogue/\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "id": "1fa634f5-50b4-482e-a712-9bb8028c39ad",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>UPC</th>\n",
+       "      <th>Title</th>\n",
+       "      <th>Price (£)</th>\n",
+       "      <th>Rating</th>\n",
+       "      <th>Genre</th>\n",
+       "      <th>Availability</th>\n",
+       "      <th>Description</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>a897fe39b1053632</td>\n",
+       "      <td>A Light in the Attic</td>\n",
+       "      <td>51.77</td>\n",
+       "      <td>3</td>\n",
+       "      <td>N/A</td>\n",
+       "      <td>In stock (22 available)</td>\n",
+       "      <td>It's hard to imagine a world without A Light i...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>90fa61229261140a</td>\n",
+       "      <td>Tipping the Velvet</td>\n",
+       "      <td>53.74</td>\n",
+       "      <td>1</td>\n",
+       "      <td>N/A</td>\n",
+       "      <td>In stock (20 available)</td>\n",
+       "      <td>\"Erotic and absorbing...Written with starling ...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>6957f44c3847a760</td>\n",
+       "      <td>Soumission</td>\n",
+       "      <td>50.10</td>\n",
+       "      <td>1</td>\n",
+       "      <td>N/A</td>\n",
+       "      <td>In stock (20 available)</td>\n",
+       "      <td>Dans une France assez proche de la nôtre, un h...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>3</th>\n",
+       "      <td>e00eb4fd7b871a48</td>\n",
+       "      <td>Sharp Objects</td>\n",
+       "      <td>47.82</td>\n",
+       "      <td>4</td>\n",
+       "      <td>N/A</td>\n",
+       "      <td>In stock (20 available)</td>\n",
+       "      <td>WICKED above her hipbone, GIRL across her hear...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4</th>\n",
+       "      <td>4165285e1663650f</td>\n",
+       "      <td>Sapiens: A Brief History of Humankind</td>\n",
+       "      <td>54.23</td>\n",
+       "      <td>5</td>\n",
+       "      <td>N/A</td>\n",
+       "      <td>In stock (20 available)</td>\n",
+       "      <td>From a renowned historian comes a groundbreaki...</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "                UPC                                  Title  Price (£)  Rating  \\\n",
+       "0  a897fe39b1053632                   A Light in the Attic      51.77       3   \n",
+       "1  90fa61229261140a                     Tipping the Velvet      53.74       1   \n",
+       "2  6957f44c3847a760                             Soumission      50.10       1   \n",
+       "3  e00eb4fd7b871a48                          Sharp Objects      47.82       4   \n",
+       "4  4165285e1663650f  Sapiens: A Brief History of Humankind      54.23       5   \n",
+       "\n",
+       "  Genre             Availability  \\\n",
+       "0   N/A  In stock (22 available)   \n",
+       "1   N/A  In stock (20 available)   \n",
+       "2   N/A  In stock (20 available)   \n",
+       "3   N/A  In stock (20 available)   \n",
+       "4   N/A  In stock (20 available)   \n",
+       "\n",
+       "                                         Description  \n",
+       "0  It's hard to imagine a world without A Light i...  \n",
+       "1  \"Erotic and absorbing...Written with starling ...  \n",
+       "2  Dans une France assez proche de la nôtre, un h...  \n",
+       "3  WICKED above her hipbone, GIRL across her hear...  \n",
+       "4  From a renowned historian comes a groundbreaki...  "
+      ]
+     },
+     "execution_count": 32,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# A list to store the dictionaries of book data\n",
+    "all_books_data = []\n",
+    "\n",
+    "# A dictionary to convert star ratings from text to numbers\n",
+    "rating_map = {\"One\": 1, \"Two\": 2, \"Three\": 3, \"Four\": 4, \"Five\": 5}\n",
+    "\n",
+    "# Find all book containers on the first page\n",
+    "book_containers = soup.find_all('article', class_='product_pod')\n",
+    "\n",
+    "# Loop through each book container\n",
+    "for book in book_containers:\n",
+    "    # Scrape data from the main page\n",
+    "    title = book.h3.a.get('title')\n",
+    "    price_str = book.find('p', class_='price_color').get_text(strip=True)\n",
+    "    rating_text = book.find('p', class_='star-rating').get('class')[1]\n",
+    "\n",
+    "    # Visit the detail page for more info\n",
+    "    partial_link = book.h3.a.get('href')\n",
+    "    full_link = base_url + partial_link\n",
+    "\n",
+    "    # Fetch and parse the detail page\n",
+    "    book_response = requests.get(full_link)\n",
+    "    book_soup = BeautifulSoup(book_response.content, \"html.parser\")\n",
+    "\n",
+    "    # Scrape data from the detail page\n",
+    "    product_table = book_soup.find('table', class_='table-striped')\n",
+    "    upc = product_table.find('td').get_text() # UPC is the first <td> in the table\n",
+    "    availability = product_table.find_all('td')[5].get_text(strip=True) # Availability is the 6th <td>\n",
+    "    \n",
+    "    description_div = book_soup.find('div', id='product_description')\n",
+    "    description_p = description_div.find_next_sibling('p')\n",
+    "    description = description_p.get_text(strip=True) if description_p else \"No description available\"\n",
+    "\n",
+    "    # --- Store the data in a dictionary ---\n",
+    "    book_data = {\n",
+    "        \"UPC\": upc,\n",
+    "        \"Title\": title,\n",
+    "        \"Price (£)\": float(price_str.replace(\"£\", \"\")),\n",
+    "        \"Rating\": rating_map.get(rating_text, 0), # Use .get() for safety\n",
+    "        \"Genre\": \"N/A\", # We'll get this later\n",
+    "        \"Availability\": availability,\n",
+    "        \"Description\": description\n",
+    "    }\n",
+    "\n",
+    "    all_books_data.append(book_data)\n",
+    "\n",
+    "# Convert the list of dictionaries to a DataFrame\n",
+    "df = pd.DataFrame(all_books_data)\n",
+    "df.head()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "id": "796f9063-9c08-4d6f-b1e5-7b4b0646b095",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Scraping page: http://books.toscrape.com/catalogue/page-1.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-2.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-3.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-4.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-5.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-6.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-7.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-8.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-9.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-10.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-11.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-12.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-13.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-14.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-15.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-16.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-17.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-18.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-19.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-20.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-21.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-22.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-23.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-24.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-25.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-26.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-27.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-28.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-29.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-30.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-31.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-32.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-33.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-34.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-35.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-36.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-37.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-38.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-39.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-40.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-41.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-42.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-43.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-44.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-45.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-46.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-47.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-48.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-49.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-50.html\n",
+      "\\nScraping complete!\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>UPC</th>\n",
+       "      <th>Title</th>\n",
+       "      <th>Price (£)</th>\n",
+       "      <th>Rating</th>\n",
+       "      <th>Genre</th>\n",
+       "      <th>Availability</th>\n",
+       "      <th>Description</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>ce6396b0f23f6ecc</td>\n",
+       "      <td>Set Me Free</td>\n",
+       "      <td>17.46</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Young Adult</td>\n",
+       "      <td>In stock (19 available)</td>\n",
+       "      <td>Aaron Ledbetter’s future had been planned out ...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>6258a1f6a6dcfe50</td>\n",
+       "      <td>The Four Agreements: A Practical Guide to Pers...</td>\n",
+       "      <td>17.66</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Spirituality</td>\n",
+       "      <td>In stock (18 available)</td>\n",
+       "      <td>In The Four Agreements, don Miguel Ruiz reveal...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>6be3beb0793a53e7</td>\n",
+       "      <td>Sophie's World</td>\n",
+       "      <td>15.94</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Philosophy</td>\n",
+       "      <td>In stock (18 available)</td>\n",
+       "      <td>A page-turning novel that is also an explorati...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>3</th>\n",
+       "      <td>657fe5ead67a7767</td>\n",
+       "      <td>Untitled Collection: Sabbath Poems 2014</td>\n",
+       "      <td>14.27</td>\n",
+       "      <td>4</td>\n",
+       "      <td>Poetry</td>\n",
+       "      <td>In stock (16 available)</td>\n",
+       "      <td>More than thirty-five years ago, when the weat...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4</th>\n",
+       "      <td>51653ef291ab7ddc</td>\n",
+       "      <td>This One Summer</td>\n",
+       "      <td>19.49</td>\n",
+       "      <td>4</td>\n",
+       "      <td>Sequential Art</td>\n",
+       "      <td>In stock (16 available)</td>\n",
+       "      <td>Every summer, Rose goes with her mom and dad t...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>...</th>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>70</th>\n",
+       "      <td>9c96cd1329fbd82d</td>\n",
+       "      <td>The Zombie Room</td>\n",
+       "      <td>19.69</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Default</td>\n",
+       "      <td>In stock (1 available)</td>\n",
+       "      <td>An unlikely bond is forged between three men f...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>71</th>\n",
+       "      <td>b78deb463531d078</td>\n",
+       "      <td>The Silent Wife</td>\n",
+       "      <td>12.34</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Fiction</td>\n",
+       "      <td>In stock (1 available)</td>\n",
+       "      <td>A chilling psychological thriller about a marr...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>72</th>\n",
+       "      <td>4280ac3eab57aa5d</td>\n",
+       "      <td>The Girl You Lost</td>\n",
+       "      <td>12.29</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Mystery</td>\n",
+       "      <td>In stock (1 available)</td>\n",
+       "      <td>Eighteen years ago your baby daughter was snat...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>73</th>\n",
+       "      <td>29fc016c459aeb14</td>\n",
+       "      <td>The Edge of Reason (Bridget Jones #2)</td>\n",
+       "      <td>19.18</td>\n",
+       "      <td>4</td>\n",
+       "      <td>Womens Fiction</td>\n",
+       "      <td>In stock (1 available)</td>\n",
+       "      <td>Monday 27 January“7:15 a.m. Hurrah! The wilder...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>74</th>\n",
+       "      <td>19fec36a1dfb4c16</td>\n",
+       "      <td>A Spy's Devotion (The Regency Spies of London #1)</td>\n",
+       "      <td>16.97</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Historical Fiction</td>\n",
+       "      <td>In stock (1 available)</td>\n",
+       "      <td>In England’s Regency era, manners and elegance...</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "<p>75 rows × 7 columns</p>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "                 UPC                                              Title  \\\n",
+       "0   ce6396b0f23f6ecc                                        Set Me Free   \n",
+       "1   6258a1f6a6dcfe50  The Four Agreements: A Practical Guide to Pers...   \n",
+       "2   6be3beb0793a53e7                                     Sophie's World   \n",
+       "3   657fe5ead67a7767            Untitled Collection: Sabbath Poems 2014   \n",
+       "4   51653ef291ab7ddc                                    This One Summer   \n",
+       "..               ...                                                ...   \n",
+       "70  9c96cd1329fbd82d                                    The Zombie Room   \n",
+       "71  b78deb463531d078                                    The Silent Wife   \n",
+       "72  4280ac3eab57aa5d                                  The Girl You Lost   \n",
+       "73  29fc016c459aeb14              The Edge of Reason (Bridget Jones #2)   \n",
+       "74  19fec36a1dfb4c16  A Spy's Devotion (The Regency Spies of London #1)   \n",
+       "\n",
+       "    Price (£)  Rating               Genre             Availability  \\\n",
+       "0       17.46       5         Young Adult  In stock (19 available)   \n",
+       "1       17.66       5        Spirituality  In stock (18 available)   \n",
+       "2       15.94       5          Philosophy  In stock (18 available)   \n",
+       "3       14.27       4              Poetry  In stock (16 available)   \n",
+       "4       19.49       4      Sequential Art  In stock (16 available)   \n",
+       "..        ...     ...                 ...                      ...   \n",
+       "70      19.69       5             Default   In stock (1 available)   \n",
+       "71      12.34       5             Fiction   In stock (1 available)   \n",
+       "72      12.29       5             Mystery   In stock (1 available)   \n",
+       "73      19.18       4      Womens Fiction   In stock (1 available)   \n",
+       "74      16.97       5  Historical Fiction   In stock (1 available)   \n",
+       "\n",
+       "                                          Description  \n",
+       "0   Aaron Ledbetter’s future had been planned out ...  \n",
+       "1   In The Four Agreements, don Miguel Ruiz reveal...  \n",
+       "2   A page-turning novel that is also an explorati...  \n",
+       "3   More than thirty-five years ago, when the weat...  \n",
+       "4   Every summer, Rose goes with her mom and dad t...  \n",
+       "..                                                ...  \n",
+       "70  An unlikely bond is forged between three men f...  \n",
+       "71  A chilling psychological thriller about a marr...  \n",
+       "72  Eighteen years ago your baby daughter was snat...  \n",
+       "73  Monday 27 January“7:15 a.m. Hurrah! The wilder...  \n",
+       "74  In England’s Regency era, manners and elegance...  \n",
+       "\n",
+       "[75 rows x 7 columns]"
+      ]
+     },
+     "execution_count": 33,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "def scrape_books(min_rating, max_price):\n",
+    "    \"\"\"\n",
+    "    Scrapes book data from books.toscrape.com across multiple pages,\n",
+    "    filters by minimum rating and maximum price, and returns a pandas DataFrame.\n",
+    "    \"\"\"\n",
+    "    \n",
+    "    # A list to store the dictionaries of book data\n",
+    "    all_books_data = []\n",
+    "    \n",
+    "    # A dictionary to convert star ratings from text to numbers\n",
+    "    rating_map = {\"One\": 1, \"Two\": 2, \"Three\": 3, \"Four\": 4, \"Five\": 5}\n",
+    "    \n",
+    "    # Start with the first page\n",
+    "    current_url = \"http://books.toscrape.com/catalogue/page-1.html\"\n",
+    "    base_url = \"http://books.toscrape.com/catalogue/\"\n",
+    "\n",
+    "    while current_url:\n",
+    "        print(f\"Scraping page: {current_url}\")\n",
+    "        response = requests.get(current_url)\n",
+    "        \n",
+    "        # Make sure the request was successful\n",
+    "        if response.status_code != 200:\n",
+    "            break\n",
+    "            \n",
+    "        soup = BeautifulSoup(response.content, \"html.parser\")\n",
+    "        \n",
+    "        # Find all book containers on the page\n",
+    "        book_containers = soup.find_all('article', class_='product_pod')\n",
+    "\n",
+    "        for book in book_containers:\n",
+    "            # Scrape rating and price first for filtering\n",
+    "            price_str = book.find('p', class_='price_color').get_text(strip=True)\n",
+    "            price = float(price_str.replace(\"£\", \"\"))\n",
+    "            \n",
+    "            rating_text = book.find('p', class_='star-rating').get('class')[1]\n",
+    "            rating = rating_map.get(rating_text, 0)\n",
+    "            \n",
+    "            # --- FILTERING LOGIC ---\n",
+    "            # If the book meets the criteria, scrape the rest of the details\n",
+    "            if rating >= min_rating and price <= max_price:\n",
+    "                \n",
+    "                # --- Scrape remaining data ---\n",
+    "                title = book.h3.a.get('title')\n",
+    "                \n",
+    "                # Visit the detail page for more info\n",
+    "                partial_link = book.h3.a.get('href')\n",
+    "                full_link = base_url + partial_link\n",
+    "                \n",
+    "                book_response = requests.get(full_link)\n",
+    "                book_soup = BeautifulSoup(book_response.content, \"html.parser\")\n",
+    "                \n",
+    "                # Scrape data from the detail page\n",
+    "                upc = book_soup.find('th', string='UPC').find_next('td').get_text()\n",
+    "                availability = book_soup.find('th', string='Availability').find_next('td').get_text(strip=True)\n",
+    "                description_p = book_soup.find('div', id='product_description').find_next_sibling('p')\n",
+    "                description = description_p.get_text(strip=True) if description_p else \"N/A\"\n",
+    "                genre = book_soup.find('ul', class_='breadcrumb').find_all('li')[2].a.get_text(strip=True)\n",
+    "\n",
+    "                # Store the data in a dictionary\n",
+    "                book_data = {\n",
+    "                    \"UPC\": upc,\n",
+    "                    \"Title\": title,\n",
+    "                    \"Price (£)\": price,\n",
+    "                    \"Rating\": rating,\n",
+    "                    \"Genre\": genre,\n",
+    "                    \"Availability\": availability,\n",
+    "                    \"Description\": description\n",
+    "                }\n",
+    "                all_books_data.append(book_data)\n",
+    "\n",
+    "        # Find the 'next' button to go to the next page\n",
+    "        next_button = soup.find('li', class_='next')\n",
+    "        if next_button:\n",
+    "            next_page_partial_url = next_button.a.get('href')\n",
+    "            current_url = base_url + next_page_partial_url\n",
+    "        else:\n",
+    "            # If no 'next' button, we are on the last page\n",
+    "            current_url = None\n",
+    "            \n",
+    "    # Convert the list of dictionaries to a DataFrame\n",
+    "    return pd.DataFrame(all_books_data)\n",
+    "\n",
+    "# --- Now, execute the function with your specified criteria ---\n",
+    "final_df = scrape_books(min_rating=4, max_price=20)\n",
+    "\n",
+    "print(\"\\\\nScraping complete!\")\n",
+    "final_df"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b3993b66-c083-49a5-ace7-f8f5508a1d72",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python [conda env:base] *",
+   "language": "python",
+   "name": "conda-base-py"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.13.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/lab-web-scraping.ipynb b/lab-web-scraping.ipynb
index e552783..e2013a6 100644
--- a/lab-web-scraping.ipynb
+++ b/lab-web-scraping.ipynb
@@ -1,148 +1,835 @@
 {
-  "cells": [
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786",
+   "metadata": {
+    "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786"
+   },
+   "source": [
+    "# Lab | Web Scraping"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ce8882fc-4815-4567-92fa-b4816358ba7d",
+   "metadata": {
+    "id": "ce8882fc-4815-4567-92fa-b4816358ba7d"
+   },
+   "source": [
+    "Welcome to the \"Books to Scrape\" Web Scraping Adventure Lab!\n",
+    "\n",
+    "**Objective**\n",
+    "\n",
+    "In this lab, we will embark on a mission to unearth valuable insights from the data available on Books to Scrape, an online platform showcasing a wide variety of books. As data analyst, you have been tasked with scraping a specific subset of book data from Books to Scrape to assist publishing companies in understanding the landscape of highly-rated books across different genres. Your insights will help shape future book marketing strategies and publishing decisions.\n",
+    "\n",
+    "**Background**\n",
+    "\n",
+    "In a world where data has become the new currency, businesses are leveraging big data to make informed decisions that drive success and profitability. The publishing industry, much like others, utilizes data analytics to understand market trends, reader preferences, and the performance of books based on factors such as genre, author, and ratings. Books to Scrape serves as a rich source of such data, offering detailed information about a diverse range of books, making it an ideal platform for extracting insights to aid in informed decision-making within the literary world.\n",
+    "\n",
+    "**Task**\n",
+    "\n",
+    "Your task is to create a Python script using BeautifulSoup and pandas to scrape Books to Scrape book data, focusing on book ratings and genres. The script should be able to filter books with ratings above a certain threshold and in specific genres. Additionally, the script should structure the scraped data in a tabular format using pandas for further analysis.\n",
+    "\n",
+    "**Expected Outcome**\n",
+    "\n",
+    "A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`. The function should scrape book data from the \"Books to Scrape\" website and return a `pandas` DataFrame with the following columns:\n",
+    "\n",
+    "**Expected Outcome**\n",
+    "\n",
+    "- A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`.\n",
+    "- The function should return a DataFrame with the following columns:\n",
+    "  - **UPC**: The Universal Product Code (UPC) of the book.\n",
+    "  - **Title**: The title of the book.\n",
+    "  - **Price (£)**: The price of the book in pounds.\n",
+    "  - **Rating**: The rating of the book (1-5 stars).\n",
+    "  - **Genre**: The genre of the book.\n",
+    "  - **Availability**: Whether the book is in stock or not.\n",
+    "  - **Description**: A brief description or product description of the book (if available).\n",
+    "  \n",
+    "You will execute this script to scrape data for books with a minimum rating of `4.0 and above` and a maximum price of `£20`. \n",
+    "\n",
+    "Remember to experiment with different ratings and prices to ensure your code is versatile and can handle various searches effectively!\n",
+    "\n",
+    "**Resources**\n",
+    "\n",
+    "- [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)\n",
+    "- [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/index.html)\n",
+    "- [Books to Scrape](https://books.toscrape.com/)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3519921d-5890-445b-9a33-934ed8ee378c",
+   "metadata": {
+    "id": "3519921d-5890-445b-9a33-934ed8ee378c"
+   },
+   "source": [
+    "**Hint**\n",
+    "\n",
+    "Your first mission is to familiarize yourself with the **Books to Scrape** website. Navigate to [Books to Scrape](http://books.toscrape.com/) and explore the available books to understand their layout and structure. \n",
+    "\n",
+    "Next, think about how you can set parameters for your data extraction:\n",
+    "\n",
+    "- **Minimum Rating**: Focus on books with a rating of 4.0 and above.\n",
+    "- **Maximum Price**: Filter for books priced up to £20.\n",
+    "\n",
+    "After reviewing the site, you can construct a plan for scraping relevant data. Pay attention to the details displayed for each book, including the title, price, rating, and availability. This will help you identify the correct HTML elements to target with your scraping script.\n",
+    "\n",
+    "Make sure to build your scraping URL and logic based on the patterns you observe in the HTML structure of the book listings!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "25a83a0d-a742-49f6-985e-e27887cbf922",
+   "metadata": {
+    "id": "25a83a0d-a742-49f6-985e-e27887cbf922"
+   },
+   "source": [
+    "\n",
+    "---\n",
+    "\n",
+    "**Best of luck! Immerse yourself in the world of books, and may the data be with you!**"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0",
+   "metadata": {
+    "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0"
+   },
+   "source": [
+    "**Important Note**:\n",
+    "\n",
+    "In the fast-changing online world, websites often update and change their structures. When you try this lab, the **Books to Scrape** website might differ from what you expect.\n",
+    "\n",
+    "If you encounter issues due to these changes, like new rules or obstacles preventing data extraction, don’t worry! Get creative.\n",
+    "\n",
+    "You can choose another website that interests you and is suitable for scraping data. Options like Wikipedia, The New York Times, or even library databases are great alternatives. The main goal remains the same: extract useful data and enhance your web scraping skills while exploring a source of information you enjoy. This is your opportunity to practice and adapt to different web environments!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "40359eee-9cd7-4884-bfa4-83344c222305",
+   "metadata": {
+    "id": "40359eee-9cd7-4884-bfa4-83344c222305"
+   },
+   "outputs": [],
+   "source": [
+    "import requests\n",
+    "from bs4 import BeautifulSoup\n",
+    "import pandas as pd"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "4b44f27a-ebe5-491c-a41f-1173e75d235c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "url = \"https://books.toscrape.com/\"\n",
+    "response = requests.get(url)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "id": "ff20a80a-a350-4184-87ae-fde78db4371d",
+   "metadata": {},
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786",
-      "metadata": {
-        "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786"
-      },
-      "source": [
-        "# Lab | Web Scraping"
-      ]
-    },
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "200\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(response.status_code)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "id": "70159878-def0-4527-bc44-dba1609ff928",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "soup = BeautifulSoup(response.content, \"html.parser\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 25,
+   "id": "b038d411-fcfd-43dd-8863-5b8f6901b013",
+   "metadata": {},
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "id": "ce8882fc-4815-4567-92fa-b4816358ba7d",
-      "metadata": {
-        "id": "ce8882fc-4815-4567-92fa-b4816358ba7d"
-      },
-      "source": [
-        "Welcome to the \"Books to Scrape\" Web Scraping Adventure Lab!\n",
-        "\n",
-        "**Objective**\n",
-        "\n",
-        "In this lab, we will embark on a mission to unearth valuable insights from the data available on Books to Scrape, an online platform showcasing a wide variety of books. As data analyst, you have been tasked with scraping a specific subset of book data from Books to Scrape to assist publishing companies in understanding the landscape of highly-rated books across different genres. Your insights will help shape future book marketing strategies and publishing decisions.\n",
-        "\n",
-        "**Background**\n",
-        "\n",
-        "In a world where data has become the new currency, businesses are leveraging big data to make informed decisions that drive success and profitability. The publishing industry, much like others, utilizes data analytics to understand market trends, reader preferences, and the performance of books based on factors such as genre, author, and ratings. Books to Scrape serves as a rich source of such data, offering detailed information about a diverse range of books, making it an ideal platform for extracting insights to aid in informed decision-making within the literary world.\n",
-        "\n",
-        "**Task**\n",
-        "\n",
-        "Your task is to create a Python script using BeautifulSoup and pandas to scrape Books to Scrape book data, focusing on book ratings and genres. The script should be able to filter books with ratings above a certain threshold and in specific genres. Additionally, the script should structure the scraped data in a tabular format using pandas for further analysis.\n",
-        "\n",
-        "**Expected Outcome**\n",
-        "\n",
-        "A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`. The function should scrape book data from the \"Books to Scrape\" website and return a `pandas` DataFrame with the following columns:\n",
-        "\n",
-        "**Expected Outcome**\n",
-        "\n",
-        "- A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`.\n",
-        "- The function should return a DataFrame with the following columns:\n",
-        "  - **UPC**: The Universal Product Code (UPC) of the book.\n",
-        "  - **Title**: The title of the book.\n",
-        "  - **Price (£)**: The price of the book in pounds.\n",
-        "  - **Rating**: The rating of the book (1-5 stars).\n",
-        "  - **Genre**: The genre of the book.\n",
-        "  - **Availability**: Whether the book is in stock or not.\n",
-        "  - **Description**: A brief description or product description of the book (if available).\n",
-        "  \n",
-        "You will execute this script to scrape data for books with a minimum rating of `4.0 and above` and a maximum price of `£20`. \n",
-        "\n",
-        "Remember to experiment with different ratings and prices to ensure your code is versatile and can handle various searches effectively!\n",
-        "\n",
-        "**Resources**\n",
-        "\n",
-        "- [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)\n",
-        "- [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/index.html)\n",
-        "- [Books to Scrape](https://books.toscrape.com/)\n"
-      ]
-    },
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "http://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html\n"
+     ]
+    }
+   ],
+   "source": [
+    "# The container for each book is an <article> tag with the class 'product_pod'\n",
+    "book_containers = soup.find_all('article', class_='product_pod')\n",
+    "\n",
+    "# Create a list to hold all the full links\n",
+    "book_links = []\n",
+    "base_url = \"http://books.toscrape.com/\"\n",
+    "\n",
+    "for book in book_containers:\n",
+    "    # Find the <a> tag inside the book's h3 tag\n",
+    "    a_tag = book.h3.a\n",
+    "    # Get the partial link from the href attribute\n",
+    "    partial_link = a_tag.get('href')\n",
+    "    # Create the full link and add it to our list\n",
+    "    full_link = base_url + partial_link\n",
+    "    book_links.append(full_link)\n",
+    "\n",
+    "print(book_links[0]) # see the first full link"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "id": "ffb0ff19-4b49-4d8c-b645-0eead4446016",
+   "metadata": {},
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "id": "3519921d-5890-445b-9a33-934ed8ee378c",
-      "metadata": {
-        "id": "3519921d-5890-445b-9a33-934ed8ee378c"
-      },
-      "source": [
-        "**Hint**\n",
-        "\n",
-        "Your first mission is to familiarize yourself with the **Books to Scrape** website. Navigate to [Books to Scrape](http://books.toscrape.com/) and explore the available books to understand their layout and structure. \n",
-        "\n",
-        "Next, think about how you can set parameters for your data extraction:\n",
-        "\n",
-        "- **Minimum Rating**: Focus on books with a rating of 4.0 and above.\n",
-        "- **Maximum Price**: Filter for books priced up to £20.\n",
-        "\n",
-        "After reviewing the site, you can construct a plan for scraping relevant data. Pay attention to the details displayed for each book, including the title, price, rating, and availability. This will help you identify the correct HTML elements to target with your scraping script.\n",
-        "\n",
-        "Make sure to build your scraping URL and logic based on the patterns you observe in the HTML structure of the book listings!"
-      ]
-    },
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Successfully fetched: http://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html\n"
+     ]
+    }
+   ],
+   "source": [
+    "first_book_url = book_links[0]\n",
+    "\n",
+    "response = requests.get(first_book_url)\n",
+    "book_soup = BeautifulSoup(response.content, \"html.parser\")\n",
+    "\n",
+    "print(\"Successfully fetched:\", first_book_url)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 30,
+   "id": "7ac15680-ae41-4810-96db-57128155ac55",
+   "metadata": {},
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "id": "25a83a0d-a742-49f6-985e-e27887cbf922",
-      "metadata": {
-        "id": "25a83a0d-a742-49f6-985e-e27887cbf922"
-      },
-      "source": [
-        "\n",
-        "---\n",
-        "\n",
-        "**Best of luck! Immerse yourself in the world of books, and may the data be with you!**"
-      ]
-    },
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Found UPC: a897fe39b1053632\n",
+      "Found Availability: In stock (22 available)\n"
+     ]
+    }
+   ],
+   "source": [
+    "product_table = book_soup.find('table', class_='table-striped')\n",
+    "\n",
+    "for row in product_table.find_all('tr'):\n",
+    "    header = row.find('th').get_text()\n",
+    "    if header == 'UPC':\n",
+    "        upc = row.find('td').get_text()\n",
+    "        print(f\"Found UPC: {upc}\")\n",
+    "    elif header == 'Availability':\n",
+    "        availability = row.find('td').get_text()\n",
+    "        print(f\"Found Availability: {availability}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "id": "faebbee1-5fb4-4f1e-b97a-3d94778936e4",
+   "metadata": {},
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0",
-      "metadata": {
-        "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0"
-      },
-      "source": [
-        "**Important Note**:\n",
-        "\n",
-        "In the fast-changing online world, websites often update and change their structures. When you try this lab, the **Books to Scrape** website might differ from what you expect.\n",
-        "\n",
-        "If you encounter issues due to these changes, like new rules or obstacles preventing data extraction, don’t worry! Get creative.\n",
-        "\n",
-        "You can choose another website that interests you and is suitable for scraping data. Options like Wikipedia, The New York Times, or even library databases are great alternatives. The main goal remains the same: extract useful data and enhance your web scraping skills while exploring a source of information you enjoy. This is your opportunity to practice and adapt to different web environments!"
-      ]
-    },
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. Silverstein's humorous and creative verse can amuse the dowdiest of readers. Lemon-faced adults and fidgety kids sit still and read these rhythmic words and laugh and smile and love th It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. Silverstein's humorous and creative verse can amuse the dowdiest of readers. Lemon-faced adults and fidgety kids sit still and read these rhythmic words and laugh and smile and love that Silverstein. Need proof of his genius? RockabyeRockabye baby, in the treetopDon't you know a treetopIs no safe place to rock?And who put you up there,And your cradle, too?Baby, I think someone down here'sGot it in for you. Shel, you never sounded so good. ...more\n"
+     ]
+    }
+   ],
+   "source": [
+    "description_div = book_soup.find('div', id='product_description')\n",
+    "\n",
+    "description_p = description_div.find_next_sibling('p')\n",
+    "\n",
+    "description = description_p.get_text(strip = True)\n",
+    "\n",
+    "print(description)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 31,
+   "id": "3b8b74c5-70ca-4aa7-9d82-49f4f49edd86",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "base_url = \"http://books.toscrape.com/\"\n",
+    "catalogue_url = \"http://books.toscrape.com/catalogue/\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 32,
+   "id": "1fa634f5-50b4-482e-a712-9bb8028c39ad",
+   "metadata": {},
+   "outputs": [
     {
-      "cell_type": "code",
-      "execution_count": null,
-      "id": "40359eee-9cd7-4884-bfa4-83344c222305",
-      "metadata": {
-        "id": "40359eee-9cd7-4884-bfa4-83344c222305"
-      },
-      "outputs": [],
-      "source": [
-        "# Your solution goes here"
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>UPC</th>\n",
+       "      <th>Title</th>\n",
+       "      <th>Price (£)</th>\n",
+       "      <th>Rating</th>\n",
+       "      <th>Genre</th>\n",
+       "      <th>Availability</th>\n",
+       "      <th>Description</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>a897fe39b1053632</td>\n",
+       "      <td>A Light in the Attic</td>\n",
+       "      <td>51.77</td>\n",
+       "      <td>3</td>\n",
+       "      <td>N/A</td>\n",
+       "      <td>In stock (22 available)</td>\n",
+       "      <td>It's hard to imagine a world without A Light i...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>90fa61229261140a</td>\n",
+       "      <td>Tipping the Velvet</td>\n",
+       "      <td>53.74</td>\n",
+       "      <td>1</td>\n",
+       "      <td>N/A</td>\n",
+       "      <td>In stock (20 available)</td>\n",
+       "      <td>\"Erotic and absorbing...Written with starling ...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>6957f44c3847a760</td>\n",
+       "      <td>Soumission</td>\n",
+       "      <td>50.10</td>\n",
+       "      <td>1</td>\n",
+       "      <td>N/A</td>\n",
+       "      <td>In stock (20 available)</td>\n",
+       "      <td>Dans une France assez proche de la nôtre, un h...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>3</th>\n",
+       "      <td>e00eb4fd7b871a48</td>\n",
+       "      <td>Sharp Objects</td>\n",
+       "      <td>47.82</td>\n",
+       "      <td>4</td>\n",
+       "      <td>N/A</td>\n",
+       "      <td>In stock (20 available)</td>\n",
+       "      <td>WICKED above her hipbone, GIRL across her hear...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4</th>\n",
+       "      <td>4165285e1663650f</td>\n",
+       "      <td>Sapiens: A Brief History of Humankind</td>\n",
+       "      <td>54.23</td>\n",
+       "      <td>5</td>\n",
+       "      <td>N/A</td>\n",
+       "      <td>In stock (20 available)</td>\n",
+       "      <td>From a renowned historian comes a groundbreaki...</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "                UPC                                  Title  Price (£)  Rating  \\\n",
+       "0  a897fe39b1053632                   A Light in the Attic      51.77       3   \n",
+       "1  90fa61229261140a                     Tipping the Velvet      53.74       1   \n",
+       "2  6957f44c3847a760                             Soumission      50.10       1   \n",
+       "3  e00eb4fd7b871a48                          Sharp Objects      47.82       4   \n",
+       "4  4165285e1663650f  Sapiens: A Brief History of Humankind      54.23       5   \n",
+       "\n",
+       "  Genre             Availability  \\\n",
+       "0   N/A  In stock (22 available)   \n",
+       "1   N/A  In stock (20 available)   \n",
+       "2   N/A  In stock (20 available)   \n",
+       "3   N/A  In stock (20 available)   \n",
+       "4   N/A  In stock (20 available)   \n",
+       "\n",
+       "                                         Description  \n",
+       "0  It's hard to imagine a world without A Light i...  \n",
+       "1  \"Erotic and absorbing...Written with starling ...  \n",
+       "2  Dans une France assez proche de la nôtre, un h...  \n",
+       "3  WICKED above her hipbone, GIRL across her hear...  \n",
+       "4  From a renowned historian comes a groundbreaki...  "
       ]
+     },
+     "execution_count": 32,
+     "metadata": {},
+     "output_type": "execute_result"
     }
-  ],
-  "metadata": {
-    "colab": {
-      "provenance": []
-    },
-    "kernelspec": {
-      "display_name": "Python 3 (ipykernel)",
-      "language": "python",
-      "name": "python3"
+   ],
+   "source": [
+    "# A list to store the dictionaries of book data\n",
+    "all_books_data = []\n",
+    "\n",
+    "# A dictionary to convert star ratings from text to numbers\n",
+    "rating_map = {\"One\": 1, \"Two\": 2, \"Three\": 3, \"Four\": 4, \"Five\": 5}\n",
+    "\n",
+    "# Find all book containers on the first page\n",
+    "book_containers = soup.find_all('article', class_='product_pod')\n",
+    "\n",
+    "# Loop through each book container\n",
+    "for book in book_containers:\n",
+    "    # Scrape data from the main page\n",
+    "    title = book.h3.a.get('title')\n",
+    "    price_str = book.find('p', class_='price_color').get_text(strip=True)\n",
+    "    rating_text = book.find('p', class_='star-rating').get('class')[1]\n",
+    "\n",
+    "    # Visit the detail page for more info\n",
+    "    partial_link = book.h3.a.get('href')\n",
+    "    full_link = base_url + partial_link\n",
+    "\n",
+    "    # Fetch and parse the detail page\n",
+    "    book_response = requests.get(full_link)\n",
+    "    book_soup = BeautifulSoup(book_response.content, \"html.parser\")\n",
+    "\n",
+    "    # Scrape data from the detail page\n",
+    "    product_table = book_soup.find('table', class_='table-striped')\n",
+    "    upc = product_table.find('td').get_text() # UPC is the first <td> in the table\n",
+    "    availability = product_table.find_all('td')[5].get_text(strip=True) # Availability is the 6th <td>\n",
+    "    \n",
+    "    description_div = book_soup.find('div', id='product_description')\n",
+    "    description_p = description_div.find_next_sibling('p')\n",
+    "    description = description_p.get_text(strip=True) if description_p else \"No description available\"\n",
+    "\n",
+    "    # --- Store the data in a dictionary ---\n",
+    "    book_data = {\n",
+    "        \"UPC\": upc,\n",
+    "        \"Title\": title,\n",
+    "        \"Price (£)\": float(price_str.replace(\"£\", \"\")),\n",
+    "        \"Rating\": rating_map.get(rating_text, 0), # Use .get() for safety\n",
+    "        \"Genre\": \"N/A\", # We'll get this later\n",
+    "        \"Availability\": availability,\n",
+    "        \"Description\": description\n",
+    "    }\n",
+    "\n",
+    "    all_books_data.append(book_data)\n",
+    "\n",
+    "# Convert the list of dictionaries to a DataFrame\n",
+    "df = pd.DataFrame(all_books_data)\n",
+    "df.head()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "id": "796f9063-9c08-4d6f-b1e5-7b4b0646b095",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Scraping page: http://books.toscrape.com/catalogue/page-1.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-2.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-3.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-4.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-5.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-6.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-7.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-8.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-9.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-10.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-11.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-12.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-13.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-14.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-15.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-16.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-17.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-18.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-19.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-20.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-21.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-22.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-23.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-24.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-25.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-26.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-27.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-28.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-29.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-30.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-31.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-32.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-33.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-34.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-35.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-36.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-37.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-38.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-39.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-40.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-41.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-42.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-43.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-44.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-45.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-46.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-47.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-48.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-49.html\n",
+      "Scraping page: http://books.toscrape.com/catalogue/page-50.html\n",
+      "\\nScraping complete!\n"
+     ]
     },
-    "language_info": {
-      "codemirror_mode": {
-        "name": "ipython",
-        "version": 3
-      },
-      "file_extension": ".py",
-      "mimetype": "text/x-python",
-      "name": "python",
-      "nbconvert_exporter": "python",
-      "pygments_lexer": "ipython3",
-      "version": "3.9.13"
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>UPC</th>\n",
+       "      <th>Title</th>\n",
+       "      <th>Price (£)</th>\n",
+       "      <th>Rating</th>\n",
+       "      <th>Genre</th>\n",
+       "      <th>Availability</th>\n",
+       "      <th>Description</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>ce6396b0f23f6ecc</td>\n",
+       "      <td>Set Me Free</td>\n",
+       "      <td>17.46</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Young Adult</td>\n",
+       "      <td>In stock (19 available)</td>\n",
+       "      <td>Aaron Ledbetter’s future had been planned out ...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>6258a1f6a6dcfe50</td>\n",
+       "      <td>The Four Agreements: A Practical Guide to Pers...</td>\n",
+       "      <td>17.66</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Spirituality</td>\n",
+       "      <td>In stock (18 available)</td>\n",
+       "      <td>In The Four Agreements, don Miguel Ruiz reveal...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>6be3beb0793a53e7</td>\n",
+       "      <td>Sophie's World</td>\n",
+       "      <td>15.94</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Philosophy</td>\n",
+       "      <td>In stock (18 available)</td>\n",
+       "      <td>A page-turning novel that is also an explorati...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>3</th>\n",
+       "      <td>657fe5ead67a7767</td>\n",
+       "      <td>Untitled Collection: Sabbath Poems 2014</td>\n",
+       "      <td>14.27</td>\n",
+       "      <td>4</td>\n",
+       "      <td>Poetry</td>\n",
+       "      <td>In stock (16 available)</td>\n",
+       "      <td>More than thirty-five years ago, when the weat...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4</th>\n",
+       "      <td>51653ef291ab7ddc</td>\n",
+       "      <td>This One Summer</td>\n",
+       "      <td>19.49</td>\n",
+       "      <td>4</td>\n",
+       "      <td>Sequential Art</td>\n",
+       "      <td>In stock (16 available)</td>\n",
+       "      <td>Every summer, Rose goes with her mom and dad t...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>...</th>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>70</th>\n",
+       "      <td>9c96cd1329fbd82d</td>\n",
+       "      <td>The Zombie Room</td>\n",
+       "      <td>19.69</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Default</td>\n",
+       "      <td>In stock (1 available)</td>\n",
+       "      <td>An unlikely bond is forged between three men f...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>71</th>\n",
+       "      <td>b78deb463531d078</td>\n",
+       "      <td>The Silent Wife</td>\n",
+       "      <td>12.34</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Fiction</td>\n",
+       "      <td>In stock (1 available)</td>\n",
+       "      <td>A chilling psychological thriller about a marr...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>72</th>\n",
+       "      <td>4280ac3eab57aa5d</td>\n",
+       "      <td>The Girl You Lost</td>\n",
+       "      <td>12.29</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Mystery</td>\n",
+       "      <td>In stock (1 available)</td>\n",
+       "      <td>Eighteen years ago your baby daughter was snat...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>73</th>\n",
+       "      <td>29fc016c459aeb14</td>\n",
+       "      <td>The Edge of Reason (Bridget Jones #2)</td>\n",
+       "      <td>19.18</td>\n",
+       "      <td>4</td>\n",
+       "      <td>Womens Fiction</td>\n",
+       "      <td>In stock (1 available)</td>\n",
+       "      <td>Monday 27 January“7:15 a.m. Hurrah! The wilder...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>74</th>\n",
+       "      <td>19fec36a1dfb4c16</td>\n",
+       "      <td>A Spy's Devotion (The Regency Spies of London #1)</td>\n",
+       "      <td>16.97</td>\n",
+       "      <td>5</td>\n",
+       "      <td>Historical Fiction</td>\n",
+       "      <td>In stock (1 available)</td>\n",
+       "      <td>In England’s Regency era, manners and elegance...</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "<p>75 rows × 7 columns</p>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "                 UPC                                              Title  \\\n",
+       "0   ce6396b0f23f6ecc                                        Set Me Free   \n",
+       "1   6258a1f6a6dcfe50  The Four Agreements: A Practical Guide to Pers...   \n",
+       "2   6be3beb0793a53e7                                     Sophie's World   \n",
+       "3   657fe5ead67a7767            Untitled Collection: Sabbath Poems 2014   \n",
+       "4   51653ef291ab7ddc                                    This One Summer   \n",
+       "..               ...                                                ...   \n",
+       "70  9c96cd1329fbd82d                                    The Zombie Room   \n",
+       "71  b78deb463531d078                                    The Silent Wife   \n",
+       "72  4280ac3eab57aa5d                                  The Girl You Lost   \n",
+       "73  29fc016c459aeb14              The Edge of Reason (Bridget Jones #2)   \n",
+       "74  19fec36a1dfb4c16  A Spy's Devotion (The Regency Spies of London #1)   \n",
+       "\n",
+       "    Price (£)  Rating               Genre             Availability  \\\n",
+       "0       17.46       5         Young Adult  In stock (19 available)   \n",
+       "1       17.66       5        Spirituality  In stock (18 available)   \n",
+       "2       15.94       5          Philosophy  In stock (18 available)   \n",
+       "3       14.27       4              Poetry  In stock (16 available)   \n",
+       "4       19.49       4      Sequential Art  In stock (16 available)   \n",
+       "..        ...     ...                 ...                      ...   \n",
+       "70      19.69       5             Default   In stock (1 available)   \n",
+       "71      12.34       5             Fiction   In stock (1 available)   \n",
+       "72      12.29       5             Mystery   In stock (1 available)   \n",
+       "73      19.18       4      Womens Fiction   In stock (1 available)   \n",
+       "74      16.97       5  Historical Fiction   In stock (1 available)   \n",
+       "\n",
+       "                                          Description  \n",
+       "0   Aaron Ledbetter’s future had been planned out ...  \n",
+       "1   In The Four Agreements, don Miguel Ruiz reveal...  \n",
+       "2   A page-turning novel that is also an explorati...  \n",
+       "3   More than thirty-five years ago, when the weat...  \n",
+       "4   Every summer, Rose goes with her mom and dad t...  \n",
+       "..                                                ...  \n",
+       "70  An unlikely bond is forged between three men f...  \n",
+       "71  A chilling psychological thriller about a marr...  \n",
+       "72  Eighteen years ago your baby daughter was snat...  \n",
+       "73  Monday 27 January“7:15 a.m. Hurrah! The wilder...  \n",
+       "74  In England’s Regency era, manners and elegance...  \n",
+       "\n",
+       "[75 rows x 7 columns]"
+      ]
+     },
+     "execution_count": 33,
+     "metadata": {},
+     "output_type": "execute_result"
     }
+   ],
+   "source": [
+    "def scrape_books(min_rating, max_price):\n",
+    "    \"\"\"\n",
+    "    Scrapes book data from books.toscrape.com across multiple pages,\n",
+    "    filters by minimum rating and maximum price, and returns a pandas DataFrame.\n",
+    "    \"\"\"\n",
+    "    \n",
+    "    # A list to store the dictionaries of book data\n",
+    "    all_books_data = []\n",
+    "    \n",
+    "    # A dictionary to convert star ratings from text to numbers\n",
+    "    rating_map = {\"One\": 1, \"Two\": 2, \"Three\": 3, \"Four\": 4, \"Five\": 5}\n",
+    "    \n",
+    "    # Start with the first page\n",
+    "    current_url = \"http://books.toscrape.com/catalogue/page-1.html\"\n",
+    "    base_url = \"http://books.toscrape.com/catalogue/\"\n",
+    "\n",
+    "    while current_url:\n",
+    "        print(f\"Scraping page: {current_url}\")\n",
+    "        response = requests.get(current_url)\n",
+    "        \n",
+    "        # Make sure the request was successful\n",
+    "        if response.status_code != 200:\n",
+    "            break\n",
+    "            \n",
+    "        soup = BeautifulSoup(response.content, \"html.parser\")\n",
+    "        \n",
+    "        # Find all book containers on the page\n",
+    "        book_containers = soup.find_all('article', class_='product_pod')\n",
+    "\n",
+    "        for book in book_containers:\n",
+    "            # Scrape rating and price first for filtering\n",
+    "            price_str = book.find('p', class_='price_color').get_text(strip=True)\n",
+    "            price = float(price_str.replace(\"£\", \"\"))\n",
+    "            \n",
+    "            rating_text = book.find('p', class_='star-rating').get('class')[1]\n",
+    "            rating = rating_map.get(rating_text, 0)\n",
+    "            \n",
+    "            # --- FILTERING LOGIC ---\n",
+    "            # If the book meets the criteria, scrape the rest of the details\n",
+    "            if rating >= min_rating and price <= max_price:\n",
+    "                \n",
+    "                # --- Scrape remaining data ---\n",
+    "                title = book.h3.a.get('title')\n",
+    "                \n",
+    "                # Visit the detail page for more info\n",
+    "                partial_link = book.h3.a.get('href')\n",
+    "                full_link = base_url + partial_link\n",
+    "                \n",
+    "                book_response = requests.get(full_link)\n",
+    "                book_soup = BeautifulSoup(book_response.content, \"html.parser\")\n",
+    "                \n",
+    "                # Scrape data from the detail page\n",
+    "                upc = book_soup.find('th', string='UPC').find_next('td').get_text()\n",
+    "                availability = book_soup.find('th', string='Availability').find_next('td').get_text(strip=True)\n",
+    "                description_p = book_soup.find('div', id='product_description').find_next_sibling('p')\n",
+    "                description = description_p.get_text(strip=True) if description_p else \"N/A\"\n",
+    "                genre = book_soup.find('ul', class_='breadcrumb').find_all('li')[2].a.get_text(strip=True)\n",
+    "\n",
+    "                # Store the data in a dictionary\n",
+    "                book_data = {\n",
+    "                    \"UPC\": upc,\n",
+    "                    \"Title\": title,\n",
+    "                    \"Price (£)\": price,\n",
+    "                    \"Rating\": rating,\n",
+    "                    \"Genre\": genre,\n",
+    "                    \"Availability\": availability,\n",
+    "                    \"Description\": description\n",
+    "                }\n",
+    "                all_books_data.append(book_data)\n",
+    "\n",
+    "        # Find the 'next' button to go to the next page\n",
+    "        next_button = soup.find('li', class_='next')\n",
+    "        if next_button:\n",
+    "            next_page_partial_url = next_button.a.get('href')\n",
+    "            current_url = base_url + next_page_partial_url\n",
+    "        else:\n",
+    "            # If no 'next' button, we are on the last page\n",
+    "            current_url = None\n",
+    "            \n",
+    "    # Convert the list of dictionaries to a DataFrame\n",
+    "    return pd.DataFrame(all_books_data)\n",
+    "\n",
+    "# --- Now, execute the function with your specified criteria ---\n",
+    "final_df = scrape_books(min_rating=4, max_price=20)\n",
+    "\n",
+    "print(\"\\\\nScraping complete!\")\n",
+    "final_df"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b3993b66-c083-49a5-ace7-f8f5508a1d72",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python [conda env:base] *",
+   "language": "python",
+   "name": "conda-base-py"
   },
-  "nbformat": 4,
-  "nbformat_minor": 5
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.13.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
 }