From db5ad1a756f481004a443edd0fedcd308b3a2041 Mon Sep 17 00:00:00 2001 From: PedroSantos Date: Fri, 29 Aug 2025 11:36:12 +0100 Subject: [PATCH] Solved lab --- lab_web_scraping_end.ipynb | 3669 ++++++++++++++++++++++++++++++++++++ 1 file changed, 3669 insertions(+) create mode 100644 lab_web_scraping_end.ipynb diff --git a/lab_web_scraping_end.ipynb b/lab_web_scraping_end.ipynb new file mode 100644 index 0000000..551dc93 --- /dev/null +++ b/lab_web_scraping_end.ipynb @@ -0,0 +1,3669 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786", + "metadata": { + "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786" + }, + "source": [ + "# Lab | Web Scraping" + ] + }, + { + "cell_type": "markdown", + "id": "ce8882fc-4815-4567-92fa-b4816358ba7d", + "metadata": { + "id": "ce8882fc-4815-4567-92fa-b4816358ba7d" + }, + "source": [ + "Welcome to the \"Books to Scrape\" Web Scraping Adventure Lab!\n", + "\n", + "**Objective**\n", + "\n", + "In this lab, we will embark on a mission to unearth valuable insights from the data available on Books to Scrape, an online platform showcasing a wide variety of books. As data analyst, you have been tasked with scraping a specific subset of book data from Books to Scrape to assist publishing companies in understanding the landscape of highly-rated books across different genres. Your insights will help shape future book marketing strategies and publishing decisions.\n", + "\n", + "**Background**\n", + "\n", + "In a world where data has become the new currency, businesses are leveraging big data to make informed decisions that drive success and profitability. The publishing industry, much like others, utilizes data analytics to understand market trends, reader preferences, and the performance of books based on factors such as genre, author, and ratings. Books to Scrape serves as a rich source of such data, offering detailed information about a diverse range of books, making it an ideal platform for extracting insights to aid in informed decision-making within the literary world.\n", + "\n", + "**Task**\n", + "\n", + "Your task is to create a Python script using BeautifulSoup and pandas to scrape Books to Scrape book data, focusing on book ratings and genres. The script should be able to filter books with ratings above a certain threshold and in specific genres. Additionally, the script should structure the scraped data in a tabular format using pandas for further analysis.\n", + "\n", + "**Expected Outcome**\n", + "\n", + "A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`. The function should scrape book data from the \"Books to Scrape\" website and return a `pandas` DataFrame with the following columns:\n", + "\n", + "**Expected Outcome**\n", + "\n", + "- A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`.\n", + "- The function should return a DataFrame with the following columns:\n", + " - **UPC**: The Universal Product Code (UPC) of the book.\n", + " - **Title**: The title of the book.\n", + " - **Price (£)**: The price of the book in pounds.\n", + " - **Rating**: The rating of the book (1-5 stars).\n", + " - **Genre**: The genre of the book.\n", + " - **Availability**: Whether the book is in stock or not.\n", + " - **Description**: A brief description or product description of the book (if available).\n", + " \n", + "You will execute this script to scrape data for books with a minimum rating of `4.0 and above` and a maximum price of `£20`. \n", + "\n", + "Remember to experiment with different ratings and prices to ensure your code is versatile and can handle various searches effectively!\n", + "\n", + "**Resources**\n", + "\n", + "- [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)\n", + "- [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/index.html)\n", + "- [Books to Scrape](https://books.toscrape.com/)\n" + ] + }, + { + "cell_type": "markdown", + "id": "3519921d-5890-445b-9a33-934ed8ee378c", + "metadata": { + "id": "3519921d-5890-445b-9a33-934ed8ee378c" + }, + "source": [ + "**Hint**\n", + "\n", + "Your first mission is to familiarize yourself with the **Books to Scrape** website. Navigate to [Books to Scrape](http://books.toscrape.com/) and explore the available books to understand their layout and structure. \n", + "\n", + "Next, think about how you can set parameters for your data extraction:\n", + "\n", + "- **Minimum Rating**: Focus on books with a rating of 4.0 and above.\n", + "- **Maximum Price**: Filter for books priced up to £20.\n", + "\n", + "After reviewing the site, you can construct a plan for scraping relevant data. Pay attention to the details displayed for each book, including the title, price, rating, and availability. This will help you identify the correct HTML elements to target with your scraping script.\n", + "\n", + "Make sure to build your scraping URL and logic based on the patterns you observe in the HTML structure of the book listings!" + ] + }, + { + "cell_type": "markdown", + "id": "25a83a0d-a742-49f6-985e-e27887cbf922", + "metadata": { + "id": "25a83a0d-a742-49f6-985e-e27887cbf922" + }, + "source": [ + "\n", + "---\n", + "\n", + "**Best of luck! Immerse yourself in the world of books, and may the data be with you!**" + ] + }, + { + "cell_type": "markdown", + "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0", + "metadata": { + "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0" + }, + "source": [ + "**Important Note**:\n", + "\n", + "In the fast-changing online world, websites often update and change their structures. When you try this lab, the **Books to Scrape** website might differ from what you expect.\n", + "\n", + "If you encounter issues due to these changes, like new rules or obstacles preventing data extraction, don’t worry! Get creative.\n", + "\n", + "You can choose another website that interests you and is suitable for scraping data. Options like Wikipedia, The New York Times, or even library databases are great alternatives. The main goal remains the same: extract useful data and enhance your web scraping skills while exploring a source of information you enjoy. This is your opportunity to practice and adapt to different web environments!" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "19ce64b7", + "metadata": {}, + "outputs": [], + "source": [ + "import requests\n", + "from bs4 import BeautifulSoup\n", + "import pandas as pd" + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "id": "40359eee-9cd7-4884-bfa4-83344c222305", + "metadata": { + "id": "40359eee-9cd7-4884-bfa4-83344c222305" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 79, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Your solution goes here\n", + "\n", + "url = \"https://books.toscrape.com/catalogue/category/books_1/index.html\"\n", + "response = requests.get(url)\n", + "response" + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "id": "0f88e196", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + " \n", + "\n", + "\n", + " Books | \n", + " Books to Scrape - Sandbox\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "
\n", + "
\n", + "
Books to Scrape We love being scraped!\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
    \n", + "
  • \n", + "Home\n", + "
  • \n", + "
  • Books
  • \n", + "
\n", + "
\n", + "\n", + "
\n", + "
\n", + "

Books

\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "1000 results - showing 1 to 20.\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + "
\n", + "
Warning! This is a demo website for web scraping purposes. Prices and ratings here were randomly assigned and have no real meaning.
\n", + "
\n", + "
    \n", + "
  1. \n", + "
    \n", + "
    \n", + "\"A\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    A Light in the ...

    \n", + "
    \n", + "

    £51.77

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  2. \n", + "
  3. \n", + "
    \n", + "
    \n", + "\"Tipping\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Tipping the Velvet

    \n", + "
    \n", + "

    £53.74

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  4. \n", + "
  5. \n", + "
    \n", + "
    \n", + "\"Soumission\"\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Soumission

    \n", + "
    \n", + "

    £50.10

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  6. \n", + "
  7. \n", + "
    \n", + "
    \n", + "\"Sharp\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Sharp Objects

    \n", + "
    \n", + "

    £47.82

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  8. \n", + "
  9. \n", + "
    \n", + "
    \n", + "\"Sapiens:\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Sapiens: A Brief History ...

    \n", + "
    \n", + "

    £54.23

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  10. \n", + "
  11. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Requiem Red

    \n", + "
    \n", + "

    £22.65

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  12. \n", + "
  13. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Dirty Little Secrets ...

    \n", + "
    \n", + "

    £33.34

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  14. \n", + "
  15. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Coming Woman: A ...

    \n", + "
    \n", + "

    £17.93

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  16. \n", + "
  17. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Boys in the ...

    \n", + "
    \n", + "

    £22.60

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  18. \n", + "
  19. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Black Maria

    \n", + "
    \n", + "

    £52.15

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  20. \n", + "
  21. \n", + "
    \n", + "
    \n", + "\"Starving\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Starving Hearts (Triangular Trade ...

    \n", + "
    \n", + "

    £13.99

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  22. \n", + "
  23. \n", + "
    \n", + "
    \n", + "\"Shakespeare's\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Shakespeare's Sonnets

    \n", + "
    \n", + "

    £20.66

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  24. \n", + "
  25. \n", + "
    \n", + "
    \n", + "\"Set\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Set Me Free

    \n", + "
    \n", + "

    £17.46

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  26. \n", + "
  27. \n", + "
    \n", + "
    \n", + "\"Scott\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Scott Pilgrim's Precious Little ...

    \n", + "
    \n", + "

    £52.29

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  28. \n", + "
  29. \n", + "
    \n", + "
    \n", + "\"Rip\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Rip it Up and ...

    \n", + "
    \n", + "

    £35.02

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  30. \n", + "
  31. \n", + "
    \n", + "
    \n", + "\"Our\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Our Band Could Be ...

    \n", + "
    \n", + "

    £57.25

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  32. \n", + "
  33. \n", + "
    \n", + "
    \n", + "\"Olio\"\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Olio

    \n", + "
    \n", + "

    £23.88

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  34. \n", + "
  35. \n", + "
    \n", + "
    \n", + "\"Mesaerion:\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Mesaerion: The Best Science ...

    \n", + "
    \n", + "

    £37.59

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  36. \n", + "
  37. \n", + "
    \n", + "
    \n", + "\"Libertarianism\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Libertarianism for Beginners

    \n", + "
    \n", + "

    £51.33

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  38. \n", + "
  39. \n", + "
    \n", + "
    \n", + "\"It's\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    It's Only the Himalayas

    \n", + "
    \n", + "

    £45.17

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  40. \n", + "
\n", + "
\n", + "
    \n", + "
  • \n", + " \n", + " Page 1 of 50\n", + " \n", + "
  • \n", + "
  • next
  • \n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ] + }, + "execution_count": 80, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "soup = BeautifulSoup(response.content)\n", + "soup" + ] + }, + { + "cell_type": "markdown", + "id": "01297ec6", + "metadata": {}, + "source": [ + "Expected Outcome** --> SO PARA GUIAR APAGAR DEPOIS\n", + "\n", + "A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`. The function should scrape book data from the \"Books to Scrape\" website and return a `pandas` DataFrame with the following columns:\n", + "\n", + "**Expected Outcome**\n", + "\n", + "- A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`.\n", + "- The function should return a DataFrame with the following columns:\n", + " - **UPC**: The Universal Product Code (UPC) of the book.\n", + " - **Title**: The title of the book.\n", + " - **Price (£)**: The price of the book in pounds.\n", + " - **Rating**: The rating of the book (1-5 stars).\n", + " - **Genre**: The genre of the book.\n", + " - **Availability**: Whether the book is in stock or not.\n", + " - **Description**: A brief description or product description of the book (if available).\n", + " \n", + "You will execute this script to scrape data for books with a minimum rating of `4.0 and above` and a maximum price of `£20`. " + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "id": "ffcf9206", + "metadata": {}, + "outputs": [], + "source": [ + "# Retrieve the book grid\n", + "\n", + "books_grid = soup.find(\"ol\", attrs = {\"class\":\"row\"})\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ffc007f", + "metadata": {}, + "outputs": [], + "source": [ + "books_grid" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8be0b8f4", + "metadata": {}, + "outputs": [], + "source": [ + "books = books_grid.find_all(\"li\", attrs ={\"col-xs-6 col-sm-4 col-md-3 col-lg-3\"})\n", + "books" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "faff16f8", + "metadata": {}, + "outputs": [], + "source": [ + "# Retrieve columns:\n", + "# -> UPC -> Genre\n", + "# -> Title. -> Availability.\n", + "# -> Price. -> Description.....\n", + "# -> Rating.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "id": "ca7c283c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'A Light in the Attic'" + ] + }, + "execution_count": 86, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Title.\n", + "\n", + "title = books[0].find_all(\"a\")[1][\"title\"]\n", + "title" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "id": "eb35cf21", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'£51.77'" + ] + }, + "execution_count": 87, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Price.\n", + "\n", + "price = books[0].find(\"p\", attrs = {\"class\": \"price_color\"}).get_text()\n", + "price" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "86910fad", + "metadata": {}, + "outputs": [], + "source": [ + "# Rating.\n", + "\n", + "books[0].find(\"article\", attrs = {\"class\": \"product_pod\"})" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "id": "d50107b0", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'star_rating Three'" + ] + }, + "execution_count": 94, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Rating.\n", + "\n", + "rating_get = books[0].find(\"article\", attrs = {\"class\": \"product_pod\"})\n", + "rating_star = rating_get.find(\"p\", attrs= {\"class\":\"star-rating Three\"})\n", + "\n", + "rating = rating_star.get_text(\"star_rating Three\").split(\"\\n\")[1]\n", + "rating" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "id": "c3b78d8f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'In stock'" + ] + }, + "execution_count": 90, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Availability.\n", + "\n", + "availability = books[0].find(\"p\", attrs = {\"class\": \"instock availability\"}).get_text().strip()\n", + "availability" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "id": "b2477418", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[\"A,\n", + " A Light in the ...]" + ] + }, + "execution_count": 91, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Description....\n", + "\n", + "books[0].find_all(\"a\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72367f69", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 92, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Books details page -> url_2 to retrieve detail information about each book\n", + "\n", + "url_2 = \"https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html\"\n", + "response_2 = requests.get(url_2)\n", + "response_2" + ] + }, + { + "cell_type": "code", + "execution_count": 93, + "id": "a7b937db", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + " \n", + "\n", + "\n", + " A Light in the Attic | Books to Scrape - Sandbox\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
\n", + "
\n", + "
\n", + "
Books to Scrape We love being scraped!\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
    \n", + "
  • \n", + "Home\n", + "
  • \n", + "
  • \n", + "Books\n", + "
  • \n", + "
  • \n", + "Poetry\n", + "
  • \n", + "
  • A Light in the Attic
  • \n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "\"A\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "

A Light in the Attic

\n", + "

£51.77

\n", + "

\n", + "\n", + " \n", + " In stock (22 available)\n", + " \n", + "

\n", + "

\n", + "\n", + "\n", + "\n", + "\n", + "\n", + " \n", + "\n", + "\n", + "

\n", + "
\n", + "
Warning! This is a demo website for web scraping purposes. Prices and ratings here were randomly assigned and have no real meaning.
\n", + "
\n", + "
\n", + "
\n", + "

Product Description

\n", + "
\n", + "

It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. Silverstein's humorous and creative verse can amuse the dowdiest of readers. Lemon-faced adults and fidgety kids sit still and read these rhythmic words and laugh and smile and love th It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. Silverstein's humorous and creative verse can amuse the dowdiest of readers. Lemon-faced adults and fidgety kids sit still and read these rhythmic words and laugh and smile and love that Silverstein. Need proof of his genius? RockabyeRockabye baby, in the treetopDon't you know a treetopIs no safe place to rock?And who put you up there,And your cradle, too?Baby, I think someone down here'sGot it in for you. Shel, you never sounded so good. ...more

\n", + "
\n", + "

Product Information

\n", + "
\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
UPCa897fe39b1053632
Product TypeBooks
Price (excl. tax)£51.77
Price (incl. tax)£51.77
Tax£0.00
AvailabilityIn stock (22 available)
Number of reviews0
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ] + }, + "execution_count": 93, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "soup_2 = BeautifulSoup(response_2.content)\n", + "soup_2" + ] + }, + { + "cell_type": "code", + "execution_count": 99, + "id": "5978e275", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\n", + "
\n", + "
\n", + "
\n", + "
Books to Scrape We love being scraped!\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
    \n", + "
  • \n", + "Home\n", + "
  • \n", + "
  • Books
  • \n", + "
\n", + "
\n", + "\n", + "
\n", + "
\n", + "

Books

\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "1000 results - showing 1 to 20.\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + "
\n", + "
Warning! This is a demo website for web scraping purposes. Prices and ratings here were randomly assigned and have no real meaning.
\n", + "
\n", + "
    \n", + "
  1. \n", + "
    \n", + "
    \n", + "\"A\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    A Light in the ...

    \n", + "
    \n", + "

    £51.77

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  2. \n", + "
  3. \n", + "
    \n", + "
    \n", + "\"Tipping\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Tipping the Velvet

    \n", + "
    \n", + "

    £53.74

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  4. \n", + "
  5. \n", + "
    \n", + "
    \n", + "\"Soumission\"\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Soumission

    \n", + "
    \n", + "

    £50.10

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  6. \n", + "
  7. \n", + "
    \n", + "
    \n", + "\"Sharp\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Sharp Objects

    \n", + "
    \n", + "

    £47.82

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  8. \n", + "
  9. \n", + "
    \n", + "
    \n", + "\"Sapiens:\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Sapiens: A Brief History ...

    \n", + "
    \n", + "

    £54.23

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  10. \n", + "
  11. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Requiem Red

    \n", + "
    \n", + "

    £22.65

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  12. \n", + "
  13. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Dirty Little Secrets ...

    \n", + "
    \n", + "

    £33.34

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  14. \n", + "
  15. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Coming Woman: A ...

    \n", + "
    \n", + "

    £17.93

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  16. \n", + "
  17. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Boys in the ...

    \n", + "
    \n", + "

    £22.60

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  18. \n", + "
  19. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Black Maria

    \n", + "
    \n", + "

    £52.15

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  20. \n", + "
  21. \n", + "
    \n", + "
    \n", + "\"Starving\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Starving Hearts (Triangular Trade ...

    \n", + "
    \n", + "

    £13.99

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  22. \n", + "
  23. \n", + "
    \n", + "
    \n", + "\"Shakespeare's\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Shakespeare's Sonnets

    \n", + "
    \n", + "

    £20.66

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  24. \n", + "
  25. \n", + "
    \n", + "
    \n", + "\"Set\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Set Me Free

    \n", + "
    \n", + "

    £17.46

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  26. \n", + "
  27. \n", + "
    \n", + "
    \n", + "\"Scott\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Scott Pilgrim's Precious Little ...

    \n", + "
    \n", + "

    £52.29

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  28. \n", + "
  29. \n", + "
    \n", + "
    \n", + "\"Rip\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Rip it Up and ...

    \n", + "
    \n", + "

    £35.02

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  30. \n", + "
  31. \n", + "
    \n", + "
    \n", + "\"Our\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Our Band Could Be ...

    \n", + "
    \n", + "

    £57.25

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  32. \n", + "
  33. \n", + "
    \n", + "
    \n", + "\"Olio\"\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Olio

    \n", + "
    \n", + "

    £23.88

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  34. \n", + "
  35. \n", + "
    \n", + "
    \n", + "\"Mesaerion:\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Mesaerion: The Best Science ...

    \n", + "
    \n", + "

    £37.59

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  36. \n", + "
  37. \n", + "
    \n", + "
    \n", + "\"Libertarianism\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Libertarianism for Beginners

    \n", + "
    \n", + "

    £51.33

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  38. \n", + "
  39. \n", + "
    \n", + "
    \n", + "\"It's\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    It's Only the Himalayas

    \n", + "
    \n", + "

    £45.17

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  40. \n", + "
\n", + "
\n", + "
    \n", + "
  • \n", + " \n", + " Page 1 of 50\n", + " \n", + "
  • \n", + "
  • next
  • \n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ] + }, + "execution_count": 99, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "grid_detail = soup.find(\"body\", attrs= {\"class\": \"default\"})\n", + "grid_detail" + ] + }, + { + "cell_type": "code", + "execution_count": 128, + "id": "f3cdc14a", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "
\n", + "
\n", + "
    \n", + "
  • \n", + "Home\n", + "
  • \n", + "
  • Books
  • \n", + "
\n", + "
\n", + "\n", + "
\n", + "
\n", + "

Books

\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "1000 results - showing 1 to 20.\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + "
\n", + "
Warning! This is a demo website for web scraping purposes. Prices and ratings here were randomly assigned and have no real meaning.
\n", + "
\n", + "
    \n", + "
  1. \n", + "
    \n", + "
    \n", + "\"A\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    A Light in the ...

    \n", + "
    \n", + "

    £51.77

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  2. \n", + "
  3. \n", + "
    \n", + "
    \n", + "\"Tipping\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Tipping the Velvet

    \n", + "
    \n", + "

    £53.74

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  4. \n", + "
  5. \n", + "
    \n", + "
    \n", + "\"Soumission\"\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Soumission

    \n", + "
    \n", + "

    £50.10

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  6. \n", + "
  7. \n", + "
    \n", + "
    \n", + "\"Sharp\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Sharp Objects

    \n", + "
    \n", + "

    £47.82

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  8. \n", + "
  9. \n", + "
    \n", + "
    \n", + "\"Sapiens:\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Sapiens: A Brief History ...

    \n", + "
    \n", + "

    £54.23

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  10. \n", + "
  11. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Requiem Red

    \n", + "
    \n", + "

    £22.65

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  12. \n", + "
  13. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Dirty Little Secrets ...

    \n", + "
    \n", + "

    £33.34

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  14. \n", + "
  15. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Coming Woman: A ...

    \n", + "
    \n", + "

    £17.93

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  16. \n", + "
  17. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Boys in the ...

    \n", + "
    \n", + "

    £22.60

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  18. \n", + "
  19. \n", + "
    \n", + "
    \n", + "\"The\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    The Black Maria

    \n", + "
    \n", + "

    £52.15

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  20. \n", + "
  21. \n", + "
    \n", + "
    \n", + "\"Starving\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Starving Hearts (Triangular Trade ...

    \n", + "
    \n", + "

    £13.99

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  22. \n", + "
  23. \n", + "
    \n", + "
    \n", + "\"Shakespeare's\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Shakespeare's Sonnets

    \n", + "
    \n", + "

    £20.66

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  24. \n", + "
  25. \n", + "
    \n", + "
    \n", + "\"Set\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Set Me Free

    \n", + "
    \n", + "

    £17.46

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  26. \n", + "
  27. \n", + "
    \n", + "
    \n", + "\"Scott\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Scott Pilgrim's Precious Little ...

    \n", + "
    \n", + "

    £52.29

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  28. \n", + "
  29. \n", + "
    \n", + "
    \n", + "\"Rip\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Rip it Up and ...

    \n", + "
    \n", + "

    £35.02

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  30. \n", + "
  31. \n", + "
    \n", + "
    \n", + "\"Our\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Our Band Could Be ...

    \n", + "
    \n", + "

    £57.25

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  32. \n", + "
  33. \n", + "
    \n", + "
    \n", + "\"Olio\"\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Olio

    \n", + "
    \n", + "

    £23.88

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  34. \n", + "
  35. \n", + "
    \n", + "
    \n", + "\"Mesaerion:\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Mesaerion: The Best Science ...

    \n", + "
    \n", + "

    £37.59

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  36. \n", + "
  37. \n", + "
    \n", + "
    \n", + "\"Libertarianism\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    Libertarianism for Beginners

    \n", + "
    \n", + "

    £51.33

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  38. \n", + "
  39. \n", + "
    \n", + "
    \n", + "\"It's\n", + "
    \n", + "

    \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "

    \n", + "

    It's Only the Himalayas

    \n", + "
    \n", + "

    £45.17

    \n", + "

    \n", + "\n", + " \n", + " In stock\n", + " \n", + "

    \n", + "
    \n", + "\n", + "
    \n", + "
    \n", + "
    \n", + "
  40. \n", + "
\n", + "
\n", + "
    \n", + "
  • \n", + " \n", + " Page 1 of 50\n", + " \n", + "
  • \n", + "
  • next
  • \n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
\n", + "
" + ] + }, + "execution_count": 128, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "page = grid_detail.find(\"div\", attrs= {\"class\": \"container-fluid page\"})\n", + "page" + ] + }, + { + "cell_type": "code", + "execution_count": 126, + "id": "6ef78412", + "metadata": {}, + "outputs": [ + { + "ename": "TypeError", + "evalue": "'str' object is not callable", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mTypeError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[126]\u001b[39m\u001b[32m, line 1\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m1\u001b[39m \u001b[43mpage\u001b[49m\u001b[43m.\u001b[49m\u001b[43mfind\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mdiv\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mattrs\u001b[49m\u001b[43m=\u001b[49m\u001b[43m \u001b[49m\u001b[43m{\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mclass\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mpage_inner\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m}\u001b[49m\u001b[43m)\u001b[49m\u001b[43m.\u001b[49m\u001b[43mtext\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[31mTypeError\u001b[39m: 'str' object is not callable" + ] + } + ], + "source": [ + "page.find(\"div\", attrs= {\"class\": \"page_inner\"}).text()" + ] + }, + { + "cell_type": "code", + "execution_count": 124, + "id": "7b21923b", + "metadata": {}, + "outputs": [ + { + "ename": "KeyError", + "evalue": "0", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mKeyError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[124]\u001b[39m\u001b[32m, line 1\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m1\u001b[39m page_1 = \u001b[43mpage\u001b[49m\u001b[43m[\u001b[49m\u001b[32;43m0\u001b[39;49m\u001b[43m]\u001b[49m.find(\u001b[33m\"\u001b[39m\u001b[33mdiv\u001b[39m\u001b[33m\"\u001b[39m, attrs= {\u001b[33m\"\u001b[39m\u001b[33mclass\u001b[39m\u001b[33m\"\u001b[39m: \u001b[33m\"\u001b[39m\u001b[33mcontent\u001b[39m\u001b[33m\"\u001b[39m})\n\u001b[32m 2\u001b[39m page_1\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\Pedro\\AppData\\Local\\Programs\\Python\\Python313\\Lib\\site-packages\\bs4\\element.py:2214\u001b[39m, in \u001b[36mTag.__getitem__\u001b[39m\u001b[34m(self, key)\u001b[39m\n\u001b[32m 2211\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34m__getitem__\u001b[39m(\u001b[38;5;28mself\u001b[39m, key: \u001b[38;5;28mstr\u001b[39m) -> _AttributeValue:\n\u001b[32m 2212\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"tag[key] returns the value of the 'key' attribute for the Tag,\u001b[39;00m\n\u001b[32m 2213\u001b[39m \u001b[33;03m and throws an exception if it's not there.\"\"\"\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m2214\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mattrs\u001b[49m\u001b[43m[\u001b[49m\u001b[43mkey\u001b[49m\u001b[43m]\u001b[49m\n", + "\u001b[31mKeyError\u001b[39m: 0" + ] + } + ], + "source": [ + "page_1 = page[0].find(\"div\", attrs= {\"class\": \"content\"})\n", + "page_1" + ] + }, + { + "cell_type": "code", + "execution_count": 123, + "id": "fc2d2288", + "metadata": {}, + "outputs": [], + "source": [ + "page.find(\"article\", attrs= {\"class\": \"product_page\"})" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "368b022d", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}