From f5309b7fad6c5a176153cca5b79bf9de0e9ebc1f Mon Sep 17 00:00:00 2001 From: Santiago Larrea Date: Mon, 4 Aug 2025 00:01:18 +0200 Subject: [PATCH] lab scraping --- README.md | 104 -------------------------- lab-web-scraping (1).ipynb | 1 + lab-web-scraping.ipynb | 148 ------------------------------------- 3 files changed, 1 insertion(+), 252 deletions(-) delete mode 100644 README.md create mode 100644 lab-web-scraping (1).ipynb delete mode 100644 lab-web-scraping.ipynb diff --git a/README.md b/README.md deleted file mode 100644 index 7cc0236..0000000 --- a/README.md +++ /dev/null @@ -1,104 +0,0 @@ -![logo_ironhack_blue 7](https://user-images.githubusercontent.com/23629340/40541063-a07a0a8a-601a-11e8-91b5-2f13e4e6b441.png) - -# LAB | Web Scraping -
- -

Learning Goals

- - - This lab allows you to practice and apply the concepts and techniques taught in class. - - Upon completion of this lab, you will be able to: - -- Use Python libraries such as requests and Beautiful Soup to extract data from websites, and convert extracted data into a suitable data structure. - -
-
- -
- - -## Introduction - -Welcome to your next challenge, aspiring data explorers! - -In today’s digital age, data serves as the backbone of many decision-making processes, and the publishing industry is no exception. Companies use vast arrays of data to gauge reader preferences, track market trends, and make informed decisions that could lead to the next bestseller. - -In this lab, you will step into the shoes of a data analyst working for a leading publishing company. Your mission? To scrape **Books to Scrape**, one of the world's most accessible online databases for books, and extract meaningful data that can aid in understanding the literary landscape during a specified period. - -Leveraging Python along with libraries such as BeautifulSoup and Pandas, you will build a script to scrape book data based on specific criteria: minimum rating and maximum price. The extracted data will be organized into a well-structured pandas DataFrame, ready for analysis. - -As you navigate through this lab, not only will you enhance your web scraping skills, but you will also explore the fascinating world of books, uncovering trends and insights that lie hidden in the extensive data available on **Books to Scrape**. - -Prepare yourself for an adventure into the world of big data and literature. Every line of code takes you a step closer to becoming a data wizard in the literary universe. Let's delve into the data, and may the insights be with you! - -
- -**Happy coding!** :heart: - - -## Requirements - -- Fork this repo -- Clone it to your machine - -## Getting Started - -Complete the challenges in the `Jupyter Notebook` file. Follow the instructions and add your code and explanations as necessary. - -## Submission - -- Upon completion, run the following commands: - -```bash -git add . -git commit -m "Solved lab" -git push origin master -``` - -- Paste the link of your lab in Student Portal. - - -## FAQs -
- I am stuck in the exercise and don't know how to solve the problem or where to start. -
- - If you are stuck in your code and don't know how to solve the problem or where to start, you should take a step back and try to form a clear question about the specific issue you are facing. This will help you narrow down the problem and come up with potential solutions. - - - For example, is it a concept that you don't understand, or are you receiving an error message that you don't know how to fix? It is usually helpful to try to state the problem as clearly as possible, including any error messages you are receiving. This can help you communicate the issue to others and potentially get help from classmates or online resources. - - - Once you have a clear understanding of the problem, you will be able to start working toward the solution. - - [Back to top](#faqs) - -
- - -
- I am unable to push changes to the repository. What should I do? -
- -There are a couple of possible reasons why you may be unable to *push* changes to a Git repository: - -1. **You have not committed your changes:** Before you can push your changes to the repository, you need to commit them using the `git commit` command. Make sure you have committed your changes and try pushing again. To do this, run the following terminal commands from the project folder: - ```bash - git add . - git commit -m "Your commit message" - git push - ``` -2. **You do not have permission to push to the repository:** If you have cloned the repository directly from the main Ironhack repository without making a *Fork* first, you do not have write access to the repository. -To check which remote repository you have cloned, run the following terminal command from the project folder: - ```bash - git remote -v - ``` -If the link shown is the same as the main Ironhack repository, you will need to fork the repository to your GitHub account first and then clone your fork to your local machine to be able to push the changes. - -**Note**: You should make a copy of your local code to avoid losing it in the process. - - [Back to top](#faqs) - -
- diff --git a/lab-web-scraping (1).ipynb b/lab-web-scraping (1).ipynb new file mode 100644 index 0000000..894720a --- /dev/null +++ b/lab-web-scraping (1).ipynb @@ -0,0 +1 @@ +{"cells":[{"cell_type":"markdown","id":"7e7a1ab8-2599-417d-9a65-25ef07f3a786","metadata":{"id":"7e7a1ab8-2599-417d-9a65-25ef07f3a786"},"source":["# Lab | Web Scraping"]},{"cell_type":"markdown","id":"ce8882fc-4815-4567-92fa-b4816358ba7d","metadata":{"id":"ce8882fc-4815-4567-92fa-b4816358ba7d"},"source":["Welcome to the \"Books to Scrape\" Web Scraping Adventure Lab!\n","\n","**Objective**\n","\n","In this lab, we will embark on a mission to unearth valuable insights from the data available on Books to Scrape, an online platform showcasing a wide variety of books. As data analyst, you have been tasked with scraping a specific subset of book data from Books to Scrape to assist publishing companies in understanding the landscape of highly-rated books across different genres. Your insights will help shape future book marketing strategies and publishing decisions.\n","\n","**Background**\n","\n","In a world where data has become the new currency, businesses are leveraging big data to make informed decisions that drive success and profitability. The publishing industry, much like others, utilizes data analytics to understand market trends, reader preferences, and the performance of books based on factors such as genre, author, and ratings. Books to Scrape serves as a rich source of such data, offering detailed information about a diverse range of books, making it an ideal platform for extracting insights to aid in informed decision-making within the literary world.\n","\n","**Task**\n","\n","Your task is to create a Python script using BeautifulSoup and pandas to scrape Books to Scrape book data, focusing on book ratings and genres. The script should be able to filter books with ratings above a certain threshold and in specific genres. Additionally, the script should structure the scraped data in a tabular format using pandas for further analysis.\n","\n","**Expected Outcome**\n","\n","A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`. The function should scrape book data from the \"Books to Scrape\" website and return a `pandas` DataFrame with the following columns:\n","\n","**Expected Outcome**\n","\n","- A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`.\n","- The function should return a DataFrame with the following columns:\n"," - **UPC**: The Universal Product Code (UPC) of the book.\n"," - **Title**: The title of the book.\n"," - **Price (£)**: The price of the book in pounds.\n"," - **Rating**: The rating of the book (1-5 stars).\n"," - **Genre**: The genre of the book.\n"," - **Availability**: Whether the book is in stock or not.\n"," - **Description**: A brief description or product description of the book (if available).\n"," \n","You will execute this script to scrape data for books with a minimum rating of `4.0 and above` and a maximum price of `£20`.\n","\n","Remember to experiment with different ratings and prices to ensure your code is versatile and can handle various searches effectively!\n","\n","**Resources**\n","\n","- [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)\n","- [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/index.html)\n","- [Books to Scrape](https://books.toscrape.com/)\n"]},{"cell_type":"markdown","id":"3519921d-5890-445b-9a33-934ed8ee378c","metadata":{"id":"3519921d-5890-445b-9a33-934ed8ee378c"},"source":["**Hint**\n","\n","Your first mission is to familiarize yourself with the **Books to Scrape** website. Navigate to [Books to Scrape](http://books.toscrape.com/) and explore the available books to understand their layout and structure.\n","\n","Next, think about how you can set parameters for your data extraction:\n","\n","- **Minimum Rating**: Focus on books with a rating of 4.0 and above.\n","- **Maximum Price**: Filter for books priced up to £20.\n","\n","After reviewing the site, you can construct a plan for scraping relevant data. Pay attention to the details displayed for each book, including the title, price, rating, and availability. This will help you identify the correct HTML elements to target with your scraping script.\n","\n","Make sure to build your scraping URL and logic based on the patterns you observe in the HTML structure of the book listings!"]},{"cell_type":"markdown","id":"25a83a0d-a742-49f6-985e-e27887cbf922","metadata":{"id":"25a83a0d-a742-49f6-985e-e27887cbf922"},"source":["\n","---\n","\n","**Best of luck! Immerse yourself in the world of books, and may the data be with you!**"]},{"cell_type":"markdown","id":"7b75cf0d-9afa-4eec-a9e2-befeac68b2a0","metadata":{"id":"7b75cf0d-9afa-4eec-a9e2-befeac68b2a0"},"source":["**Important Note**:\n","\n","In the fast-changing online world, websites often update and change their structures. When you try this lab, the **Books to Scrape** website might differ from what you expect.\n","\n","If you encounter issues due to these changes, like new rules or obstacles preventing data extraction, don’t worry! Get creative.\n","\n","You can choose another website that interests you and is suitable for scraping data. Options like Wikipedia, The New York Times, or even library databases are great alternatives. The main goal remains the same: extract useful data and enhance your web scraping skills while exploring a source of information you enjoy. This is your opportunity to practice and adapt to different web environments!"]},{"cell_type":"code","execution_count":null,"id":"40359eee-9cd7-4884-bfa4-83344c222305","metadata":{"id":"40359eee-9cd7-4884-bfa4-83344c222305"},"outputs":[],"source":["# Your solution goes here\n","import requests\n","from bs4 import BeautifulSoup\n","import pandas as pd"]},{"cell_type":"code","source":["url = 'https://books.toscrape.com/'\n","\n","response = requests.get(url)\n","\n","response"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"tofcRN92X2vk","executionInfo":{"status":"ok","timestamp":1754248814019,"user_tz":-120,"elapsed":74,"user":{"displayName":"Santiago Larrea","userId":"01956628674237458372"}},"outputId":"4058cb2c-31f2-4de7-be2c-e4dc0b65a7f1"},"id":"tofcRN92X2vk","execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":[""]},"metadata":{},"execution_count":4}]},{"cell_type":"code","source":["print(response.content)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"fzEhHBx_YLom","executionInfo":{"status":"ok","timestamp":1754248836889,"user_tz":-120,"elapsed":343,"user":{"displayName":"Santiago Larrea","userId":"01956628674237458372"}},"outputId":"3e331600-6861-492b-aa08-473d2592ebc9"},"id":"fzEhHBx_YLom","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["b'\\n\\n\\n\\n \\n \\n \\n All products | Books to Scrape - Sandbox\\n\\n\\n \\n \\n \\n \\n \\n\\n \\n \\n\\n \\n \\n \\n\\n \\n \\n \\n \\n \\n \\n \\n \\n\\n\\n \\n \\n\\n \\n\\n \\n \\n \\n\\n \\n \\n\\n \\n \\n \\n \\n \\n
\\n
\\n
\\n
Books to Scrape We love being scraped!\\n
\\n\\n \\n
\\n
\\n
\\n\\n \\n \\n
\\n
\\n \\n
    \\n
  • \\n Home\\n
    • \\n
  • All products
    • \\n
\\n\\n
\\n\\n \\n\\n
\\n \\n
\\n

All products

\\n
\\n \\n\\n \\n\\n\\n\\n
\\n\\n
\\n\\n\\n
\\n \\n
\\n\\n \\n
\\n \\n
\\n \\n \\n
\\n\\n \\n \\n \\n 1000 results - showing 1 to 20.\\n \\n \\n \\n \\n
\\n \\n
\\n
Warning! This is a demo website for web scraping purposes. Prices and ratings here were randomly assigned and have no real meaning.
\\n\\n
\\n
    \\n \\n
  1. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"A\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    A Light in the ...

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa351.77

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    2. \\n \\n
  2. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Tipping\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Tipping the Velvet

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa353.74

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    3. \\n \\n
  3. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Soumission\"\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Soumission

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa350.10

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    4. \\n \\n
  4. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Sharp\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Sharp Objects

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa347.82

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    5. \\n \\n
  5. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Sapiens:\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Sapiens: A Brief History ...

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa354.23

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    6. \\n \\n
  6. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"The\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    The Requiem Red

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa322.65

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    7. \\n \\n
  7. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"The\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    The Dirty Little Secrets ...

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa333.34

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    8. \\n \\n
  8. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"The\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    The Coming Woman: A ...

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa317.93

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    9. \\n \\n
  9. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"The\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    The Boys in the ...

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa322.60

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    10. \\n \\n
  10. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"The\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    The Black Maria

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa352.15

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    11. \\n \\n
  11. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Starving\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Starving Hearts (Triangular Trade ...

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa313.99

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    12. \\n \\n
  12. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Shakespeare's\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Shakespeare's Sonnets

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa320.66

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    13. \\n \\n
  13. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Set\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Set Me Free

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa317.46

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    14. \\n \\n
  14. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Scott\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Scott Pilgrim's Precious Little ...

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa352.29

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    15. \\n \\n
  15. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Rip\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Rip it Up and ...

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa335.02

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    16. \\n \\n
  16. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Our\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Our Band Could Be ...

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa357.25

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    17. \\n \\n
  17. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Olio\"\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Olio

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa323.88

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    18. \\n \\n
  18. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Mesaerion:\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Mesaerion: The Best Science ...

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa337.59

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    19. \\n \\n
  19. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"Libertarianism\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    Libertarianism for Beginners

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa351.33

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    20. \\n \\n
  20. \\n\\n\\n\\n\\n\\n\\n
    \\n \\n
    \\n \\n \\n \"It's\\n \\n \\n
    \\n \\n\\n \\n \\n

    \\n \\n \\n \\n \\n \\n

    \\n \\n \\n\\n \\n

    It's Only the Himalayas

    \\n \\n\\n \\n
    \\n \\n\\n\\n\\n\\n\\n\\n \\n

    \\xc2\\xa345.17

    \\n \\n\\n

    \\n \\n \\n In stock\\n \\n

    \\n\\n \\n \\n\\n\\n\\n\\n\\n\\n \\n
    \\n \\n
    \\n\\n\\n \\n
    \\n \\n
    \\n\\n
    21. \\n \\n
\\n \\n\\n\\n\\n
\\n
    \\n \\n
  • \\n \\n Page 1 of 50\\n \\n
    • \\n \\n
  • next
    • \\n \\n
\\n
\\n\\n\\n
\\n
\\n \\n\\n\\n
\\n\\n
\\n
\\n
\\n\\n\\n \\n
\\n \\n \\n \\n
\\n\\n\\n \\n \\n \\n \\n \\n \\n \\n \\n\\n\\n \\n \\n \\n \\n \\n \\n \\n \\n\\n \\n \\n\\n\\n \\n \\n \\n\\n \\n\\n\\n \\n \\n\\n \\n \\n \\n \\n\\n'\n"]}]},{"cell_type":"code","source":["soup = BeautifulSoup(response.content)\n","soup"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"collapsed":true,"id":"AZyIwkm6YRQc","executionInfo":{"status":"ok","timestamp":1754248907781,"user_tz":-120,"elapsed":114,"user":{"displayName":"Santiago Larrea","userId":"01956628674237458372"}},"outputId":"93c47bc9-53b2-4985-bb24-7e22fa03fbc0"},"id":"AZyIwkm6YRQc","execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["\n"," \n","\n","\n"," All products | Books to Scrape - Sandbox\n","\n","\n","\n","\n","\n","\n","\n","\n","\n","\n","\n","\n","\n","\n","
\n","
\n","
\n","
Books to Scrape We love being scraped!\n","
\n","
\n","
\n","
\n","
\n","
\n","
    \n","
  • \n","Home\n","
    • \n","
  • All products
    • \n","
\n","
\n","\n","
\n","
\n","

All products

\n","
\n","
\n","
\n","
\n","
\n","
\n","
\n","
\n","1000 results - showing 1 to 20.\n"," \n"," \n"," \n"," \n","
\n","
\n","
Warning! This is a demo website for web scraping purposes. Prices and ratings here were randomly assigned and have no real meaning.
\n","
\n","
    \n","
  1. \n","
    \n","
    \n","\"A\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    A Light in the ...

    \n","
    \n","

    £51.77

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    2. \n","
  2. \n","
    \n","
    \n","\"Tipping\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Tipping the Velvet

    \n","
    \n","

    £53.74

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    3. \n","
  3. \n","
    \n","
    \n","\"Soumission\"\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Soumission

    \n","
    \n","

    £50.10

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    4. \n","
  4. \n","
    \n","
    \n","\"Sharp\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Sharp Objects

    \n","
    \n","

    £47.82

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    5. \n","
  5. \n","
    \n","
    \n","\"Sapiens:\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Sapiens: A Brief History ...

    \n","
    \n","

    £54.23

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    6. \n","
  6. \n","
    \n","
    \n","\"The\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    The Requiem Red

    \n","
    \n","

    £22.65

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    7. \n","
  7. \n","
    \n","
    \n","\"The\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    The Dirty Little Secrets ...

    \n","
    \n","

    £33.34

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    8. \n","
  8. \n","
    \n","
    \n","\"The\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    The Coming Woman: A ...

    \n","
    \n","

    £17.93

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    9. \n","
  9. \n","
    \n","
    \n","\"The\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    The Boys in the ...

    \n","
    \n","

    £22.60

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    10. \n","
  10. \n","
    \n","
    \n","\"The\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    The Black Maria

    \n","
    \n","

    £52.15

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    11. \n","
  11. \n","
    \n","
    \n","\"Starving\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Starving Hearts (Triangular Trade ...

    \n","
    \n","

    £13.99

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    12. \n","
  12. \n","
    \n","
    \n","\"Shakespeare's\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Shakespeare's Sonnets

    \n","
    \n","

    £20.66

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    13. \n","
  13. \n","
    \n","
    \n","\"Set\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Set Me Free

    \n","
    \n","

    £17.46

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    14. \n","
  14. \n","
    \n","
    \n","\"Scott\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Scott Pilgrim's Precious Little ...

    \n","
    \n","

    £52.29

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    15. \n","
  15. \n","
    \n","
    \n","\"Rip\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Rip it Up and ...

    \n","
    \n","

    £35.02

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    16. \n","
  16. \n","
    \n","
    \n","\"Our\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Our Band Could Be ...

    \n","
    \n","

    £57.25

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    17. \n","
  17. \n","
    \n","
    \n","\"Olio\"\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Olio

    \n","
    \n","

    £23.88

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    18. \n","
  18. \n","
    \n","
    \n","\"Mesaerion:\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Mesaerion: The Best Science ...

    \n","
    \n","

    £37.59

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    19. \n","
  19. \n","
    \n","
    \n","\"Libertarianism\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Libertarianism for Beginners

    \n","
    \n","

    £51.33

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    20. \n","
  20. \n","
    \n","
    \n","\"It's\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    It's Only the Himalayas

    \n","
    \n","

    £45.17

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    21. \n","
\n","
\n","
    \n","
  • \n"," \n"," Page 1 of 50\n"," \n","
    • \n","
  • next
    • \n","
\n","
\n","
\n","
\n","
\n","
\n","
\n","
\n","
\n","
\n","\n","\n","\n","\n","\n","\n","\n","\n","\n","\n","\n","\n",""]},"metadata":{},"execution_count":6}]},{"cell_type":"code","source":["grid = soup.find('ol', attrs={'class':'row'})\n","\n","grid"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"collapsed":true,"id":"eNRA5K7pYjxs","executionInfo":{"status":"ok","timestamp":1754256567848,"user_tz":-120,"elapsed":68,"user":{"displayName":"Santiago Larrea","userId":"01956628674237458372"}},"outputId":"529e3bd1-3cf1-49a3-ea0a-beb961eee8ed"},"id":"eNRA5K7pYjxs","execution_count":31,"outputs":[{"output_type":"execute_result","data":{"text/plain":["
    \n","
  1. \n","
    \n","
    \n","\"A\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    A Light in the ...

    \n","
    \n","

    £51.77

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    2. \n","
  2. \n","
    \n","
    \n","\"Tipping\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Tipping the Velvet

    \n","
    \n","

    £53.74

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    3. \n","
  3. \n","
    \n","
    \n","\"Soumission\"\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Soumission

    \n","
    \n","

    £50.10

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    4. \n","
  4. \n","
    \n","
    \n","\"Sharp\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Sharp Objects

    \n","
    \n","

    £47.82

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    5. \n","
  5. \n","
    \n","
    \n","\"Sapiens:\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Sapiens: A Brief History ...

    \n","
    \n","

    £54.23

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    6. \n","
  6. \n","
    \n","
    \n","\"The\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    The Requiem Red

    \n","
    \n","

    £22.65

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    7. \n","
  7. \n","
    \n","
    \n","\"The\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    The Dirty Little Secrets ...

    \n","
    \n","

    £33.34

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    8. \n","
  8. \n","
    \n","
    \n","\"The\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    The Coming Woman: A ...

    \n","
    \n","

    £17.93

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    9. \n","
  9. \n","
    \n","
    \n","\"The\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    The Boys in the ...

    \n","
    \n","

    £22.60

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    10. \n","
  10. \n","
    \n","
    \n","\"The\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    The Black Maria

    \n","
    \n","

    £52.15

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    11. \n","
  11. \n","
    \n","
    \n","\"Starving\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Starving Hearts (Triangular Trade ...

    \n","
    \n","

    £13.99

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    12. \n","
  12. \n","
    \n","
    \n","\"Shakespeare's\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Shakespeare's Sonnets

    \n","
    \n","

    £20.66

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    13. \n","
  13. \n","
    \n","
    \n","\"Set\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Set Me Free

    \n","
    \n","

    £17.46

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    14. \n","
  14. \n","
    \n","
    \n","\"Scott\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Scott Pilgrim's Precious Little ...

    \n","
    \n","

    £52.29

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    15. \n","
  15. \n","
    \n","
    \n","\"Rip\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Rip it Up and ...

    \n","
    \n","

    £35.02

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    16. \n","
  16. \n","
    \n","
    \n","\"Our\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Our Band Could Be ...

    \n","
    \n","

    £57.25

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    17. \n","
  17. \n","
    \n","
    \n","\"Olio\"\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Olio

    \n","
    \n","

    £23.88

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    18. \n","
  18. \n","
    \n","
    \n","\"Mesaerion:\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Mesaerion: The Best Science ...

    \n","
    \n","

    £37.59

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    19. \n","
  19. \n","
    \n","
    \n","\"Libertarianism\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    Libertarianism for Beginners

    \n","
    \n","

    £51.33

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    20. \n","
  20. \n","
    \n","
    \n","\"It's\n","
    \n","

    \n","\n","\n","\n","\n","\n","

    \n","

    It's Only the Himalayas

    \n","
    \n","

    £45.17

    \n","

    \n","\n"," \n"," In stock\n"," \n","

    \n","
    \n","\n","
    \n","
    \n","
    \n","
    21. \n","
"]},"metadata":{},"execution_count":31}]},{"cell_type":"code","source":["products = grid.findAll('li', attrs = {'class':'col-xs-6 col-sm-4 col-md-3 col-lg-3'})\n","\n","products"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"collapsed":true,"id":"uz91Ng3vZQyu","executionInfo":{"status":"ok","timestamp":1754256573366,"user_tz":-120,"elapsed":68,"user":{"displayName":"Santiago Larrea","userId":"01956628674237458372"}},"outputId":"8e817a0c-d5e3-4828-93c6-34d0ec066e71"},"id":"uz91Ng3vZQyu","execution_count":32,"outputs":[{"output_type":"stream","name":"stderr","text":["/tmp/ipython-input-3071271258.py:1: DeprecationWarning: Call to deprecated method findAll. (Replaced by find_all) -- Deprecated since version 4.0.0.\n"," products = grid.findAll('li', attrs = {'class':'col-xs-6 col-sm-4 col-md-3 col-lg-3'})\n"]},{"output_type":"execute_result","data":{"text/plain":["[
  • \n","
    \n","
    \n"," \"A\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    A Light in the ...

    \n","
    \n","

    £51.77

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Tipping\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Tipping the Velvet

    \n","
    \n","

    £53.74

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Soumission\"\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Soumission

    \n","
    \n","

    £50.10

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Sharp\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Sharp Objects

    \n","
    \n","

    £47.82

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Sapiens:\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Sapiens: A Brief History ...

    \n","
    \n","

    £54.23

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"The\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    The Requiem Red

    \n","
    \n","

    £22.65

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"The\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    The Dirty Little Secrets ...

    \n","
    \n","

    £33.34

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"The\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    The Coming Woman: A ...

    \n","
    \n","

    £17.93

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"The\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    The Boys in the ...

    \n","
    \n","

    £22.60

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"The\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    The Black Maria

    \n","
    \n","

    £52.15

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Starving\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Starving Hearts (Triangular Trade ...

    \n","
    \n","

    £13.99

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Shakespeare's\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Shakespeare's Sonnets

    \n","
    \n","

    £20.66

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Set\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Set Me Free

    \n","
    \n","

    £17.46

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Scott\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Scott Pilgrim's Precious Little ...

    \n","
    \n","

    £52.29

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Rip\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Rip it Up and ...

    \n","
    \n","

    £35.02

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Our\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Our Band Could Be ...

    \n","
    \n","

    £57.25

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Olio\"\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Olio

    \n","
    \n","

    £23.88

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Mesaerion:\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Mesaerion: The Best Science ...

    \n","
    \n","

    £37.59

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"Libertarianism\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    Libertarianism for Beginners

    \n","
    \n","

    £51.33

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ,\n","
  • \n","
    \n","
    \n"," \"It's\n","
    \n","

    \n"," \n"," \n"," \n"," \n"," \n","

    \n","

    It's Only the Himalayas

    \n","
    \n","

    £45.17

    \n","

    \n"," \n"," \n"," In stock\n"," \n","

    \n","
    \n"," \n","
    \n","
    \n","
    \n","
    • ]"]},"metadata":{},"execution_count":32}]},{"cell_type":"code","source":["products[0].get_text()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":35},"id":"obzadQ4hZ-Vo","executionInfo":{"status":"ok","timestamp":1754256577234,"user_tz":-120,"elapsed":8,"user":{"displayName":"Santiago Larrea","userId":"01956628674237458372"}},"outputId":"fa659fe6-5981-4e85-da7b-4c4e19c7a29b"},"id":"obzadQ4hZ-Vo","execution_count":33,"outputs":[{"output_type":"execute_result","data":{"text/plain":["'\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nA Light in the ...\\n\\n£51.77\\n\\n\\n \\n In stock\\n \\n\\n\\nAdd to basket\\n\\n\\n\\n'"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":33}]},{"cell_type":"code","source":["#UPC\n","products[0].find_all('a')[1]['title']\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":35},"id":"6SZA7PfoaMHV","executionInfo":{"status":"ok","timestamp":1754256578107,"user_tz":-120,"elapsed":44,"user":{"displayName":"Santiago Larrea","userId":"01956628674237458372"}},"outputId":"5f4d074e-c17a-48d4-ff46-002d8d0a8d12"},"id":"6SZA7PfoaMHV","execution_count":34,"outputs":[{"output_type":"execute_result","data":{"text/plain":["'A Light in the Attic'"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":34}]},{"cell_type":"code","source":["def get_title(book):\n"," title = book.find_all('a')[1]['title']\n"," return title\n","\n","def get_url(book):\n"," url = book.find_all('a')[1]['href']\n"," domain = 'https://books.toscrape.com/'\n"," return domain+url\n","def get_price(book):\n"," price = book.find('p', attrs = {'class':'price_color'}).get_text()\n"," return price"],"metadata":{"id":"x19--r9Oy49I","executionInfo":{"status":"ok","timestamp":1754256579787,"user_tz":-120,"elapsed":9,"user":{"displayName":"Santiago Larrea","userId":"01956628674237458372"}}},"id":"x19--r9Oy49I","execution_count":35,"outputs":[]},{"cell_type":"code","source":["books_dict = {}\n","key = 0\n","\n","for book in products:\n"," title = get_title(book)\n"," price = get_price(book)\n"," url = get_url(book)\n","\n"," books_dict[key] = {'title': title,\n"," 'price': price,\n"," 'url':url\n"," }\n"," key += 1\n","\n","books_df = pd.DataFrame.from_dict(books_dict, orient = 'index')\n","\n","books_df"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":677},"id":"np_HEEZwzuTA","executionInfo":{"status":"ok","timestamp":1754256627927,"user_tz":-120,"elapsed":34,"user":{"displayName":"Santiago Larrea","userId":"01956628674237458372"}},"outputId":"d9fb371b-1a83-4c30-8489-783130f18e09"},"id":"np_HEEZwzuTA","execution_count":38,"outputs":[{"output_type":"execute_result","data":{"text/plain":[" title price \\\n","0 A Light in the Attic £51.77 \n","1 Tipping the Velvet £53.74 \n","2 Soumission £50.10 \n","3 Sharp Objects £47.82 \n","4 Sapiens: A Brief History of Humankind £54.23 \n","5 The Requiem Red £22.65 \n","6 The Dirty Little Secrets of Getting Your Dream... £33.34 \n","7 The Coming Woman: A Novel Based on the Life of... £17.93 \n","8 The Boys in the Boat: Nine Americans and Their... £22.60 \n","9 The Black Maria £52.15 \n","10 Starving Hearts (Triangular Trade Trilogy, #1) £13.99 \n","11 Shakespeare's Sonnets £20.66 \n","12 Set Me Free £17.46 \n","13 Scott Pilgrim's Precious Little Life (Scott Pi... £52.29 \n","14 Rip it Up and Start Again £35.02 \n","15 Our Band Could Be Your Life: Scenes from the A... £57.25 \n","16 Olio £23.88 \n","17 Mesaerion: The Best Science Fiction Stories 18... £37.59 \n","18 Libertarianism for Beginners £51.33 \n","19 It's Only the Himalayas £45.17 \n","\n"," url \n","0 https://books.toscrape.com/catalogue/a-light-i... \n","1 https://books.toscrape.com/catalogue/tipping-t... \n","2 https://books.toscrape.com/catalogue/soumissio... \n","3 https://books.toscrape.com/catalogue/sharp-obj... \n","4 https://books.toscrape.com/catalogue/sapiens-a... \n","5 https://books.toscrape.com/catalogue/the-requi... \n","6 https://books.toscrape.com/catalogue/the-dirty... \n","7 https://books.toscrape.com/catalogue/the-comin... \n","8 https://books.toscrape.com/catalogue/the-boys-... \n","9 https://books.toscrape.com/catalogue/the-black... \n","10 https://books.toscrape.com/catalogue/starving-... \n","11 https://books.toscrape.com/catalogue/shakespea... \n","12 https://books.toscrape.com/catalogue/set-me-fr... \n","13 https://books.toscrape.com/catalogue/scott-pil... \n","14 https://books.toscrape.com/catalogue/rip-it-up... \n","15 https://books.toscrape.com/catalogue/our-band-... \n","16 https://books.toscrape.com/catalogue/olio_984/... \n","17 https://books.toscrape.com/catalogue/mesaerion... \n","18 https://books.toscrape.com/catalogue/libertari... \n","19 https://books.toscrape.com/catalogue/its-only-... "],"text/html":["\n","
    \n","
    \n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
    titlepriceurl
    0A Light in the Attic£51.77https://books.toscrape.com/catalogue/a-light-i...
    1Tipping the Velvet£53.74https://books.toscrape.com/catalogue/tipping-t...
    2Soumission£50.10https://books.toscrape.com/catalogue/soumissio...
    3Sharp Objects£47.82https://books.toscrape.com/catalogue/sharp-obj...
    4Sapiens: A Brief History of Humankind£54.23https://books.toscrape.com/catalogue/sapiens-a...
    5The Requiem Red£22.65https://books.toscrape.com/catalogue/the-requi...
    6The Dirty Little Secrets of Getting Your Dream...£33.34https://books.toscrape.com/catalogue/the-dirty...
    7The Coming Woman: A Novel Based on the Life of...£17.93https://books.toscrape.com/catalogue/the-comin...
    8The Boys in the Boat: Nine Americans and Their...£22.60https://books.toscrape.com/catalogue/the-boys-...
    9The Black Maria£52.15https://books.toscrape.com/catalogue/the-black...
    10Starving Hearts (Triangular Trade Trilogy, #1)£13.99https://books.toscrape.com/catalogue/starving-...
    11Shakespeare's Sonnets£20.66https://books.toscrape.com/catalogue/shakespea...
    12Set Me Free£17.46https://books.toscrape.com/catalogue/set-me-fr...
    13Scott Pilgrim's Precious Little Life (Scott Pi...£52.29https://books.toscrape.com/catalogue/scott-pil...
    14Rip it Up and Start Again£35.02https://books.toscrape.com/catalogue/rip-it-up...
    15Our Band Could Be Your Life: Scenes from the A...£57.25https://books.toscrape.com/catalogue/our-band-...
    16Olio£23.88https://books.toscrape.com/catalogue/olio_984/...
    17Mesaerion: The Best Science Fiction Stories 18...£37.59https://books.toscrape.com/catalogue/mesaerion...
    18Libertarianism for Beginners£51.33https://books.toscrape.com/catalogue/libertari...
    19It's Only the Himalayas£45.17https://books.toscrape.com/catalogue/its-only-...
    \n","
    \n","
    \n","\n","
    \n"," \n","\n"," \n","\n"," \n","
    \n","\n","\n","
    \n"," \n","\n","\n","\n"," \n","
    \n","\n","
    \n"," \n"," \n"," \n","
    \n","\n","
    \n","
    \n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"books_df","summary":"{\n \"name\": \"books_df\",\n \"rows\": 20,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 20,\n \"samples\": [\n \"A Light in the Attic\",\n \"Mesaerion: The Best Science Fiction Stories 1800-1849\",\n \"Our Band Could Be Your Life: Scenes from the American Indie Underground, 1981-1991\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 20,\n \"samples\": [\n \"\\u00a351.77\",\n \"\\u00a337.59\",\n \"\\u00a357.25\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"url\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 20,\n \"samples\": [\n \"https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html\",\n \"https://books.toscrape.com/catalogue/mesaerion-the-best-science-fiction-stories-1800-1849_983/index.html\",\n \"https://books.toscrape.com/catalogue/our-band-could-be-your-life-scenes-from-the-american-indie-underground-1981-1991_985/index.html\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"}},"metadata":{},"execution_count":38}]}],"metadata":{"colab":{"provenance":[]},"kernelspec":{"display_name":"Python 3 (ipykernel)","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.13"}},"nbformat":4,"nbformat_minor":5} \ No newline at end of file diff --git a/lab-web-scraping.ipynb b/lab-web-scraping.ipynb deleted file mode 100644 index e552783..0000000 --- a/lab-web-scraping.ipynb +++ /dev/null @@ -1,148 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786", - "metadata": { - "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786" - }, - "source": [ - "# Lab | Web Scraping" - ] - }, - { - "cell_type": "markdown", - "id": "ce8882fc-4815-4567-92fa-b4816358ba7d", - "metadata": { - "id": "ce8882fc-4815-4567-92fa-b4816358ba7d" - }, - "source": [ - "Welcome to the \"Books to Scrape\" Web Scraping Adventure Lab!\n", - "\n", - "**Objective**\n", - "\n", - "In this lab, we will embark on a mission to unearth valuable insights from the data available on Books to Scrape, an online platform showcasing a wide variety of books. As data analyst, you have been tasked with scraping a specific subset of book data from Books to Scrape to assist publishing companies in understanding the landscape of highly-rated books across different genres. Your insights will help shape future book marketing strategies and publishing decisions.\n", - "\n", - "**Background**\n", - "\n", - "In a world where data has become the new currency, businesses are leveraging big data to make informed decisions that drive success and profitability. The publishing industry, much like others, utilizes data analytics to understand market trends, reader preferences, and the performance of books based on factors such as genre, author, and ratings. Books to Scrape serves as a rich source of such data, offering detailed information about a diverse range of books, making it an ideal platform for extracting insights to aid in informed decision-making within the literary world.\n", - "\n", - "**Task**\n", - "\n", - "Your task is to create a Python script using BeautifulSoup and pandas to scrape Books to Scrape book data, focusing on book ratings and genres. The script should be able to filter books with ratings above a certain threshold and in specific genres. Additionally, the script should structure the scraped data in a tabular format using pandas for further analysis.\n", - "\n", - "**Expected Outcome**\n", - "\n", - "A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`. The function should scrape book data from the \"Books to Scrape\" website and return a `pandas` DataFrame with the following columns:\n", - "\n", - "**Expected Outcome**\n", - "\n", - "- A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`.\n", - "- The function should return a DataFrame with the following columns:\n", - " - **UPC**: The Universal Product Code (UPC) of the book.\n", - " - **Title**: The title of the book.\n", - " - **Price (£)**: The price of the book in pounds.\n", - " - **Rating**: The rating of the book (1-5 stars).\n", - " - **Genre**: The genre of the book.\n", - " - **Availability**: Whether the book is in stock or not.\n", - " - **Description**: A brief description or product description of the book (if available).\n", - " \n", - "You will execute this script to scrape data for books with a minimum rating of `4.0 and above` and a maximum price of `£20`. \n", - "\n", - "Remember to experiment with different ratings and prices to ensure your code is versatile and can handle various searches effectively!\n", - "\n", - "**Resources**\n", - "\n", - "- [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)\n", - "- [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/index.html)\n", - "- [Books to Scrape](https://books.toscrape.com/)\n" - ] - }, - { - "cell_type": "markdown", - "id": "3519921d-5890-445b-9a33-934ed8ee378c", - "metadata": { - "id": "3519921d-5890-445b-9a33-934ed8ee378c" - }, - "source": [ - "**Hint**\n", - "\n", - "Your first mission is to familiarize yourself with the **Books to Scrape** website. Navigate to [Books to Scrape](http://books.toscrape.com/) and explore the available books to understand their layout and structure. \n", - "\n", - "Next, think about how you can set parameters for your data extraction:\n", - "\n", - "- **Minimum Rating**: Focus on books with a rating of 4.0 and above.\n", - "- **Maximum Price**: Filter for books priced up to £20.\n", - "\n", - "After reviewing the site, you can construct a plan for scraping relevant data. Pay attention to the details displayed for each book, including the title, price, rating, and availability. This will help you identify the correct HTML elements to target with your scraping script.\n", - "\n", - "Make sure to build your scraping URL and logic based on the patterns you observe in the HTML structure of the book listings!" - ] - }, - { - "cell_type": "markdown", - "id": "25a83a0d-a742-49f6-985e-e27887cbf922", - "metadata": { - "id": "25a83a0d-a742-49f6-985e-e27887cbf922" - }, - "source": [ - "\n", - "---\n", - "\n", - "**Best of luck! Immerse yourself in the world of books, and may the data be with you!**" - ] - }, - { - "cell_type": "markdown", - "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0", - "metadata": { - "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0" - }, - "source": [ - "**Important Note**:\n", - "\n", - "In the fast-changing online world, websites often update and change their structures. When you try this lab, the **Books to Scrape** website might differ from what you expect.\n", - "\n", - "If you encounter issues due to these changes, like new rules or obstacles preventing data extraction, don’t worry! Get creative.\n", - "\n", - "You can choose another website that interests you and is suitable for scraping data. Options like Wikipedia, The New York Times, or even library databases are great alternatives. The main goal remains the same: extract useful data and enhance your web scraping skills while exploring a source of information you enjoy. This is your opportunity to practice and adapt to different web environments!" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "40359eee-9cd7-4884-bfa4-83344c222305", - "metadata": { - "id": "40359eee-9cd7-4884-bfa4-83344c222305" - }, - "outputs": [], - "source": [ - "# Your solution goes here" - ] - } - ], - "metadata": { - "colab": { - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.13" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -}