From 2aba074d8c7e2ff4c0d35cefa736fa9c008d53d1 Mon Sep 17 00:00:00 2001 From: Macarena Date: Sun, 3 Aug 2025 22:49:19 +0200 Subject: [PATCH] lab --- .../lab-web-scraping-checkpoint.ipynb | 148 ++++ books_data.csv | 21 + lab-web-scraping.ipynb | 681 ++++++++++++++---- 3 files changed, 714 insertions(+), 136 deletions(-) create mode 100644 .ipynb_checkpoints/lab-web-scraping-checkpoint.ipynb create mode 100644 books_data.csv diff --git a/.ipynb_checkpoints/lab-web-scraping-checkpoint.ipynb b/.ipynb_checkpoints/lab-web-scraping-checkpoint.ipynb new file mode 100644 index 0000000..e552783 --- /dev/null +++ b/.ipynb_checkpoints/lab-web-scraping-checkpoint.ipynb @@ -0,0 +1,148 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786", + "metadata": { + "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786" + }, + "source": [ + "# Lab | Web Scraping" + ] + }, + { + "cell_type": "markdown", + "id": "ce8882fc-4815-4567-92fa-b4816358ba7d", + "metadata": { + "id": "ce8882fc-4815-4567-92fa-b4816358ba7d" + }, + "source": [ + "Welcome to the \"Books to Scrape\" Web Scraping Adventure Lab!\n", + "\n", + "**Objective**\n", + "\n", + "In this lab, we will embark on a mission to unearth valuable insights from the data available on Books to Scrape, an online platform showcasing a wide variety of books. As data analyst, you have been tasked with scraping a specific subset of book data from Books to Scrape to assist publishing companies in understanding the landscape of highly-rated books across different genres. Your insights will help shape future book marketing strategies and publishing decisions.\n", + "\n", + "**Background**\n", + "\n", + "In a world where data has become the new currency, businesses are leveraging big data to make informed decisions that drive success and profitability. The publishing industry, much like others, utilizes data analytics to understand market trends, reader preferences, and the performance of books based on factors such as genre, author, and ratings. Books to Scrape serves as a rich source of such data, offering detailed information about a diverse range of books, making it an ideal platform for extracting insights to aid in informed decision-making within the literary world.\n", + "\n", + "**Task**\n", + "\n", + "Your task is to create a Python script using BeautifulSoup and pandas to scrape Books to Scrape book data, focusing on book ratings and genres. The script should be able to filter books with ratings above a certain threshold and in specific genres. Additionally, the script should structure the scraped data in a tabular format using pandas for further analysis.\n", + "\n", + "**Expected Outcome**\n", + "\n", + "A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`. The function should scrape book data from the \"Books to Scrape\" website and return a `pandas` DataFrame with the following columns:\n", + "\n", + "**Expected Outcome**\n", + "\n", + "- A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`.\n", + "- The function should return a DataFrame with the following columns:\n", + " - **UPC**: The Universal Product Code (UPC) of the book.\n", + " - **Title**: The title of the book.\n", + " - **Price (£)**: The price of the book in pounds.\n", + " - **Rating**: The rating of the book (1-5 stars).\n", + " - **Genre**: The genre of the book.\n", + " - **Availability**: Whether the book is in stock or not.\n", + " - **Description**: A brief description or product description of the book (if available).\n", + " \n", + "You will execute this script to scrape data for books with a minimum rating of `4.0 and above` and a maximum price of `£20`. \n", + "\n", + "Remember to experiment with different ratings and prices to ensure your code is versatile and can handle various searches effectively!\n", + "\n", + "**Resources**\n", + "\n", + "- [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)\n", + "- [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/index.html)\n", + "- [Books to Scrape](https://books.toscrape.com/)\n" + ] + }, + { + "cell_type": "markdown", + "id": "3519921d-5890-445b-9a33-934ed8ee378c", + "metadata": { + "id": "3519921d-5890-445b-9a33-934ed8ee378c" + }, + "source": [ + "**Hint**\n", + "\n", + "Your first mission is to familiarize yourself with the **Books to Scrape** website. Navigate to [Books to Scrape](http://books.toscrape.com/) and explore the available books to understand their layout and structure. \n", + "\n", + "Next, think about how you can set parameters for your data extraction:\n", + "\n", + "- **Minimum Rating**: Focus on books with a rating of 4.0 and above.\n", + "- **Maximum Price**: Filter for books priced up to £20.\n", + "\n", + "After reviewing the site, you can construct a plan for scraping relevant data. Pay attention to the details displayed for each book, including the title, price, rating, and availability. This will help you identify the correct HTML elements to target with your scraping script.\n", + "\n", + "Make sure to build your scraping URL and logic based on the patterns you observe in the HTML structure of the book listings!" + ] + }, + { + "cell_type": "markdown", + "id": "25a83a0d-a742-49f6-985e-e27887cbf922", + "metadata": { + "id": "25a83a0d-a742-49f6-985e-e27887cbf922" + }, + "source": [ + "\n", + "---\n", + "\n", + "**Best of luck! Immerse yourself in the world of books, and may the data be with you!**" + ] + }, + { + "cell_type": "markdown", + "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0", + "metadata": { + "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0" + }, + "source": [ + "**Important Note**:\n", + "\n", + "In the fast-changing online world, websites often update and change their structures. When you try this lab, the **Books to Scrape** website might differ from what you expect.\n", + "\n", + "If you encounter issues due to these changes, like new rules or obstacles preventing data extraction, don’t worry! Get creative.\n", + "\n", + "You can choose another website that interests you and is suitable for scraping data. Options like Wikipedia, The New York Times, or even library databases are great alternatives. The main goal remains the same: extract useful data and enhance your web scraping skills while exploring a source of information you enjoy. This is your opportunity to practice and adapt to different web environments!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "40359eee-9cd7-4884-bfa4-83344c222305", + "metadata": { + "id": "40359eee-9cd7-4884-bfa4-83344c222305" + }, + "outputs": [], + "source": [ + "# Your solution goes here" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/books_data.csv b/books_data.csv new file mode 100644 index 0000000..16fd9de --- /dev/null +++ b/books_data.csv @@ -0,0 +1,21 @@ +Title,UPC,Price,Rating,Genre,Availability,Description +A Light in the Attic,a897fe39b1053632,£51.77,3,Poetry,In stock (22 available),"It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. Silverstein's humorous and creative verse can amuse the dowdiest of readers. Lemon-faced adults and fidgety kids sit still and read these rhythmic words and laugh and smile and love th It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. Silverstein's humorous and creative verse can amuse the dowdiest of readers. Lemon-faced adults and fidgety kids sit still and read these rhythmic words and laugh and smile and love that Silverstein. Need proof of his genius? RockabyeRockabye baby, in the treetopDon't you know a treetopIs no safe place to rock?And who put you up there,And your cradle, too?Baby, I think someone down here'sGot it in for you. Shel, you never sounded so good. ...more" +Tipping the Velvet,90fa61229261140a,£53.74,1,Historical Fiction,In stock (20 available),"""Erotic and absorbing...Written with starling power.""--""The New York Times Book Review "" Nan King, an oyster girl, is captivated by the music hall phenomenon Kitty Butler, a male impersonator extraordinaire treading the boards in Canterbury. Through a friend at the box office, Nan manages to visit all her shows and finally meet her heroine. Soon after, she becomes Kitty's ""Erotic and absorbing...Written with starling power.""--""The New York Times Book Review "" Nan King, an oyster girl, is captivated by the music hall phenomenon Kitty Butler, a male impersonator extraordinaire treading the boards in Canterbury. Through a friend at the box office, Nan manages to visit all her shows and finally meet her heroine. Soon after, she becomes Kitty's dresser and the two head for the bright lights of Leicester Square where they begin a glittering career as music-hall stars in an all-singing and dancing double act. At the same time, behind closed doors, they admit their attraction to each other and their affair begins. ...more" +Soumission,6957f44c3847a760,£50.10,1,Fiction,In stock (20 available),"Dans une France assez proche de la nôtre, un homme s’engage dans la carrière universitaire. Peu motivé par l’enseignement, il s’attend à une vie ennuyeuse mais calme, protégée des grands drames historiques. Cependant les forces en jeu dans le pays ont fissuré le système politique jusqu’à provoquer son effondrement. Cette implosion sans soubresauts, sans vraie révolution, s Dans une France assez proche de la nôtre, un homme s’engage dans la carrière universitaire. Peu motivé par l’enseignement, il s’attend à une vie ennuyeuse mais calme, protégée des grands drames historiques. Cependant les forces en jeu dans le pays ont fissuré le système politique jusqu’à provoquer son effondrement. Cette implosion sans soubresauts, sans vraie révolution, se développe comme un mauvais rêve.Le talent de l’auteur, sa force visionnaire nous entraînent sur un terrain ambigu et glissant ; son regard sur notre civilisation vieillissante fait coexister dans ce roman les intuitions poétiques, les effets comiques, une mélancolie fataliste.Ce livre est une saisissante fable politique et morale. ...more" +Sharp Objects,e00eb4fd7b871a48,£47.82,4,Mystery,In stock (20 available),"WICKED above her hipbone, GIRL across her heart Words are like a road map to reporter Camille Preaker’s troubled past. Fresh from a brief stay at a psych hospital, Camille’s first assignment from the second-rate daily paper where she works brings her reluctantly back to her hometown to cover the murders of two preteen girls. NASTY on her kneecap, BABYDOLL on her leg Since WICKED above her hipbone, GIRL across her heart Words are like a road map to reporter Camille Preaker’s troubled past. Fresh from a brief stay at a psych hospital, Camille’s first assignment from the second-rate daily paper where she works brings her reluctantly back to her hometown to cover the murders of two preteen girls. NASTY on her kneecap, BABYDOLL on her leg Since she left town eight years ago, Camille has hardly spoken to her neurotic, hypochondriac mother or to the half-sister she barely knows: a beautiful thirteen-year-old with an eerie grip on the town. Now, installed again in her family’s Victorian mansion, Camille is haunted by the childhood tragedy she has spent her whole life trying to cut from her memory. HARMFUL on her wrist, WHORE on her ankle As Camille works to uncover the truth about these violent crimes, she finds herself identifying with the young victims—a bit too strongly. Clues keep leading to dead ends, forcing Camille to unravel the psychological puzzle of her own past to get at the story. Dogged by her own demons, Camille will have to confront what happened to her years before if she wants to survive this homecoming.With its taut, crafted writing, Sharp Objects is addictive, haunting, and unforgettable. ...more" +Sapiens: A Brief History of Humankind,4165285e1663650f,£54.23,5,History,In stock (20 available),"From a renowned historian comes a groundbreaking narrative of humanity’s creation and evolution—a #1 international bestseller—that explores the ways in which biology and history have defined us and enhanced our understanding of what it means to be “human.”One hundred thousand years ago, at least six different species of humans inhabited Earth. Yet today there is only one—h From a renowned historian comes a groundbreaking narrative of humanity’s creation and evolution—a #1 international bestseller—that explores the ways in which biology and history have defined us and enhanced our understanding of what it means to be “human.”One hundred thousand years ago, at least six different species of humans inhabited Earth. Yet today there is only one—homo sapiens. What happened to the others? And what may happen to us?Most books about the history of humanity pursue either a historical or a biological approach, but Dr. Yuval Noah Harari breaks the mold with this highly original book that begins about 70,000 years ago with the appearance of modern cognition. From examining the role evolving humans have played in the global ecosystem to charting the rise of empires, Sapiens integrates history and science to reconsider accepted narratives, connect past developments with contemporary concerns, and examine specific events within the context of larger ideas.Dr. Harari also compels us to look ahead, because over the last few decades humans have begun to bend laws of natural selection that have governed life for the past four billion years. We are acquiring the ability to design not only the world around us, but also ourselves. Where is this leading us, and what do we want to become?Featuring 27 photographs, 6 maps, and 25 illustrations/diagrams, this provocative and insightful work is sure to spark debate and is essential reading for aficionados of Jared Diamond, James Gleick, Matt Ridley, Robert Wright, and Sharon Moalem. ...more" +The Requiem Red,f77dbf2323deb740,£22.65,1,Young Adult,In stock (19 available),"Patient Twenty-nine.A monster roams the halls of Soothing Hills Asylum. Three girls dead. 29 is endowed with the curse…or gift of perception. She hears messages in music, sees lyrics in paintings. And the corn. A lifetime asylum resident, the orchestral corn music is the only constant in her life.Mason, a new, kind orderly, sees 29 as a woman, not a lunatic. And as his bel Patient Twenty-nine.A monster roams the halls of Soothing Hills Asylum. Three girls dead. 29 is endowed with the curse…or gift of perception. She hears messages in music, sees lyrics in paintings. And the corn. A lifetime asylum resident, the orchestral corn music is the only constant in her life.Mason, a new, kind orderly, sees 29 as a woman, not a lunatic. And as his belief in her grows, so does her self- confidence. That perhaps she might escape, might see the outside world. But the monster has other plans. The missing girls share one common thread...each was twenty-nine's cell mate. Will she be next? ...more" +The Dirty Little Secrets of Getting Your Dream Job,2597b5a345f45e1b,£33.34,4,Business,In stock (19 available),"Drawing on his extensive experience evaluating applicants for his marketing agency, and featuring stories based on real-life situations, sample cover letters, resumes, and straightforward advice, Don Raskin’s The Dirty Little Secrets of Getting Your Dream Job offers all the necessary tools for navigating the tough job market and securing your dream job.Don Raskin owns and Drawing on his extensive experience evaluating applicants for his marketing agency, and featuring stories based on real-life situations, sample cover letters, resumes, and straightforward advice, Don Raskin’s The Dirty Little Secrets of Getting Your Dream Job offers all the necessary tools for navigating the tough job market and securing your dream job.Don Raskin owns and operates MME, an advertising and marketing agency in New York City. During his twenty-five years at the agency he has interviewed hundreds of new college graduates for positions within his agency and has placed a strong emphasis on entry-level recruitment for positions in creative, account management, traffic, and production. Raskin has also mentored countless students and their parents on best practices for the job search. Over the years, Raskin has kept exceptionally detailed notes on the interviews he has conducted, observing the good, the bad, the ridiculous, the irreverent. He also has a treasure trove of over-the-top cover letters, resumes, interviews, and post interview follow-ups he has conducted and received. Now, he wants to share all the wisdom and insider secrets he has gathered to help students and first-time job seekers find a job in this economy.Based on his remarkable expertise, Raskin's book provides exclusive insight into the job search process and lets readers in on all of the dirty little secrets to landing their first job—or a new one—and finding career success. ...more" +"The Coming Woman: A Novel Based on the Life of the Infamous Feminist, Victoria Woodhull",e72a5dfc7e9267b2,£17.93,3,Default,In stock (19 available),"""If you have a heart, if you have a soul, Karen Hicks' The Coming Woman will make you fall in love with Victoria Woodhull.""-Kinky Friedman, author and Governor of the Heart of Texas ""What kind of confidence would it take for a woman to buck the old boy's club of politics in 1872? More than 140 years pre-Hillary, there was Victoria Woodhull. This book takes you back with a ""If you have a heart, if you have a soul, Karen Hicks' The Coming Woman will make you fall in love with Victoria Woodhull.""-Kinky Friedman, author and Governor of the Heart of Texas ""What kind of confidence would it take for a woman to buck the old boy's club of politics in 1872? More than 140 years pre-Hillary, there was Victoria Woodhull. This book takes you back with a breathtaking, present-tense bird's eye view into a time when women's liberation was primarily confined to one woman's very capable, independent mind. I couldn't put it down.""---Ruth Buzzi, Golden Globe Award winner and Television Hall of Fame inductee""Sadly, too many Americans have never heard of Victoria Woodhull, let alone learned of her story: her revolutionary campaign for the presidency at a time when women weren't even allowed to vote, her support for worker's rights, or her feminist commitment to equality, a century before the official battle over the Equal Rights Amendment. But in The Coming Woman, Karen Hicks brings Woodhull's efforts to life, and reminds us that some of our nation's greatest figures aren't always featured in the history books. It is a riveting account of an amazing woman and her struggle for justice and human dignity, told in an engaging and eminently readable style.""-Tim Wise, author, ""White Like Me: Reflections on Race from a Privileged Son""""The Coming Woman"" is a novel based on the life of feminist Victoria Woodhull, the first woman to run for U.S. President, 50 years before women could even vote!Running for President wasn't Victoria's only first as a woman. She was also the first to own a successful Wall Street firm, the first to publish a successful national newspaper, and the first to head the two-million-member Spiritualist Association. She was the first woman to enter the Senate Judiciary Committee chambers to petition for woman's suffrage, her argument changing the entire focus of the suffragist movement by pointing out that the 14th and 15th Amendments already gave women the vote.In her campaign for the Presidency, Victoria Woodhull boldly addressed many of the issues we still face today: equal pay for equal work; freedom in love; corporate greed and political corruption fueled by powerful lobbyists; and the increasing disparity between the rich and the poor, to name only a few. Her outspoken and common-sense ideas may shed a new perspective on the parallel conundrums of today's world.This bold, beautiful, and sexually progressive woman dared to take on society and religion. To make an example of the hypocrisy in what Mark Twain dubbed The Gilded Age, she exposed the extramarital affairs of the most popular religious figure of the day (Henry Ward Beecher). This led to her persecution and imprisonment and the longest, most infamous trial of the 19th century. But it did not stop her fight for equality.Victoria's epic story, set in the late 1800s, comes to life in a modern, fictional style, while staying true to the actual words and views of the many well-known characters. ...more" +The Boys in the Boat: Nine Americans and Their Epic Quest for Gold at the 1936 Berlin Olympics,e10e1e165dc8be4a,£22.60,4,Default,In stock (19 available),"For readers of Laura Hillenbrand's Seabiscuit and Unbroken, the dramatic story of the American rowing team that stunned the world at Hitler's 1936 Berlin Olympics Daniel James Brown’s robust book tells the story of the University of Washington’s 1936 eight-oar crew and their epic quest for an Olympic gold medal, a team that transformed the sport and grabbed the attention o For readers of Laura Hillenbrand's Seabiscuit and Unbroken, the dramatic story of the American rowing team that stunned the world at Hitler's 1936 Berlin Olympics Daniel James Brown’s robust book tells the story of the University of Washington’s 1936 eight-oar crew and their epic quest for an Olympic gold medal, a team that transformed the sport and grabbed the attention of millions of Americans. The sons of loggers, shipyard workers, and farmers, the boys defeated elite rivals first from eastern and British universities and finally the German crew rowing for Adolf Hitler in the Olympic games in Berlin, 1936. The emotional heart of the story lies with one rower, Joe Rantz, a teenager without family or prospects, who rows not for glory, but to regain his shattered self-regard and to find a place he can call home. The crew is assembled  by an enigmatic coach and mentored by a visionary, eccentric British boat builder, but it is their trust in each other that makes them a victorious team. They remind the country of what can be done when everyone quite literally pulls together—a perfect melding of commitment, determination, and optimism. Drawing on the boys’ own diaries and journals, their photos and memories of a once-in-a-lifetime shared dream, The Boys in the Boat is an irresistible story about beating the odds and finding hope in the most desperate of times—the improbable, intimate story of nine working-class boys from the American west who, in the depths of the Great Depression, showed the world what true grit really meant. It will appeal to readers of Erik Larson, Timothy Egan, James Bradley, and David Halberstam's The Amateurs. ...more" +The Black Maria,1dfe412b8ac00530,£52.15,1,Poetry,In stock (19 available),"Praise for Aracelis Girmay:""[Girmay's] every loss—she calls them estrangements—is a yearning for connection across time and place; her every fragment is a bulwark against ruin."" — O, The Oprah Magazine ""In Aracelis Girmay we have a poet who collects, polishes, and shares stories with such brilliant invention, tenderness, and intellectual liveliness that it is understandabl Praise for Aracelis Girmay:""[Girmay's] every loss—she calls them estrangements—is a yearning for connection across time and place; her every fragment is a bulwark against ruin."" — O, The Oprah Magazine ""In Aracelis Girmay we have a poet who collects, polishes, and shares stories with such brilliant invention, tenderness, and intellectual liveliness that it is understandable that we think of her as the blessed curator of our collective histories. There is in her art the vulnerability of one who lives inside of the stories that she gathers in this remarkable collection. Her poems set off alarms even as they transform the world she inhabits, showing us, in the process, exactly what she asks of Romare Bearden’s art: ‘…how not to // assign all blackness near the sea / a captivity.’ This is one of the many sweet contradictions in the black maria, which ‘is a black flag / wounding the pastoral.’ I am deeply thankful that we have a poet of her unique and singular talent writing today."" —Kwame DawesTaking its name from the moon's dark plains, misidentified as seas by early astronomers, the black maria investigates African diasporic histories, the consequences of racism within American culture, and the question of human identity. Central to this project is a desire to recognize the lives of Eritrean refugees who have been made invisible by years of immigration crisis, refugee status, exile, and resulting statelessness. The recipient of a 2015 Whiting Award for Poetry, Girmay's newest collection elegizes and celebrates life, while wrestling with the humanistic notion of seeing beyond: seeing violence, seeing grace, and seeing each other better.""to the sea""great storage house, historyon which we rode, we touchedthe brief pulse of your flutteringpages, spelled with salt & life,your rage, your indifferenceyour gentleness washing our feet,all of you going onwhether or not we live,to you we bring our carnationsyellow & pink, how they floatlike bright sentences atopyour memory's dark hairAracelis Girmay is the author of three poetry collections, the black maria; Kingdom Animalia, which won the Isabella Gardner Award and was a finalist for the NBCC Award; and Teeth. The recipient of a 2015 Whiting Award, she has received grants and fellowships from the Jerome, Cave Canem, and Watson foundations, as well as Civitella Ranieri and the NEA. She currently teaches at Hampshire College's School for Interdisciplinary Arts and in Drew University's low residency MFA program. Originally from Santa Ana, California, she splits her time between New York and Amherst, Massachusetts. ...more" +"Starving Hearts (Triangular Trade Trilogy, #1)",0312262ecafa5a40,£13.99,2,Default,In stock (19 available),"Since her assault, Miss Annette Chetwynd has been plagued by nightmares and worries about an arranged marriage. But she yearns to find her anonymous rescuer. Unfortunately, her health and intellect prevent it. Both repel suitors and cause Annette to doubt God's existence, at least until He answers her prayers in an unusual way ... Mr. Peter Adsley is joining the clergy, an Since her assault, Miss Annette Chetwynd has been plagued by nightmares and worries about an arranged marriage. But she yearns to find her anonymous rescuer. Unfortunately, her health and intellect prevent it. Both repel suitors and cause Annette to doubt God's existence, at least until He answers her prayers in an unusual way ... Mr. Peter Adsley is joining the clergy, and he desires a godly wife by his side. After his failed attempt to obtain one, he engages in a clandestine meeting with the bewitching young woman who keeps crossing his path. But she is so unsettling. Destined for disappointments, Peter and Annette endure disgraceful situations. Will Peter's faith sustain him through overwhelming setbacks? Can Annette overcome her doubts? Or will their starving hearts yield to the machinations of a mad man? ...more" +Shakespeare's Sonnets,30a7f60cd76ca58c,£20.66,4,Poetry,In stock (19 available),"This book is an important and complete collection of the Sonnets of William Shakespeare. Most readers are aware of the great plays and manuscripts written for the stage, but are unaware of the magnificent Sonnets which were written around the same period. This is an excellent, complete collection of the Sonnets and poetry of William Shakespeare and should not be missed by This book is an important and complete collection of the Sonnets of William Shakespeare. Most readers are aware of the great plays and manuscripts written for the stage, but are unaware of the magnificent Sonnets which were written around the same period. This is an excellent, complete collection of the Sonnets and poetry of William Shakespeare and should not be missed by those interested in the completion of a collection of his writings and those interested in early poetic works. ...more" +Set Me Free,ce6396b0f23f6ecc,£17.46,5,Young Adult,In stock (19 available),"Aaron Ledbetter’s future had been planned out for him since before he was born. Each year, the Ledbetter family vacation on Tybee Island gave Aaron a chance to briefly free himself from his family’s expectations. When he meets Jonas “Lucky” Luckett, a caricature artist in town with the traveling carnival, he must choose between the life that’s been mapped out for him, and Aaron Ledbetter’s future had been planned out for him since before he was born. Each year, the Ledbetter family vacation on Tybee Island gave Aaron a chance to briefly free himself from his family’s expectations. When he meets Jonas “Lucky” Luckett, a caricature artist in town with the traveling carnival, he must choose between the life that’s been mapped out for him, and the chance at true love. ...more" +Scott Pilgrim's Precious Little Life (Scott Pilgrim #1),3b1c02bac2a429e6,£52.29,5,Sequential Art,In stock (19 available),"Scott Pilgrim's life is totally sweet. He's 23 years old, he's in a rockband, he's ""between jobs"" and he's dating a cute high school girl. Nothing could possibly go wrong, unless a seriously mind-blowing, dangerously fashionable, rollerblading delivery girl named Ramona Flowers starts cruising through his dreams and sailing by him at parties. Will Scott's awesome life get Scott Pilgrim's life is totally sweet. He's 23 years old, he's in a rockband, he's ""between jobs"" and he's dating a cute high school girl. Nothing could possibly go wrong, unless a seriously mind-blowing, dangerously fashionable, rollerblading delivery girl named Ramona Flowers starts cruising through his dreams and sailing by him at parties. Will Scott's awesome life get turned upside-down? Will he have to face Ramona's seven evil ex-boyfriends in battle? The short answer is yes. The long answer is Scott Pilgrim, Volume 1: Scott Pilgrim's Precious Little Life ...more" +Rip it Up and Start Again,a34ba96d4081e6a4,£35.02,5,Music,In stock (19 available),"Punk's raw power rejuvenated rock, but by the summer of 1977 the movement had become a parody of itself. RIP IT UP AND START AGAIN is a celebration of what happened next.Post-punk bands like PiL, Joy Division, Talking Heads, The Fall and The Human League dedicated themselves to fulfilling punk's unfinished musical revolution. The post-punk groups were fervent modernists; w Punk's raw power rejuvenated rock, but by the summer of 1977 the movement had become a parody of itself. RIP IT UP AND START AGAIN is a celebration of what happened next.Post-punk bands like PiL, Joy Division, Talking Heads, The Fall and The Human League dedicated themselves to fulfilling punk's unfinished musical revolution. The post-punk groups were fervent modernists; whether experimenting with electronics and machine rhythm or adapting ideas from dub reggae and disco, they were totally confident they could invent a whole new future for music. ...more" +"Our Band Could Be Your Life: Scenes from the American Indie Underground, 1981-1991",deda3e61b9514b83,£57.25,3,Music,In stock (19 available),"This is the never-before-told story of the musical revolution that happened right under the nose of the Reagan Eighties--when a small but sprawling network of bands, labels, fanzines, radio stations, and other subversives reenergized American rock with punk rock's do-it-yourself credo and created music that was deeply personal, often brilliant, always challenging, and imme This is the never-before-told story of the musical revolution that happened right under the nose of the Reagan Eighties--when a small but sprawling network of bands, labels, fanzines, radio stations, and other subversives reenergized American rock with punk rock's do-it-yourself credo and created music that was deeply personal, often brilliant, always challenging, and immensely influential. This sweeping chronicle of music, politics, drugs, fear, loathing, and faith has been recognized as an indie rock classic in its own right. Among the bands profiled: Mission of Burma, Butthole Surfers, The Minutemen, Sonic Youth, Black Flag, Big Black, Hüsker Dü, Fugazi, Minor Threat, Mudhoney, The Replacements, Beat Happening, and Dinosaur Jr. ...more" +Olio,feb7cc7701ecf901,£23.88,1,Poetry,In stock (19 available),"Part fact, part fiction, Tyehimba Jess's much anticipated second book weaves sonnet, song, and narrative to examine the lives of mostly unrecorded African American performers directly before and after the Civil War up to World War I. Olio is an effort to understand how they met, resisted, complicated, co-opted, and sometimes defeated attempts to minstrelize them.So, while Part fact, part fiction, Tyehimba Jess's much anticipated second book weaves sonnet, song, and narrative to examine the lives of mostly unrecorded African American performers directly before and after the Civil War up to World War I. Olio is an effort to understand how they met, resisted, complicated, co-opted, and sometimes defeated attempts to minstrelize them.So, while I lead this choir, I still find thatI'm being led…I'm a missionarymending my faith in the midst of this flock…I toil in their fields of praise. When folks seethese freedmen stand and sing, they hear their Godspeak in tongues. These nine dark mouths sing shelter;they echo a hymn's haven from slavery's weather.Detroit native Tyehimba Jess' first book of poetry, leadbelly, was a winner of the 2004 National Poetry Series. Jess, a Cave Canem and NYU Alumni, has received fellowships from the Whiting Foundation, National Endowment for the Arts, Illinois Arts Council, and the Provincetown Fine Arts Work Center. Jess is also a veteran of the 2000 and 2001 Green Mill Poetry Slam Team. He exhibited his poetry at the 2011 TEDxNashville Conference. Jess is an Associate Professor of English at College of Staten Island. ...more" +Mesaerion: The Best Science Fiction Stories 1800-1849,e30f54cea9b38190,£37.59,1,Science Fiction,In stock (19 available),"Andrew Barger, award-winning author and engineer, has extensively researched forgotten journals and magazines of the early 19th century to locate groundbreaking science fiction short stories in the English language. In doing so, he found what is possibly the first science fiction story by a female (and it is not from Mary Shelley). Andrew located the first steampunk short Andrew Barger, award-winning author and engineer, has extensively researched forgotten journals and magazines of the early 19th century to locate groundbreaking science fiction short stories in the English language. In doing so, he found what is possibly the first science fiction story by a female (and it is not from Mary Shelley). Andrew located the first steampunk short story, which has not been republished since 1844. There is the first voyage to the moon in a balloon, republished for the first time since 1820 that further tells of a darkness machine and a lunarian named Zuloc. Other sci-stories include the first robotic insect and an electricity gun. Once again, Andrew has searched old texts to find the very best science fiction stories from the period when the genre automated to life, some of the stories are published for the first time in nearly 200 years. Read these fantastic stories today!OUR OWN COUNTRY So mechanical has the age become, that men seriously talk of flying machines, to go by steam, --not your air-balloons, but real Daedalian wings, made of wood and joints, nailed to your shoulder, --not wings of feathers and wax like the wings of Icarus, who fell into the Cretan sea, but real, solid, substantial, rock-maple wings with wrought-iron hinges, and huge concavities, to propel us through the air. Knickerbocker Magazine, May 18 ...more" +Libertarianism for Beginners,a18a4f574854aced,£51.33,2,Politics,In stock (19 available),"Libertarianism isn't about winning elections; it is first and foremost a political philosophy--a description of how, in the opinion of libertarians, free people ought to treat one another, at least when they use the law, which they regard as potentially dangerous. If libertarians are correct, the law should intrude into people's lives as little as possible, rarely telling Libertarianism isn't about winning elections; it is first and foremost a political philosophy--a description of how, in the opinion of libertarians, free people ought to treat one another, at least when they use the law, which they regard as potentially dangerous. If libertarians are correct, the law should intrude into people's lives as little as possible, rarely telling them what to do or how to live.A political and economic philosophy as old as John Locke and John Stuart Mill, but as alive and timely as Rand Paul, the Tea Party, and the novels of Ayn Rand, libertarianism emphasizes individual rights and calls for a radical reduction in the power and size of government. ""Libertarianism For Beginners"" lays out the history and principles of this often-misunderstood philosophy in lucid, dispassionate terms that help illuminate today's political dialogue."" ...more" +It's Only the Himalayas,a22124811bfa8350,£45.17,2,Travel,In stock (19 available),"“Wherever you go, whatever you do, just . . . don’t do anything stupid.” —My MotherDuring her yearlong adventure backpacking from South Africa to Singapore, S. Bedford definitely did a few things her mother might classify as ""stupid."" She swam with great white sharks in South Africa, ran from lions in Zimbabwe, climbed a Himalayan mountain without training in Nepal, and wa “Wherever you go, whatever you do, just . . . don’t do anything stupid.” —My MotherDuring her yearlong adventure backpacking from South Africa to Singapore, S. Bedford definitely did a few things her mother might classify as ""stupid."" She swam with great white sharks in South Africa, ran from lions in Zimbabwe, climbed a Himalayan mountain without training in Nepal, and watched as her friend was attacked by a monkey in Indonesia.But interspersed in those slightly more crazy moments, Sue Bedfored and her friend ""Sara the Stoic"" experienced the sights, sounds, life, and culture of fifteen countries. Joined along the way by a few friends and their aging fathers here and there, Sue and Sara experience the trip of a lifetime. They fall in love with the world, cultivate an appreciation for home, and discover who, or what, they want to become.It's Only the Himalayas is the incredibly funny, sometimes outlandish, always entertaining confession of a young backpacker that will inspire you to take your own adventure. ...more" diff --git a/lab-web-scraping.ipynb b/lab-web-scraping.ipynb index e552783..f11e312 100644 --- a/lab-web-scraping.ipynb +++ b/lab-web-scraping.ipynb @@ -1,148 +1,557 @@ { - "cells": [ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786" + }, + "source": [ + "# Lab | Web Scraping" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ce8882fc-4815-4567-92fa-b4816358ba7d" + }, + "source": [ + "Welcome to the \"Books to Scrape\" Web Scraping Adventure Lab!\n", + "\n", + "**Objective**\n", + "\n", + "In this lab, we will embark on a mission to unearth valuable insights from the data available on Books to Scrape, an online platform showcasing a wide variety of books. As data analyst, you have been tasked with scraping a specific subset of book data from Books to Scrape to assist publishing companies in understanding the landscape of highly-rated books across different genres. Your insights will help shape future book marketing strategies and publishing decisions.\n", + "\n", + "**Background**\n", + "\n", + "In a world where data has become the new currency, businesses are leveraging big data to make informed decisions that drive success and profitability. The publishing industry, much like others, utilizes data analytics to understand market trends, reader preferences, and the performance of books based on factors such as genre, author, and ratings. Books to Scrape serves as a rich source of such data, offering detailed information about a diverse range of books, making it an ideal platform for extracting insights to aid in informed decision-making within the literary world.\n", + "\n", + "**Task**\n", + "\n", + "Your task is to create a Python script using BeautifulSoup and pandas to scrape Books to Scrape book data, focusing on book ratings and genres. The script should be able to filter books with ratings above a certain threshold and in specific genres. Additionally, the script should structure the scraped data in a tabular format using pandas for further analysis.\n", + "\n", + "**Expected Outcome**\n", + "\n", + "A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`. The function should scrape book data from the \"Books to Scrape\" website and return a `pandas` DataFrame with the following columns:\n", + "\n", + "**Expected Outcome**\n", + "\n", + "- A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`.\n", + "- The function should return a DataFrame with the following columns:\n", + " - **UPC**: The Universal Product Code (UPC) of the book.\n", + " - **Title**: The title of the book.\n", + " - **Price (£)**: The price of the book in pounds.\n", + " - **Rating**: The rating of the book (1-5 stars).\n", + " - **Genre**: The genre of the book.\n", + " - **Availability**: Whether the book is in stock or not.\n", + " - **Description**: A brief description or product description of the book (if available).\n", + " \n", + "You will execute this script to scrape data for books with a minimum rating of `4.0 and above` and a maximum price of `£20`. \n", + "\n", + "Remember to experiment with different ratings and prices to ensure your code is versatile and can handle various searches effectively!\n", + "\n", + "**Resources**\n", + "\n", + "- [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)\n", + "- [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/index.html)\n", + "- [Books to Scrape](https://books.toscrape.com/)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3519921d-5890-445b-9a33-934ed8ee378c" + }, + "source": [ + "**Hint**\n", + "\n", + "Your first mission is to familiarize yourself with the **Books to Scrape** website. Navigate to [Books to Scrape](http://books.toscrape.com/) and explore the available books to understand their layout and structure. \n", + "\n", + "Next, think about how you can set parameters for your data extraction:\n", + "\n", + "- **Minimum Rating**: Focus on books with a rating of 4.0 and above.\n", + "- **Maximum Price**: Filter for books priced up to £20.\n", + "\n", + "After reviewing the site, you can construct a plan for scraping relevant data. Pay attention to the details displayed for each book, including the title, price, rating, and availability. This will help you identify the correct HTML elements to target with your scraping script.\n", + "\n", + "Make sure to build your scraping URL and logic based on the patterns you observe in the HTML structure of the book listings!" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "25a83a0d-a742-49f6-985e-e27887cbf922" + }, + "source": [ + "\n", + "---\n", + "\n", + "**Best of luck! Immerse yourself in the world of books, and may the data be with you!**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0" + }, + "source": [ + "**Important Note**:\n", + "\n", + "In the fast-changing online world, websites often update and change their structures. When you try this lab, the **Books to Scrape** website might differ from what you expect.\n", + "\n", + "If you encounter issues due to these changes, like new rules or obstacles preventing data extraction, don’t worry! Get creative.\n", + "\n", + "You can choose another website that interests you and is suitable for scraping data. Options like Wikipedia, The New York Times, or even library databases are great alternatives. The main goal remains the same: extract useful data and enhance your web scraping skills while exploring a source of information you enjoy. This is your opportunity to practice and adapt to different web environments!" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "id": "40359eee-9cd7-4884-bfa4-83344c222305" + }, + "outputs": [ { - "cell_type": "markdown", - "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786", - "metadata": { - "id": "7e7a1ab8-2599-417d-9a65-25ef07f3a786" - }, - "source": [ - "# Lab | Web Scraping" + "data": { + "text/plain": [ + "" ] - }, + }, + "execution_count": 1, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Your solution goes here\n", + "\n", + "# !conda install requests beautifulsoup4 -y\n", + "\n", + "import requests\n", + "from bs4 import BeautifulSoup\n", + "import pandas as pd\n", + "\n", + "url = \"https://books.toscrape.com/\"\n", + "response = requests.get(url)\n", + "response\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "2658aec3", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "ce8882fc-4815-4567-92fa-b4816358ba7d", - "metadata": { - "id": "ce8882fc-4815-4567-92fa-b4816358ba7d" - }, - "source": [ - "Welcome to the \"Books to Scrape\" Web Scraping Adventure Lab!\n", - "\n", - "**Objective**\n", - "\n", - "In this lab, we will embark on a mission to unearth valuable insights from the data available on Books to Scrape, an online platform showcasing a wide variety of books. As data analyst, you have been tasked with scraping a specific subset of book data from Books to Scrape to assist publishing companies in understanding the landscape of highly-rated books across different genres. Your insights will help shape future book marketing strategies and publishing decisions.\n", - "\n", - "**Background**\n", - "\n", - "In a world where data has become the new currency, businesses are leveraging big data to make informed decisions that drive success and profitability. The publishing industry, much like others, utilizes data analytics to understand market trends, reader preferences, and the performance of books based on factors such as genre, author, and ratings. Books to Scrape serves as a rich source of such data, offering detailed information about a diverse range of books, making it an ideal platform for extracting insights to aid in informed decision-making within the literary world.\n", - "\n", - "**Task**\n", - "\n", - "Your task is to create a Python script using BeautifulSoup and pandas to scrape Books to Scrape book data, focusing on book ratings and genres. The script should be able to filter books with ratings above a certain threshold and in specific genres. Additionally, the script should structure the scraped data in a tabular format using pandas for further analysis.\n", - "\n", - "**Expected Outcome**\n", - "\n", - "A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`. The function should scrape book data from the \"Books to Scrape\" website and return a `pandas` DataFrame with the following columns:\n", - "\n", - "**Expected Outcome**\n", - "\n", - "- A function named `scrape_books` that takes two parameters: `min_rating` and `max_price`.\n", - "- The function should return a DataFrame with the following columns:\n", - " - **UPC**: The Universal Product Code (UPC) of the book.\n", - " - **Title**: The title of the book.\n", - " - **Price (£)**: The price of the book in pounds.\n", - " - **Rating**: The rating of the book (1-5 stars).\n", - " - **Genre**: The genre of the book.\n", - " - **Availability**: Whether the book is in stock or not.\n", - " - **Description**: A brief description or product description of the book (if available).\n", - " \n", - "You will execute this script to scrape data for books with a minimum rating of `4.0 and above` and a maximum price of `£20`. \n", - "\n", - "Remember to experiment with different ratings and prices to ensure your code is versatile and can handle various searches effectively!\n", - "\n", - "**Resources**\n", - "\n", - "- [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)\n", - "- [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/index.html)\n", - "- [Books to Scrape](https://books.toscrape.com/)\n" - ] - }, + "name": "stdout", + "output_type": "stream", + "text": [ + "['A Light in the Attic', 'Tipping the Velvet', 'Soumission', 'Sharp Objects', 'Sapiens: A Brief History of Humankind', 'The Requiem Red', 'The Dirty Little Secrets of Getting Your Dream Job', 'The Coming Woman: A Novel Based on the Life of the Infamous Feminist, Victoria Woodhull', 'The Boys in the Boat: Nine Americans and Their Epic Quest for Gold at the 1936 Berlin Olympics', 'The Black Maria', 'Starving Hearts (Triangular Trade Trilogy, #1)', \"Shakespeare's Sonnets\", 'Set Me Free', \"Scott Pilgrim's Precious Little Life (Scott Pilgrim #1)\", 'Rip it Up and Start Again', 'Our Band Could Be Your Life: Scenes from the American Indie Underground, 1981-1991', 'Olio', 'Mesaerion: The Best Science Fiction Stories 1800-1849', 'Libertarianism for Beginners', \"It's Only the Himalayas\"]\n", + "['£51.77', '£53.74', '£50.10', '£47.82', '£54.23', '£22.65', '£33.34', '£17.93', '£22.60', '£52.15', '£13.99', '£20.66', '£17.46', '£52.29', '£35.02', '£57.25', '£23.88', '£37.59', '£51.33', '£45.17']\n" + ] + } + ], + "source": [ + "#price\n", + "titulos = [h3.a['title'] for h3 in soup.find_all('h3')]\n", + "precios = [p.text for p in soup.find_all('p', class_='price_color')]\n", + "\n", + "print(titulos)\n", + "print(precios)\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cf052e83", + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "3519921d-5890-445b-9a33-934ed8ee378c", - "metadata": { - "id": "3519921d-5890-445b-9a33-934ed8ee378c" - }, - "source": [ - "**Hint**\n", - "\n", - "Your first mission is to familiarize yourself with the **Books to Scrape** website. Navigate to [Books to Scrape](http://books.toscrape.com/) and explore the available books to understand their layout and structure. \n", - "\n", - "Next, think about how you can set parameters for your data extraction:\n", - "\n", - "- **Minimum Rating**: Focus on books with a rating of 4.0 and above.\n", - "- **Maximum Price**: Filter for books priced up to £20.\n", - "\n", - "After reviewing the site, you can construct a plan for scraping relevant data. Pay attention to the details displayed for each book, including the title, price, rating, and availability. This will help you identify the correct HTML elements to target with your scraping script.\n", - "\n", - "Make sure to build your scraping URL and logic based on the patterns you observe in the HTML structure of the book listings!" - ] - }, + "name": "stdout", + "output_type": "stream", + "text": [ + "['A Light in the Attic', 'Tipping the Velvet', 'Soumission', 'Sharp Objects', 'Sapiens: A Brief History of Humankind', 'The Requiem Red', 'The Dirty Little Secrets of Getting Your Dream Job', 'The Coming Woman: A Novel Based on the Life of the Infamous Feminist, Victoria Woodhull', 'The Boys in the Boat: Nine Americans and Their Epic Quest for Gold at the 1936 Berlin Olympics', 'The Black Maria', 'Starving Hearts (Triangular Trade Trilogy, #1)', \"Shakespeare's Sonnets\", 'Set Me Free', \"Scott Pilgrim's Precious Little Life (Scott Pilgrim #1)\", 'Rip it Up and Start Again', 'Our Band Could Be Your Life: Scenes from the American Indie Underground, 1981-1991', 'Olio', 'Mesaerion: The Best Science Fiction Stories 1800-1849', 'Libertarianism for Beginners', \"It's Only the Himalayas\"]\n" + ] + } + ], + "source": [] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "25a83a0d-a742-49f6-985e-e27887cbf922", - "metadata": { - "id": "25a83a0d-a742-49f6-985e-e27887cbf922" - }, - "source": [ - "\n", - "---\n", - "\n", - "**Best of luck! Immerse yourself in the world of books, and may the data be with you!**" - ] - }, + "name": "stdout", + "output_type": "stream", + "text": [ + "Title: A Light in the Attic, Rating: 3\n", + "Title: Tipping the Velvet, Rating: 1\n", + "Title: Soumission, Rating: 1\n", + "Title: Sharp Objects, Rating: 4\n", + "Title: Sapiens: A Brief History of Humankind, Rating: 5\n", + "Title: The Requiem Red, Rating: 1\n", + "Title: The Dirty Little Secrets of Getting Your Dream Job, Rating: 4\n", + "Title: The Coming Woman: A Novel Based on the Life of the Infamous Feminist, Victoria Woodhull, Rating: 3\n", + "Title: The Boys in the Boat: Nine Americans and Their Epic Quest for Gold at the 1936 Berlin Olympics, Rating: 4\n", + "Title: The Black Maria, Rating: 1\n", + "Title: Starving Hearts (Triangular Trade Trilogy, #1), Rating: 2\n", + "Title: Shakespeare's Sonnets, Rating: 4\n", + "Title: Set Me Free, Rating: 5\n", + "Title: Scott Pilgrim's Precious Little Life (Scott Pilgrim #1), Rating: 5\n", + "Title: Rip it Up and Start Again, Rating: 5\n", + "Title: Our Band Could Be Your Life: Scenes from the American Indie Underground, 1981-1991, Rating: 3\n", + "Title: Olio, Rating: 1\n", + "Title: Mesaerion: The Best Science Fiction Stories 1800-1849, Rating: 1\n", + "Title: Libertarianism for Beginners, Rating: 2\n", + "Title: It's Only the Himalayas, Rating: 2\n" + ] + } + ], + "source": [ + "# Find all book containers\n", + "books = soup.find_all('article', class_='product_pod')\n", + "\n", + "# Prepare lists to store titles and ratings\n", + "titles = []\n", + "ratings = []\n", + "\n", + "# Mapping rating words to numbers (optional)\n", + "rating_map = {\n", + " 'One': 1,\n", + " 'Two': 2,\n", + " 'Three': 3,\n", + " 'Four': 4,\n", + " 'Five': 5\n", + "}\n", + "\n", + "for book in books:\n", + " # Extract full title from

tag's title attribute\n", + " title = book.h3.a['title']\n", + " titles.append(title)\n", + " \n", + " # Extract rating from the

tag with class 'star-rating'\n", + " p_rating = book.find('p', class_='star-rating')\n", + " classes = p_rating.get('class') # e.g. ['star-rating', 'Three']\n", + " \n", + " # The second class is the rating word\n", + " rating_word = classes[1] if len(classes) > 1 else 'Not rated'\n", + " \n", + " # Convert to numeric rating if desired\n", + " rating_num = rating_map.get(rating_word, 0)\n", + " ratings.append(rating_num)\n", + "\n", + "# Print results\n", + "for t, r in zip(titles, ratings):\n", + " print(f'Title: {t}, Rating: {r}')\n" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0", - "metadata": { - "id": "7b75cf0d-9afa-4eec-a9e2-befeac68b2a0" - }, - "source": [ - "**Important Note**:\n", - "\n", - "In the fast-changing online world, websites often update and change their structures. When you try this lab, the **Books to Scrape** website might differ from what you expect.\n", - "\n", - "If you encounter issues due to these changes, like new rules or obstacles preventing data extraction, don’t worry! Get creative.\n", - "\n", - "You can choose another website that interests you and is suitable for scraping data. Options like Wikipedia, The New York Times, or even library databases are great alternatives. The main goal remains the same: extract useful data and enhance your web scraping skills while exploring a source of information you enjoy. This is your opportunity to practice and adapt to different web environments!" - ] - }, + "name": "stdout", + "output_type": "stream", + "text": [ + "{'UPC': 'a897fe39b1053632', 'Title': 'A Light in the Attic', 'Price': '£51.77', 'Rating': 3, 'Genre': 'Poetry', 'Availability': 'In stock (22 available)', 'Description': \"It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. Silverstein's humorous and creative verse can amuse the dowdiest of readers. Lemon-faced adults and fidgety kids sit still and read these rhythmic words and laugh and smile and love th It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. Silverstein's humorous and creative verse can amuse the dowdiest of readers. Lemon-faced adults and fidgety kids sit still and read these rhythmic words and laugh and smile and love that Silverstein. Need proof of his genius? RockabyeRockabye baby, in the treetopDon't you know a treetopIs no safe place to rock?And who put you up there,And your cradle, too?Baby, I think someone down here'sGot it in for you. Shel, you never sounded so good. ...more\"}\n", + "{'UPC': '90fa61229261140a', 'Title': 'Tipping the Velvet', 'Price': '£53.74', 'Rating': 1, 'Genre': 'Historical Fiction', 'Availability': 'In stock (20 available)', 'Description': '\"Erotic and absorbing...Written with starling power.\"--\"The New York Times Book Review \" Nan King, an oyster girl, is captivated by the music hall phenomenon Kitty Butler, a male impersonator extraordinaire treading the boards in Canterbury. Through a friend at the box office, Nan manages to visit all her shows and finally meet her heroine. Soon after, she becomes Kitty\\'s \"Erotic and absorbing...Written with starling power.\"--\"The New York Times Book Review \" Nan King, an oyster girl, is captivated by the music hall phenomenon Kitty Butler, a male impersonator extraordinaire treading the boards in Canterbury. Through a friend at the box office, Nan manages to visit all her shows and finally meet her heroine. Soon after, she becomes Kitty\\'s dresser and the two head for the bright lights of Leicester Square where they begin a glittering career as music-hall stars in an all-singing and dancing double act. At the same time, behind closed doors, they admit their attraction to each other and their affair begins. ...more'}\n", + "{'UPC': '6957f44c3847a760', 'Title': 'Soumission', 'Price': '£50.10', 'Rating': 1, 'Genre': 'Fiction', 'Availability': 'In stock (20 available)', 'Description': 'Dans une France assez proche de la nôtre, un homme sâ\\x80\\x99engage dans la carrière universitaire. Peu motivé par lâ\\x80\\x99enseignement, il sâ\\x80\\x99attend Ã\\xa0 une vie ennuyeuse mais calme, protégée des grands drames historiques. Cependant les forces en jeu dans le pays ont fissuré le système politique jusquâ\\x80\\x99Ã\\xa0 provoquer son effondrement. Cette implosion sans soubresauts, sans vraie révolution, s Dans une France assez proche de la nôtre, un homme sâ\\x80\\x99engage dans la carrière universitaire. Peu motivé par lâ\\x80\\x99enseignement, il sâ\\x80\\x99attend Ã\\xa0 une vie ennuyeuse mais calme, protégée des grands drames historiques. Cependant les forces en jeu dans le pays ont fissuré le système politique jusquâ\\x80\\x99Ã\\xa0 provoquer son effondrement. Cette implosion sans soubresauts, sans vraie révolution, se développe comme un mauvais rêve.Le talent de lâ\\x80\\x99auteur, sa force visionnaire nous entraînent sur un terrain ambigu et glissant ; son regard sur notre civilisation vieillissante fait coexister dans ce roman les intuitions poétiques, les effets comiques, une mélancolie fataliste.Ce livre est une saisissante fable politique et morale. ...more'}\n", + "{'UPC': 'e00eb4fd7b871a48', 'Title': 'Sharp Objects', 'Price': '£47.82', 'Rating': 4, 'Genre': 'Mystery', 'Availability': 'In stock (20 available)', 'Description': 'WICKED above her hipbone, GIRL across her heart Words are like a road map to reporter Camille Preakerâ\\x80\\x99s troubled past. Fresh from a brief stay at a psych hospital, Camilleâ\\x80\\x99s first assignment from the second-rate daily paper where she works brings her reluctantly back to her hometown to cover the murders of two preteen girls. NASTY on her kneecap, BABYDOLL on her leg Since WICKED above her hipbone, GIRL across her heart Words are like a road map to reporter Camille Preakerâ\\x80\\x99s troubled past. Fresh from a brief stay at a psych hospital, Camilleâ\\x80\\x99s first assignment from the second-rate daily paper where she works brings her reluctantly back to her hometown to cover the murders of two preteen girls. NASTY on her kneecap, BABYDOLL on her leg Since she left town eight years ago, Camille has hardly spoken to her neurotic, hypochondriac mother or to the half-sister she barely knows: a beautiful thirteen-year-old with an eerie grip on the town. Now, installed again in her familyâ\\x80\\x99s Victorian mansion, Camille is haunted by the childhood tragedy she has spent her whole life trying to cut from her memory. HARMFUL on her wrist, WHORE on her ankle As Camille works to uncover the truth about these violent crimes, she finds herself identifying with the young victimsâ\\x80\\x94a bit too strongly. Clues keep leading to dead ends, forcing Camille to unravel the psychological puzzle of her own past to get at the story. Dogged by her own demons, Camille will have to confront what happened to her years before if she wants to survive this homecoming.With its taut, crafted writing, Sharp Objects is addictive, haunting, and unforgettable. ...more'}\n", + "{'UPC': '4165285e1663650f', 'Title': 'Sapiens: A Brief History of Humankind', 'Price': '£54.23', 'Rating': 5, 'Genre': 'History', 'Availability': 'In stock (20 available)', 'Description': 'From a renowned historian comes a groundbreaking narrative of humanityâ\\x80\\x99s creation and evolutionâ\\x80\\x94a #1 international bestsellerâ\\x80\\x94that explores the ways in which biology and history have defined us and enhanced our understanding of what it means to be â\\x80\\x9chuman.â\\x80\\x9dOne hundred thousand years ago, at least six different species of humans inhabited Earth. Yet today there is only oneâ\\x80\\x94h From a renowned historian comes a groundbreaking narrative of humanityâ\\x80\\x99s creation and evolutionâ\\x80\\x94a #1 international bestsellerâ\\x80\\x94that explores the ways in which biology and history have defined us and enhanced our understanding of what it means to be â\\x80\\x9chuman.â\\x80\\x9dOne hundred thousand years ago, at least six different species of humans inhabited Earth. Yet today there is only oneâ\\x80\\x94homo sapiens. What happened to the others? And what may happen to us?Most books about the history of humanity pursue either a historical or a biological approach, but Dr. Yuval Noah Harari breaks the mold with this highly original book that begins about 70,000 years ago with the appearance of modern cognition. From examining the role evolving humans have played in the global ecosystem to charting the rise of empires, Sapiens integrates history and science to reconsider accepted narratives, connect past developments with contemporary concerns, and examine specific events within the context of larger ideas.Dr. Harari also compels us to look ahead, because over the last few decades humans have begun to bend laws of natural selection that have governed life for the past four billion years. We are acquiring the ability to design not only the world around us, but also ourselves. Where is this leading us, and what do we want to become?Featuring 27 photographs, 6 maps, and 25 illustrations/diagrams, this provocative and insightful work is sure to spark debate and is essential reading for aficionados of Jared Diamond, James Gleick, Matt Ridley, Robert Wright, and Sharon Moalem. ...more'}\n", + "{'UPC': 'f77dbf2323deb740', 'Title': 'The Requiem Red', 'Price': '£22.65', 'Rating': 1, 'Genre': 'Young Adult', 'Availability': 'In stock (19 available)', 'Description': \"Patient Twenty-nine.A monster roams the halls of Soothing Hills Asylum. Three girls dead. 29 is endowed with the curseâ\\x80¦or gift of perception. She hears messages in music, sees lyrics in paintings. And the corn. A lifetime asylum resident, the orchestral corn music is the only constant in her life.Mason, a new, kind orderly, sees 29 as a woman, not a lunatic. And as his bel Patient Twenty-nine.A monster roams the halls of Soothing Hills Asylum. Three girls dead. 29 is endowed with the curseâ\\x80¦or gift of perception. She hears messages in music, sees lyrics in paintings. And the corn. A lifetime asylum resident, the orchestral corn music is the only constant in her life.Mason, a new, kind orderly, sees 29 as a woman, not a lunatic. And as his belief in her grows, so does her self- confidence. That perhaps she might escape, might see the outside world. But the monster has other plans. The missing girls share one common thread...each was twenty-nine's cell mate. Will she be next? ...more\"}\n", + "{'UPC': '2597b5a345f45e1b', 'Title': 'The Dirty Little Secrets of Getting Your Dream Job', 'Price': '£33.34', 'Rating': 4, 'Genre': 'Business', 'Availability': 'In stock (19 available)', 'Description': \"Drawing on his extensive experience evaluating applicants for his marketing agency, and featuring stories based on real-life situations, sample cover letters, resumes, and straightforward advice, Don Raskinâ\\x80\\x99s The Dirty Little Secrets of Getting Your Dream Job offers all the necessary tools for navigating the tough job market and securing your dream job.Don Raskin owns and Drawing on his extensive experience evaluating applicants for his marketing agency, and featuring stories based on real-life situations, sample cover letters, resumes, and straightforward advice, Don Raskinâ\\x80\\x99s The Dirty Little Secrets of Getting Your Dream Job offers all the necessary tools for navigating the tough job market and securing your dream job.Don Raskin owns and operates MME, an advertising and marketing agency in New York City. During his twenty-five years at the agency he has interviewed hundreds of new college graduates for positions within his agency and has placed a strong emphasis on entry-level recruitment for positions in creative, account management, traffic, and production. Raskin has also mentored countless students and their parents on best practices for the job search. Over the years, Raskin has kept exceptionally detailed notes on the interviews he has conducted, observing the good, the bad, the ridiculous, the irreverent. He also has a treasure trove of over-the-top cover letters, resumes, interviews, and post interview follow-ups he has conducted and received. Now, he wants to share all the wisdom and insider secrets he has gathered to help students and first-time job seekers find a job in this economy.Based on his remarkable expertise, Raskin's book provides exclusive insight into the job search process and lets readers in on all of the dirty little secrets to landing their first jobâ\\x80\\x94or a new oneâ\\x80\\x94and finding career success. ...more\"}\n", + "{'UPC': 'e72a5dfc7e9267b2', 'Title': 'The Coming Woman: A Novel Based on the Life of the Infamous Feminist, Victoria Woodhull', 'Price': '£17.93', 'Rating': 3, 'Genre': 'Default', 'Availability': 'In stock (19 available)', 'Description': '\"If you have a heart, if you have a soul, Karen Hicks\\' The Coming Woman will make you fall in love with Victoria Woodhull.\"-Kinky Friedman, author and Governor of the Heart of Texas \"What kind of confidence would it take for a woman to buck the old boy\\'s club of politics in 1872? More than 140 years pre-Hillary, there was Victoria Woodhull. This book takes you back with a \"If you have a heart, if you have a soul, Karen Hicks\\' The Coming Woman will make you fall in love with Victoria Woodhull.\"-Kinky Friedman, author and Governor of the Heart of Texas \"What kind of confidence would it take for a woman to buck the old boy\\'s club of politics in 1872? More than 140 years pre-Hillary, there was Victoria Woodhull. This book takes you back with a breathtaking, present-tense bird\\'s eye view into a time when women\\'s liberation was primarily confined to one woman\\'s very capable, independent mind. I couldn\\'t put it down.\"---Ruth Buzzi, Golden Globe Award winner and Television Hall of Fame inductee\"Sadly, too many Americans have never heard of Victoria Woodhull, let alone learned of her story: her revolutionary campaign for the presidency at a time when women weren\\'t even allowed to vote, her support for worker\\'s rights, or her feminist commitment to equality, a century before the official battle over the Equal Rights Amendment. But in The Coming Woman, Karen Hicks brings Woodhull\\'s efforts to life, and reminds us that some of our nation\\'s greatest figures aren\\'t always featured in the history books. It is a riveting account of an amazing woman and her struggle for justice and human dignity, told in an engaging and eminently readable style.\"-Tim Wise, author, \"White Like Me: Reflections on Race from a Privileged Son\"\"The Coming Woman\" is a novel based on the life of feminist Victoria Woodhull, the first woman to run for U.S. President, 50 years before women could even vote!Running for President wasn\\'t Victoria\\'s only first as a woman. She was also the first to own a successful Wall Street firm, the first to publish a successful national newspaper, and the first to head the two-million-member Spiritualist Association. She was the first woman to enter the Senate Judiciary Committee chambers to petition for woman\\'s suffrage, her argument changing the entire focus of the suffragist movement by pointing out that the 14th and 15th Amendments already gave women the vote.In her campaign for the Presidency, Victoria Woodhull boldly addressed many of the issues we still face today: equal pay for equal work; freedom in love; corporate greed and political corruption fueled by powerful lobbyists; and the increasing disparity between the rich and the poor, to name only a few. Her outspoken and common-sense ideas may shed a new perspective on the parallel conundrums of today\\'s world.This bold, beautiful, and sexually progressive woman dared to take on society and religion. To make an example of the hypocrisy in what Mark Twain dubbed The Gilded Age, she exposed the extramarital affairs of the most popular religious figure of the day (Henry Ward Beecher). This led to her persecution and imprisonment and the longest, most infamous trial of the 19th century. But it did not stop her fight for equality.Victoria\\'s epic story, set in the late 1800s, comes to life in a modern, fictional style, while staying true to the actual words and views of the many well-known characters. ...more'}\n", + "{'UPC': 'e10e1e165dc8be4a', 'Title': 'The Boys in the Boat: Nine Americans and Their Epic Quest for Gold at the 1936 Berlin Olympics', 'Price': '£22.60', 'Rating': 4, 'Genre': 'Default', 'Availability': 'In stock (19 available)', 'Description': \"For readers of Laura Hillenbrand's Seabiscuit and Unbroken, the dramatic story of the American rowing team that stunned the world at Hitler's 1936 Berlin Olympics Daniel James Brownâ\\x80\\x99s robust book tells the story of the University of Washingtonâ\\x80\\x99s 1936 eight-oar crew and their epic quest for an Olympic gold medal, a team that transformed the sport and grabbed the attention o For readers of Laura Hillenbrand's Seabiscuit and Unbroken, the dramatic story of the American rowing team that stunned the world at Hitler's 1936 Berlin Olympics Daniel James Brownâ\\x80\\x99s robust book tells the story of the University of Washingtonâ\\x80\\x99s 1936 eight-oar crew and their epic quest for an Olympic gold medal, a team that transformed the sport and grabbed the attention of millions of Americans. The sons of loggers, shipyard workers, and farmers, the boys defeated elite rivals first from eastern and British universities and finally the German crew rowing for Adolf Hitler in the Olympic games in Berlin, 1936. The emotional heart of the story lies with one rower, Joe Rantz, a teenager without family or prospects, who rows not for glory, but to regain his shattered self-regard and to find a place he can call home. The crew is assembledÂ\\xa0 by an enigmatic coach and mentored by a visionary, eccentric British boat builder, but it is their trust in each other that makes them a victorious team. They remind the country of what can be done when everyone quite literally pulls togetherâ\\x80\\x94a perfect melding of commitment, determination, and optimism. Drawing on the boysâ\\x80\\x99 own diaries and journals, their photos and memories of a once-in-a-lifetime shared dream, The Boys in the Boat is an irresistible story about beating the odds and finding hope in the most desperate of timesâ\\x80\\x94the improbable, intimate story of nine working-class boys from the American west who, in the depths of the Great Depression, showed the world what true grit really meant. It will appeal to readers of Erik Larson, Timothy Egan, James Bradley, and David Halberstam's The Amateurs. ...more\"}\n", + "{'UPC': '1dfe412b8ac00530', 'Title': 'The Black Maria', 'Price': '£52.15', 'Rating': 1, 'Genre': 'Poetry', 'Availability': 'In stock (19 available)', 'Description': 'Praise for Aracelis Girmay:\"[Girmay\\'s] every lossâ\\x80\\x94she calls them estrangementsâ\\x80\\x94is a yearning for connection across time and place; her every fragment is a bulwark against ruin.\" â\\x80\\x94 O, The Oprah Magazine \"In Aracelis Girmay we have a poet who collects, polishes, and shares stories with such brilliant invention, tenderness, and intellectual liveliness that it is understandabl Praise for Aracelis Girmay:\"[Girmay\\'s] every lossâ\\x80\\x94she calls them estrangementsâ\\x80\\x94is a yearning for connection across time and place; her every fragment is a bulwark against ruin.\" â\\x80\\x94 O, The Oprah Magazine \"In Aracelis Girmay we have a poet who collects, polishes, and shares stories with such brilliant invention, tenderness, and intellectual liveliness that it is understandable that we think of her as the blessed curator of our collective histories. There is in her art the vulnerability of one who lives inside of the stories that she gathers in this remarkable collection. Her poems set off alarms even as they transform the world she inhabits, showing us, in the process, exactly what she asks of Romare Beardenâ\\x80\\x99s art: â\\x80\\x98â\\x80¦how not to // assign all blackness near the sea / a captivity.â\\x80\\x99 This is one of the many sweet contradictions in the black maria, which â\\x80\\x98is a black flag / wounding the pastoral.â\\x80\\x99 I am deeply thankful that we have a poet of her unique and singular talent writing today.\" â\\x80\\x94Kwame DawesTaking its name from the moon\\'s dark plains, misidentified as seas by early astronomers, the black maria investigates African diasporic histories, the consequences of racism within American culture, and the question of human identity. Central to this project is a desire to recognize the lives of Eritrean refugees who have been made invisible by years of immigration crisis, refugee status, exile, and resulting statelessness. The recipient of a 2015 Whiting Award for Poetry, Girmay\\'s newest collection elegizes and celebrates life, while wrestling with the humanistic notion of seeing beyond: seeing violence, seeing grace, and seeing each other better.\"to the sea\"great storage house, historyon which we rode, we touchedthe brief pulse of your flutteringpages, spelled with salt & life,your rage, your indifferenceyour gentleness washing our feet,all of you going onwhether or not we live,to you we bring our carnationsyellow & pink, how they floatlike bright sentences atopyour memory\\'s dark hairAracelis Girmay is the author of three poetry collections, the black maria; Kingdom Animalia, which won the Isabella Gardner Award and was a finalist for the NBCC Award; and Teeth. The recipient of a 2015 Whiting Award, she has received grants and fellowships from the Jerome, Cave Canem, and Watson foundations, as well as Civitella Ranieri and the NEA. She currently teaches at Hampshire College\\'s School for Interdisciplinary Arts and in Drew University\\'s low residency MFA program. Originally from Santa Ana, California, she splits her time between New York and Amherst, Massachusetts. ...more'}\n", + "{'UPC': '0312262ecafa5a40', 'Title': 'Starving Hearts (Triangular Trade Trilogy, #1)', 'Price': '£13.99', 'Rating': 2, 'Genre': 'Default', 'Availability': 'In stock (19 available)', 'Description': \"Since her assault, Miss Annette Chetwynd has been plagued by nightmares and worries about an arranged marriage. But she yearns to find her anonymous rescuer. Unfortunately, her health and intellect prevent it. Both repel suitors and cause Annette to doubt God's existence, at least until He answers her prayers in an unusual way ... Mr. Peter Adsley is joining the clergy, an Since her assault, Miss Annette Chetwynd has been plagued by nightmares and worries about an arranged marriage. But she yearns to find her anonymous rescuer. Unfortunately, her health and intellect prevent it. Both repel suitors and cause Annette to doubt God's existence, at least until He answers her prayers in an unusual way ... Mr. Peter Adsley is joining the clergy, and he desires a godly wife by his side. After his failed attempt to obtain one, he engages in a clandestine meeting with the bewitching young woman who keeps crossing his path. But she is so unsettling. Destined for disappointments, Peter and Annette endure disgraceful situations. Will Peter's faith sustain him through overwhelming setbacks? Can Annette overcome her doubts? Or will their starving hearts yield to the machinations of a mad man? ...more\"}\n", + "{'UPC': '30a7f60cd76ca58c', 'Title': \"Shakespeare's Sonnets\", 'Price': '£20.66', 'Rating': 4, 'Genre': 'Poetry', 'Availability': 'In stock (19 available)', 'Description': 'This book is an important and complete collection of the Sonnets of William Shakespeare. Most readers are aware of the great plays and manuscripts written for the stage, but are unaware of the magnificent Sonnets which were written around the same period. This is an excellent, complete collection of the Sonnets and poetry of William Shakespeare and should not be missed by This book is an important and complete collection of the Sonnets of William Shakespeare. Most readers are aware of the great plays and manuscripts written for the stage, but are unaware of the magnificent Sonnets which were written around the same period. This is an excellent, complete collection of the Sonnets and poetry of William Shakespeare and should not be missed by those interested in the completion of a collection of his writings and those interested in early poetic works. ...more'}\n", + "{'UPC': 'ce6396b0f23f6ecc', 'Title': 'Set Me Free', 'Price': '£17.46', 'Rating': 5, 'Genre': 'Young Adult', 'Availability': 'In stock (19 available)', 'Description': 'Aaron Ledbetterâ\\x80\\x99s future had been planned out for him since before he was born. Each year, the Ledbetter family vacation on Tybee Island gave Aaron a chance to briefly free himself from his familyâ\\x80\\x99s expectations. When he meets Jonas â\\x80\\x9cLuckyâ\\x80\\x9d Luckett, a caricature artist in town with the traveling carnival, he must choose between the life thatâ\\x80\\x99s been mapped out for him, and Aaron Ledbetterâ\\x80\\x99s future had been planned out for him since before he was born. Each year, the Ledbetter family vacation on Tybee Island gave Aaron a chance to briefly free himself from his familyâ\\x80\\x99s expectations. When he meets Jonas â\\x80\\x9cLuckyâ\\x80\\x9d Luckett, a caricature artist in town with the traveling carnival, he must choose between the life thatâ\\x80\\x99s been mapped out for him, and the chance at true love. ...more'}\n", + "{'UPC': '3b1c02bac2a429e6', 'Title': \"Scott Pilgrim's Precious Little Life (Scott Pilgrim #1)\", 'Price': '£52.29', 'Rating': 5, 'Genre': 'Sequential Art', 'Availability': 'In stock (19 available)', 'Description': 'Scott Pilgrim\\'s life is totally sweet. He\\'s 23 years old, he\\'s in a rockband, he\\'s \"between jobs\" and he\\'s dating a cute high school girl. Nothing could possibly go wrong, unless a seriously mind-blowing, dangerously fashionable, rollerblading delivery girl named Ramona Flowers starts cruising through his dreams and sailing by him at parties. Will Scott\\'s awesome life get Scott Pilgrim\\'s life is totally sweet. He\\'s 23 years old, he\\'s in a rockband, he\\'s \"between jobs\" and he\\'s dating a cute high school girl. Nothing could possibly go wrong, unless a seriously mind-blowing, dangerously fashionable, rollerblading delivery girl named Ramona Flowers starts cruising through his dreams and sailing by him at parties. Will Scott\\'s awesome life get turned upside-down? Will he have to face Ramona\\'s seven evil ex-boyfriends in battle? The short answer is yes. The long answer is Scott Pilgrim, Volume 1: Scott Pilgrim\\'s Precious Little Life ...more'}\n", + "{'UPC': 'a34ba96d4081e6a4', 'Title': 'Rip it Up and Start Again', 'Price': '£35.02', 'Rating': 5, 'Genre': 'Music', 'Availability': 'In stock (19 available)', 'Description': \"Punk's raw power rejuvenated rock, but by the summer of 1977 the movement had become a parody of itself. RIP IT UP AND START AGAIN is a celebration of what happened next.Post-punk bands like PiL, Joy Division, Talking Heads, The Fall and The Human League dedicated themselves to fulfilling punk's unfinished musical revolution. The post-punk groups were fervent modernists; w Punk's raw power rejuvenated rock, but by the summer of 1977 the movement had become a parody of itself. RIP IT UP AND START AGAIN is a celebration of what happened next.Post-punk bands like PiL, Joy Division, Talking Heads, The Fall and The Human League dedicated themselves to fulfilling punk's unfinished musical revolution. The post-punk groups were fervent modernists; whether experimenting with electronics and machine rhythm or adapting ideas from dub reggae and disco, they were totally confident they could invent a whole new future for music. ...more\"}\n", + "{'UPC': 'deda3e61b9514b83', 'Title': 'Our Band Could Be Your Life: Scenes from the American Indie Underground, 1981-1991', 'Price': '£57.25', 'Rating': 3, 'Genre': 'Music', 'Availability': 'In stock (19 available)', 'Description': \"This is the never-before-told story of the musical revolution that happened right under the nose of the Reagan Eighties--when a small but sprawling network of bands, labels, fanzines, radio stations, and other subversives reenergized American rock with punk rock's do-it-yourself credo and created music that was deeply personal, often brilliant, always challenging, and imme This is the never-before-told story of the musical revolution that happened right under the nose of the Reagan Eighties--when a small but sprawling network of bands, labels, fanzines, radio stations, and other subversives reenergized American rock with punk rock's do-it-yourself credo and created music that was deeply personal, often brilliant, always challenging, and immensely influential. This sweeping chronicle of music, politics, drugs, fear, loathing, and faith has been recognized as an indie rock classic in its own right. Among the bands profiled: Mission of Burma, Butthole Surfers, The Minutemen, Sonic Youth, Black Flag, Big Black, Hüsker Dü, Fugazi, Minor Threat, Mudhoney, The Replacements, Beat Happening, and Dinosaur Jr. ...more\"}\n", + "{'UPC': 'feb7cc7701ecf901', 'Title': 'Olio', 'Price': '£23.88', 'Rating': 1, 'Genre': 'Poetry', 'Availability': 'In stock (19 available)', 'Description': \"Part fact, part fiction, Tyehimba Jess's much anticipated second book weaves sonnet, song, and narrative to examine the lives of mostly unrecorded African American performers directly before and after the Civil War up to World War I. Olio is an effort to understand how they met, resisted, complicated, co-opted, and sometimes defeated attempts to minstrelize them.So, while Part fact, part fiction, Tyehimba Jess's much anticipated second book weaves sonnet, song, and narrative to examine the lives of mostly unrecorded African American performers directly before and after the Civil War up to World War I. Olio is an effort to understand how they met, resisted, complicated, co-opted, and sometimes defeated attempts to minstrelize them.So, while I lead this choir, I still find thatI'm being ledâ\\x80¦I'm a missionarymending my faith in the midst of this flockâ\\x80¦I toil in their fields of praise. When folks seethese freedmen stand and sing, they hear their Godspeak in tongues. These nine dark mouths sing shelter;they echo a hymn's haven from slavery's weather.Detroit native Tyehimba Jess' first book of poetry, leadbelly, was a winner of the 2004 National Poetry Series. Jess, a Cave Canem and NYU Alumni, has received fellowships from the Whiting Foundation, National Endowment for the Arts, Illinois Arts Council, and the Provincetown Fine Arts Work Center. Jess is also a veteran of the 2000 and 2001 Green Mill Poetry Slam Team. He exhibited his poetry at the 2011 TEDxNashville Conference. Jess is an Associate Professor of English at College of Staten Island. ...more\"}\n", + "{'UPC': 'e30f54cea9b38190', 'Title': 'Mesaerion: The Best Science Fiction Stories 1800-1849', 'Price': '£37.59', 'Rating': 1, 'Genre': 'Science Fiction', 'Availability': 'In stock (19 available)', 'Description': 'Andrew Barger, award-winning author and engineer, has extensively researched forgotten journals and magazines of the early 19th century to locate groundbreaking science fiction short stories in the English language. In doing so, he found what is possibly the first science fiction story by a female (and it is not from Mary Shelley). Andrew located the first steampunk short Andrew Barger, award-winning author and engineer, has extensively researched forgotten journals and magazines of the early 19th century to locate groundbreaking science fiction short stories in the English language. In doing so, he found what is possibly the first science fiction story by a female (and it is not from Mary Shelley). Andrew located the first steampunk short story, which has not been republished since 1844. There is the first voyage to the moon in a balloon, republished for the first time since 1820 that further tells of a darkness machine and a lunarian named Zuloc. Other sci-stories include the first robotic insect and an electricity gun. Once again, Andrew has searched old texts to find the very best science fiction stories from the period when the genre automated to life, some of the stories are published for the first time in nearly 200 years. Read these fantastic stories today!OUR OWN COUNTRY So mechanical has the age become, that men seriously talk of flying machines, to go by steam, --not your air-balloons, but real Daedalian wings, made of wood and joints, nailed to your shoulder, --not wings of feathers and wax like the wings of Icarus, who fell into the Cretan sea, but real, solid, substantial, rock-maple wings with wrought-iron hinges, and huge concavities, to propel us through the air. Knickerbocker Magazine, May 18 ...more'}\n", + "{'UPC': 'a18a4f574854aced', 'Title': 'Libertarianism for Beginners', 'Price': '£51.33', 'Rating': 2, 'Genre': 'Politics', 'Availability': 'In stock (19 available)', 'Description': 'Libertarianism isn\\'t about winning elections; it is first and foremost a political philosophy--a description of how, in the opinion of libertarians, free people ought to treat one another, at least when they use the law, which they regard as potentially dangerous. If libertarians are correct, the law should intrude into people\\'s lives as little as possible, rarely telling Libertarianism isn\\'t about winning elections; it is first and foremost a political philosophy--a description of how, in the opinion of libertarians, free people ought to treat one another, at least when they use the law, which they regard as potentially dangerous. If libertarians are correct, the law should intrude into people\\'s lives as little as possible, rarely telling them what to do or how to live.A political and economic philosophy as old as John Locke and John Stuart Mill, but as alive and timely as Rand Paul, the Tea Party, and the novels of Ayn Rand, libertarianism emphasizes individual rights and calls for a radical reduction in the power and size of government. \"Libertarianism For Beginners\" lays out the history and principles of this often-misunderstood philosophy in lucid, dispassionate terms that help illuminate today\\'s political dialogue.\" ...more'}\n", + "{'UPC': 'a22124811bfa8350', 'Title': \"It's Only the Himalayas\", 'Price': '£45.17', 'Rating': 2, 'Genre': 'Travel', 'Availability': 'In stock (19 available)', 'Description': 'â\\x80\\x9cWherever you go, whatever you do, just . . . donâ\\x80\\x99t do anything stupid.â\\x80\\x9d â\\x80\\x94My MotherDuring her yearlong adventure backpacking from South Africa to Singapore, S. Bedford definitely did a few things her mother might classify as \"stupid.\" She swam with great white sharks in South Africa, ran from lions in Zimbabwe, climbed a Himalayan mountain without training in Nepal, and wa â\\x80\\x9cWherever you go, whatever you do, just . . . donâ\\x80\\x99t do anything stupid.â\\x80\\x9d â\\x80\\x94My MotherDuring her yearlong adventure backpacking from South Africa to Singapore, S. Bedford definitely did a few things her mother might classify as \"stupid.\" She swam with great white sharks in South Africa, ran from lions in Zimbabwe, climbed a Himalayan mountain without training in Nepal, and watched as her friend was attacked by a monkey in Indonesia.But interspersed in those slightly more crazy moments, Sue Bedfored and her friend \"Sara the Stoic\" experienced the sights, sounds, life, and culture of fifteen countries. Joined along the way by a few friends and their aging fathers here and there, Sue and Sara experience the trip of a lifetime. They fall in love with the world, cultivate an appreciation for home, and discover who, or what, they want to become.It\\'s Only the Himalayas is the incredibly funny, sometimes outlandish, always entertaining confession of a young backpacker that will inspire you to take your own adventure. ...more'}\n" + ] + } + ], + "source": [ + "import requests\n", + "from bs4 import BeautifulSoup\n", + "\n", + "base_url = \"http://books.toscrape.com/\"\n", + "\n", + "# Get the listing page\n", + "response = requests.get(base_url)\n", + "soup = BeautifulSoup(response.text, 'html.parser')\n", + "\n", + "books = soup.find_all('article', class_='product_pod')\n", + "\n", + "book_details_list = []\n", + "\n", + "rating_map = {\n", + " 'One': 1,\n", + " 'Two': 2,\n", + " 'Three': 3,\n", + " 'Four': 4,\n", + " 'Five': 5\n", + "}\n", + "\n", + "for book in books:\n", + " # Title\n", + " title = book.h3.a['title']\n", + " \n", + " # Price\n", + " price = book.find('p', class_='price_color').text\n", + " \n", + " # Rating\n", + " rating_classes = book.find('p', class_='star-rating')['class']\n", + " rating_word = [r for r in rating_classes if r != 'star-rating'][0]\n", + " rating = rating_map.get(rating_word, 0)\n", + " \n", + " # Link to detail page\n", + " relative_url = book.h3.a['href']\n", + " book_url = base_url + relative_url.replace('../../../', 'catalogue/')\n", + " \n", + " # Request detail page\n", + " detail_response = requests.get(book_url)\n", + " detail_soup = BeautifulSoup(detail_response.text, 'html.parser')\n", + " \n", + " # UPC and Availability are in a table with class 'table table-striped'\n", + " table = detail_soup.find('table', class_='table table-striped')\n", + " upc = None\n", + " availability = None\n", + " for row in table.find_all('tr'):\n", + " header = row.th.text.strip()\n", + " value = row.td.text.strip()\n", + " if header == 'UPC':\n", + " upc = value\n", + " elif header == 'Availability':\n", + " availability = value\n", + " \n", + " # Genre is the second breadcrumb link (first is Home)\n", + " genre = detail_soup.find('ul', class_='breadcrumb').find_all('li')[2].a.text.strip()\n", + " \n", + " # Description is in

followed by a

\n", + " description_div = detail_soup.find('div', id='product_description')\n", + " if description_div:\n", + " description = description_div.find_next_sibling('p').text.strip()\n", + " else:\n", + " description = ''\n", + " \n", + " book_details_list.append({\n", + " 'UPC': upc,\n", + " 'Title': title,\n", + " 'Price': price,\n", + " 'Rating': rating,\n", + " 'Genre': genre,\n", + " 'Availability': availability,\n", + " 'Description': description\n", + " })\n", + "\n", + "for book in book_details_list:\n", + " print(book)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ { - "cell_type": "code", - "execution_count": null, - "id": "40359eee-9cd7-4884-bfa4-83344c222305", - "metadata": { - "id": "40359eee-9cd7-4884-bfa4-83344c222305" - }, - "outputs": [], - "source": [ - "# Your solution goes here" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + " Title UPC \\\n", + "0 A Light in the Attic a897fe39b1053632 \n", + "1 Tipping the Velvet 90fa61229261140a \n", + "2 Soumission 6957f44c3847a760 \n", + "3 Sharp Objects e00eb4fd7b871a48 \n", + "4 Sapiens: A Brief History of Humankind 4165285e1663650f \n", + "5 The Requiem Red f77dbf2323deb740 \n", + "6 The Dirty Little Secrets of Getting Your Dream... 2597b5a345f45e1b \n", + "7 The Coming Woman: A Novel Based on the Life of... e72a5dfc7e9267b2 \n", + "8 The Boys in the Boat: Nine Americans and Their... e10e1e165dc8be4a \n", + "9 The Black Maria 1dfe412b8ac00530 \n", + "10 Starving Hearts (Triangular Trade Trilogy, #1) 0312262ecafa5a40 \n", + "11 Shakespeare's Sonnets 30a7f60cd76ca58c \n", + "12 Set Me Free ce6396b0f23f6ecc \n", + "13 Scott Pilgrim's Precious Little Life (Scott Pi... 3b1c02bac2a429e6 \n", + "14 Rip it Up and Start Again a34ba96d4081e6a4 \n", + "15 Our Band Could Be Your Life: Scenes from the A... deda3e61b9514b83 \n", + "16 Olio feb7cc7701ecf901 \n", + "17 Mesaerion: The Best Science Fiction Stories 18... e30f54cea9b38190 \n", + "18 Libertarianism for Beginners a18a4f574854aced \n", + "19 It's Only the Himalayas a22124811bfa8350 \n", + "\n", + " Price Rating Genre Availability \\\n", + "0 £51.77 3 Poetry In stock (22 available) \n", + "1 £53.74 1 Historical Fiction In stock (20 available) \n", + "2 £50.10 1 Fiction In stock (20 available) \n", + "3 £47.82 4 Mystery In stock (20 available) \n", + "4 £54.23 5 History In stock (20 available) \n", + "5 £22.65 1 Young Adult In stock (19 available) \n", + "6 £33.34 4 Business In stock (19 available) \n", + "7 £17.93 3 Default In stock (19 available) \n", + "8 £22.60 4 Default In stock (19 available) \n", + "9 £52.15 1 Poetry In stock (19 available) \n", + "10 £13.99 2 Default In stock (19 available) \n", + "11 £20.66 4 Poetry In stock (19 available) \n", + "12 £17.46 5 Young Adult In stock (19 available) \n", + "13 £52.29 5 Sequential Art In stock (19 available) \n", + "14 £35.02 5 Music In stock (19 available) \n", + "15 £57.25 3 Music In stock (19 available) \n", + "16 £23.88 1 Poetry In stock (19 available) \n", + "17 £37.59 1 Science Fiction In stock (19 available) \n", + "18 £51.33 2 Politics In stock (19 available) \n", + "19 £45.17 2 Travel In stock (19 available) \n", + "\n", + " Description \n", + "0 It's hard to imagine a world without A Light i... \n", + "1 \"Erotic and absorbing...Written with starling ... \n", + "2 Dans une France assez proche de la nôtre, un ... \n", + "3 WICKED above her hipbone, GIRL across her hear... \n", + "4 From a renowned historian comes a groundbreaki... \n", + "5 Patient Twenty-nine.A monster roams the halls ... \n", + "6 Drawing on his extensive experience evaluating... \n", + "7 \"If you have a heart, if you have a soul, Kare... \n", + "8 For readers of Laura Hillenbrand's Seabiscuit ... \n", + "9 Praise for Aracelis Girmay:\"[Girmay's] every l... \n", + "10 Since her assault, Miss Annette Chetwynd has b... \n", + "11 This book is an important and complete collect... \n", + "12 Aaron Ledbetter’s future had been planned ou... \n", + "13 Scott Pilgrim's life is totally sweet. He's 23... \n", + "14 Punk's raw power rejuvenated rock, but by the ... \n", + "15 This is the never-before-told story of the mus... \n", + "16 Part fact, part fiction, Tyehimba Jess's much ... \n", + "17 Andrew Barger, award-winning author and engine... \n", + "18 Libertarianism isn't about winning elections; ... \n", + "19 “Wherever you go, whatever you do, just . . ... \n" + ] } - ], - "metadata": { - "colab": { - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.13" + ], + "source": [ + "import requests\n", + "from bs4 import BeautifulSoup\n", + "import pandas as pd\n", + "\n", + "base_url = \"http://books.toscrape.com/\"\n", + "\n", + "response = requests.get(base_url)\n", + "soup = BeautifulSoup(response.text, 'html.parser')\n", + "\n", + "books = soup.find_all('article', class_='product_pod')\n", + "\n", + "rating_map = {'One':1, 'Two':2, 'Three':3, 'Four':4, 'Five':5}\n", + "\n", + "book_details_list = []\n", + "\n", + "for book in books:\n", + " # Extract Title and Price from listing page\n", + " title = book.h3.a['title']\n", + " price = book.find('p', class_='price_color').text\n", + " \n", + " # Extract Rating from class names\n", + " rating_classes = book.find('p', class_='star-rating')['class']\n", + " rating_word = [c for c in rating_classes if c != 'star-rating'][0]\n", + " rating = rating_map.get(rating_word, 0)\n", + " \n", + " # Build full URL for the book detail page\n", + " relative_url = book.h3.a['href']\n", + " book_url = base_url + relative_url.replace('../../../', 'catalogue/')\n", + " \n", + " # Request and parse detail page\n", + " detail_response = requests.get(book_url)\n", + " detail_soup = BeautifulSoup(detail_response.text, 'html.parser')\n", + " \n", + " # Extract UPC and Availability from product information table\n", + " table = detail_soup.find('table', class_='table table-striped')\n", + " upc = None\n", + " availability = None\n", + " for row in table.find_all('tr'):\n", + " header = row.th.text.strip()\n", + " value = row.td.text.strip()\n", + " if header == 'UPC':\n", + " upc = value\n", + " elif header == 'Availability':\n", + " availability = value\n", + " \n", + " # Extract Genre from breadcrumb navigation (3rd li)\n", + " genre = detail_soup.find('ul', class_='breadcrumb').find_all('li')[2].a.text.strip()\n", + " \n", + " # Extract Description (if present)\n", + " description_div = detail_soup.find('div', id='product_description')\n", + " if description_div:\n", + " description = description_div.find_next_sibling('p').text.strip()\n", + " else:\n", + " description = ''\n", + " \n", + " # Append all data as a dictionary\n", + " book_details_list.append({\n", + " 'Title': title,\n", + " 'UPC': upc,\n", + " 'Price': price,\n", + " 'Rating': rating,\n", + " 'Genre': genre,\n", + " 'Availability': availability,\n", + " 'Description': description\n", + " })\n", + "\n", + "# Create DataFrame with Title as first column\n", + "df = pd.DataFrame(book_details_list, columns=['Title', 'UPC', 'Price', 'Rating', 'Genre', 'Availability', 'Description'])\n", + "\n", + "# Display the DataFrame\n", + "print(df)\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CSV file 'books_data.csv' created successfully.\n" + ] } + ], + "source": [ + "# After creating the DataFrame 'df', save it to a CSV file\n", + "df.to_csv('books_data.csv', index=False)\n", + "\n", + "print(\"CSV file 'books_data.csv' created successfully.\")\n" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" }, - "nbformat": 4, - "nbformat_minor": 5 + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 }