The purpose of the project is to learn how to gather data in a database for your research.

computer science

Description

Data (Web) Scraping Project


The purpose of the project is to learn how to gather data in a database for your research. 


Steps: 


1. Choose any website that you want to scrape for your research or fun. (e.g., Games, Billboard Charts, Movies and so on).


2. Design your database schema for the website. Well-designed database schema gets more points. 


3. Do scraping the website. You can use any language or parsing libraries. Python and the BeautifulSoup library are recommended. Use a sleep method to avoid interrupting the web service. 


4. Store the web data in your relational DBMS (e.g., MySQL) that can be any local or remote machine. The number of rows must be greater than 10,000. 


Submission: 


- Submit your source code files 


- A word file that describes what you have done for the project. Explain your database schema and queries. Include screenshots of the data in the DBMS. 


Related Questions in computer science category