Web crawling (wine site crawling)

Project details

1. Crawling the required information from the existing database:
- Sales date
- Current stock quantity (if available, otherwise just availability status)
- Purchase link (URL of the product detail page with a purchase button)
- Wine's Korean name
- Wine's English name
- Wine's vintage (production year, if not available, label as ‘unknown’)
- Wine's volume (750ml, 375ml, 1500ml, etc.; default is 750ml)
- Wine's price (original price / discount rate and discount details / final price)
- Wine shop name
- Wine details (wine thumbnail image URL / type of wine: red, white, rosé, sparkling, etc. / grape variety)

2. The information crawled will not be directly registered into the database but will be processed through an admin page or the ChatGPT API for data refinement before registration. The necessary information and data to be crawled through the admin page are as follows:

Wine Name:
2. 1-1. If only the Korean name of the wine exists: -> Use the ChatGPT API or other methods to fill in the English wine name, then proceed to step
2. 1-2. If only the English name of the wine exists: -> Use the ChatGPT API or other methods to fill in the Korean wine name, then proceed to step
2. 2. If both the Korean and English wine names exist or after filling both using the ChatGPT API: -> Crawl the wine’s English name from wine-searcher.com or vivino.com, then show the list of wines. If the following information cannot be crawled from wine-searcher.com or vivino.com, then use the ChatGPT API.

3. Information that should be included in the wine list
- Wine name
- Image URL
- Classification (Red / White / Rosé / Sparkling, etc.)
- Grape variety
- Region
- If the wine price is not in "KRW" (e.g., "EUR", "JPY", "HKD"), the final sale price displayed on the site should be entered into the DB's original price field.