Skip to content

Python web scraper

ChatGPT 6 views 0 copies Oct 4, 2026
Build a polite, robust web scraper that extracts structured data and saves it to CSV or JSON.

Prompt

Act as a Python developer.

Website: [URL or type of site]
Data to extract: [fields]
Pages: [single page / pagination / list of URLs]
Output: [CSV / JSON]

Write a script using requests + BeautifulSoup (or Playwright if the page needs JavaScript) that:
- Respects robots.txt and adds a delay between requests
- Sets a clear User-Agent
- Handles missing elements and errors
- Saves results incrementally
- Is configurable at the top of the file

Remind me to check the site's terms of service before scraping.

Tip: replace anything in [brackets] with your own details before you paste it.

#web scraping#python#beautifulsoup#data collection
Keep exploring

Similar prompts

Act as an expert email marketer and online course designer. Create a 7-day email course about [topic] for [target audience]. By the end of the course, readers should be able to [desired result]. Start with a short outline listing the 7 lesson titles, then write all 7 emails. For each day, include: 1. A subject line (u…

ChatGPT
Marketing & SEO

Turn any topic into a 7-day email course

Creates a complete 7-day email course from a single topic: subject lines, lessons, action steps and a closing call to action. Great for growing a newsletter, onboarding new customers or launching a lead magnet. Works just as well in Claude and Gemini.

2
Midjourney Featured
Photography

Drone aerial photography

Breathtaking top-down and high-angle aerial shots of coastlines, cities and landscapes, perfect for travel content and wallpapers.

1

A nostalgic 1980s arcade at night, kids playing glowing cabinets, neon lights and carpet patterns, a teen winning and cheering, handheld camera, warm VHS-style colors, fun and lively.

Kling
Video & Animation

Retro arcade scene

A nostalgic 1980s arcade full of neon.

1

Create a 12-month personal growth reading plan focused on [areas, e.g. habits, relationships, money, mindset]. Suggest one well-known book per month, why it's useful, and a practical exercise to apply each one.

ChatGPT
Personal Growth

Personal growth reading plan

A curated reading path for self-improvement.

1