web-scraper

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.

🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

Python scraper based on AI

Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust

Turn any website into a structured API. Extract, automate, search and monitor the web.

A collection of awesome web crawler,spider in different languages

scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place

Self-hosted webscraper.

A list of practical knowledge-building projects.

Scrape Google AI Mode responses without blocks on a large scale.

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.

Generate and download e-books from online sources.

Web Scraper in Go, similar to BeautifulSoup

A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.

A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.

Professional-grade, local-first browser extension that transforms any webpage into high-fidelity, LLM-friendly Markdown entirely within the browser's runtime. Zero-cloud, privacy-centric, and designed specifically for AI workflows.

CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)