+
Skip to content
@scrapy

Scrapy project

An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.

Pinned Loading

  1. scrapy scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    Python 57.5k 11k

  2. scrapyd scrapyd Public

    A service daemon to run Scrapy spiders

    Python 3k 572

  3. parsel parsel Public

    Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

    Python 1.2k 152

  4. w3lib w3lib Public

    Python library of web-related functions

    Python 409 107

  5. protego protego Public

    A pure-Python robots.txt parser with support for modern conventions.

    DIGITAL Command Language 70 28

  6. itemadapter itemadapter Public

    Common interface for data container classes

    Python 68 12

Repositories

Showing 10 of 29 repositories
  • scrapyd Public

    A service daemon to run Scrapy spiders

    scrapy/scrapyd’s past year of commit activity
    Python 3,043 BSD-3-Clause 572 9 0 Updated Jul 7, 2025
  • flake8-scrapy Public

    A Flake8 plugin to catch common issues in Scrapy projects.

    scrapy/flake8-scrapy’s past year of commit activity
    Python 19 MIT 4 0 0 Updated Jul 7, 2025
  • scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    scrapy/scrapy’s past year of commit activity
    Python 57,490 BSD-3-Clause 10,951 456 (19 issues need help) 197 Updated Jul 6, 2025
  • itemloaders Public

    Library to populate items using XPath and CSS with a convenient API

    scrapy/itemloaders’s past year of commit activity
    Python 48 BSD-3-Clause 16 17 5 Updated Jun 28, 2025
  • itemadapter Public

    Common interface for data container classes

    scrapy/itemadapter’s past year of commit activity
    Python 68 BSD-3-Clause 12 5 2 Updated Jun 29, 2025
  • scrapy-bench Public

    A CLI for benchmarking Scrapy.

    scrapy/scrapy-bench’s past year of commit activity
    Python 31 MIT 15 6 1 Updated Jun 28, 2025
  • protego Public

    A pure-Python robots.txt parser with support for modern conventions.

    scrapy/protego’s past year of commit activity
    DIGITAL Command Language 70 BSD-3-Clause 28 7 (3 issues need help) 0 Updated Jun 24, 2025
  • sphinx-scrapy Public

    Sphinx extension for documentation in the Scrapy ecosystem

    scrapy/sphinx-scrapy’s past year of commit activity
    Python 1 BSD-3-Clause 1 0 0 Updated Jun 16, 2025
  • scrapyd-client Public

    Command line client for Scrapyd server

    scrapy/scrapyd-client’s past year of commit activity
    Python 776 BSD-3-Clause 146 5 0 Updated May 23, 2025
  • parsel Public

    Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

    scrapy/parsel’s past year of commit activity
    Python 1,241 BSD-3-Clause 152 32 (1 issue needs help) 12 Updated May 12, 2025
View all repositories
点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载