crawler:scrapyarchitecturecode
Differences
This shows you the differences between two versions of the page.
| crawler:scrapyarchitecturecode [2017/01/06 17:23] – [Integrate Scrapy with Other Systems] admin | crawler:scrapyarchitecturecode [2022/10/29 16:15] (current) – external edit 127.0.0.1 | ||
|---|---|---|---|
| Line 862: | Line 862: | ||
| Integrate via below systems: | Integrate via below systems: | ||
| * Database: MySQL, MongoDB | * Database: MySQL, MongoDB | ||
| - | * Cache: Redis Cache, Cm Cache | + | * Cache: Redis Cache, Cm Cache -> You can **start multiple spider instances that share a single redis queue**. Best suitable for **broad multi-domain crawls**. |
crawler/scrapyarchitecturecode.1483723396.txt.gz · Last modified: 2022/10/29 16:15 (external edit)