by David | Jan 18, 2023 | Automation
I was trying to pull out a big description block for an item in a recent scraping project. Of course, this contains all kinds of weird and wonderful HTML formatting as it is probably built in a WYSIWYG editor. I found that Scrapy doesn’t have a good way to...
by David | Apr 25, 2021 | Automation
Scrapy is great, debugging Scrapy less so Are you adding print statements and then rerunning your scraper time and time again to get that one selector right? Do you have chrome open in the background and using jQuery to test those selectors live on the website you are...
