Best proprietary search tactics and web scraping tools

searcher profile

September 21, 2023

by a searcher from INSEAD in Toronto, ON, Canada

Hi! I am looking for tips on how to be the most efficient in searching for deals and learning about your experience with web scraping tools.

2
6
202
Replies
6
commentor profile
Reply by a searcher
from Warwick Business School in Royal Tunbridge Wells, Tunbridge Wells, UK
redacted‌^ Depending on your level of programming expertise, I would generally recommend investing in a provider who does it right, rather than trying to build new tools (especially if this is not your main focus). The self-service approach typically involves some Python coding or using out-of-the-box tools, such as the ChatGPT web scraper. However, this may not be very scalable. I'm sure there are lots of other "easy" tools available, but the reality is that it can be pretty tedious work. Hope this helps.
commentor profile
Reply by a searcher
from Seattle University in Meridian, ID, USA
https://www.octoparse.com/ It's relatively user friendly. It took me a couple tries to get it to work with some government sites, but eventually had it working. There are also some very nice out-of-the-box site scraping for Google, Yelp, Yellowpages, etc. that turned out to work well.
commentor profile
+4 more replies.
Join the discussion