I'm building a price-tracking tool for a small US retail business, pulling publicly listed product prices from several competing stores. Basic Python requests work on some sites, but others rely heavily on JavaScript or keep rejecting automated requests. I've tried running headless browsers, although managing proxies, retries, and browser crashes is taking more time than I'd planned. I'm considering a managed scraping API, but I'm not sure how much of that work it actually handles or how expensive it gets with regular daily checks. What should I look for before choosing one?

Comments (2)
No login
gif
color_lens
Login or register to post your comment
Arthyron
Managed scraping APIs can save you from running browsers and proxy logic yourself, but the pricing can get complicated when JavaScript rendering and retries enter the picture. ZenRows is one of the services you could compare with your current setup, since it handles scraping requests through an API. There's a review explaining what ZenRows is and how it works here: https://gologin.com/blog/zenrows-review/ . Estimate how many requests you'd make in a normal month and check how the service counts ...  more
Heartman
A 200 response doesn't tell you much if the page still comes back without the product data you need. Build a small test set with ordinary pages, JavaScript-heavy pages, and a few of the stores that currently fail. For every request, check whether you actually received the product name, current price, availability, and any other fields your tracker depends on. That gives you a much better measure of reliability than counting successful HTTP requests alone.
arrow_back
Cookies on WhereWeChat.
This site uses cookies to store your information on your computer.
Read more east Accept done