Job Description

At Profitmind, we're building massive-scale ecommerce datasets to use in AI training/inference, and we need a junior engineer to help develop the scraping infrastructure for product data. You'll be reverse-engineering undocumented APIs, handling anti-bot systems, and dealing with edge cases like pagination limits, rate limiting, and sites that change their protection schemes without warning. The technical side involves analyzing a site's network requests, deobfuscating and reading obfuscated javascript, and implementing simple HTTP request scraping to full browser automation. You'll also work on the infrastructure layer: state management for resumable scrapes, deduplicating products, data integrity, and monitoring systems to detect when sites change. You should have solid Python skills and experience scraping ecommerce websites and APIs.