A quick weekend project to get experience with more Python modules and HTML. When I started this project, I had issues with SSL certificate and the Java script that the Indeed site was running on. I was concerned I'd need to request access to the Indeed API, which they only give to legit businesses. Thankfully there's the opensource cloudscraper module for scraping websites built with Cloudflare. From there, I imported the tkinter module for a basic GUI to enter "Job" and "Positions" to generate the HTML address from user input. I was thinking of also working in an Excel export or automatic emailing of listings, but I'll didn't want to spam myself.
https://github.com/sirhclee/Web-Scraper-Indeed
04SEP22: Upon later testing, I determined an issue where the scraper was unable to bypass the Captcha; I was concerned the scraper module was a limited-duration trial. I found out the created scraper browser just needed to be updated to Chrome. Nice!
Comments
Post a Comment