Dear Webscraping Community,
Scrapoxy 4, the super proxies aggregator, is currently in heavy preparation and will soon be released. This latest version represents a full rewrite, the results of almost two years of dedicated investment, and is tailored for the webscraping industry.
I would like to thanks all the contributors who have helped me during these years to make this project a reality. Today, I am now fully dedicated to this project, for the next years ahead thanks to Wiremind.
The tech stack is built on the latest NodeJS, Typescript, utilizing the NestJS and Angular frameworks.
Here are the included features:
- Support for Cloud providers such as AWS, Azure, GCP, OVH, Digital Ocean, and many more on the horizon
- Compatibility with Proxies providers like Zyte, Rayobyte, IPRoyal, Proxyrack, with additional providers being added regularly
- Integration with Hardware providers like Proxidize
- Seamless compatibility with free HTTP/HTTPS proxies
- Intelligent traffic routing across proxies, following a robust anti-ban strategy
- Effortless management of sticky sessions (including web browser support)
- Autoscaling proxies to optimize costs with both upscale and downscale capabilities
- Real-time request and response rewriting on-the-fly
- Modern authentication methods, including Google and Github
- End-to-end encryption of traffic between the master and proxies
- Distributed architecture on Docker, Kubernetes, RabbitMQ and MongoDB
- Stability with more than 500 end-to-end tests
- And there's a lot more exciting features in the pipeline!
As always, Scrapoxy remains open source, ensuring that the web scraping community can continue to benefit from its capabilities.
Stay tuned Github for the official release announcement, and I can't wait to see how Scrapoxy 4 elevates your webscraping projects to new heights!
Fabien.
PS: Here are some screenshots of the 4.0.0 version running in a production environment
Modern Authentication with Google (or Github)
Support multiple projects
Support multiple connectors and regions in a single project
Support also proxies providers (residential network here with proxyrack)
Many options are availables
Advanced metrics
Proxies coverage over the world