Just here for good conversation with good people.

  • 3 Posts
  • 16 Comments
Joined 2 years ago
cake
Cake day: July 20th, 2023

help-circle










  • OP, you’re looking for something called “Bot as a Service”. There are more and more companies that cater to those needing a bot infrastructure. Bright data, ScrapingBee, ZenRows, and Apify are some of the more common services I typically work against that offer what you’re looking for.

    *Edit: If you’re just looking for performance testing, you can use services like Loadster.





  • 👋 Hi, I really hope iocaine works for you and I think it still might be wise to temper expectations. Some background, I work in bot detection and mitigation.

    I quickly tried reading through their code and documentation but I don’t see the main detection mechanism that determines human vs bot other than what you mentioned as an example. If it’s user agent based, it is trivially easy to spoof as you already know. I am finding in my work that these companies do not keep the user agent they report in their documentation when challenged.

    My second concern was the page the reverse proxy served when spoofing my user agent. The DOM was nowhere close to that of Lemmy and I think it’s important to point out that a simple check for specific elements on the page will keep the bot from poisoning itself.

    I admit I could be too close to this problem to see other solutions, and I really hope it works. It sucks that this is a problem. I wish there were more open source options too.

    If for some reason this solution doesn’t work, and if anyone is interested in help, I am more than happy to freely offer my knowledge.