11·
3 days ago@[email protected] that’s great, some days ago i saw some people on r/selfhosting discussing how to stop AI crawlers that don’t respect robots.txt (so all of them) and there were a lot of people basically reinventing the tarpit idea, having a dedicated tool for that is great, combined with a simple logging of all ip ranges falling for it to get blacklisted we might get a fighting chance, there were even people serving zip bombs to ai bots, but i don’t believe they would bother to open it
@[email protected] interesting, the hackernews thread linked in that article has someone talking about similar tools
https://news.ycombinator.com/item?id=42726426