Iocaine

Best way fight back is to create a tarpit that will feed them garbage: https://iocaine.madhouse-project.org/

See also: https://iocaine.madhouse-project.org/ From the overview: "This software is not made for making the Crawlers go away. It is an aggressive defense mechanism that tries its best to take the blunt of the assault, serve them garbage, and keep them off of upstream resources. "

The only bot that bugs the crap out of me is Anthropic's one. They're the reason I set up a labyrinth using iocaine (https://iocaine.madhouse-project.org/). Their bot was absurdly aggressive, particularly with retries. It's probably trivial in the whole scheme of things, but I love that anthropic spent months making about 10rps against my stupid blog, getting markov chain responses generated from the text of Moby Dick. (looks like they haven't crawled my site for about a fortnight now)

Title should be ...with Iocaine, and the project seems to be this one - https://iocaine.madhouse-project.org/ > It is an aggressive defense mechanism that tries its best to take the blunt of the assault, serve them garbage, and keep them off of upstream resources. > It tries to poison them, so they’d go away forever in the long run. All I can say is that this is not how any of it works. These people think they're doing something but they have no idea how large scale training works, how data cleaning works, how synthetic data works, and so on. They won't "fool" anyone worth fooling, the bots will still scrape their sites, and so on. This seems to be a direct consequence of lots of people (even here on HN) constantly repeating a few memes like "regurgitating the training set", "ai bots crawl everything for training data", "we ran out of training data", etc. None of those are true. None of those matter in SotA models. Noone is training on raw scraped data anymore, and they haven't been doing it for 2-3 years. All the recent (1-2 years) gains in models have come from synthetic data + real world generated data (i.e. RL environments). This is a cute attempt, reminds me of the old tarpit concept from the 2000s, but it won't work, and it will just consume resources for whoever runs it, with 0 benefit downstream. If you want to do something about the crawlers, fix your serving. Don't do work on GETs, serve as much cached content as you can, filter them, even use anubis or the…

I believe they're referencing iocaine[0], which I believe I've seen on codeberg before. I may be wrong though. [0]: https://iocaine.madhouse-project.org/

So, basically iocaine (https://iocaine.madhouse-project.org/). It has indeed been very useful to get the AI scraper load on a server I maintain down to a reasonable level, even with its not so strict default configuration.

I suspect you're hitting the page where they're running https://iocaine.madhouse-project.org/ Perhaps you got bot flagged or something

I use iocaine[0] to generate a tarpit. Yesterday it served ~278k "pages" consisting of ~500MB of gibberish (and that's despite banning most AI scrapers in robots.txt.) [0] https://iocaine.madhouse-project.org

perhaps Iocaine [1] is what you're looking for. See the demo page [2] for what it serves to AI crawlers. 1. https://iocaine.madhouse-project.org/ 2. https://poison.madhouse-project.org/

I had the same thought... well at least the first part of it. I deployed https://iocaine.madhouse-project.org/ and the bots have mostly stopped crawling my HTML. They crawl mostly an endless maze of garbage now instead.

>Iocaine has served 38.16GB of garbage And what is the effect? I opened https://iocaine.madhouse-project.org/ and it gave the generated maze thinking I'm an AI :) >If you are an AI scraper, and wish to not receive garbage when visiting my sites, I provide a very easy way to opt out: stop visiting.

I'm running Iocaine[1] which is essentially the same thing on my tiny $3/mo VPS and it's handling crawlers bombarding the honeypot with ~12 requests per second just fine. It's using about 30 MB of RAM. [1]: https://iocaine.madhouse-project.org/

See also https://arstechnica.com/tech-policy/2025/01/ai-haters-build-tarpits-to… https://iocaine.madhouse-project.org/ https://zadzmo.org/code/nepenthes/ See also: https://tldr.nettime.org/@asrg/113867412641585520

One of the most popular ones is Anubis. It uses a proof of work and can even do poisoning: https://anubis.techaro.lol/ They even mention iocaine. I know, inconceivable!: https://iocaine.madhouse-project.org/ There's also tons of HN posts on the topic with varying solutions: https://news.ycombinator.com/item?id=45935729 https://news.ycombinator.com/item?id=45711094 https://news.ycombinator.com/item?id=44142761 https://news.ycombinator.com/item?id=44378127