What bizarre idiocy has lead to Anubus gatekeeping access to the Wiki?

I ran into the following internet-destroying stupidity today checking a wiki page:
1762346299485.png

Checking with Javascript disabled, I discovered it is part of a trend of false-virtue driven enshitificaton of the internet, this by anti-AI zealots destroying the internet to protect "their" content from "misuse" by "AI"

Obviously, no rational person could care less if an AI engine "trains" on the data they've gifted to someone else's hardware, such as freebsd.org's servers or whatnot, there's zero harm, no loss, only net gain if it proves useful on the AI platofrm and zero impact if not. Stopping AI company scrapers is also neutral, don't care, makes zero difference, tempest in a teapot idiocy until it some overzealous moron blocks actual human access. Then it moves beyond performative virtue signaling to performative self-harm.

Look at this idiotic, utterly inane sputtering stupidity attempting to justify censoring access:
Why am I seeing this?
You are seeing this because the administrator of this website has set up Anubis to protect the server against the scourge of AI companies aggressively scraping websites. This can and does cause downtime for the websites, which makes their resources inaccessible for everyone.

Anubis is a compromise. Anubis uses a Proof-of-Work scheme in the vein of Hashcash, a proposed proof-of-work scheme for reducing email spam. The idea is that at individual scales the additional load is ignorable, but at mass scraper levels it adds up and makes scraping much more expensive.

Ultimately, this is a hack whose real purpose is to give a "good enough" placeholder solution so that more time can be spent on fingerprinting and identifying headless browsers (EG: via how they do font rendering) so that the challenge proof of work page doesn't need to be presented to users that are much more likely to be legitimate.

Please note that Anubis requires the use of modern JavaScript features that plugins like JShelter will disable. Please disable JShelter or other such plugins for this domain.

This website is running Anubis version .


Sadly, you must enable JavaScript to get past this challenge. This is required because AI companies have changed the social contract around how website hosting works. A no-JS solution is a work-in-progress.

This sort of challenge, like the dumbassss at cloudflare and the idiotic web-admins who enable their more aggressive filtering options taking sledgehammers to their own feet in self-righteous outrage that a "bot" might 'scrape" their precious datas, OMG! This breaks the website. It drives legitimate users away. But you know what just works? Claude and ChatGPT et al.

The irony is that while these forums, the humans who populate them, the historical data that they have created have always been an excellent resource and are for now, and likely going forward, a theoretically superior resource that any LLM trained on similar data, by attempting gatekeep data resources away from LLMs or other uses, the thus utterly broken or at least enshitificated resources become less convenient and less accessible to actual humans, who turn to the much lower friction LLMs for more fluid access to critical information and by so doing move the very "training data" that the AI haters are desperate to cling to away from the once thriving communities to the very platforms they had hoped to throw sand into.

Just stop. It is dumb. You're not "under attack by AI bots." AI companies have NOT changed the "social contract" around how website hosting works. At all. That's the dumbest thing I've read all day, and it is late in my day here. If an IP block starts dominating a traffic pull to sufficiently compromise access for other users, rate limit or block it. Otherwise, what possible legitiamte reason is there to try to gatekeep the data away from anyone or anything, AI or human? That's utterly, unbelievably, absolutely idiotic. The social contract I made bothering to write this, bothering to contribute to the site, is to return some help in exchange for the help provided me by others, human or algorithmic, and it was not, is not, part of that contract to allow someone else decide who or what is sufficiently virutous to deserve access to it.

THAT is changing the social contract.
 
I honestly don't know enough about it to make a judgment, but accessing the wiki, from New York in the US, I only get a flash of checking you're not a bot before it goes right to the page. I don't know where either of you are located, but could it possibly be a location based thing. (I didn't even know what Anubis was, had to startpage (less intrusive alternative to google) it).
 
well i blocked 5E6 ips over several days for creating useless load.
they all had forged user agents like ancient versions of current browsers, ie for xp and what not.
0.1-0.2% of the whole ipv4 space in a week.
there are proxy services that offer you scan bots with unlimited ips
 
could it possibly be a location based thing
Most likely, you have not disabled JavaScript and cookies, either for all websites or for individual websites. Those two elements are required to pass the Anubis check. If enabling Js and cookies for all sites is not an option, permissions can be granted site by site, for example.

Anubis-JS.png Anubis-cookie.png
 
As a maintainer of hobby forums in other areas, I can assure you that this has *NOTHING* to do with zealotry of any kind. Bandwidth and compute do cost real money. AI bots don't respect robots.txt. The stuff I run for myself and a few small communities on a shoestring budget gets completely DDoS'ed out of existence by AI bots hammering small sites with tens of thousands of hits per day. It's gone far enough that I'm currently blackholing whole hyperscalers at a time at the firewall level to make my forums usable to real humans again. I'm not affiliated with this particular forum, so I can't speak for the maintainers here, but small sites are quite definitely under attack.
 
  • Thanks
Reactions: mro
Or an AI bot?
I would like to think AI Bots would have moved out of Oakland and upgraded their address to San Jose....

Seriously though. This user joined in 2009. Before there were AI scrapers. Is he they a time traveling AI bot?

$edited for proper pronoun usage
 
I would like to think AI Bots would have moved out of Oakland and upgraded their address to San Jose....

Seriously though. This user joined in 2009. Before there were AI scrapers. Is he they a time traveling AI bot?

$edited for proper pronoun usage
Well time is an artifical concept so maybe? :) and I apologize if I forgot the appropriate "sarcasm" tags
 
Most likely, you have not disabled JavaScript and cookies, either for all websites or for individual websites. Those two elements are required to pass the Anubis check. If enabling Js and cookies for all sites is not an option, permissions can be granted site by site, for example.

View attachment 24119 View attachment 24120
No, in this case the site is running an older version of the check with a bug that is known and fixed in newer versions. Disabling Javascript results in the block page considering that you are probably human but not playing along with the challenge and reveals a down toggle for the meta message, which includes the text I take some issue with because it aligns specifically with an opinion I don't share (no we don't all have to agree, that's what makes the internet interesting):

This is required because AI companies have changed the social contract around how website hosting works.

This is an opinion statement and while I'm sure there are well thought-out arguments for or against it by people who've spent far more time engaged with the issue than I, I disagree. To me, LLMs are just a tool, one I find useful if not nearly as useful as is often claimed and one annoyingly often used inappropriately, especially by people who don't understand the limitations of semantic prediction models.

But I have battle scars from people taking umbrage at thumbnailing images for search results - remember when that was a drama? Even a google adsense ban-able violation (even though image search did the same)? I don't think I'm the only one that still, 17 years later, finds image search useful.

And, sure, managing/limiting bots is a real thing, they vastly outnumber real traffic. That's a very practical concern not remotely limited to AI tin-skins. I do not, would not unless forced, run a challenge myself; the point of a web server is to serve content, not to NOT serve content. I do have a Rube Goldberg fail2ban->pfBlockerNG script that blocks 2x as many packets with only 874 IPs (as of this moment) as the next most triggered IP block list of 7,909 IPs listed as naughty. Happy to share that. It mostly lets polite bots through while mostly banning the ones that act inappropriately.

That this is framed as an "AI" issue results in a non-linear emotional response relative to more familiar practical issues of exactly the same consequence. Since this is already "off topic," flame on. And no, I'm clearly not a bot, LLM's can only wish to be so erudite as I and little chance of that if they are deprived of training by my sparkling wit thanks to the robophobists at Anubus. Oakland was never the same after Ask Jeeves and Dog Pile.

Reaching back to a DefCon 6 talk: in the US we gift inventors a temporary monopoly on their inventions solely for the purpose of promoting the progress of science and the useful arts. Any limitation on anyone's natural right to freely spread ideas from one to another over the world wide globe by claiming exclusive appropriation in a manner that retards progress unconstitutionally darkens others without increasing their own light. At least in the US.

It would be a hard row to hoe to claim that LLMs, despite all their risks and limitations, which surely are still legion, are not the sort of progress intended to be promoted. The practical requirements of managing multitudinous swarms of automated scrapers and indexers is neither new nor AI specific. Arguing that AI clankers are specifically somehow more abhorrent for what their owners do with the data is a political position not shared by everyone, after all there is difference between BSD and AGPLv3.

LLMs are useful tools when used properly. To the extent that they are less likely to hallucinate catastrophic code given access to this site's extraordinary repository of expertise is a less awful outcome than many truly nefarious applications of the interwebs.

But that aside, even if one is so robophobist as to be overcome by shaking hatred for all digital systems that might pass a Turing test, the problem with the proliferation of human-papers-please demands is that it breaks the usability of the internet. Few of us would know, fewer even still care were the enforcers of meat-based cognitive access transparent in their actions, but these algo-thugs are far too confident of their condemnations and I, a real human, no really, am tired of being mis-speciated by the proliferating gate-keeper bots.

I am frequently annoyed by coworkers who confuse LLMs for search engines and try to pass off copy-pasta, stupid emoji-bullets intact, for thoughtful analysis yet when the links I chase after a well-worded search query throw one bouncer after another demanding my notabot papers please, I turn ever more often to the polite engagement and occasionally bizarre hallucination of my favorite plastic pal, surely not the outcome intended by those hoping to grit the gears of AI developers with their challenge scripts.
 
LLMs are useful tools when used properly. To the extent that they are less likely to hallucinate catastrophic code given access to this site's extraordinary repository of expertise is a less awful outcome than many truly nefarious applications of the interwebs.
What the hell are they useful for besides violating copyright, collecting personal data, pushing teenagers to commit suicide, using massive amounts of electricity, and just generally being useless?
 
I call my neutered/spayed pets "it" all the time. I'm hoping that does not make be bigoted.
Ugh. What you have in your pants has nothing to do with who you are, especially if you are something other than a man or a woman. I can tell this is somesort of transphobic joke. Imagine if you were in pain because you were a different person than everyone saw you as. I will not go into who I am, but I deal with things myself that I would not wish on anyone else. Be glad you don't have to deal with bigotry and other things and don't attack other people for who they are. There are not only 2 genders (it's a very varied spectrum), and birth sex ≠ gender. End of story.
 
I call my neutered/spayed pets "it" all the time. I'm hoping that does not make be bigoted.
I never really thought about it, but I figure the cat was born with a gender.

That reminds me of an old-school game switching Male/Female to Body type A/B (I didn't agree to be neutered to a generic body type to someone else's benefit :p)
 
Back
Top