In my free time, I help run a small Mastodon server for roughly six hundred queer leatherfolk. When a new member signs up, we require them to write a short application—just a sentence or two. There’s a small text box in the signup form which says:

Please tell us a bit about yourself and your connection to queer leather/kink/BDSM. What kind of play or gear gets you going?

This serves a few purposes. First, it maintains community focus. Before this question, we were flooded with signups from straight, vanilla people who wandered in to the bar (so to speak), and that made things a little awkward. Second, the application establishes a baseline for people willing and able to read text. This helps in getting people to follow server policy and talk to moderators when needed. Finally, it is remarkably effective at keeping out spammers. In almost six years of operation, we’ve had only a handful of spam accounts.

I was talking about this with Erin Kissane last year, as she and Darius Kazemi conducted research for their report on Fediverse governance. We shared a fear that Large Language Models (LLMs) would lower the cost of sophisticated, automated spam and harassment campaigns against small servers like ours in ways we simply couldn’t defend against.

  • memfree@beehaw.orgEnglish
    8·
    3 days ago

    These attacks do not have to be reliable to be successful. They only need to work often enough to be cost-effective, and the cost of LLM text generation is cheap and falling. Their sophistication will rise. Link-spam will be augmented by personal posts, images, video, and more subtle, influencer-style recommendations—“Oh my god, you guys, this new electro plug is incredible.” Networks of bots will positively interact with one another, throwing up chaff for moderators. I would not at all be surprised for LLM spambots to contest moderation decisions via email.

    I don’t know how to run a community forum in this future. I do not have the time or emotional energy to screen out regular attacks by Large Language Models, with the knowledge that making the wrong decision costs a real human being their connection to a niche community.

    Ouch. I’d never want to tell someone ‘Denied. I think you’re a bot.’ – but I really hate the number of bots already out there. I was fine with the occasional bots that would provide a wiki-link and even the ones who would reply to movie quotes with their own quotes. Those were obvious and you could easily opt to ignore/hide their accounts. As the article states, the particular bot here was also easy to spot once they got in the door, but the initial contact could easily have been human and we can expect bots to continuously seem human as AI improves.

    Bots are already driving policy decisions in government by promoting/demoting particular posts and writing their own comments that can redirect conversations. They make it look like there is broad consensus for the views they’re paid to promote, and at least some people will take that as a sign that the view is a valid option (ad populum).

    Sometimes it feels like the internet is a crowd of bots all shouting at one another and stifling the humans trying to get a word in. The tricky part is that I WANT actual unpaid humans to tell me what they actually: like/hate/do/avoid. I WANT to hear actual stories from real humans. I don’t want to find out the ‘Am I the A-hole?’ story getting everyone so worked up was an ‘AI-hole’ experiment in manipulating emotions.

    I wish I could offer some means to successfully determine human vs. generated content, but the only solutions I’ve come up with require revealing real-world identities to sites, and that feels as awful as having bots. Otherwise, I imagine that identifying bots will be an ever escalating war akin to Search Engine Optimization wars.