Apparently one of the lemmy.ml admins was overzealous in banning all User-Agent strings that contained the string “bot”. Bans were entered for all of the individual strings containing that word which were observed in their webserver logs, which impacted kbin’s reported agent of “kbinBot”.

The issue has been fixed, and I observed that one of my kbin posts to a lemmy.ml community was successfully pushed to the original instance.

  • DarkThoughts@kbin.social
    link
    fedilink
    arrow-up
    3
    ·
    1 year ago

    Reeks of competency if you do a blanket ban on that term, as if malicious bots would announce themselves as such.

    • blightbow@kbin.socialOP
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      1 year ago

      There are plenty of high volume, non-malicious bots that do. robots.txt is a thing for a reason, and we can see here that lemmy.ml has implemented it. Not all bots that ignore robots.txt are malicious though, just poorly designed. You can basically lump them into three categories:

      • Well-behaved bots that announce that they’re bots in the User-Agent header and obey robots.txt (note that they may still slam the server even if they obey it)
      • Mediocre bots that announce that they’re bots in the User-Agent header but ignore robots.txt (or vice versa)
      • Bad bots (malicious or otherwise) that announce their User-Agent as other things, often pretending to be other software.

      Their logs told them they had a lot of traffic from stuff identifying itself as bots. Throwing that traffic out wouldn’t break lemmy but would help them deal with the capacity problems that all of the mainstream lemmy/kbin instances had to deal with shortly after the Reddit exodus began. They fucked up and tagged kbin in the process, which definitely would have been one of the highest volume ActivityPub consumers matching their criteria.