- 0 Posts
- 9 Comments
except it doesn’t work well for the rest of lemmy/the fediverse.
many other instances seem to be getting hit by this, but they don’t have as many activities generated locally for this to become much of a problem. additionally, this is mostly affecting instances with high latency to the instance that is being flooded by kbin, as lemmy currently has an issue where activity throughput between instances with high latency can’t keep up with too many activities being sent. the impact of this is can be a bit less on smaller instances with smaller communities often not having as many subscribers on remote instances, although we’ve seen problems reported by some other admins as well. this includes e.g. kbin.earth, which i suspect to have been hit by responses from a lemmy instance, while the lemmy instance was actually only answering the requests sent from that kbin instance.
during the last peak, when we decided to pull the plug for now, kbin.social was sending us more than 20 activities per second for 7 hours straight. lemmy.world can easily handle this amount of activities, but the problem arises when this impacts our federation towards other (lemmy) instances, as e.g. votes will get relayed by the community (magazine) instance, which means, depending on the type of activity being sent, we might have to be sending out the same 20 requests per second to up to 4,000+ other fediverse instances that are subscribed/following the community this is happening in. trying to send 20 requests per second, which lemmy does not do in parallel, requires us to use at most 50ms per activity total sending time to avoid creating lag. when the instance is in australia, with 200ms+ latency, this is simply not possible.
looking at the activity generation rates of some popular lemmy instances, anything that is significantly above lemmy.world is likely not just sending legitimate activities.
ps: if you’re wondering how i’m seeing this post, you can search for a post url and comment urls on lemmy to make lemmy fetch them, even if they haven’t been directly submitted through normal federation processes. this requires a logged in user on lemmy’s end.
for a magazine to show up on lemmy, a logged-in user needs to visit it first. afterwards, to ensure that new content is published to lemmy instances, someone from that instance needs to subscribe to the magazine. this needs to happen on every instance as far as i know. this is one of the reasons services like https://lemmy-federate.com/ or https://browse.feddit.de/ exist.
it is indeed mostly like related activities we’re seeing
it’s not just this community getting hit.
it’s mostly [email protected], [email protected], [email protected] and [email protected], though some of the latest spam also started arriving in [email protected]
5·5 months agoMost of these are unfortunately limited to local instance moderation, which means all instances that don’t run these bots don’t benefit from them.
The posts have already been modified in ways that they aren’t as easy to reliably filter anymore, though still possible with fairly low false positive rate.
To add to this, depending on how content is removed, removal may or may not federate properly. on Lemmy.World, we’ve been removing content in a way that reliably federates, so while a lot of this spam does arrive in [email protected] and [email protected], the removals on Lemmy.World should federate to all other instances (0.18.5+).
The other portion of the spam is mostly on Lemmy.ml, in [email protected] and [email protected], and not all of their removals have been done in a way that federates.
they appear to be advertising their “criminal organization”, listing some of the illegal activities they do and where to find them.
11·5 months agoposting and commenting.
voting should work: https://lemmy.world/post/11967676
this is unfortunately correct for the time being.
while we still have aggressive rate limits in place to limit federation impact from kbin bugs, which started with the measures that @[email protected] mentioned, this wouldn’t impact activities coming from lemmy.world towards kbin.social.
while kbin.social used to break down every now and then based on what i saw people comment, service was typically restored within a short period of time. more recently however, any time i’ve looked at kbin.social in the past couple weeks, it’s only been showing an error page. i suspect it may have been unavailable the entire time, not just at the times i looked at it. looking at our federation stats, the last successfully sent activity from lemmy.world to kbin.social was dated 2024-06-18 00:12:25 UTC, although the actual send date may have been later. successful is also not necessarily guaranteed, as some error codes might be misinterpreted as success due to how servers can be set up and how response status codes are interpreted on the sending side.
if activities sent from lemmy.world don’t reach kbin.social then the posts and comments won’t be relayed to other instances. this is generally an issue in activitypub when instances are down, as such “orphaned” (at the time) communities effectively become local-only communities, isolated islands on all instances that already know about them.
at this point, the last time we’ve received an activity submission (federation traffic) from kbin.social as on 18th of June, so it seems like it was working for some time on that day and has been broken since.
at the start of this month, @[email protected] (kbin.social owner, main kbin dev) said that he was going to hand over management of kbin.social to someone else, as he’s currently unable to take care of it. presumably this hasn’t happened yet.