Our prod exposed Redis until I noticed. Apparently we had to reset caches after releases, and our devOPs forgot to remove access to it after creating scripts to handle that.

It happens, but it really shouldn’t. At least ours was at a different subdomain (something like app-region-redis.domain.com or whatever).

So I guess musl/busybox/Linux?

And yet, if someone asks, I will link it. I’m not proud of it, but I am helpful to a fault.

Ok, but what if there isn’t any GNU? Musl/Linux?

I’ve considered it, yeah. There aren’t many posts, and I’ve been thinking of posting a bunch there to get it more popular.

Or maybe I’ll make a new, related community with a little broader appeal, idk. I find avoiding lemmy.ml is usually more enjoyable.

You don’t have to convince me that Rust rocks. I just need to convince my team that it’s worth the investment in terms of time to onboard everyone, time to port out application, and risk of introducing bugs.

We have a complex mix of CRUD, math-heavy algorithms, and data transformation logic. Fortunately, each of those are largely broken up into microservices, so they can be replaced as needed. If we decide to port, we can at least do it a little at a time.

The real question is, does the team want to maintain a Python or Rust app, and since almost nobody on the team has professional experience with low-level languages and our load is pretty small (a few thousand users; b2b), Python is preferred.

Here’s what I do:

• occasionally post stuff and comment

I really haven’t had any issues on the two I mod, but they’re pretty small communities. And honestly, at Lemmy’s scale, if you’re feeling the need to use tools, you’re probably moderating too strictly, or you’re moderating a massive community.

Ooh, I’ll have to get cracking, I still have half of the year to catch up! What’s the grand prize?

I’m certainly biased, but we use macOS at work and nearly everyone is familiar with the terminal. We’re developers though, but even our less technical people (product owners and whatnot) know what it is and what it can do.

But yeah, I wouldn’t be opposed to turning on a dev option to enable it though. I use it every day, but most don’t need to (even our devs could configure commands in their IDE).

Yup, I guess not. But if I was on the product team, the customers and director ate the bosses. And on it goes up to the CEO, where the board and shareholders are the boss.

If I can justify the change, we’ll do it. That’s close enough for me. And I did do a POC w/ Rust and could’ve switched one service over, but I campaigned against myself since we got good enough perf w/ Python (numpy + numba) and I was the only one who wanted it. That has changed, so I might try again with another service (prob our gateway, we have 3 and they all kinda suck).

I’ll have to check out Deno again. I remember looking at it (or something like it) a couple years ago when first announced on Reddit.

Well, I’m kind of the boss, but I inherited the Python codebase. The original reasoning was it’s easier to hire/on-board people, which I think is largely true.

If it was up to me, I’d rewrite a bunch of our code to Rust. I use it for personal projects already, so I know the ecosystem. But that’s a tough sale to the product team, so it’s probably not happening anytime soon. I’d also have to retrain everyone, which doesn’t sound fun…

However, any change I make needs to work smoothly for our devs, and we have a few teams across 3 regions. So it needs clear advantages and whatnot to go through the pain of addressing everyone’s concerns.

That’s pretty impressive! We have a bunch of a bunch of compiled stuff (numpy, tensorflow, etc), so I’m guessing we wouldn’t see as dramatic of an improvement.

Then again, <1 min is “good enough” for me, certainly good enough to not warrant a rewrite. But I’ll have to try uv out, maybe we’ll switch to it. We switched from requirements.txt -> pyproject.toml using poetry, so maybe it’s worth trying out the improved pyproject.toml support. Our microservices each take ~30s to install (I think w/o cache?), which isn’t terrible and it’s a relatively insignificant part of our build pipelines, but rebuilding everything from scratch when we upgrade Python is a pain.

Both of those are largely bound by i/o, but with some processing in between, so the best way to speed things up is probably am async i/o loop that feeds a worker pool. In Python, you’d use processes, which can be expensive and a little complicated, but workable.

And as you pointed out, scons and pip exist, and they’re fast enough. I actually use poetry, and it’s completely fine.

You could go all out and build something like cargo, but it’s the architecture decisions that matter most in something i/o bound like that.

Yeah, looks like it’s just bypassing a security feature that doesn’t exist anyway on most phones. So yeah, low priority sounds exactly right. I have that feature enabled on my phone, so I’m interested in seeing it get fixed though.

Money?

Honestly, I prefer C to C++ anyway. If they’re going to switch to something, I’d prefer Rust.

But whatever, not my project, not my concern.

Yup, we do it for Python and Javascript at work, and I do it on my Rust projects (and my older C projects). I don’t see why C++ should be any more difficult.

Exactly. We have hundreds of thousands of lines of code that work reasonably well. I think we made the important decisions correctly, so performance issues in one area rarely impact others.

We rewrote ~1k lines of poorly running Fortran code into well-written Python code, and that worked because we got the important parts right (reduced big-O CPU from O(n³) to O(n² log n) and memory from O(n⁴) to O(n³)). Runtime went from minutes to seconds in medium size data sets, and made large data sets possible to run (those would OOM due to O(n⁴) storage in RAM). If you get the important parts right, Python is probably good enough, and you can get linear optimizations from there by moving parts to a compiled language (or use a JIT like numba). Python wasn’t why we could make it fast, it’s just what we prototyped with so we could focus on the architecture, and we stopped optimizing when it was fast enough.

Yup, which is why you should try to limit the copying by designing your parallel processing algorithm around it. If you can’t, you would handle threading with a native library or something and scale vertical instead of horizontal. Or pick a different language if it’s a huge part of your app.

But in a lot of cases, it’s reasonable to stick with Python and scale horizontally. That has value if you’re otherwise a Python shop.

Then you need to break up your problem into processes. Python doesn’t really do multi-threading (hopefully that changes with the GIL going away), but most things can scale reasonably well in a process pool if you manage the worker queue properly (e.g. RabbitMQ works well).

It’s not as good as proper threadimg, but it’s a lot simpler and easier to scale horizontally. You can later rewrite certain parts if hosting costs become a larger issue than dev costs.