That is possible indeed! For more context, you can look up “static linking vs dynamic linking”
Tldr: Static linking: all dependencies get baked into the final binary Dynamic linking: the binary searches for libraries in your system’s PATH and loads them dynamically at runtime
The hard part is getting used to it. How do I share my public keys? How do I use GPG (the program)? How do I access them easily? What do I do when I want to encrypt my mails on desktop (maybe Windows+Linux), laptop, and phone? It’s just relatively much work to gather the knowledge.
+ the fact there’re not many people using it
Tl;dr: Stick in a USB cable and the other side gets your console.