I want to add users via manifests, so these users could use logins/passwords/pubkeys, and that's out of the box, without installing dex, keycloak or delegating to other systems.
Think about Linux installation. I don't need to add IDP to create unix users for various people.
Right now it's super complicated in Kubernetes and even requires third-party extensions for kubectl.
You can create service accounts and tokens... Although long lived tokens are discouraged, that's as simple as it gets.
Sorry I think you're in the minority here. Most people don't want what you are talking about, they want to use SSO. Even with plain Linux machines, they want SSO.
Service accounts can't belong to groups, so they are super not convenient for human operators. You can't just create group "developers", assign roles for this group and add service accounts to this group. You must assign role for every user in every namespace, etc.
Having SSO is fine as long as it's built-in. Installing and configuring separate SSO software is not fine.
Think about Linux installation. I don't need to add IDP to create unix users for various people.
Right now it's super complicated in Kubernetes and even requires third-party extensions for kubectl.