Labeling Theory[1] suggests that referring to people as criminals increases the chance of them committing crimes. When people's existence is referred to as criminal (or when they're referred to as criminals for having committed other crimes, many of which are not dangerous to society or others, such as using drugs) their social and economic prospects are harmed and according to this theory, they then become more likely to engage in other activities which (in addition to violating laws) are actually more detrimental to society.

So I think there's a strong argument that we should be much more conservative with the application of labels like "criminal" to people.

[1]: https://en.wikipedia.org/wiki/Labeling_theory#The_%22crimina...