On the practical side, dot products are great, but break in mixed precision and integer representations, where accurately normalizing to unit length isn't feasible.

In other cases people prefer L2 distances for embeddings, where the magnitude can have a serious impact on the distance between a pair of points.