I accidentally did this to a vacation home we stayed in once, like weeks after we checked out. It took me a solid minute to piece together 'where the music was going'. I felt so bad; hopefully the house was vacant.
About a month or two ago Alexa started telling us it didn't understand "stop" while it played the audio we didn't ask for (Alexa's voice recognition is really poor). I had to unplug it.
I probably could have used the app to stop it but I didn't think about that at the time.
I unplugged it and am not too sure about plugging it back in.