Yes, there is the concept of a "frakenmerge" and folks have also bolted on vision and audio models to LLMs.