I am definitely slower - 4.5mph or so - on my 42mm gravel bike on road than I am on my 32mm road bike.
But my road bike has slick tyres and the gravel bike’s tyres are more knobbly, and I run the gravel bike’s tyres at much lower pressure. (And there is the difference in geometry of each bike, too.)
I’m not even sure you can buy 42mm slick tyres, so a like-for-like comparison is probably very hard.