Interesting thing about Summit and Sierra: the CPUs and GPUs are connected via nvlink rather than PCIe, which reduces the well known cost to move data to GPU.