Let's compare to see if it's really the same market:
HGX B200: 36 petaflops at FP16. 14.4 terabytes/second bandwidth.
RX4060 (similar to Intel): 15 teraflops at FP16. 272 gigabytes/second bandwidth
Hmmm.... Note the prefixes (peta versus tera)
A lot of that is apples-to-oranges, but that's kind of the point. It's a different market.
A low-performance high-RAM product would not cut into the server market since performance matters. What it would do is open up a world of diverse research, development, and consumer applications.
Critically, even if it were, Intel doesn't play in this market. If what you're saying were to happen, it should be a no-brainer for Intel to launch a low-cost alternative. That said, it wouldn't happen. What would happen is a lot of business, researchers, and individuals would be able to use ≈200GB models on their own hardware for low-scale use.
> By the way, the CUDA moat is overrated since people already implement support for alternatives.
No. It's not. It's perhaps overrated if you're building a custom solution and making the next OpenAI or Anthropic. It's very much not overrated if you're doing general-purpose work and want things to just work.
What treprinum suggested was AMD selling their current 192GB enterprise card (MI300X) for $2000, not a low end card. Everything you said makes sense but it is beside the point that was raised above. You want the other discussion about attaching 128GB to a basic GPU. I agree that would be disruptive, but that is a different discussion entirely. In fact, I beat you to saying that would be disruptive by about 16 hours:
HGX B200: 36 petaflops at FP16. 14.4 terabytes/second bandwidth.
RX4060 (similar to Intel): 15 teraflops at FP16. 272 gigabytes/second bandwidth
Hmmm.... Note the prefixes (peta versus tera)
A lot of that is apples-to-oranges, but that's kind of the point. It's a different market.
A low-performance high-RAM product would not cut into the server market since performance matters. What it would do is open up a world of diverse research, development, and consumer applications.
Critically, even if it were, Intel doesn't play in this market. If what you're saying were to happen, it should be a no-brainer for Intel to launch a low-cost alternative. That said, it wouldn't happen. What would happen is a lot of business, researchers, and individuals would be able to use ≈200GB models on their own hardware for low-scale use.
> By the way, the CUDA moat is overrated since people already implement support for alternatives.
No. It's not. It's perhaps overrated if you're building a custom solution and making the next OpenAI or Anthropic. It's very much not overrated if you're doing general-purpose work and want things to just work.
https://www.nvidia.com/en-us/data-center/hgx/ https://www.techpowerup.com/gpu-specs/geforce-rtx-4060.c4107