If this Gemma tokenizer I found online is accurate then my Pixel 10 Pro XL is getting ~22 tok/s on Gemma 4 E2B using the NPU, vs. 40 tok/s is what people are saying the MLX version gets on iPhone.
Actually I found official performance numbers from Google saying iPhone gets 56 tok/s and Qualcomm gets 52. They don't even bother listing Tensor in their table. Maybe because it would be too embarrassing. Ouch! https://ai.google.dev/edge/litert-lm/overview
Nice! Tried on iPhone 16 pro with 30 TPS from Gemma-4-E2B-it model.
Although the phone got considerably hot while inferencing. It’s quite an impressive performance and cannot wait to try it myself in one of my personal apps.
It's at least somewhat limited in non-English content. It knows how to make lentil soup, so I was happy that I never need to look up recipe sites with awful UX and ads, but then it couldn't find a recipe for "Kalter Hund"/"Kalte Schnauze". So sad ;)
Still, absolutely fabulous. What a time to be alive!
It's strange that my iPhone 14 is at regular temperature when using the E2B model. But also it's a lot slower (not sure how to measure the exact tokens per second, ~12 if I had to guess)
Ditto. Almost all of my internal tools are built in the same fashion and deployed on cloudflare itself.
It is so much liberating to have no overload of CMS and the bloated code they come with.
Wish the best for Sid! My father was unlucky with his TP53 mutation which resulted in AEL and he passed away suddenly. 100% fit to vanishing away straight in 8 months. It was pretty rough for me especially to see how these doctors (most of them but bot all) are so non-experimental in their attitude.
Sid is right, Staying alive is our own job and definitely what he is doing will give him and his loved ones enough hope to get through this and sometimes he eventually he will get through.
Hopefully the deflation scenario due to technology will help with this shift gracefully. Else, it will be extremely difficult.
But if we had deflation in the economy, how will the investment scenario will Peter out is anyone’s guess.
I wonder The people who invented this financial growth will, why they didn’t thought about this in the long term? I guess, I have asked a question which has quite a generic answer already…
Fyi, my account was registered under my father’s name (I had his permission ofcourse)