I appreciate your enthusiasm, but the history of ML shows that your approach is less likely to work. Maybe you’ll be the one to prove everyone else wrong. Architectural breakthroughs are few and far between, and it’s incredibly difficult to reason about. I came up with the Lion optimizer while Google was using random tree search across 300 TPUs to discover the same thing, and it’s just five lines or so.
“The bitter lesson” would like to have a word. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
I appreciate your enthusiasm, but the history of ML shows that your approach is less likely to work. Maybe you’ll be the one to prove everyone else wrong. Architectural breakthroughs are few and far between, and it’s incredibly difficult to reason about. I came up with the Lion optimizer while Google was using random tree search across 300 TPUs to discover the same thing, and it’s just five lines or so.