So, Khan Academy is experimenting with providing every student with (what they call) a "personal tutor".
Examples given are: in coding and math, identifying potential mistakes in partial solutions, and suggesting how the student might get unstuck; in literature, an AI impersonates characters in a novel so the student can ask them questions (e.g. "Mr. Gatsby, why do you keep staring at that green light?"); in writing, the AI writes with the student, rather than for students.
The flip side is teachers saving time on lesson planning, grading, etc, all the work that's adjacent to actually teaching.
A lot of this is a feature preview for Khan Academy, and a big part of it is just the ReACT pattern[1] (have people settled on this name?).
To me, even all of the above is just an efficiency increase, which means workloads will increase to fill the available time, and we might see student to teacher ratios in the order of tens of thousands to one. How much of a future teacher's career will be about maintaining their AI systems? How much of learning will be like that too?
I don’t think it’s ever safe to procnosticate on the basis that something is “Just an efficiency increase”. Plenty of tech that is just an efficiency increase has ended up having profound impact on society.
>just an efficiency increase has ended up having profound impact on society.
truer words haven't been spoken. one big one off the top of my mind was the us governments decision to allow civillians to access gps at the same resolution as the arms. with that, all sorts of applications that require realtime location tracking. uber, lyft, rappi, gojeck off the top of my head.
More fundamentally, the telegraph was "just" a more efficient way to send messages over sending a message via pony express, and the Internet evolved from there.
And efficiency is everything. People defending bad education system using "gov has no money" "country has no teacher" excuses while the higher education is pumping out unemployed graduates every year. They can act more actively to make teacher as an job option to people. They just don't like problem being solved. Now GPT4 personal tutor exist, make them no excuse to not provide better education to people.
Do teachers as we know them even exist in a world where that idea of tutoring goes to scale ?
I don't think they do. It sounds more like a world, where in person testing and certification is extremely important when a human skill needs to be evaluated.
Do teachers as we know them even exist in a world where that idea of tutoring goes to scale ?
Do students as we know them even exist in such a world ? I'm thinking if AI can effectively teach (AI asks questions to the student, gives feedback and leads the lesson, reacts to situational events, maintains discipline etc.) then AI can also do. Aside from the romantic take of learning for the pleasure of learning why would we assume that such AI teachers aren't also going to be AI employees. Why would humans even have the drive to learn stuff on a mass scale if the main or one of the main motivators for education is the market (or is it ?).
> Why would humans even have the drive to learn stuff
Because someone still has to make decisions. Much as the educational-industrial complex would have you believe otherwise, living the good life does not equate with getting the right answers on standardized tests.
One of the unfortunate aspects of the education system we've built over the last several hundred years is that it conceals this from most people. This is because the purpose of the system is to feed a societal system where a very small number of people make decisions and the rest are expected simply to obey orders. There is nothing necessarily wrong with that, but it's really important not to lose sight of the fact that replacing the current education system with AI is not the same as replacing educated people with AI, or even the same as replacing all of education with AI.
The problem is that decisions can only be made with respect to some quality metric, and deciding what that should be cannot be delegated. Only you can know what you want.
The other problem is we often say "it's common sense!" as a result of someone (or something) else following our metrics in a technically correct way that's totally not what we meant.
The tutors scale up,
but so can the testing. Instead of standardized testing, every student can get interviewed as if the teacher had only them as a student and be given an in-depth assessment of their skills and knowledge, rather than being simplified to a letter grade.
Pedagogy is an area of study outside of my expertise, but I do know there are institutions that didn't give grades.
One of the biggest factors impacting education outcomes is students per teacher. Students tutored one-on-one learn better than students in small classes, and students in small classes learn better than students in large classes. Because if there are fewer students per teacher, the teacher can devote more time and attention to each student. AI promises to put a decent one-on-one tutor in the hands of every student for a fraction of the cost of human tuition, potentially bringing about a huge leap in educational attainment. No longer will private tutors be restricted to the rich only.
>No longer will private tutors be restricted to the rich only.
This is probably the number one thing that excites me about AI. Throughout all of history, when you look back at famous scientists, there's a clearly obvious pattern. Almost without exception they were children of enormous privilege who were given the opportunities to study and be tutored individually throughout their life. Unlocking that for every child on earth is going to accelerate human progress faster than anything else.
You are either being dishonest or you're in for a major disappointment. There's a lot more involved in that "opportunity" than a mechanism for low cost feedback. One's material circumstances predominates their ability to take advantage of such mechanisms, hence why strictly increasing school budgets or teacher quality has never succeeded in significantly improving overall outcomes for children living in poverty. I suppose maybe you have to experience it to grasp the extent to which the havoc and insecurity that living in poverty in brutally individualized Western countries ensures, but it's been shown over and over again that these shortcuts simply don't work. You have to address the poverty itself.
I think ultimately it just comes down to the ability to ask a question. Googling did a lot for this, but LLM agents are on a whole other level. When someone can just endlessly ask questions and have them answered directly and comprehensibly, they can get past that initial stage of confusion in learning much quicker. It used to be you needed a human tutor to do this. But now an AI system can track and manage your knowledge graph and slowly level you up through direct natural language interaction.
That is so incredibly naive. When you and your family is distracted with trauma and turmoil at every turn, there is no precedent for curiosity about anything except how to survive the current state of affairs. This is extremely basic stuff. It's literally just rational prioritization of concerns. If you can't grasp that, you have other problems that aren't being discussed in this discussion.
Children living in poverty will surely use LLMs, but they will continue to lack the capacity for curiosity and intentional learning. They will use LLMs to assist with the same things they invest their energy in without LLMs, which is their immediate life challenges and psychological escapism. And insufferable people like you will blame them when your ignorant plan inevitably fails.
That's one factor. But the way I've learned best, is through stories. My teachers told all sorts of stories, and it made me very engaged in my classes. How can the same be said with AI? How can it make an engaging learning experience?
Even if you want it to, how could it make a very engaging environment when deep down inside you know it's fake?
You guys are making it seem like this is a common occurence and I've never heard of anything like this. I've never heard of teachers frequently sharing personalized stories that are also true to help people learn a subject.
In fact, the opposite, most subjects like math or physics have made-up problems to isolate and make simple the concept they're trying to teach. Adding a constraint for realism would just be needlessly complicated.
Also not scalable - if these stories are so effective, why not put them in a book to share with everyone?
The only stories any of my teachers ever shared with me were historical events that were grounded in reality.
Are there just a whole bunch of educational institutions that are teaching things around a campfire and a long-form stories like in ancient times?
Decent? These current LLMs have more knowledge than every teacher on the planet combined. I’m very hopeful to see what finetuned models come out on top of the base gpt4 or lama
That's a little like saying Wikipedia has more knowledge than every teacher on the planet combined. It's being able to bring the relevant knowledge to bear on the student's understandings that matters, and that depends not only on the prompts the student provides the teacher/tutor/LLM, but on the teacher/tutor/LLM's understanding of the domain, the student's point on the learning curve, the student's level of frustration and receptivity, how people learn, and probably a dozen things that aren't coded into how LLMs respond.
But is it just knowledge or is it the human connection that makes a difference?
Students have access to all the information they need on Calculus through Google and other resources, for instance, but actually sitting down with a living, breathing human being and having them help you through your troubles...to me, that cannot be replaced with an AI.
If access to knowledge was all we needed to acquire and maintain intelligence in a specific area, we would all be a lot smarter...
'Disproven' is too strong, it's trivially true at the extreme edge cases. But smaller class sizes fail to eliminate all the performance gaps between schools, so the idea of class size (and/or school funding) being the explanation for all school performance gaps has been disproven. Particularly, smaller class sizes can't eliminate a performance gap caused by different conditions at home, whether students bully each other for trying to learn, etc.
But there's really no question that a small class where teachers can know each individual student is better than a large class of hundreds of students or more, where the teachers can't realistically remember who is who.
Scott E Page - The Model thinker. He talks about school size rather than class size, my mistake. He says “Failure to take sample size into account and inferring causality from outliers can lead to incorrect policy actions. For this reason, Howard Wainer refers to the formula for the standard deviation of the mean the “most dangerous equation in the world.” For example, in the 1990s the Gates Foundation and other nonprofits advocated breaking up schools into smaller schools based on evidence that the best schools were small. To see the flawed reasoning, imagine that schools come in two sizes—small schools with 100 students and large schools with 1,600 students—and that student scores at both types of schools are drawn from the same distribution with a mean score of 100 and a standard deviation of 80. At small schools, the standard deviation of the mean equals 8 (the standard deviation of the student scores, 80, divided by 10, the square root of the number of students). At large schools, the standard deviation of the mean equals 2. If we assign the label “high-performing” to schools with means above 110 and the label “exceptional” to schools with means above 120, then only small schools will meet either threshold. For the small schools, an average score of 110 is 1.25 standard deviations above the mean; such events occur about 10% of the time. A mean score of 120 is 2.5 standard deviations above the mean; an event of that size should occur about once in 150 schools. When we do these same calculations for large schools, we find that the “high-performing” threshold lies five standard deviations above the mean and the “exceptional” threshold lies ten standard deviations above the mean. Such events would, in practice, never occur. Thus, the fact that the very best schools are small is not evidence that smaller schools perform better. The very best schools will be small even if size has no effect solely because of the square root rules.”
Jesus, no! Not all of my teachers were great, and most of the knowledge I got, I earned on my own. But they were part of a world that had nothing to do with the world of my parents--my teachers were learned, where my immediate family was not. My teachers were, if nothing else, an example to follow, a light, a guide.
>> No longer will private tutors be restricted to the rich only.
It's not as simple. I've met plenty of poor people who leave well-tutored rich folk to bite the dust, and the other way around too. That's neither here nor there. A kid with all the tutors but with parents or an environment that doesn't favor learning, gets nowhere. That I have seen too, more times than I can count. And, let's face it, we have had the Internet for a few decades now, bursting to the seams with free knowledge. What difference will it make an AI system of dubious trustworthiness?
Just ignore the AI hype herd. People who are pulling predictions out of their ass, have never had to deal day to day with the bugs these systems produce. You will never see a single person who works on production AI(not toy ai or research ai) making predictions cuz they are too busy staring at exploding bug lists.
Maybe AI is going to save education, but the next few years are going to be very rough. I built a plugin for Chrome that helps detect cheating [1] so I've been talking to a bunch of teachers lately and pretty much everyone acknowledges that ChatGPT is a game changer for how schools will need to run. Plagiarism, tutors and parent help have been around for a while, but they still required most students to put in some effort. Now, any student can have chatGPT generate a book report for Lord of the Flies in less than 5 minutes. The common theme I hear from teachers is that the education model will need to flip, where students do more writing in class and more learning outside of class. It sounds simple, but it's actually a massive change to implement and things are gonna be very tricky over the next few years until the dust settles more.
[1] https://www.revisionhistory.com, which I originally thought would be most useful for cheating, but interestingly, many teachers are more interested in using it to help students work through revisions. I've gotten feedback from teachers that they plan to have students use ChatGPT to start their assignments and then task them with updating, modifying and annotating them, so I'm trying to figure out which features to build to support that.
I hope it will force the current system which favours a specific subset of learners to change. I hope it will evolve towards a collaborative system where learners are not tested as specimens in a vacuum. The current system is all about making it easier for the system to do its thing instead of the user(learner) getting something valuable out of it.
> I hope it will force the current system which favours a specific subset of learners to change. I hope it will evolve towards a collaborative system where learners are not tested as specimens in a vacuum.
That will also favor a specific subset of learners.
Perhaps, but learners that are good in the current system will do well regardless imo. They are good at self directed, isolated learning which I think is the minority.
Those same 'independent' learners end up carrying the groups they're placed in. In the best case they're essentially drafted as unpaid tutors for the rest of the group, but often they end doing most of the work themselves while sharing the credit with the others. In either case they end up overworked, but particularly in the later case it becomes a system for gaming statistics to make the whole class look like it's performing better than it really is. Teachers like setting up this kind of system when their compensation/advancement is tied to student performance, a manifestation of Goodhart's Law. The sort of equality that you can achieve by giving student the average grade of a group is just a trick with numbers. Why not grade the whole class as one single group? Average every test score together and give every student that average score. Now you have excellent equality as the whole class performs adequately... but only on paper.
Anyway, there's no "one size fits all" in education, any scheme will favor some students more than others.
Current education system is broken. You can't learn anything. Learning need feedback loop, but this do not exist.
In school, you hand in your homework, get the feedback only after Christmas. Because human cannot evaluate 100 students' homework efficiently.
Majority of academic textbook are trash. Some author even say "we have 300 new exercises in this edition" proudly, while the solution is often either nonexistent, or at the end of the book. Why at the end, not next to that problem? Or next page? Why wasting my time to flip pages back and forth?
If the content of exercise is important, why not include that into the text?
If instructor is needed, why not making every concepts into Q&A as they had already been explained in office hour or email somewhere else? Why zero effort being put into knowledge accumulation?
Human failed at education. ChatGPT obviously is a more knowledgeable entity, personally I prefer that more than human instructor.
I'm sorry, but I've taught a lot of students at the university level. Some of them have exactly this attitude. You're simply wrong. And on the long term path to failure.
The exercises are invaluable. You don't get the answers because then you look at them. The whole point of the exercises is for you to do them, and then for you to figure out if you are right or wrong!
The content of the exercises isn't in the book because you need to actually do things to learn. You cannot learn math, physics, and many other topics passively.
You're ignoring the main lessons and features of the system.
And no. There are countless amazing books. The fact that you can't see that is because you don't understand how to learn.
Yeah reading and doing stuff is how you learn any topic, getting the right solution is not really the point (and arguably should not be in the book in the first place).
The main issue with education is grades and how obsessed people are with those as a sorting mechanism for students.
And why are they not doing the exercises? Are they lazy? The moment a significant percentage of learners are alienated by the system then there is something wrong with the system. Currently a specific subset of learners fit into this system. The rest either needs extraordinary grit or fail since it is not setup for them to succeed. I for one welcome AI to burn some of our old decrepit institutions to the ground.
The problem with linking success to attitude is the counterfactual; Would students with the right "attitude" been successful regardless?
That's why more formal academic studies are needed for topics such as these rather than anecdotes, because we'll find a whole range of contradictory opinions.
If you choose to fight with me, I fight with you. You dig your only grave. You just jump out and admitted that you are the problematic human teacher that causing social problems.
I do my homework when I feel like I need them. Accessibility to the solutions is a problem. Learning though examples is another prominent way that you don't see. People can choose to investigate into the problem right now or absorb knowledge then think about stuffs. I have 80 years in my life expectancy to do exercise and indeed solving problems everyday is that. What is the point of "figure it out yourself" about some useless math at the moment when people have their whole life to think about it? People know nothing because you never tell anything. You failed your job as a teacher. You are doing excellent job of not documenting stuff and being anti knowledge accumulation.
I'm sorry to read you've had such an awful educational experience. Where was this? It does not resonate with my experience at all. I had my fair share of awful teachers, but the handful of excellent teachers really made a difference. The course material was generally pretty decent as well.
My formal education ended some 20 years ago, and I'm sure things have changed, and from what I hear from younger friends and relatives, it doesn't seem to be for the better ...
I think I read more textbook then average person to a extend that can call myself textbook reviewer. People usually stuck with only one book that is being assigned to them at class, But as a self-learner, I read all, literally all, linear algebra, discrete math, stuff like that, all just terribly written.
I think it's broken, but for other reasons. In the UK at least, secondary school exams are overwhelmingly about memorisation and prediction of what questions there will be rather than applying logical or investigative thinking. There's very little practical work, you sit in a chair and listen to the teacher or things on the whiteboard most of the time. Also, there's a majority feminine influence in education. All of these factors lead to me believe that education simply does not cater to males, and soaring grades of girls confirms this. In the UK, white males have been the poorest performing demographic on exams for the past decade. But nobody wants to talk about it interestingly.
I was fortunate to get an apprenticeship at the age of 16, which enabled me to get into the real world of work and build experience years ahead of everyone else. It's a completed overlooked path - most students still think they need a degree to get a good job.
> secondary school exams are overwhelmingly about memorisation
That's depressing. It was starting to move away from that when I did my physics and chemistry A-levels in 1974, had been for some years in fact with Nuffield courses emphasizing exploration and comprehension rather than simply getting the 'right' answer. The Physics A-level had a substantial section that tested the student's ability to understand the results of experiments.
Paper is more focused on "feedback benefit learning" instead of how long should people wait for feedback.
Seems that you are quoting another paper instead. But I don't think it is definitive, as you see, n=27. Also the experiment setting is not about how long after initial test, it is about how short before the next test.
Problem with education is that it has a dual goal of both educating everyone and "filtering the cream of the crop". On the long term such goals begin to clash, but it's much easier to do the latter than the former.
It's easy to blame the individual, but I wonder what are the collective social costs of such stringent attitudes on the long term.
The AI I want in education generates a test and then based on my responses to questions figures out the exact content I need from a huge library. Imagine asking a student 15 math questions with adaptive difficulty in a dynamic test to figure out the next content they need to learn from complete videos of JK-Uni mathematics? Instead of teaching to an age, we give students exactly the content they need right now.
This is gross posturing and indicates future rent seeking behaviour. Imagine getting in front of people and telling them that some corporation trained the font of wisdom, and that you'd be happy to charge fuckin' admission! What absolute dogshit. All that future rent seeking for a glorified DAN prompt. This is the digital equivalent to making everyone do bullshit jobs because we never figured out how to deal with increased worker productivity from manufacturing.
I really see the potential here (if it is not already happening) for fine-tuned LLMs to start generating bespoke defence and economic policies. Think about the amount of data you could feed a system about orthodox management strategies, current supply chains, demographics etc.
Probably at least 50% of current global governments / bureaucracy could be wholesale replaced by a single decent LLM, and the economic and social outcomes would be far better.
I think teachers should probably use ChatGPT for a couple days before starting this conversation. It is the common ground. Not knowing what ChatGPT is able to do hinder valuable debate.
I don't see any reason to listen to Sal Khan. As far as I can tell, he became a major figure in education, with ample resources and real power, because he is good at explaining things. Why would I assume he knows anything about how education works in general because people liked his approach to one very small part of it?
At the gym, people assume anyone with big muscles must be an expert in medicine. Let's not make the same mistake here.
Many parts of Khan Academy are great precisely because Sal is good at explaning things. In an engaging fashion, in clear language, with frequent exercises. It's benefited many people over the years (including myself).
Building a succesful learning platform which has impacted many students, certainly qualifies someone to have an opinion on education in my book. Would you be happier if he were some elected Minister Of Education -- who, from what I've seen, are generally about as qualified as a box of donuts?
Literally no one at a gym assumes that muscular people are experts at medicine. But it is reasonable to assume that they know a thing or two about building muscle ...
Indeed, and I see plenty of reasons to listen to him. Khan Academy helped me immensely in college for free and Sal Khan has been at it for what, 17 years now?
I'm pretty sure he has a better than average grasp on what it means to learn and how to improve the process. And even more importantly, he's trying to! If not him, then who should we listen to?
I don't have a clue why GP reduces Sal to "good at explaining things" when Khan Academy is so much bigger than just him for a long time now. And I, for one, am very grateful that he exists. People in first world countries can't imagine the impact that free access to great education resources has in the rest of the world, like in my case.
> At the gym, people assume anyone with big muscles must be an expert in medicine. Let's not make the same mistake here.
The gym analogy falls short to me. The mistake here is assuming that someone that worked in education for almost two decades, and has achieved extraordinary results in the mean time, doesn't have any clue about education in general.
Examples given are: in coding and math, identifying potential mistakes in partial solutions, and suggesting how the student might get unstuck; in literature, an AI impersonates characters in a novel so the student can ask them questions (e.g. "Mr. Gatsby, why do you keep staring at that green light?"); in writing, the AI writes with the student, rather than for students.
The flip side is teachers saving time on lesson planning, grading, etc, all the work that's adjacent to actually teaching.
A lot of this is a feature preview for Khan Academy, and a big part of it is just the ReACT pattern[1] (have people settled on this name?).
To me, even all of the above is just an efficiency increase, which means workloads will increase to fill the available time, and we might see student to teacher ratios in the order of tens of thousands to one. How much of a future teacher's career will be about maintaining their AI systems? How much of learning will be like that too?
[1] https://til.simonwillison.net/llms/python-react-pattern