> If the SAT is not predictive, as some claim, we've been selecting our elites and professionals the wrong way for three generations.
Yes, and?
The easiest way to understand the shortcomings of these tests (at least their older versions) is to realize that students who use paid SAT/ACT-prep materials and services get higher scores than students who don't. Yes, there are confounding factors, but this fact alone is fairly damning evidence that these tests can be "defeated" using techniques beyond simply learning the things taught in high school. A better-designed test would not yield higher scores to test-takers with more specific knowledge of how the test itself is constructed. (In software terms, think "property based testing" as opposed to "unit test cases written in a predictable manner", with the assumption that your implementation under test has adversarial motives to obtain passing builds.)
This complaint is addressed in footnote 10 of the article.
It would be great to design a test that's less gameable than the current version, and we should certainly try to do that, but the current version is already less gameable than basically anything else colleges consider for admission.
Whereas "holistic" admissions processes are probably the _most_ gameable option out there. If you were designing a process specifically to be able to game it (for example to keep out the Jews) then turning down the transparency would be the way to go.
> The easiest way to understand the shortcomings of these tests (at least their older versions) is to realize that students who use paid SAT/ACT-prep materials and services get higher scores than students who don't.
Admission criteria do not exist in a vacuum. Since throwing up our hands and saying "we're just not going to admit anyone since this is an imperfect criteria" is obviously not the answer, It only makes sense to compare to the alternatives. I can't think of a cheaper (and more accessible) measuring method than standardized tests. Compared to the cost of participating in leadership roles, sports, learning an instrument, etc, a $20 SAT prep book is an incredibly accessible option.
Or put another way, out of all admission criteria, is standardized testing really the least useful / fair? If not, why should it get cut first?
> The easiest way to understand the shortcomings of these tests (at least their older versions) is to realize that students who use paid SAT/ACT-prep materials and services get higher scores than students who don't.
Yes, and affluent people with family friends on the boards of non-profits can show volunteer work that’s way more interesting in an application than being actively involved in your church or mosque. Rich people can hire essay consultants to understand what the (mostly white American) admissions officers are looking for: https://cepa.stanford.edu/content/essay-content-strongly-rel...
Of all these measures, the SATs show the smallest benefits from paying to prep.
The issue with this comment and opponents of standardized testing more generally is that they compare standardized tests against some purely theoretical ideal metric that perfectly captures "merit," however it's defined.
The reality is that standardized testing doesn't need to be absolutely perfectly fair to be something worth keeping; it just needs to be fairer than selecting students who had the opportunity to be captain of the lacrosse team. That is a far lower bar that it easily exceeds.
This is the point where standardized testing opponents will typically rhetorically concede that they also are opposed to that unfairness, but they never manage to get particularly incensed about athletic or legacy admissions.
I'm not opposed to standardized testing nor am I among those saying to just eliminate the test; I'm just defending opinions critical of these tests from knee-jerk responses that I think are overly protective of them.
Yes, and?
The easiest way to understand the shortcomings of these tests (at least their older versions) is to realize that students who use paid SAT/ACT-prep materials and services get higher scores than students who don't. Yes, there are confounding factors, but this fact alone is fairly damning evidence that these tests can be "defeated" using techniques beyond simply learning the things taught in high school. A better-designed test would not yield higher scores to test-takers with more specific knowledge of how the test itself is constructed. (In software terms, think "property based testing" as opposed to "unit test cases written in a predictable manner", with the assumption that your implementation under test has adversarial motives to obtain passing builds.)