In case you haven't heard, #Texas schools are set to start using natural language processing tools—computers—to automatically score some standardised tests (STAAR).
But no worries, it also assigns itself a confidence score that might send it to humans to rescore if it's too low (how opaque will that scoring be?).
But surely the STAAR tests can't be that important? "The commissioner can appoint a conservator to oversee campuses and school districts. State law also allows the commissioner to suspend and replace elected school boards with an appointed board of managers. If a campus receives failing grades for five years in a row, the commissioner is required to appoint a board of managers or close that school."
Surely you can appeal?
"For $50, students and their parents can request a rescore if they think the computer or the human got it wrong. The fee is waived if the new score is higher than the initial score. "
@joop@pluralistic Wow, this is bullshit. Looking past everything else, just the fact that you need to pay $50 to request a re-evaluation is insane. The groups that will be negatively affected by this will be the same people who can't just fork over $50. Really upsetting.
@technotion@joop@pluralistic
The biggest losers will be the kids pressured by parents to score higher who put that $50 on a high-interest credit card in pursuit of college scholarships at institutions those parents can't afford.
Yes, that's one big run-on sentence. Call it a prison sentence.
@joop@pluralistic having known people who scored these kinds of tests in Pearson test centers, the inter-rater reliability was likely scandalous to begin with (I was hearing stats like 60% agreement rate). Which means the model training data may very well be trash. Using LLMs to do scoring just adds one more layer of garbage to the dumpster fire that is standardized testing - biased questions, vague scoring rubrics, low-payed & poorly trained scorers…
Add comment