: : : L u n a d e l C o r v o : : : (![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
lunadelcorvo) wrote2024-12-31 06:05 pm
lunadelcorvo) wrote2024-12-31 06:05 pm
Adventures in the uncanny valley of AI
The implications of AI are huge and very, very worrying. In the realm of social media and internet culture, AI has become a tool of misinformation and disinformation, both intentional and simply as a result of how AI works.
One major element of concern is just that - they way AI works. AI draws from whatever it can find, in most cases, the massive amount of material existing online. Because AI is not truly a thinking entity, it cannot distinguish between real, true, and accurate information and nonsense. It doesn't matter why the nonsense is there, whether because of honest error, because of malicious intent, satire, fiction, or propaganda. What matters is that AI is not able to distinguish the difference. Even worse, when an AI is programmed to prioritize 'engagement' (as is the case with Meta's AI, for example) over accuracy, then there isn't even an impetus to try to distinguish fact from fancy.
Certainly, it's easy to see the issues that can arise from the large-scale propagation of untested, unverified, infiltered information. News, current events, science, history; it doesn't even matter the topic. The potential for disaster is clear. Our society has already gone well down a path of 'alternative facts,' and has more or less discarded the belief that truth and accuracy matters, or that truth of objective reality is even discoverable. On the heels of that trend, the dangers of AI are amplified.
My personal adventures with AI are in education, however. There is a rapidly growing trend in education of reliance on AI to provide source material for teaching. yes, teachers are being directed to go to common AI sources to generate content, readings, and materials for teaching students. That alone is problematic enough, for reasons I will get back to momentarily. But I should also note that the resources teachers are being encouraged to use are not education-specific, they are not programmed for the purpose of creating educational materials, and they are not AIs that operate within carefully curated datasets. Teachers are being told to use ChatGPT.
Two issues need clarification here: why the datasets matter, and why it's such a problem that teachers are being directed to use them.
The 'dataset' is a fancy word for all the material that an AI draws from when 'creating' content. I use 'creating' in quotes because AI does not actually create anything. It collates, copies, amalgamates, and assembles whatever bits and pieces it can find. In essence, it plagiarizes. Now if one were to set up an AI with a limited dataset, comprised of say, peer-reviewed journals, authoritative sources, and material published by experts, that may not be a problem. There has been some discussion of whether AI can 'understand' the more com-plex written structure of something like a scholarly or scientific artticle, and that, too, is an issue. Still, if one could limit the dataset to only material known to be accurate, factual, and reliable, would one get better output? Maybe, but not necessarily. That is because of the other problem with AI. Regardless of the quality of information going in, the AI has to understand how to assemble that information. It must draw inferences, understand context, see connections, and all the other skills we so often despair of our human students ever learning. Consider the AI that advised the use of superglue to prevent the cheese from sliding off a pizza. It's absurd to us because we understand context, and can make inferences that superglue, while it may 'hold anything in place,' is not for use in food. AI cannot do that, so even with only the very best information in the dataset, AI can still produce nonsense.
That brings us to the other problem, and this is not a problem with AI as such, but with how most teachers are trained. Most teachers in the US are trained in education. That may not seem like a problem until one looks closely at what that training actually entails. In the majority of graduate programs in education, the preponderance of the coursework is in pedagogy (not particularly sound pedagogy, but that's my opinion,on which I shall also elaborate shortly). What most teachers do not study in any depth at all is the subjects they teach.
I am a bit of an odd bird in that I find myself in this system without any training in education, but with multiple masters' degrees in the content that I teach (in this case, history, culture, and civics). In fact, my teaching certification comes via an alternate route, which is teaching a requisite number of hours at the college level, where only content training matters. Beyond tooting my own horn, however, there is a reason I am pointing this out. The consequence of this education format is that most teachers have little to no actual training in their subjects. Generally, teachers take, at most, two or three undergraduate (200-300-level) courses in their subject areas.
In working with my peers, I have seen first-hand how dismally little many teachers actually know about what they teach. They can spin whole paragraphs of pedagogical buzzword bingo, though! Now, that's troubling enough, but the obvious implication of this fact in light of the growth of AI in education is that most teachers don't know enough about their subjects to even recognize when the AI-generated lesson they just had ChatGPT spit out is full of inaccuracies.
So the nonsense that floods the social media sphere will be the same that underpins what little is left of public education. For me, as a historian, this is especially disturbing. Historians know how important history is to democracy, civic dialog, and critical thinking. Orwell masterfully presented the dangers of erasing the past in Nineteen Eighty-Four: "Who controls the past controls the future." When we no longer know the past, we have no referrent for what is wrong with the now. There can be no rebellion without memory. There can be no truth without objective reality. There can be no civil rights without a past.
One major element of concern is just that - they way AI works. AI draws from whatever it can find, in most cases, the massive amount of material existing online. Because AI is not truly a thinking entity, it cannot distinguish between real, true, and accurate information and nonsense. It doesn't matter why the nonsense is there, whether because of honest error, because of malicious intent, satire, fiction, or propaganda. What matters is that AI is not able to distinguish the difference. Even worse, when an AI is programmed to prioritize 'engagement' (as is the case with Meta's AI, for example) over accuracy, then there isn't even an impetus to try to distinguish fact from fancy.
Certainly, it's easy to see the issues that can arise from the large-scale propagation of untested, unverified, infiltered information. News, current events, science, history; it doesn't even matter the topic. The potential for disaster is clear. Our society has already gone well down a path of 'alternative facts,' and has more or less discarded the belief that truth and accuracy matters, or that truth of objective reality is even discoverable. On the heels of that trend, the dangers of AI are amplified.
My personal adventures with AI are in education, however. There is a rapidly growing trend in education of reliance on AI to provide source material for teaching. yes, teachers are being directed to go to common AI sources to generate content, readings, and materials for teaching students. That alone is problematic enough, for reasons I will get back to momentarily. But I should also note that the resources teachers are being encouraged to use are not education-specific, they are not programmed for the purpose of creating educational materials, and they are not AIs that operate within carefully curated datasets. Teachers are being told to use ChatGPT.
Two issues need clarification here: why the datasets matter, and why it's such a problem that teachers are being directed to use them.
The 'dataset' is a fancy word for all the material that an AI draws from when 'creating' content. I use 'creating' in quotes because AI does not actually create anything. It collates, copies, amalgamates, and assembles whatever bits and pieces it can find. In essence, it plagiarizes. Now if one were to set up an AI with a limited dataset, comprised of say, peer-reviewed journals, authoritative sources, and material published by experts, that may not be a problem. There has been some discussion of whether AI can 'understand' the more com-plex written structure of something like a scholarly or scientific artticle, and that, too, is an issue. Still, if one could limit the dataset to only material known to be accurate, factual, and reliable, would one get better output? Maybe, but not necessarily. That is because of the other problem with AI. Regardless of the quality of information going in, the AI has to understand how to assemble that information. It must draw inferences, understand context, see connections, and all the other skills we so often despair of our human students ever learning. Consider the AI that advised the use of superglue to prevent the cheese from sliding off a pizza. It's absurd to us because we understand context, and can make inferences that superglue, while it may 'hold anything in place,' is not for use in food. AI cannot do that, so even with only the very best information in the dataset, AI can still produce nonsense.
That brings us to the other problem, and this is not a problem with AI as such, but with how most teachers are trained. Most teachers in the US are trained in education. That may not seem like a problem until one looks closely at what that training actually entails. In the majority of graduate programs in education, the preponderance of the coursework is in pedagogy (not particularly sound pedagogy, but that's my opinion,on which I shall also elaborate shortly). What most teachers do not study in any depth at all is the subjects they teach.
I am a bit of an odd bird in that I find myself in this system without any training in education, but with multiple masters' degrees in the content that I teach (in this case, history, culture, and civics). In fact, my teaching certification comes via an alternate route, which is teaching a requisite number of hours at the college level, where only content training matters. Beyond tooting my own horn, however, there is a reason I am pointing this out. The consequence of this education format is that most teachers have little to no actual training in their subjects. Generally, teachers take, at most, two or three undergraduate (200-300-level) courses in their subject areas.
In working with my peers, I have seen first-hand how dismally little many teachers actually know about what they teach. They can spin whole paragraphs of pedagogical buzzword bingo, though! Now, that's troubling enough, but the obvious implication of this fact in light of the growth of AI in education is that most teachers don't know enough about their subjects to even recognize when the AI-generated lesson they just had ChatGPT spit out is full of inaccuracies.
So the nonsense that floods the social media sphere will be the same that underpins what little is left of public education. For me, as a historian, this is especially disturbing. Historians know how important history is to democracy, civic dialog, and critical thinking. Orwell masterfully presented the dangers of erasing the past in Nineteen Eighty-Four: "Who controls the past controls the future." When we no longer know the past, we have no referrent for what is wrong with the now. There can be no rebellion without memory. There can be no truth without objective reality. There can be no civil rights without a past.