Read this quick blog/comment (https://news.ycombinator.com/item?id=33841672)
ChatGPT (and Its Relatives) and College Writing: A Quick Guide for Students, by Erik Simpson (https://eriksimpson.sites.grinnell.edu/Connections/Documents/WritingWithLLMs.pdf)
MLA-CCC Joint Task Force on Writing and AI Working Paper: Overview of the Issues, Statement of Principles, and Recommendations(https://hcommons.org/app/uploads/sites/1003160/2023/07/MLA-CCCC-Joint-Task-Force-on-Writing-and-AI-Working-Paper-1.pdf)
In reading these 3 documents, I was particularly drawn to the Simpson piece, mostly because it has relevance to our Grinnell environment (being written by a Grinnell professor and all). Like all of the pieces, it drew emphasis to hallucination (aka stochastic parroting), but it also discussed why, even if factually accurate, that writing will often sound “off”, which was interesting to see as far as knowing what these tools are likely to produce goes. However, I felt this text was in need of an update in light of the general policy towards ChatGPT in the new Academic Handbook. It may have been written afterwards, but somewhat unclear. The update I would suggest would be an expansion of the short end part, specifically to the point of 3. How will using an LLM affect my ability to learn what this assignment intends to teach me?. Perhaps Simpson does not believe there is such a case, in which case this would make sense. However, in my 213, it’s emphasized often to use ChatGPT to support learning, not undermining, and if Simpson believes this can ever be the case, a structure for how to differentiate would be interesting
LLMs and specifically, ChatGPT are posing the greatest potential and changes that I have ever witnessed in education. Syllabi and entire courses have shifted a heavy focus on not using ChatGPT or other LLMs in ways that inhibit the learning process. I have been considering this largely from the perspective of a student, but it was interesting to hear the costs from the instructor’s side of things. The time and effort that comes with understanding these LLMs requires additional training for teachers. Teachers, coming from the son of one, are often already overworked and underpaid. Additional training on LLMs might fall on a full plate.
I have already witnessed and experienced some of the risks and benefits of LLMs to students as discussed in MLA-CCCC Joint Task Force on Writing and AI Working Paper: Overview of the Issues, Statement of Principles, and Recommendations. I have seen students skip readings and assignments by just hopping onto a ChatGPT chatbot. Multiple syllabi each have a different approach and set of principles for using “AIs” in the classroom and students can easily access these powerful tools only if they have up-to-date internet devices. On the contrary, I have seen ChatGPT being used for interview prep, presentation prep, essay outlines, and quick questions that allow students to keep up and progress in their learning. Evidently, LLMs have already ingrained their place in education; they are not going away. Still, it is important that administrators, instructors, and students are aware of both the benefits and costs of using LLMs in education. As LLMs only progress and get even more complex, it is important that there is a foundational literacy on the good, bad, and ugly that comes with using them in the classroom.
In the Hacker News thread assigned for class, there is a discussion on whether these Artificial Intelligences can discern pieces of information for their truth. A number of commenters mentioned that artificial intelligence like ChatGPT are trained to produce plausible results for the prompts provided to them. Thus, some argue that these generative AI’s are not trained to produce true information because they are not sensitive to the fact true information is not probabilistic. At the same time, others argue that databases of true information should be built into these generative artificial intelligences that filter for false information. In fact, one commenter linked to a paper written by researchers at Google, where they learn that the incorporation of database of “ground truth” can limit the hallucination of factual information.
However, these commenters assume that it is possible to establish a database of “ground truth” in the first place. In a number of circumstances, the factual basis for information relies on a limited group of witnesses and whether their reporting is to be believed. In the case of politics, much of what is considered true depends on journalism. If only one journalist witnessed an event, they are the only person that can be sure of the event taking place without relying on the credibility of others. Everyone else would only know the event took place through the journalist’s reports and whether they can be believed. Part of this belief depends on the reputation of the journalist’s publication, their previous work, and whether their methods are believed to be conducive to truth. Regardless, whether a piece of information is true depends in part on trust, something that cannot be entirely vetted in an objective manner. Thus, any database that serves as ground truth for a generative artificial intelligence will depend on a subjective standard for what makes a source and their work trustworthy.
The chat GPT discourse gave me a lot of different insights into people’s understanding of how chat GPT interacts with us. For example, some of them thought the interaction caused hallucinogenic effects because chat GPT appears to be knowledgeable and provides trustworthy sources. It appears as all-knowing and never really simply says “I don’t know” instead it always tries to give something to the user whether that’s correct or not. Which is a tricky gamble. It was also interesting to know that chat GPT doesn’t have the ability to browse the Internet and get the most recent data available. So now it makes way more sense that some of these articles and citations are just made up. One of the people on the thread said how they appreciated that chat GPT “keeps you on your toes” but the intention of this algorithm was to sweep us off our feet. In the second article ChatGPT (and Its Relatives) and College Writing: A Quick Guide for Students, by Erik Simpson further pointed out how chat GPT makes up their own quotations and articles instead of doing nothing or giving nothing it’s very misleading. In those instances, it’s hard to keep yourself on your toes if you come to chat GPT, not knowing anything and wanting a quick answer so from that standpoint there’s no way to know that this information is clearly incorrect. This also reminds me of the point another article we read from Monday made about how dangerous it is to have AI voices in a place where they are supposed to be real human voices because if someone were to write a paper using chat, GPT, their sourcing ideas that may not even exist and are just synthesized from a computer it’s a lot of misinformation which is dangerous. It’s crazy to know that the more chat GPT improves the harder it will be to discern what is legitimate information and what is not when the algorithm was created to add clarification.
I really liked trilbyglens’ quote that the real issue when it comes to stochastic parroting is that “it’s a language model, and not a knowledge model.” I do wonder sometimes, if OpenAI was just working on their chatbot/virtual friend/whatever, one day realized how good it was at being “smart,” and just got swept along for the ride in seeing how good they could make it at something it wasn’t really meant to be. I have no evidence for this, but that’s how I picture it mentally. It was also really helpful to learn about the word “confabulation” which is not only fun to say, but also really fits for what these models are doing.
Also, I thought the third question at the end of Professor Simpson’s memo was really interesting. It’s always baffled me why people come to college only to skip class or not do assignments, and like I get it, you want to have fun, a degree, a stable financial future, and honestly, I support it in some cases (the stats class I’m mentoring has so much busy work that sometimes I advocate that my students only do the ones that are officially graded, even if more are assigned, just because if they burn themselves out they won’t retain the content). But it just seems absurd how many people don’t like learning! Everything you interact with, from a blade of grass to your computer is complex on a level that one person could never truly understand all the way down. And because of that, it really baffles me when someone takes 211, for example, and just doesn’t do the assignments. Or who uses Chat GPT to make their philosophy presentation. There’s so much to learn! Why wouldn’t you want to learn as much as you can. I mean that gets to death and philosophy, so I guess it depends.
The threshold of confidence that chatGPT has when giving wrong answers is mildly concerning when thinking of a student just trying to get something turned in. It is easy to picture a scenario in which a student may put themselves at risk of failing assignments—or worse, getting into an academic dishonesty case—if they abuse chatGPT and submit work that is verifiably false. It is also amusing to consider that figuring out how to properly use chatGPT to write a “good” college paper might take as much time in finding a prompt that the machine interprets properly, to ensuring the sources it gives are real, to replacing those sources with actual sources, to editing the grammar and syntax to feel more real, etc, as it does to just write the paper on your own.
I remember when chatGPT first came out, I used it to try to find new music that was similar to songs that I enjoyed, and was amazed when it spat out other songs that I knew and liked. What was curious was that some of the additional titles were songs that didn’t exist, or were by a different artist, and I was perplexed by this. It makes sense now, knowing that chatGPT is not a “knowing” entity. It is doing essentially the same thing that any automated playlist maker is doing—looking for associations with whatever input you give it and producing some output. This sort of practice is what makes chatGPT so misleading for education, and a problem that is very worth bringing up to students in class. Students unfamiliar with the inner workings of chatGPT need to be told that it is not the answer to hard work, and that treating it as though it can truly understand—like it is a human—is a process that short changes their education and also may land them in some precarious situations.
Today’s readings mainly discuss the impacts of LLMs, particularly ChatGPT, on the education system.
The blog post highlights the phenomenon of “hallucination” in the ChatGPT where the system generates inaccurate information such as nonexistent citations and words that can have negative impacts on users. Professor Simpson writes an article cautioning students to use the ChatGPT for academic purposes because it does not generate sentences as proficiently as students can. The MLA-CCCC Joint Task Force offers insightful perspectives on the risks and benefits of the ChatGPT for students and teachers along with recommendations for its use.
I agree with the arguments of the readings and I appreciate how some of the articles offer both negative and positive impacts on the education system because a lot of people never mention how ChatGPT can be useful for learning if used wisely.
While the ChatGPT and other LLMs undeniably provide benefits to students, as Kissinger argued in last week’s reading, students always need to have “the confidence and ability to challenge the outputs of AI systems” by developing “skepticism and interrogatory skills” to live and learn in the era of AI.
Today we focused on LLM’s effects on academia specifically. ChatGPT is still a very new tool for students to use, and professors are still attempting to formulate what is considered “fair use” of the resource. Some are saying that students should not use it at all. Others say that they cannot stop them from using ChatGPT. Then there is the middle case where professors encourage students to use it in moderation and as an add-on rather than a replacement.
When it comes to the differences between the speech patterns that chatbots and humans create, humans are more inconsistent with sentence length and their choices of connector words like “but”. So regarding creating a tool that can detect the use of LLM for professors, that is something that the tool could potentially analyze. However, all student’s writing voices vary, and it has been proven that the tools that already exist for checking against the use of LLMs like ChatGPT are very inaccurate, and are getting students who did not use them in trouble, and vice versa.
For now, the best solution for students is to ensure they retain the material they are writing about, and only use ChatGPT in moderation to enhance their learning. If students choose to supplement LLMs for their writing skills, they will be the only ones who suffer from that choice in the long run.
Connecting stochastic parrots to the readings we had today, we are continuing with the ideas about unreliable LLMs and possibly giving misinformation. We notice that we are given some pros and cons about LLMs in how they help with possibly understanding a reading, translating certain lines, and helping those who struggle with speech. However, some cons that we see with LLMs is that they generate reasonable words together and produce fake information sometimes. For example, we notice that in ChatGPT – 4, the bot connects authors and writers with articles that never existed. By creating such information, we see that in the “Task Force on Writing and AI Working Paper” reading, they go further in-depth about how students and teachers are affected. We see that there are benefits for students and teachers as well as risks. I feel like by acknowledging these issues, there is a possibility to be able to find a middle-ground of these LLMs and how they affect schoolwork. I feel like we would be able to figure out a way where LLMs are available to the students and teachers, where they could be able to ask about gaining a better understanding of what is going on as well as figuring out how to approach certain problems, but not ask about writing essays for them or researching specific topics of their work.
I thought there were a lot of interesting comments on the first reading. One of the top comments replied saying that these models “must predict the next word, even if it means that they have to make something up.” This much is true, but there are also plenty of cases where LLMs typically refuse to respond, like questions on sensitive topics. I think it is better for LLMs to be directed not to answer the types of questions where they are most likely to “hallucinate” information that does not exist. After all, what is the point of these models if they are made practically useless by the amount of skepticism you have to apply to each answer?
Simpson’s article definitely makes some interesting points about LLMs in education, but I feel that the evaluation of the “blandness” of LLM language is a little misguided. These models are meant to be easy to interact with and comprehend, not spit out a chunk of formal, scholarly writing in response to every inquiry. I think their distinctness from human scholars is perfectly fine, as writing scholarly papers is not the current aim of many LLMs.
I really appreciated the initial perspective of the third reading. Writing, in all of its forms, has always been an incredibly human way of communicating ideas and feelings. There is so much that can be lost by over relying on a homogeneous tool to produce language. I think that the final principles proposed at the end of the article hit on this pretty hard. It’s impossible to make people not use these types of tools to circumvent the difficulty of writing original work, so it’s crucial to impress people with the importance of being a good writer not only in education, but in life as a whole.
I remember that during my first semester of college, the academic policies at the end of each syllabus mostly emphasized the importance of citing and giving proper credit for the ideas used in assignments. However, this semester, all of my classes included a significant section on policies regarding the use of ChatGPT and other Large Language Models (LLMs). All of the readings provided valuable insights into understanding the use of LLMs in academia. Personally, I believe that LLMs can be really helpful for explaining grammar and possibly correcting some spelling and grammar errors, but they should not be used to formulate arguments. The readings showed that ChatGPT is not 100% reliable and can indeed provide misinformation and produce fake information when it doesn’t know the answer to the given prompt. I consider that arguing requires an opinion on a topic, and these LLMs aren’t capable of generating new ideas and positions to contribute to ongoing conversations in academia. I found the analysis of the text produced by LLMs in the MLA-CCC Joint Task Force on Writing and AI Working Paper: Overview of the Issues and Statement of Principles to be particularly insightful. I appreciated how the authors approached the problem by analyzing the benefits and problems from the perspective of everyone involved in this debate, including teachers, students, and academic authors. The article does an excellent job of highlighting how writing has been an important tool and does well in emphasizing the dangers of it disappearing or declining in quality because of LLMs. However, it also highlights how these LLMs can be very helpful, especially for people who aren’t native English speakers. I appreciated how they also included the risks already associated with these models, such as inequities between those who can use them and those who can’t.
These readings gave some great language with which to describe the issues we’ve been describing with LLMs this week. Confabulation is one that I think really fits.
In conversation with a friend about using Chat GPT to help write papers and learn and what not, I heard them say that the classes were all about the ideas and that Chat GPT allowed the to interact with and put the ideas together regardless of whether or not they wrote the paper. I understand that idea, but I wonder if they would still think the same things understanding the likelihood of confabulation on the part of the model. I don’t think I’ve heard anybody discuss any sort of doubts about what Chat spits out. For the most part, I get that it actually does spit out useful information in the right contexts. However, my main worry is that we as students might not really be able to apply the proper amount of scrutiny towards the ideas Chat spits out while simultaneously learning about said ideas.
I found the joint task force paper to be very illuminating on some of the ways in which educators can approach ChatGPT and their students to avoid policies of surveillance and disbelief as a default. It was also interesting to see the ways they predicted/ theorized about possible uses for LLMs, and how it could be used as a tool to support critical thinking development, rather than as a crutch. I think the framing and discussion / characterization of LLMs is critical to the ways in which students will learn to interact with and use them, though this will certainly be a difficult task, as basic internet literacy is already relatively highly priveleged and difficult to access.
I also really liked both discussions of AI “hallucinations”. I think the more people realize what kind of things AI often gets wrong, the more likely they are to be critical of the results it produces.
In the last class, the video/readings discussed how human to human interaction differs from stochastic parrots; human to human interaction is co-constructed and leads to a shared model/understanding of the world, whereas language models combine “linguistic forms” from training data without references to meaning, communicative intent, or intent to construct a shared model of the world. Simpson’s discussion of college-level writing “creating a written conversation with other scholars” in contrast to LLM production of writing that sounds like scholarship but is ridden with false information is a good example of how the responses constructed by LLMs differ from human conversation/interaction. There are limitations LLMs have when dealing with scholarship, as they don’t necessarily have the motivations to do directed research, find factually correct information, and further conversation and understanding; rather, they answer a prompt given to them by putting together data fed to them in a coherent way. I also found it interesting to read about how LLMs can be used to further academic writing and conversation; often, I feel like I encounter LLMs either not being used at all, or LLMs being used in place of an actual person writing.
The readings bring up one of the most glaring issues of LLMs that I frequently notice, in that they can be confidently wrong, in ways that sometimes seem more creative than when they’re right. The “hallucination” seen in the forum post was almost impressive, it just made up a number of titles and links in an incredibly convincing way. ChatGPT has mastered creating text that looks “real” or convincing, but hasn’t mastered accuracy.
The other two readings acknowledge the shortcomings of these LLMs, and advocate for careful usage. The Simpson article attempts to teach the reader about the dangers of putting faith into LLMs, as they can often be deceitful or just incorrect, and their use is often not worth the risk of academic dishonesty or a poor grade. The MLA article has a similar purpose, as it also acts as a warning. For the most part I agree with the sentiment that frequent use of LLMs leads to increased discrimination against marginalized communities, as LLMs frequently parrot hegemonic viewpoints. Not only that, it hurts students’ ability to think critically and their reading/writing skills in general. But it feels to me like these authors understand the futility of trying to stop the use of LLMs. Instead of trying to limit usage, they argue for more careful AI practices in writing. I can’t imagine this was the MLA’s ideal scenario, but it’s better than staying quiet on the issue.
There is a comment on the blog that says “The fact that it’s only a language model probably means that this is just out of scope,” in which someone else replies “That doesn’t stop the companies churning out these models to pretend otherwise.” If ChatGPT is marketed towards the masses as a comprehensive predictive tool, it doesn’t really matter what the intended scope is if no one understands it to be such. I also thought Simpson’s article illuminated quite a lot about how and why LLMs are used. We’ve talked a bit about how general critical thinking skills have collectively gone down, which is exacerbated by misinformation online and the rise of “digestible” information instead of thorough analysis of cultural and social events (i.e. TikToks about the Israeli-Palestine conflict). And if students 1) deprive themselves of reading and comprehending great literary texts like Toni Morrison and 2) are being told what to think by inaccurate or fraudulent sources, their reliance on this kind of presentation of info only deepens.
I did like how MLA-CCCC highlighted benefits of AI and writing, because I think it is important to consider why students even turn to tools like ChatGPT for assignments. Perhaps there is a need that is not being addressed in the classroom.
The blog post showed the first example that I had heard from general media about one of the biggest flaws of ChatGPT. Seeing that these programs are generating sources is incredibly misleading. Bing Chat which uses ChatGPT in its algorithm fixes this in some way by summarizing results, but Bing’s version is also less versatile in other ways, and I have also not tried to ask for citations, so I do not know how well it would perform. I worry a lot about the citation issue brought up in the Quick Guide reading. I think we have a long history of leaving out women and minorities in our history and citation, and I think that these algorithms could exacerbate this issue. One of the big pillars that I learned about Black feminism is their field’s attention to citation because of this issue, and I think we need to be able to know where our information comes from. I thought the suggestions mentioned in the MLA-CCC reading for editing ChatGPT essays interesting. Comparing this idea of educational writing ideas in the MLA-CCC reading with the note from the Quick Guide about how LLM’s tend to use additive vs comparative language I thought was interesting, and I would like to see more study of the differences between human writing and LLM writing.
The blog talks about the phenomenon that chatGPT produces made-up nonexistent references and links when one asks it about mathematical treatment books. In the blog, some of the commenters refer to it as a “hallucination” effect since with the black box of machine learning the GPT cannot guarantee the accuracy of what it is produced. And some people think of it as “stochastic parroting”, since “large language models are trained to predict the next word for a given input. And they don’t have a choice about this; they must predict the next word, even if it means that they have to make something up”. Originally, I did not agree with the hallucination opinion, since I thought it to be more like a human action. However, don’t humans hallucinate in the same way? We learn from what we sense and accidentally mix them and make up a new reference that is not been given before. And confidently convince others with our thoughts.
The two other papers talk about how people should treat LLM rather than just trust whatever result they give. It is definitely beneficial for people to use the LLM as a data filtration. However, they could only provide some more mechanical rules like grammar or provide a broader picture of certain areas.
After reviewing the MLA-CCCC Joint Task Force on Writing and AI Working Paper and the guide for students on ChatGPT and college writing, it is evident that AI language models like GPT have both risks and benefits in the realm of writing. These models have the potential to aid in drafting, genre modeling, creative work, and language learning, offering valuable assistance to writers. However, they also pose risks such as plagiarism, decreased learning, and the need for teacher training without adequate support.
One key concern highlighted in the papers is the issue of fake scholarship generated by AI text generators. These models may produce formulaic and capable text in expected genres but lack the nuanced understanding and context necessary to produce valid scholarly work. Citing non-existent sources and creating academic fraud undermines the integrity of academic writing. Therefore, students must exercise caution and carefully consider the restrictions set by their teachers, the importance of accuracy, and the impact on their own learning before relying on AI text generators for assignments.
The blandness often observed in AI-generated text is attributed to an overuse of additive words like “and” rather than the nuanced contrast that human writers can provide. This limitation further emphasizes the need for critical evaluation and discernment when utilizing AI text generators in a college writing context.
In light of these considerations, it is crucial for educators and institutions to adopt collaborative approaches that prioritize teacher support, value the writing process, and promote critical AI literacy. Punitive surveillance measures should be avoided, and policies related to AI text generation should be developed with faculty involvement, allowing for iterative review as technology evolves.
In conclusion, while AI text generators offer potential benefits in various aspects of writing, they also present challenges and risks that must be addressed. By recognizing the limitations of these models and approaching their use with care and critical thinking, students can navigate the intersection of AI and writing responsibly, ensuring academic integrity and fostering meaningful learning experiences.
The “hallucination” by GPT-based LLMs, as discussed in Professor Simpson’s article and on Hacker News, is a fundamental problem that limits the utility of generative AI. One of the most important points, however, is that hallucination must really be seen as a feature of LLMs and not a bug. That is, hallucination is the primary mechanism for LLMs to generate novel or creative responses, so it can’t be easily fixed without fundamentally changing the type of output generated by the LLM.
I see this issue as a fundamental roadblock for bridging between generative AI and AGI. It’s also why I see AGI as much further off than it may seem to a layperson, since while impressive, generative AI systems still have extremely limited capacity to reason. When presented with any kind prompt that requires logical thinking, such as the “b’s in blueberry” question, the LLMs answer should really be seen as a kind of highly informed guess — a statistical approximation of the answer rather than a logical endpoint.
Erik Simpson’s “ChatGPT (and Its Relatives) and College Writing: A Quick Guide for Students” offers a comprehensive and accurate portrayal of Large Language Models (LLMs), specifically focusing on their capabilities and limitations in academic settings. His assessment underscores the advantages of LLMs while also highlighting their pitfalls when used in the domain of scholarly writing. The fact that LLMs, like GPT-4, can produce coherent and grammatically correct sentences is rightly emphasized. Indeed, the underlying algorithms are designed to understand and replicate human-like text patterns based on vast amounts of data. The ability of LLMs to generate text that fits the expectations of specific writing genres, as Simpson notes, is commendable and has various applications, particularly in repetitive or formulaic writing tasks. The MLA-CCCC Joint Task Force’s working paper on Writing and AI is a much-needed document in today’s rapidly changing technological landscape. The task force underscores the importance of viewing writing as both a process and a product. They advocate for the recognition of human endeavors in humanities education and caution against the unbridled adoption of AI technologies, which might jeopardize the quality and essence of writing and language learning programs.
The matters addressed by today’s readings go back to an earlier discussion and re-emphasize the fact that Large Language Models do not “know” or “understand” language. Instead, they merely use statistics and probability to predict and generate words or punctuations based on patterns that they notice in their training data. Although I do not use generative AI technologies myself and am not the most up-to-date with the recent developments, I’m sure I’ve heard of how ChatGPT and its equivalents are now able to perform much more advanced writing tasks, even cheating online plagiarism checker tools. This may be the result of what the reading referred to when it said generative AI mixes some random pattern for more unpredictable outputs, but I think it might be interesting to discuss how this relates to the claims made in the readings and how concerning this might be to education.
Regarding the analysis of risks and benefits that generative AI poses to education, I think it’s fair to say that this is an overall positive development. All the risks to education, specifically literary studies and writing instruction, are expected side effects, and can be prevented with carefully thought-out and implemented policies on the part of institutions and college professors themselves. What I’m most concerned about is the effect this has on the general public view on the importance of writing and critical thinking. The fact that we are becoming more reliant on the technologies and are under the misapprehension that they can produce critical quality works of writing might eventually change how we think about the necessity of education as a whole or change how such skills are evaluated in all job markets.
I was not surprised by ChatGPT’s tendency to make up references and fabricate other kinds of information. There are many stories online about Bing’s AI chatbot gaslighting its users about facts which it obviously has wrong. I distinctly remember a news story in which a beta tester for the Bing chatbot tried asking Bing about when a movie came out, to which the AI began to aggressively try to convince the user that the movie actually had not come out yet, and that it was still 2022. It was incredible to see how frustrated and passive-aggressive Bing got with the user over the course of the interaction as the user tried to prove that the AI was wrong, and Bing refused to acknowledge its mistake. Someone pointed out in the comments of the Hacker News post that LLMs are not, in fact, answer engines. Providing accurate information is essentially out of scope. It still creates an enormous problem for the kind of things people try to use LLMs for, but I think that it can in some sense be considered a non-problem as long as you are informed about the capabilities and limitations of LLMs. I also found the long-winded discussion in the comments section about whether “hallucination” is an appropriate term amusing.