Writing with ChatGPT: An Illustration of its Capacity, Limitations & Implications for Academic Writers

Abstract

In the writer’s craft section we offer simple tips to improve your writing in one of three areas: Energy, Clarity and Persuasiveness. Each entry focuses on a key writing feature or strategy, illustrates how it commonly goes wrong, teaches the grammatical underpinnings necessary to understand it and offers suggestions to wield it effectively. We encourage readers to share comments on or suggestions for this section on Twitter, using the hashtag: #how’syourwriting? ChatGPT and other artificial intelligence (AI) tools are raising alarm bells across academia. Much of the alarm centers on how ChatGPT will affect the educational mission. How will it affect student learning? Will it lead to rampant student cheating? Will it mean the death of traditional knowledge assessments [1]? Recently, the alarm has reached our scholarly mission as well. Is it a new technological resource, or a threat to scientific integrity? What uses are appropriate, and how should they be acknowledged? These are not abstract questions. ChatGPT has already been credited with authorship in preprints and peer-reviewed published articles since January 2023 [2]. Concerns have been raised about its uncredited or fraudulent use [3], and major journals are now declaring their positions on the issue. For instance, the Springer Nature journals have declared that ChatGPT cannot be a co-author because it cannot take responsibility for the work, and they require that researchers document any use of ChatGPT in their Methods or Acknowledgements sections [4]. Academic Medicine guides authors to disclose the use of AI tools in scholarship, describe transparently the nature of that use, and be aware of limitations that affect accuracy and integrity [5]. A recent systematic review in the domain of healthcare education, research and practice acknowledged ChatGPT’s promise but concluded that it should be embraced with “extreme caution” considering concerns with “ethical, copyright, transparency, and legal issues, the risk of bias, plagiarism, lack of originality, inaccurate content with risk of hallucination, limited knowledge, incorrect citations, cybersecurity issues, and risk of infodemics.” [6]. We already use technology to assist our research and writing. Imagine how you’d function without SPSS or NVivo to manage your data analysis, Reference Manager to organize your citations, or Grammarly editing software to correct your spelling and grammar. This Writer’s Craft aims to familiarize writers with ChatGPT so that they might use it effectively and appropriately. Drawing on chats I had with ChatGPT4 in March and April 2023 to illustrate its capacity and its limitations, I extract a series of Cautions and Insights and guide writers in how to use incremental prompting to ‘train’ the software, how to use it for brainstorming and generating content like outlines and summaries, and how to employ it as an editor. What is ChatGPT, exactly? ChatGPT is an AI large language model. Current versions (3.5 and 4.0) have been pre-trained on massive amounts of data pre-2021, allowing it to learn language patterns and associations which it uses to generate human-like conversational text when prompted. When you input a prompt into ChatGPT, it analyzes the input and rapidly generates a response drawn from information acquired via machine learning in engagement with the internet. It excels at processing, distilling and presenting information verbally in human-like text [7]; it has the potential, therefore, to be an important asset for academic writers, particularly if we can outsource some of the labor of writing [8] given time constraints and other resource limitations. ChatGPT generates responses that are grammatically correct and semantically meaningful. They are not, however, always accurate. This is because AI large language models don’t have ‘knowledge’ in the usual sense of the word; they don’t store or retrieve data; they don’t crawl the web like a search engine for information. They are “just good at predicting the next word(s) in a sequence” [9] based on what they have learned. Thus, ChatGPT has “uneven factual accuracy” [10]; moreover, it does not try to ensure that the content of its text is true, robust, verifiably, valid, generalizable, etc. [11]. In machine learning terms, it can “hallucinate”, confidently presenting legitimate-sounding material that it is not real [12]. Training ChatGPT through incremental prompting ChatGPT’s default is to offer generic and descriptive responses. Incremental prompting is the process by which you gradually focus its attention and train it to give you responses tailored to your interests and your level of understanding. General prompts are fine as a starting point, such as my question below: It goes on to explain active and passive voice, but that’s not really what I’m interested in. I try to get a more specific answer by telling it about myself: That’s somewhat better: it’s talking about style and tone, and the example of pronouns is more specific. But it is still more generic than I would like. I ask it to: This response employs more formal linguistic features to talk about voice in academic writing, and offers illustrations of deixis, modality, and coherence. However, these aren’t the primary features that scholars are concerned with when they talk about academic voice. So my next prompt asks about a prominent linguistic scholar of academic voice: This is accurate, but still generic and selective: Hyland’s theory has three main components, only one of which is explicitly represented in its response (“stance”). My next incremental prompt tries to focus it further: ChatGPT is now able, in a few seconds, to offer a 360-word description of Hyland’s theory with helpful, illustrative examples of its three components. But now, I wonder, what other linguistic theories of voice did it not tell me about, due to my focusing in on Hyland in my prompting? So I ask: Now ChatGPT describes four important theories of voice, which I could probe further with incremental prompting. I could also ask it to compare these theories, or to describe tensions among them, or to suggest how more recent theorists have built on them. Once you get to this level of specificity, ChatGPT can be very helpful in pointing out connections among ideas for you to pursue in your own reading and writing. As this conversation illustrates, incremental prompting is a key to maximizing the value of ChatGPT’s responses. But effective incremental prompting requires domain-specific knowledge: you need to know the domain quite well already, so that you can judge the extent to which ChatGPT’s response is accurate, selective or comprehensive, and use follow up prompts to improve it. If you ask it about domains you don’t know well, it will be unclear what information ChatGPT has selected from, and why it has selected some things and not others. One final point about ‘training’ ChatGPT: what it learns does not appear to consistently transfer beyond the current chat. When I opened a separate chat and asked again “What do you understand about Ken Hyland’s theory of academic voice in writing”, the first response included described all three features of this model: that seemed to represent learning transferred from the previous chat. But this is not always the case, as I will discuss in the next section. Caution: ChatGPT’s default responses are both generic and (invisibly) selective. Insight: Incremental prompting can increase specificity, guide selections & reveal exclusions. Using ChatGPT for brainstorming ChatGPT is better at some things than others. The main distinction is between content generation and structural support. In terms of content generation, as the previous conversation about academic writing voice illustrated, you need to know the content well to judge the quality of what ChatGPT is giving you. One form of content generation that the software has been identified to be good at is the creation of outlines [13]. I wanted it to provide an outline for this Writer’s Craft for me, so I began by asking what it knew about the Writer’s Craft genre: Stop right there: the Writer’s Craft is not a series of books, and I have never written books with these titles. As this illustrates, at each stage of the incremental prompting, you should be alert for false responses. ChatGPT is a text generator, not a brain: it is putting together words that are likely to be found together around the topic you’ve asked about. That doesn’t mean these words ‘belong’ together or that they are ‘true’. In fact, ChatGPT seems to enjoy making sh*t up. You absolutely cannot trust the references it gives you. And not only because it doesn’t have access to material post-2021; the Writer’s Craft series extends back to 2015, but it doesn’t know them and apparently doesn’t try to find them. Instead, it “hallucinates”, making up a book series. Happily, you can stop it in its tracks when it does this, by hitting the “Stop Generating” button, which I did. Then I corrected it: This answer too, is mostly repetition of what I told it in the prompt, so I try to focus it down and get specifics by asking for the titles of the Writer’s Craft series: The problem is, these aren’t real titles: they’re made up. This is one of ChatGPT’s main weaknesses: when you prompt it to provide specific references, it often invents them [14]. Because these inventions are based on word associations, they will have realistic combinations of keywords and coauthors. But they are as likely as not to be bogus. Techniques for addressing this problem are emerging, such as extensions that improve ChatGPT’s access to the Web or restrict it to Google Scholar references; however, reports of their effectiveness vary [15]. Caution:  ChatGPT will lie to you, a phenomenon known as an AI hallucination. Insight:  Double check any content it generates, particularly references. I will say, though, that ChatGPT is rather creative when it comes to titles. I wish, for instance, that I had written this fake Writer’s Craft: “Lingard L. Convince me: the art of argumentation. Perspectives on Medical Education. 2013; 2(2): 75–78.” This suggests that one way to use the software effectively would be to ask it for title suggestions. As an experiment, I give it the introductory paragraphs from a recent Writer’s Craft (that I had entitled “Writing for the Reader: Using Reader Expectation Principles to Maximize Clarity) and I ask it for possible titles: Some of these are pretty good: I especially like 1 and 6, and if I were going to write a new title it would likely combine pieces of these favorites. Back to my attempt to get ChatGPT to create an outline for this Writer’s Craft on ChatGPT. I tried to get it to familiarize itself with actual Writer’s Craft pieces, but it resisted. At first I’m puzzled that it resists actually reading the entire series, as it surely does have the ability and the time (!). Perhaps if I gave it all the references it would respond better. Here though, it reverts again to generalities: “The series consists of approximately 20 articles, each of which likely focuses on a different aspect of academic writing…”; The series likely provides practical advice and strategies for researchers to use in their academic writing…”, and so on. So far, I can’t have confidence that ChatGPT has actually read any of the Writer’s Crafts, even the specific one that I provided as an example. But then I remember that ChatGPT is neither reading nor analyzing – it’s just recognizing language patterns. Thus, I prompt again: That’s getting closer. There is sufficient detail that I can tell ChatGPT has picked up the main structure of this Writer’s Craft. Notice though, how important my own existing knowledge is: I know this article well (I wrote it!) and so I have been able to readily discern when ChatGPT is spouting generic stuff and when it is actually talking about the article(s) I’ve directed it to. Caution:  Don’t rely on ChatGPT to read articles for you. Insight:  Use ChatGPT in relation to familiar domains rather than unfamiliar ones. Now that I’ve (finally) gotten it to recognize the features of a specific Writer’s Craft, I can get to my original aim: getting ChatGPT to brainstorm an outline for this paper: This is useful. Admittedly, it’s not precisely the format from the Writer’s Craft I trained it on: it has reverted to a somewhat more generic structure. And it is not (you will have noticed) how I actually decided to organize this piece. It does, however, help me conceptualize some of the necessary sections. And it was useful to return to during the drafting process, to see what I was emphasizing and overlooking. Once you get ChatGPT to this point through incremental prompting, you’re on the cusp of all kinds of brainstorming bounty. Let’s say you want some specifics to help you flush out each section: Insight:  If you dread the blank page, a ChatGPT outline could jumpstart your drafting process. Caution:  Its ideas are generic; use them as a starting place, not a replacement for your own. What if you wanted help weaving in some additional ideas, to deepen the piece of writing? Just ask, remembering to be as specific as you can be: These are all relevant points, and they provide me with search terms I could input into Google Scholar to round out my understanding of each. As my chat above illustrates, ChatGPT can be used to create solid outlines. You need to train it on the genre you’re going to write in and you need to judge its knowledge about the subject areas you’ll cover, but once you’ve taken those steps you can quickly request a series of outlines with different orders, sections, emphases. This can help you imagine different ways of approaching the manuscript: choose the best one, and start drafting. ChatGPT is fast once you get it pointed in the right direction (this whole chat took less than 10 minutes), but that training effort doesn’t transfer to new chats. The system saves all your chatlogs: you can see them on the sidebar and go back and access them, but they are discrete entities. “Contextual memory only applies to your current conversation. ChatGPT’s stateless architecture treats conversations as independent instances; it can’t reference information from previous ones. Starting new chats always resets the model’s state” [16]. Not knowing this, a few days later I started a new chat and asked it again “Tell me what you know about the Writer’s Craft series written (predominantly) by Lorelei Lingard to help researchers improve their academic writing”, only to be told again about 5 books I had never written. When I went back into saved chat logs and picked up my prompting where I’d left off, the result was better but not consistently so, which may be due either to limits on ChatGPT’s contextual memory or to its tendency to “break character” due to “dropping instructions it deems irrelevant” [16]. Caution:  ChatGPT doesn’t transfer the training you’ve done across chats. Insight:  Try returning to saved chat logs; you may be able to build on the training you’ve done through previous prompting. Generating counterarguments, summaries, and abstracts Outlines are not the only useful way to use ChatGPT for content generation. You can also ask it to review a section of your argument and suggest counter arguments. Keep in mind that the version of ChatGPT you use matters. The free ChatGPT 3.5 has a limit of about 500 words on what it can read and respond to, so if you input your whole results or discussion section you’ll get this error message: ChatGPTPlus (the paid version) is supposed to handle up to 25,000 words at a time, but I still received the error message when I tried to input more than a few paragraphs for it to read and respond to. Therefore, I think it’s better to give it a rough summary of a section (say, while you’re still drafting it) and ask it for counterarguments. In this prompt I summarize the gist of the results of a paper I’m currently working on, tell ChatGPT what I want to argue based on those results, and ask it to suggest counterarguments: In about 3 seconds (it still leaves me breathless how quickly it works), ChatGPT offers this: This is enough to get my wheels turning: I could readily start writing a ‘counterargument’ section of my discussion. If I don’t understand fully some of these ideas or I want more specificity, further prompting would focus these responses. Insight: ChatGPT can be a good brainstorming resource. Caution: But don’t accept its suggestions blindly. You are smarter than it is (at least in its current form). AI tools like ChatGPT can also help you to make your writing more accessible and inclusive of a wider audience. For instance, I gave it the opening paragraphs of a recent grant application and asked it to generate a 100-word lay summary, to which it responded: This was still a bit formal, suggesting that the meaning of ‘lay summary’ was perhaps not self-evident in my initial prompt, so I prompted it further: This is an acceptable first draft that I can now rework; for instance, I would probably reinstate a few of the keywords (like “scientific integrity”) that were removed. But having a workable draft to start from has probably saved me at least 30 minutes of work. Abstracts are another piece of writing labor that ChatGPT can help with. It can’t help you with your first abstract draft, because it can’t read your entire paper due to limits on the words you can enter in a prompt. But it can help you take your abstract’s first draft, which is invariably too long, and reduce it to the required word limit. This is labor most of us would be happy to outsource: I have yet to meet a writer who cherishes the task (and time!) of whittling words from their abstract. But even with this task which seems perfectly suited to AI, you can’t entirely trust it. As illustration, I asked it to remove 27 words from a structured abstract which I provided, adding that I wanted it to rewrite as little as possible as I liked the content as it was. It produced an unstructured version about 100 words long. I responded: Now it returned a structured abstract, but still much shorter than I had requested. I prompted again (I confess, a bit irritated), this time not asking it to do the mathematical task of subtracting the number of words, but setting a word limit (300) for its response: It seems like it understands, but the next version wasn’t 300 words either (yes, by this time I was copying and pasting all the attempts into a Word document to check the word count): Finally, success: the last version was 310 words (we’ll give ChatGPT the point, because 10 of those were the structured headings). Insight:  ChatGPT apparently can’t count. Caution:  Even with concrete tasks, be alert for failure. ChatGPT can count, of course: if you ask it to solve math problems, it can do so. But exact word counts clearly aren’t its forte. Nevertheless, ChatGPT is still useful for reformatting existing abstracts for new purposes. I have a structured conference abstract of 250 words, which I want to submit to another conference that requires unstructured, 100-word abstracts: It still hasn’t counted accurately (this is 108 words), but it has reformatted to an unstructured abstract and retained the key ideas. I’ll trim the extra 8 words when I rework this version; that’s certainly faster than doing the whole reformat myself. Editing with ChatGPT to improve clarity and coherence Another way to use the tool is to strengthen the clarity and coherence of sections of your draft, particularly those dense spots where you think you might lose the reader. To strengthen internal coherence, you could input a single paragraph and ask it to rewrite so that the ideas develop more convincingly, including suggesting where you should add token sentences to illustrate your points. I inputted a paragraph I’d drafted and asked it for three possible topic sentences, to which it responded: Seeing the different emphases in each topic sentence helped me to identify the issues that were vying for attention in the paragraph, and make it more coherent. Insight:  Asking ChatGPT for topic sentences can help reveal issues with paragraph coherence. Caution:  Don’t use those sentences verbatim. They are a signal, not a solution. Theoretically, you can also strengthen external coherence with ChatGPT, by inputting a series of paragraphs and asking it to suggest new topic and transition sentences. I asked ChatGPT for help with the opening paragraphs I had drafted for this Writer’s Craft: Here’s what it suggested: I noticed two things immediately: first, ChatGPT changed all the sentences, not only the topic and transition sentences as requested. And second, it also changed the writing style: for instance, it uses many passive voice constructions (e.g., “has sparked”, “has ignited”, “has been acknowledged”), and changed out my simple subjects for more elaborate constructions (e.g., my “ChatGPT” has been changed to “the rising influence of ChatGPT within academia”). Now, I will be the first to admit that I can be a bit precious about my writing, but this is not what I asked it to do. Thus, I clarify: Ugh. It has overdone – and mixed! – the metaphors. And the tone has swung from stuffy academic to effusive adolescent. My next prompt reveals my irritation: This is closer to my writing style, and the topic and transition sentences are effective at connecting and developing the opening argument. If you’re struggling with internal or external coherence in a piece of writing, this could be a helpful resource. But if you’re not struggling with coherence (as I wasn’t particularly in this piece), its suggestions are unlikely to excite you – and some of them may frustrate you. Caution:  ChatGPT will edit your writing style as well as your content. Insight:  As part of your rewrite of ChatGPT-generated material, make the style your own. Whatever you want ChatGPT to help improve in your writing, you need to ensure that it understands the grammatical/rhetorical/linguistic concepts behind that feature of your writing. You don’t know what it “knows” until you ask it. I wanted to see if it could help one of my students identify and improve their tendency to write left-branching sentences (those that introduce a lot of detail early, leaving the main idea until late and thus potentially creating confusion for the reader who needs the main idea to organize all the other details). I started by asking it: As you can see, it had them backwards. I corrected it: I don’t like all of the sentences it has created, but they are more right-branching. This would be a useful coaching resource, once a writer’s habits are identified and we have ensured that ChatGPT has accurate knowledge of the grammatical features we’re interested in. For instance, many writers struggle to expand their repertoire of strong verbs. We could give ChatGPT a few paragraphs of their writing and ask it to rewrite with stronger, more dynamic verbs. Ask it for a few different versions and suddenly you have a nice catalogue of new verbs to choose from. More generally, ChatGPT could also serve as a free language editor for scholars writing in English as an additional language (EAL). Many EAL writers now incur the costs (both time/effort and financial) of language editing: it could alleviate some of those costs, particularly during the drafting and revision stages, and free writers to focus on the ideas and worry less about the grammar. A note on ethics Much of the alarm about ChatGPT has to do with the ethics of its use: is it ‘fair’ to have it write for you? As you will have noticed, I don’t advise having it write for you. Most of my examples involve putting my own writing into ChatGPT and asking it to make suggestions (here’s my introduction, please suggest some good titles), to do some tiresome labor (here’s my abstract, please cut it in half), to illustrate grammatical changes (here’s my left branching sentence pattern, please suggest right branching alternatives). I would argue that these are ethical and appropriate uses of ChatGPT. I’m not asking it to do all the intellectual, creative work, I’m outsourcing some of the labor [14]. Where I have asked ChatGPT to create something for me (an outline, a list of possible counterarguments, a passage improved with stronger topic and transition sentences), I treat it as a starting point for my next round of revisions. This isn’t only to avoid presenting ChatGPT’s writing as my own, although that’s of course important. It is also because I don’t want to outsource the writing craft, which (on some days, at least) gives me joy. And I certainly don’t want to ‘sound’ like ChatGPT – I want my writing to sound like me. Based on my experiences so far, it will take less time (and be more satisfying) to work on my voice than to work on getting ChatGPT to mimic me. In Summary Rather than being alarmed or anxious, writers need to understand ChatGPT’s strengths and weaknesses. It is better at structure than it is at content. It is a good brainstorming tool (think titles, outlines, counter-arguments), but you must double check everything it tells you, especially if you’re outside your domain of expertise. It can provide summaries of complex ideas, and connect them with other ideas, but only if you have put a lot of thought into the incremental prompting needed to shift it from its generic default and train it to focus on what you care about. Its access to information is limited to what it was originally trained on, therefore your own training phase is essential to identify gaps and inaccuracies. It can be used for labor, such as reformatting abstracts or reducing the length of sections, but it can’t replace the thinking a writer does to determine why some paragraphs or ideas deserve more words and others can be cut back. It can be inaccurate: in fact, rather stubbornly so, persisting with inaccuracies even after they are pointed out, while at the same time presenting its next attempt as corrected. I know it isn’t sentient and doesn’t have motivations or emotions, but I can’t help but think in some of our exchanges that it was being sullen, intractable, even deliberately insincere. Still, writers can harness its power to make our processes more efficient and our products more robust. Do check your target journal, as policies about writing with AI tools are emerging and evolving. Within journal parameters, however, leverage ChatGPT to your advantage. Identify the moments in your writing process where you get stuck: can ChatGPT help you there by generating an outline or brainstorming the next points in the storyline? Use it to help address your grammar challenges (e.g., if you default to passive voice, ask it to change sentences to active so you can compare); use it to strengthen coherence of a complex section of your argument; get it to increase clarity by converting your right-branching sentences to left-branching. Distinguish the laborious from the creative writing tasks: use ChatGPT to support the former, and keep the latter for yourself. And always view what it has generated as a first draft which you will refine and rework, infusing it with your own particular emphases, your unique voice and style.