How might generative text tools be useful in academia?

Last updated Jun 16, 2023 | Originally published Jun 16, 2023

• Futures
• University
• Academia
• Research

Today I hosted a small lunchtime workshop at the MUN Faculty of Business Administration on the futures of generative text tools in research and teaching.

I presented some early strategic foresight work on signals and trends of change in this space, culminating in four scenarios and some artifacts from these generative text/academia futures.

Download the slides!

These scenarios were developed using a conventional critical uncertainties matrix (Ogilvy & Schwartz, 1998).

The first critical uncertainty is acceptability. There are two possible extremes here: either use of AI tools become accepted and rewarded, or they become completely unacceptable and any use is viciously punished.

The second critical uncertainty is exclusivity. AI tools may continue to proliferate, and free-to-access tools may get easier and easier, such that they are ubiquitous. Or, the regulatory pressures, costs, and other issues of offering and maintaining language models mount. Meanwhile, reliability decreases and the learning curve to effectively use these tools increases. This combination of factors make models exclusionary and inaccessible, such that only those who put the effort and resources in can use them effectively.

These two critical uncertainties produce four scenarios:

1. AI everywhere: acceptable x ubiquitous
2. Bootleg AI: unacceptable x exclusive
3. AI is a joke: unacceptable x ubiquitous
4. AI experts: acceptable x exclusive

Below I briefly discuss each scenario. I’ve also used reverse archaeology to define some artifacts from each future (Candy, 2013).

Generative text and related tools become so ubiquitous and acceptable that everyone simply uses them with/for everything.

Spellcheck → Autocorrect → Grammarly → Thinkcheck. AI assistants are embedded in everything and always helpful. So, when you write a dumb paragraph or miss a key reference, the assistant makes sure to check your thinking.

Every student depends on text generation and chatbot interactions. Academic institutions follow suit, eliminating any notion of plagiarism or cheating related to these tools.

In a world where using these tools is difficult and shameful, AI use might be hidden (and hunted out). Practices for covering your tracks after logging into a speakeasy server to use the latest model become part of the skillset of being a user.

If research writ large cannot stand AI-based contributions, we might see journals that do everything they can to screen them out. “AI-free journals” start to engage in interviews with authors of submissions as a kind of oral peer review, before submissions are sent out to reviewers for consideration in a future publication.

Oral exams may see a return in this scenario as teachers look for ways to assess student performance free of AI supports.

AI use is common, but the output is easily recognized and of laughable quality.

Researchers are proud to label their work as free of any AI influences. (Conversely, researchers who associate themselves with AI use lose reputation.)

Universities and colleges compete for the recognition of one another and employers by doubling down on anti-AI teaching and learning. AI-free degrees are certified by a third-party audit and regularly reviewed to assure the world that graduates are both brilliant and have no AI dependencies.

Effective use of AI is very valuable, but very difficult. Teachers and researchers who develop the skillset and and tools necessary to take advantage are able to set themselves apart.

Large labs and researchers with a lot of funding build focused AI research assistants: finely-tuned models that are always up-to-date in the literature, well-versed in their researchers’ data, and are constantly fed new notes and ideas from their team. These tools are able to answer prompts and identify serendipitous connections, becoming invaluable tools for their operators.

AI becomes so essential that dedicated courses are developed to ensure graduates have the skills necessary to use and learn from these tools. These courses are offered early in undergraduate degrees to ensure that students have the prerequisites to take on more challenging AI-augmented tasks later in their programs.

In the discussion after the presentation, a framework emerged for assessing AI. Teachers trying to evaluate student work in the AI era have at least these five options:

• Grade the process, not the product. Have students describe and share proof of work by including evidence of each step of the process — essay outlines, notes, draft versions, and so on.
• Use invigilation to ensure that work is done without any help from AI tools. (E.g., even written assignments are completed in a classroom with an observing invigilator.)
• Fold AI tools into assessments. Ask students to use these tools as a part of their coursework. (E.g., have them use AI tools to summarize a course reading, then have them critique the summary.)
• Make assessments harder. Create assignments that are more challenging than they would have been in the pre-AI era. Expect that AI tools will help students succeed at the basic elements of these assignments, but design them such that getting good marks would be impossible if the student simply depends on AI tools.
• Oral exams. Similar to using invigilation. Add short oral examinations to course assessments, designing student interviews and follow-up questions that effectively reveal when students cannot complete course objectives of their own accord.