Research, curriculum and grading: new data sheds light on how professors are using AI

When Georgia State University professor G. Sue Kasun taught a new course this summer, she used generative artificial intelligence to help her brainstorm.

Kasun, a professor of language, culture and education, teaches current and future language educators. And she used Gemini — Google’s generative AI chatbot — to come up with ideas for readings and activities for a course on integrating identity and culture in language education.

“There were suggestions of offering different choices like having students generate an image, having students write a poem. And these are things that I could maybe think of but we have limits on our time, which is probably our most valuable resource as faculty.”

Kasun also uses Gemini to create grading rubrics. She says she always checks to make sure that what it generates is accurate “and importantly representative of what my learning objectives are.”

It’s a massive time-saver, she says.

Kasun is one of an increasing number of higher education faculty using generative AI models in their work.

One national survey of more than 1,800 higher education staff members conducted by consulting firm Tyton Partners earlier this year found that about 40% of administrators and 30% of instructions use generative AI daily or weekly — that’s up from just 2% and 4%, respectively, in the spring of 2023.

New research from Anthropic — the company behind the AI chatbot Claude — suggests professors around the world are using AI for curriculum development, designing lessons, conducting research, writing grant proposals, managing budgets, grading student work and designing their own interactive learning tools, among other uses.

“When we looked into the data late last year, we saw that of all the ways people were using Claude, education made up two out of the top four use cases,” says Drew Bent, education lead at Anthropic and one of the researchers who led the study.

That includes both students and professors. Bent says those findings inspired a report on how university students use the AI chatbot and the most recent research on professor use of Claude.

How professors are using AI 

Anthropic’s report is based on roughly 74,000 conversations that users with higher education email addresses had with Claude over an 11-day period in late May and early June of this year. The company used an automated tool to analyze the conversations.

The majority — or 57% of the conversations analyzed — related to curriculum development, like designing lesson plans and assignments. Bent says one of the more surprising findings was professors using Claude to develop interactive simulations for students, like web-based games.

“It’s helping write the code so that you can have an interactive simulation that you as an educator can share with students in your class for them to help understand a concept,” Bent says.

The second most common way professors used Claude was for academic research — this comprised 13% of conversations. Educators also used the AI chatbot to complete administrative tasks, including budget plans, drafting letters of recommendation and creating meeting agendas.

Their analysis suggests professors tend to automate more tedious and routine work, including financial and administrative tasks.

“But for other areas like teaching and lesson design, it was much more of a collaborative process, where the educators and the AI assistant are going back and forth and collaborating on it together,” Bent says.

The data comes with caveats – Anthropic published its findings but did not release the full data behind them – including how many professors were in the analysis.

And the research captured a snapshot in time; the period studied encompassed the tail end of the academic year. Had they analyzed an 11-day period in October, Bent says, for example, the results could have been different.

Grading student work with AI

About 7% of the conversations Anthropic analyzed were about grading student work.

“When educators use AI for grading, they often automate a lot of it away, and they have AI do significant parts of the grading,” Bent says.

The company partnered with Northeastern University on this research – surveying 22 faculty members about how and why they use Claude. In their survey responses, university faculty said grading student work was the task the chatbot was least effective at.

It’s not clear whether any of the assessments Claude produced actually factored into the grades and feedback students received.

Nevertheless, Marc Watkins, a lecturer and researcher at the University of Mississippi, fears that Anthropic’s findings signal a disturbing trend. Watkins studies the impact of AI on higher education.

“This sort of nightmare scenario that we might be running into is students using AI to write papers and teachers using AI to grade the same papers. If that’s the case, then what’s the purpose of education?”

Watkins says he’s also alarmed by the use of AI in ways that he says, devalue professor-student relationships.

“If you’re just using this to automate some portion of your life, whether that’s writing emails to students, letters of recommendation, grading or providing feedback, I’m really against that,” he says.

Professors and faculty need guidance 

Kasun — the professor from Georgia State — also doesn’t believe professors should use AI for grading.

She wishes colleges and universities had more support and guidance on how best to use this new technology.

“We are here, sort of alone in the forest, fending for ourselves,” Kasun says.

Drew Bent, with Anthropic, says companies like his should partner with higher education institutions. He cautions: “Us as a tech company, telling educators what to do or what not to do is not the right way.”

But educators and those working in AI, like Bent, agree that the decisions made now over how to incorporate AI in college and university courses will impact students for years to come.