New AI Technology Lets Students Evaluate Professors by ‘Chatting’

Artificial intelligence, or AI, has slowly begun to influence higher education around the world. Now, one new AI tool could change the way university students evaluate their professors.

The tool is called Hubert. It is a teacher evaluation 1 tool that appears as an AI-powered chatbot. Instead of filling out a form, students use a chat window to give feedback on the course and their professor. Afterwards, Hubert categorizes the students’ comments for the professor to review.

The beta version of Hubert launched about seven months ago. It is free for educators to use. More than 600 teachers use it already.

Making evaluations 2 easier, and more human

The goal of the new Hubert program is to improve education by giving teachers detailed 3, organized feedback from students.

Viktor Nordmark, from Sweden, started the company in 2015, with several friends. Before creating Hubert, the founders 4 asked teachers what would improve their teaching skills. The answer, Nordmark said, was qualitative 5 feedback from their students. In other words, teachers wanted students to write detailed answers to open-ended questions about their experiences in class.

Qualitative feedback is different from the survey responses students are often asked to provide at the end of a class. “This kind of data takes a lot of time for professors to collect and analyze 6,” Nordmark said. And for the students, it can also be “really boring to fill out.”

So Nordmark and his friends designed Hubert as a compromise between a traditional survey and a personal interview. “You can reach a really large crowd, but you can also get really qualitative data back,” Nordmark said.

First, they ran a program to train Hubert to understand human conversation. Hubert learned different types of comments, and how to respond to them. Then Hubert learned how to organize the comments it received into categories.

Now, when Hubert receives comments from students, it compares them with the information already in its system to organize the data.

This idea, Nordmark says, is that Hubert will continue to get smarter as it receives more comments.

How it changes classrooms

What makes Hubert special is its similarity to communicating with a human. For some professors, this feature helps them receive higher-quality feedback.

“It allows students to zero in on things that are of interest to them… of concern to them, and a chance to say something good… they can essentially 8 present what their concerns are…and it’s not threatening,” said John Munro, an associate professor of business at the University of the Virgin 9 Islands.

Munro believed traditional methods of evaluating professors were not very helpful. They happened only at the end of the course, and the questions were formed around a four-part ranking system based on traditional lecture classes. These kinds of surveys, Munro said, did not give him the right information to make effective changes in his classroom.

In contrast, professors can offer the Hubert evaluation anytime they want. The chatbot asks students four questions: What could the instructor 10 do to improve the course? Is there anything the instructor should stop doing? What's working well? How has the class been over all?

Munro thinks the “chat” format 7 of the evaluation helps get more information out of students.

“Everyone, both students and professors, are already comfortable with chats,” Munro said.

If students respond with no information, Hubert knows to try again by asking follow-up questions. For example, if a student responds with “no,” Hubert might ask, “Are you sure there is nothing that could be different?”

Hubert evaluations might also be helpful for English language learners. Samuel Adams is an instructional technology specialist in English language programs at Temple University in Philadelphia, Pennsylvania. He tested Hubert with his English teacher training course.

“On average, we get very little written feedback from language learners,” Adams said.

Adams found that Hubert was effective in getting written feedback about his course. But it did have its limitations. Adams found that the pre-set interview questions made it impossible to get information about other parts of his teaching. And the feedback is also organized into pre-established categories.

An artificially human future

Nordmark says he and his co-founders have plans to make Hubert more flexible and accurate. They would like to give teachers the possibility of selecting their own evaluation questions. And they will teach Hubert to create its own categories, so that it can more correctly organize information. This feature would also allow Hubert to be used outside of education systems.

Its creators also want to make Hubert more human-like and possess qualities such as humor in the interviews.

Another goal is to let Hubert users share the information they are learning. If teachers are discovering strategies that work well in their classrooms, Nordmark wants to develop a way for them to share that advice with other Hubert users.

I’m Phil Dierking.

Words in This Story

accurate - adj. free from mistakes or errors?

analyze - v. to study (something) closely and carefully?

artificial intelligence - n. an area of computer science that deals with giving machines the ability to seem like they have human intelligence?

boring - adj. dull and uninteresting ?

categorize - v. to put (someone or something) into a group of similar people or things?

chatbot - n. a computer program designed to simulate conversation with human users, especially over the Internet.?

contrast - v. to be different especially in a way that is very obvious

course - n. a series of classes about a particular subject in a school

comfortable - adj. not causing any physically 11 unpleasant feelings

data - n. facts or information used usually to calculate, analyze, or plan something

essential - adj. extremely important and necessary

evaluate - v. to judge the value or condition of (someone or something) in a careful and thoughtful way?

feature - n. an interesting or important part, quality, ability, etc.

feedback - n. helpful information or criticism that is given to someone to say what can be done to improve a performance, product, etc.?

flexible - adj. an interesting or important part, quality, ability, etc.

format - n. the form, design, or arrangement of something (such as a book, magazine, or television or radio program)

lecture - n. a talk or speech given to a group of people to teach them about a particular subject

interview - n. a meeting at which people talk to each other in order to ask questions and get information

instructor - n. a person who teaches a subject or skill

ranking - adj. having a high position or the highest position in a group or organization

response - n. something that is said or written as a reply to something

review - v. an act of carefully looking at or examining the quality or condition of something or someone

select - v. to choose (someone or something) from a group

strategy - n. a careful plan or method for achieving a particular goal usually over a long period of time?

survey - n. ?an activity in which many people are asked a question or a series of questions in order to gather information about what most people do or think about something

type - n. type

zero in - phrasal v. to direct all of one's attention to (someone or something)