Forum Discussion
AI Tutor not grounded only in course content? Uses general knowledge?
I've been playing around with AI Tutor since it was in Beta and now that it's out of Beta, I'm noticing something different.
During early testing, my understanding was that AI Tutor was grounded in the content of the Rise course and would only use information available within the course itself. It was positioned as course-based support rather than a general-purpose AI assistant.
Recently, while testing a course, I asked a question about a role that was not covered in the lesson. AI Tutor responded with an answer and then explained that the information came from "general knowledge" rather than the course content. When I challenged it, it acknowledged that it should be focusing on the course content.
Has anyone else noticed similar behaviour since AI Tutor came out of Beta?
Did something change in how grounding works, or is this just expected behaviour when the course doesn't contain enough information to answer a question?
I'm trying to understand whether AI Tutor is now intended to supplement course content with general knowledge, or if this is a limitation/bug that the Articulate is aware of. Any insights from others who have been testing it would be appreciated.
4 Replies
It's still being tweaked and worked on so this all may change, but it does have to rely on some general knowledge. Otherwise it wouldn't know that the Earth is round unless my Rise course said so, right? With how much information the average course is missing, the tutor would be useless if it was literally restricted to only the information in the course.
If someone was taking a Rise course about marketing principles and someone asked the tutor something like "Does Coca-Cola fit that model?" and the AI tutor said "What's Coca-Cola?" the perceived utility would quickly crater. I think the AI tutor probably tries to bias its focus toward the course content, but we should expect that the LLM behind the scenes is encountering guard rails that help it "decide" what it should and should not speak to. That doesn't mean it doesn't "know" those things but rather that it's choosing its words carefully. It's probably similar in some ways to how other commercial LLMs self-censor about other topics.
I see your point. Some general knowledge is probably necessary to make the tutor useful and provide relevant examples or context.
My concern is less about whether the model has general knowledge and more about where it draws the line. If learners can't tell whether a response is based on approved course content or the model's broader training, there's a risk of introducing information that hasn't been reviewed or validated.
Ideally, the tutor would use general knowledge to clarify and provide examples while keeping its guidance grounded in the course content and learning objectives.
Just to clarify my example, my prompt was: "What roles are involved in this process?"
AI Tutor responded by identifying multiple roles, including one that wasn't referenced in the course and doesn't exist in my organization. When I asked where that role and definition came from, it indicated that it was relying on general knowledge rather than the course content.
That is a bad thing. You already have the content knowledge to question that AI hallucination. You can't guarantee that every student will.
