Forum Discussion
Scaling feedback?
Do you have any way of providing feedback to your learners? If so, how do you think about scaling it?
Hey all! I recently had a coffee chat with JulieDirksen and she said something that got me thinking (well, she said a lot of things that got me thinking but this was one I wanted to share!).
She was talking about how for a long time, feedback was one of the last things in learning that you couldn't scale. But, she noted that with AI that is now changing—it's now more possible to scale the experience of receiving feedback on your work.
As a community professional, scaling feedback is something I've long thought about doing via nominating experts from the community to help spread the expertise of an "instructor" beyond one single person—I'm actually going to be giving a session at Articuland next week in Orlando that touches on this!
But, all of this got me thinking about different forms of providing feedback and how they have the potential to scale, and I wanted to pitch this question to the community and get your thoughts.
6 Replies
Scaling feedback is such a succinct way to phrase the problem. All effective learning seems to require some kind of feedback, so how can we add more as well as multiply the impact of what's there?
Scaling could be along channels of information. We can expect an audio-less black-and-white educational video to have missed some opportunities to reinforce key ideas compared to a video that consciously uses color to show relationships between objects on screen and uses audio to add emotional impact.
To me lately, scaling feedback has meant a scripted system of procedurally generated feedback. The answers chosen in an interactive eLearning could be tracked by holistic variables like "amountOfRiskTaken" or "customerSatisfaction," and then the quiz can present conditional feedback to the learner according to where they fell on that measure. For learners who chose more answers that were technically correct but also resulted in a negative customer satisfaction, the eLearning could provide additional feedback reminding them of the customer. While this kind of functionality doesn't help scale access to the content, it does scale the effectiveness of the content by giving some of the most immediate feedback the learner can receive.
I think the type of feedback matters a lot when deciding whether it can actually scale. For objective tasks, automated feedback can work well, but for open-ended work, the value often comes from understanding the learner's reasoning rather than just identifying whether the answer is correct.
AI could help with the first layer of feedback, such as pointing out patterns or suggesting areas to revisit, while peer or instructor feedback could handle the parts that require more context and judgment. That might make scaling more realistic without relying on AI to replace the human side completely.
How do others decide which parts of learner feedback are suitable for automation and which should remain human-led?
I really like this answer alextaylorone — I like the idea of being critical about which parts of the feedback motion are possible to automate and which require a human in the loop. It allows us to not be so black and white about saying "AI can't replace human feedback" so that a piece of the motion can scale with the help of technology, while still ensuring quality.
Have we connected in the community before? I like how you think! It would be great to have a virtual coffee chat some time :)
Julie's right that with AI something's changed, but I think the hard part is now task design rather than the feedback itself. Most Rise and SL courses give the learner nothing to get feedback on. A tick or a cross tells you nothing about what they were thinking. Ask them to put the steps of a process in order, or to spot what's wrong in a worked example, and now you can see where they went wrong and say so. None of that needs AI either, it's just triggers and conditions.
Open response is where the AI could come in, and I'm a bit torn on it. I built a weekly challenge course that highlighted this, The Red Pen, and came away less convinced than I went in. Give a model a generic rubric and it produces confident, even-handed comments on everything, including the bits that don't matter. Learners trust that. At scale I suspect it's worse than no feedback at all, because nobody's questioning it…
Really interesting point. I wonder whether this is really just where community comes in. I suspect peer feedback is better than AI feedback. I think about something like the weekly challenges here that DavidAnderson runs and how those make such a difference to learning in comparison to just taking async training. I think the social learning and feedback is an aspect of that.
Maybe the honest answer is that it doesn't scale… I guess this is why using AI is tempting, and why I'm still wary of it!
