Forum Discussion
Voice Recognition and Storyline Example
UPDATE: To answer some questions, I have created a better, improved example that doesn't require any external files and wrote a step to step guide on how to implement this - https://elearningdifferently.com/speech-recognition-and-storyline
Dear all,
This morning, I thought it would be nice to make some quizes I build more fun and make learners to answer by voice. There are already some topics created on this from awhile ago, but most links are dead.
I have build a fresh example - https://elearningdifferently.com/voice/story.html
I am attaching source files for 360 with scripts in the ZIP below. Hope you will find it useful. And thank you to this blog post on explaining how to web speech API work.
P.S. Web speech API is supported only with Chrome.
Hi all,
Apologies, and thank you to @Matthew Bibby. It should be sorted now. Site were this demo was hosted is now in the Maintenance mode for redesign, so links died. I have made the links from original post to work again. So it should work now in Chrome ;)
https://elearningdifferently.com/voice/story_html5.html
If you are using this in your work, I would advice to download .zip example from the top post and host all elements on your own site/server. vars.html needs to be hosted online and referenced in the Storyline's project iframe, to evaluate voice using Speech API. Hosting this on your own location means you will have full control and prevent it breaking when my links eventually expire....
Matt.
Of all of the potential applications with Storyline 360, this is the one I am most interested in learning more about...
Please keep up the great work, Matt.
Something to note: when hosting web speech api example or using web speech API, your files need to be hosted over secure HTTPS connection. When using HTTP the domain/browser will not be allowed to access user's microphone.
Here is an example of one I did using the web speech API. I think the example is on the fourth slide. I tried to make it respond to German with mixed results.
Hi Matthew! Thanks for the link! I exported the web-demo.story file to SCORM ... but I can't get the mic to work even when I use Chrome. May I know your export setting?
Links are dead again. It seems there is sufficient interest in voice recognition for it to be included as an in-built future feature request... I am also looking forward to a time when results can be sent to email (yes, I know the arguments against this that have been made in the past) or at least a simplified way to export them to a spreadsheet (for those who don't need / want the added work of setting up or even going through an LMS.
Thanks for your feedback! Would you be up for logging a feature request to tell us more about your specific needs?
In the meantime, if there’s anything else I can do to help, please let me know!
I have a project where there is voice controlled software and looking to emulate in Storyline. If there's any updated convos, examples, code snippets, etc, it would be appreciated!
Hello! Somebody could help me to define one language as default please!
This is the code :
<!DOCTYPE html>
<meta charset="utf-8">
<title>Web Speech API Demo</title>
<style>
body {
background: #5A185B;
}
* {
font-family: Verdana, Arial, sans-serif;
}
a:link {
color:#000;
text-decoration: none;
}
a:visited {
color:#000;
}
a:hover {
color:#33F;
}
.button {
background: -webkit-linear-gradient(top,#008dfd 0,#0370ea 100%);
border: 1px solid #076bd2;
border-radius: 3px;
color: #fff;
display: none;
font-size: 13px;
font-weight: bold;
line-height: 1.3;
padding: 8px 25px;
text-align: center;
text-shadow: 1px 1px 1px #076bd2;
letter-spacing: normal;
}
.center {
padding: 10px;
text-align: center;
display: none;
}
.final {
color: #fff;
padding-right: 3px;
}
.interim {
color: #66bdc9;
}
.info {
font-size: 14px;
text-align: center;
color: #777;
display: none;
}
.right {
float: right;
}
.sidebyside {
display: inline-block;
width: 45%;
min-height: 40px;
text-align: left;
vertical-align: top;
}
#headline {
font-size: 40px;
font-weight: 300;
}
#info {
font-size: 20px;
text-align: center;
color: #777;
display: none;
}
#results {
font-size: 14px;
font-weight: bold;
border: 1px solid #ddd;
padding: 15px;
text-align: left;
min-height: 150px;
color: #fff !important;
background-color: #5A185B;
}
#start_button {
border: 0;
background-color:transparent;
padding: 0;
}
</style>
<div id="info">
<p id="info_start">Click on the microphone icon and begin speaking.</p>
<p id="info_speak_now">Speak now.</p>
<p id="info_no_speech">No speech was detected. You may need to adjust your
<a href="//support.google.com/chrome/bin/answer.py?hl=en&answer=1407892">
microphone settings</a>.</p>
<p id="info_no_microphone" style="display:none">
No microphone was found. Ensure that a microphone is installed and that
<a href="//support.google.com/chrome/bin/answer.py?hl=en&answer=1407892">
microphone settings</a> are configured correctly.</p>
<p id="info_allow">Click the "Allow" button above to enable your microphone.</p>
<p id="info_denied">Permission to use microphone was denied.</p>
<p id="info_blocked">Permission to use microphone is blocked. To change,
go to chrome://settings/contentExceptions#media-stream</p>
<p id="info_upgrade">Web Speech API is not supported by this browser.
Upgrade to <a href="//www.google.com/chrome">Chrome</a>
version 25 or later.</p>
</div>
<div class="right">
<button id="start_button" onclick="startButton(event)">
<img id="start_img" src="mic.gif" alt="Start"></button>
</div>
<div id="results">
<span id="final_span" class="final"></span>
<span id="interim_span" class="interim"></span>
<p>
</div>
<div class="center">
<div class="sidebyside" style="text-align:right">
<button id="copy_button" class="button" onclick="copyButton()">
Copy and Paste</button>
<div id="copy_info" class="info">
Press Control-C to copy text.<br>(Command-C on Mac.)
</div>
</div>
<div class="sidebyside">
<button id="email_button" class="button" onclick="emailButton()">
Create Email</button>
<div id="email_info" class="info">
Text sent to default email application.<br>
(See chrome://settings/handlers to change.)
</div>
</div>
<p>
</div><div id="div_language">
<center>
<select id="select_language" onchange="updateCountry()"></select>
<select id="select_dialect"></select>
</center>
</div><script>
var langs =
[['Afrikaans', ['af-ZA']],
['Bahasa Indonesia',['id-ID']],
['Bahasa Melayu', ['ms-MY']],
['Català', ['ca-ES']],
['Čeština', ['cs-CZ']],
['Deutsch', ['de-DE']],
['English', ['en-AU', 'Australia'],
['en-CA', 'Canada'],
['en-IN', 'India'],
['en-NZ', 'New Zealand'],
['en-ZA', 'South Africa'],
['en-GB', 'United Kingdom'],
['en-US', 'United States']],
['Español', ['es-AR', 'Argentina'],
['es-BO', 'Bolivia'],
['es-CL', 'Chile'],
['es-CO', 'Colombia'],
['es-CR', 'Costa Rica'],
['es-EC', 'Ecuador'],
['es-SV', 'El Salvador'],
['es-ES', 'España'],
['es-US', 'Estados Unidos'],
['es-GT', 'Guatemala'],
['es-HN', 'Honduras'],
['es-MX', 'México'],
['es-NI', 'Nicaragua'],
['es-PA', 'Panamá'],
['es-PY', 'Paraguay'],
['es-PE', 'Perú'],
['es-PR', 'Puerto Rico'],
['es-DO', 'República Dominicana'],
['es-UY', 'Uruguay'],
['es-VE', 'Venezuela']],
['Euskara', ['eu-ES']],
['Français', ['fr-FR']],
['Galego', ['gl-ES']],
['Hrvatski', ['hr_HR']],
['IsiZulu', ['zu-ZA']],
['Íslenska', ['is-IS']],
['Italiano', ['it-IT', 'Italia'],
['it-CH', 'Svizzera']],
['Magyar', ['hu-HU']],
['Nederlands', ['nl-NL']],
['Norsk bokmål', ['nb-NO']],
['Polski', ['pl-PL']],
['Português', ['pt-BR', 'Brasil'],
['pt-PT', 'Portugal']],
['Română', ['ro-RO']],
['Slovenčina', ['sk-SK']],
['Suomi', ['fi-FI']],
['Svenska', ['sv-SE']],
['Türkçe', ['tr-TR']],
['български', ['bg-BG']],
['Pусский', ['ru-RU']],
['Српски', ['sr-RS']],
['한국어', ['ko-KR']],
['中文', ['cmn-Hans-CN', '普通话 (中国大陆)'],
['cmn-Hans-HK', '普通话 (香港)'],
['cmn-Hant-TW', '中文 (台灣)'],
['yue-Hant-HK', '粵語 (香港)']],
['日本語', ['ja-JP']],
['Lingua latīna', ['la']]];for (var i = 0; i < langs.length; i++) {
select_language.options[i] = new Option(langs[i][0], i);
}
select_language.selectedIndex = 6;
updateCountry();
select_dialect.selectedIndex = 6;
showInfo('info_start');function updateCountry() {
for (var i = select_dialect.options.length - 1; i >= 0; i--) {
select_dialect.remove(i);
}
var list = langs[select_language.selectedIndex];
for (var i = 1; i < list.length; i++) {
select_dialect.options.add(new Option(list[i][1], list[i][0]));
}
select_dialect.style.visibility = list[1].length == 1 ? 'hidden' : 'visible';
}var create_email = false;
var final_transcript = '';
var recognizing = false;
var ignore_onend;
var start_timestamp;
if (!('webkitSpeechRecognition' in window)) {
upgrade();
} else {
start_button.style.display = 'inline-block';
var recognition = new webkitSpeechRecognition();
recognition.continuous = true;
recognition.interimResults = true;recognition.onstart = function() {
recognizing = true;
showInfo('info_speak_now');
start_img.src = 'mic-animate.gif';
};recognition.onerror = function(event) {
if (event.error == 'no-speech') {
start_img.src = 'mic.gif';
showInfo('info_no_speech');
ignore_onend = true;
}
if (event.error == 'audio-capture') {
start_img.src = 'mic.gif';
showInfo('info_no_microphone');
ignore_onend = true;
}
if (event.error == 'not-allowed') {
if (event.timeStamp - start_timestamp < 100) {
showInfo('info_blocked');
} else {
showInfo('info_denied');
}
ignore_onend = true;
}
};recognition.onend = function() {
recognizing = false;
if (ignore_onend) {
return;
}
start_img.src = 'mic.gif';
if (!final_transcript) {
showInfo('info_start');
return;
}
showInfo('');
if (window.getSelection) {
window.getSelection().removeAllRanges();
var range = document.createRange();
range.selectNode(document.getElementById('final_span'));
window.getSelection().addRange(range);
}
if (create_email) {
create_email = false;
createEmail();
}
};recognition.onresult = function(event) {
var interim_transcript = '';
for (var i = event.resultIndex; i < event.results.length; ++i) {
if (event.results[i].isFinal) {
final_transcript += event.results[i][0].transcript;
} else {
interim_transcript += event.results[i][0].transcript;
}
}
final_transcript = capitalize(final_transcript);
final_span.innerHTML = linebreak(final_transcript);
interim_span.innerHTML = linebreak(interim_transcript);
var player=parent.GetPlayer();
player.SetVar("storyline_rec_result",final_transcript);
console.log(final_transcript);
if (final_transcript || interim_transcript) {
showButtons('inline-block');
}
};
}function upgrade() {
start_button.style.visibility = 'hidden';
showInfo('info_upgrade');
}var two_line = /\n\n/g;
var one_line = /\n/g;
function linebreak(s) {
return s.replace(two_line, '<p></p>').replace(one_line, '<br>');
}var first_char = /\S/;
function capitalize(s) {
return s.replace(first_char, function(m) { return m.toUpperCase(); });
}/*function createEmail() {
var n = final_transcript.indexOf('\n');
if (n < 0 || n >= 80) {
n = 40 + final_transcript.substring(40).indexOf(' ');
}
var subject = encodeURI(final_transcript.substring(0, n));
var body = encodeURI(final_transcript.substring(n + 1));
window.location.href = 'mailto:?subject=' + subject + '&body=' + body;
}*/function copyButton() {
if (recognizing) {
recognizing = false;
recognition.stop();
}
copy_button.style.display = 'none';
copy_info.style.display = 'inline-block';
showInfo('');
}function emailButton() {
if (recognizing) {
create_email = true;
recognizing = false;
recognition.stop();
} else {
createEmail();
}
email_button.style.display = 'none';
email_info.style.display = 'inline-block';
showInfo('');
}function startButton(event) {
if (recognizing) {
recognition.stop();
return;
}
final_transcript = '';
recognition.lang = select_dialect.value;
recognition.start();
ignore_onend = false;
final_span.innerHTML = '';
interim_span.innerHTML = '';
start_img.src = 'mic-slash.gif';
showInfo('info_allow');
showButtons('none');
start_timestamp = event.timeStamp;
}function showInfo(s) {
if (s) {
for (var child = info.firstChild; child; child = child.nextSibling) {
if (child.style) {
child.style.display = child.id == s ? 'inline' : 'none';
}
}
info.style.display = 'block';
} else {
info.style.display = 'none';
}
}var current_style;
function showButtons(style) {
if (style == current_style) {
return;
}
current_style = style;
copy_button.style.display = style;
email_button.style.display = style;
copy_info.style.display = 'none';
email_info.style.display = 'none';
}
</script>Somebody could help to know how to put a language as default?
What can I do to make it works in IOS?
