Forum Discussion
Drawing Annotation on Storyline Slide
Demo: https://360.articulate.com/review/content/518383b2-1161-408d-b9f5-adb9a6f57a11/review
Inspired by code discussed on:
https://img.ly/blog/how-to-draw-on-an-image-with-javascript/
Using Charts in your Storyline | Articulate - Community
About
This is a brief example of using an HTML canvas element to enable annotation on a Storyline slide. The example displays an image on a slide, sets a few variables, and the accessibility tags of a some slide objects. It then triggers a JavaScript trigger to create a canvas over the image, and then watches the mouse buttons and movement to allow drawing on the canvas.
How it works
A canvas element is created, filled with the specified base image, and inserted below a small rectangle (canvasAnchor) that is the same width as and placed directly above the image.
Another rectangle (canvasClickArea) overlays and is sized to match the image. This is the area that allows drawing on the canvas (where the mouse is watched).
Brush width and color can be controlled. The drawing can be cleared. It also resizes with the slide.
To improve
The events to watch the mouse and clear button should be better handled to allow removal when a new canvas is created.
A mechanism to allow a blank base (clear) should be reinstated. Right now it just relies on the use of the initial image from the slide. Both options have their uses.
Since the canvas is a raster image, resizing to very small and then very large results in poor image quality.
The image can be extracted from the canvas. This could be saved or printed.
More drawing options are available with the canvas element.
Credit: X-ray images from https://learningradiology.com/
Offcourse i cannot resist a challenge. So to detect whether a user's drawing is valid you need to convert the image to base64. Do try it with 2 similar but different images.
https://www.base64-image.de/
Here you can create base64 encoding.
Then you compare the outputs like here.
https://codepen.io/rosca/pen/NWLqLJ
And you get a result in amounts of pixels that differ. If you would want this to work in Storyline, you need to do some testruns...how much pixels the endresult may vary. I made 2 images, i with an ok-selection...that varied 750 pixels, and one completely off..that varied > 8000 pixels.
If time permits i make a demo of this.I recently started looking at this project again. I adapted the annotation example I posted earlier and added some test code for assessing the user-drawn input versus a couple of predetermined keys. This example used Resemble.js to compare the images. It has a numer of options that could be useful for finetuning comparisons.
Demo- https://360.articulate.com/review/content/868a0b98-e757-4355-909f-85d09337f2eb/review
The script is a hot mess right now and it is a bit slow due to a lot of extraneous debugging code, but it does a fair job at assigning a score to the user input. This example is contrived, but think of somethng like asking someone to draw in the missing path (in this case, the Arkansas River).
When I get a chance to clean this up, I will post a more complete example.
Good example math. If we wanted to incorporate that into this canvas drawing demo, it might be good to just forego the base image and just use the user drawn components to compare to a similar key overlay. This way, we could make generic keys that could be more easily applied to multiple image scenarios.
Indeed. I do think i have parts for the code for that available in some older project 😀
Nice Nathaniel, not only the endresult, but also the way you describe the process and things to improve upon.
What challenges me most, is...could we think of a way to detect what the user has drawn...and then compare that with the correct answer and give him/her clues on what should be drawn. And make the range somewhat fuzzy... so are you close... or not... :-)
One approach for that could be something like this.
https://codepen.io/lavrton/pen/OWKYMr
Invisible canvas elements, when the user clicks/draws on enough of them you know the correct area is drawn.
And this library might do the trick too.
https://observablehq.com/@mourner/pixelmatch-demoHi Nathan! Wow, you are amazing, and thank you for taking the time to figure this out for me. This is EXACTLY what I was trying to figure out for my current project!
