Extending Articulate Storyline with the SDKs - a Case Study
My boss came to me with a programming job. He wanted to be able to extend the functionality of PowerPoint-based teaching tools for teachers with the power of Articulate Storyline. He knew that Storyline can import PowerPoint files, and that a developer could add much interactivity to the lessons by using built-in and extended functionality in Storyline.
He initially asked me to add the ability to:
- Add buttons to show/hide the main menu, and to show the main Lesson Index.
- Add a separate window in a separate screen for Notes.
- Make the program open up two windows: the main window was to be shown typically in an overhead projector, the other was to be on the teacher’s laptop and contain the Notes window.
- The main screen of the program would need to automatically open up to use the largest amount of screen real estate, while still adhering to the correctly published aspect ratio.
- Open up PDFs and other files types, in their native format.
- Add a custom timer.
- Add an Annotation tool, so the teacher could draw upon the surface of a slide.
- He also gave me a couple of things I didn’t implement because they would have dramatically expanded the scope of the program.
So I went ahead with this mandate. The only way I could attempt to do these things within the confines of what I knew at the time, was to build an application with a Visual Basic executable shell, and use a linked Flash Shockwave file as the main window. I suppose I could have gone in the HTML5 direction, but I knew how to get the job done (more or less) with the VB/Flash version. I also knew that I would have to build a few things in the Articulate SDK in order to build the Annotation tool and the custom Timer. These further guided my path toward the VB/Flash version, because the SDK is only and forevermore only usable with Flash.
The Articulate SDK
The SDK is a set of classes that, when published in Flash, creates a new frame.swf file. This frame.swf takes the place of your Storyline player, and allows you to expand its functionality. You copy this new version of the frame.swf over the old frame.swf file that is in your “story_content” sub-folder in order to test the new functionality.
The most important file to edit is Frame.as. This file is where access to the “root” of the Flash code lives.
Here are the files you need to open and edit, at a bare minimum, in order to add new functionality to your Storyline project using the SDK:
1. Open the project file (customFrame/CustomFrame.xfl).
2. Open up Frame.as (src/customframe/Frame.as) for editing.
3. Compile the project, and it will generate a file called frame.swf. This file ends up at the root of the folder structure of your SDK.
4. Copy this frame.swf file into the “story_content” folder of your published Storyline protect, over your old frame.swf file. (You should probably rename the old frame.swf just in case you really mess up the new frame.swf.)
5. Test & repeat.
The frame.xml file (within the story_content folder) is pretty important too. Much of the stylistic settings are here, plus all the slide & notes text. You can set certain CSS-type default settings here, like whether or not the menu defaults to open or closed, or the menu text wraps etc. Slide and notes text are also contained within this file. I'll talk about the frame.xml file more below.
Flash / VB communication and KeyDown Events
All controls in this combo-program are created on the Flash side; some are internal commands to Actionscript code, and some are for the VB shell.
The published Flash story.swf file is externally loaded into an ActiveX control in my Visual Basic code. Since the ActiveX control was created at an earlier stage of Flash development, the interface to external control at that time was in using the fscommand functionality, rather than the later ExternalInterface structure. So when the Flash needs to communicate with the VB shell, it uses fscommand. When the VB shell needs to communicate with the embedded Flash control, it uses the VBControlname.CallFunction XML string mechanism, where VBControlname is the name of the Visual Basic control on the form.
The “fscommand” communication between Flash and HTML is fairly well documented. The “AxShockwaveFlash.CallFunction” XML string communication, where Visual Basic communicates with the embedded Flash ActiveX SWF file, is really not very well documented at all. [Within this document, I refer to the ActiveX object with the VB name I defined it with, “AxShockwaveFlash”.] Below is an example of how I send keystrokes from the VB shell to the embedded ActiveX Flash SWF.
First, I create a function in Flash called ReceiveKeystroke. I give it one parameter, a string, which will represent what key the user presses. In my case I am looking for “a”, “,”(<), and “.”(>), to help simulate PowerPoint functionality:
"a" - reveals all bullets on a slide
“<” - goes back one bullet on a slide
“>” – goes forward one bullet on a slide
In Visual Basic, you create a function that handles the
KeyDown event. Within that function, you start the communication with "AxShockwaveFlash" by creating an XML string based upon these three strings:
You eventually concatenate them into
Here’s how they are declared:
Dim TheFrontString As String = "<invoke name='ReceiveKeystroke' returntype='xml'><arguments><string>"
Dim TheMiddleString As String = ""
Dim TheEndString As String = "</string></arguments></invoke>"
Dim TheWholeString As String = “”
If your VB
KeyDown function has found that an “a” key has been pressed,
TheMiddleString gets set to “a”. You then process this bit of code at the end of the function:
If TheMiddleString IsNot "" Then
TheWholeString = TheFrontString & TheMiddleString & TheEndString
This calls the Flash Function ReceiveKeystroke with the parameter “a”. You then handle this information in Flash however you need to. The Flash function needs to be public, not private, and it needs to be contained within the frame.as file (see below.)
Program Launching and Sizing
When the program opens, I need to provide that it opens with automatic re-sizing, to get the screen as large as possible within the default aspect ratio. I made the automatic screen re-sizing look for about 10 popular screen dimensions, then re-size the screen to fit best. In testing, it was desired that I add a few more control keys in order for the teacher to tweak out the size and position of the main screen – in case the automatic re-sizing mentioned above didn’t work to their satisfaction. Here's a listing of these keystrokes and how they function in the program:
- The arrow keys (up, down, left, right) are used to move the main screen around for best viewing.
- <Shift-Up Arrow> and <Shift-Down Arrow> are used to scale the screen to be taller or shorter.
- <Shift-Left Arrow> and <Shift-Right Arrow> are used to scale the screen wider or thinner.
- <Esc> (the Escape key) is used to reset the dimensions back to their original, automatically-sized dimensions.
NOTE: These keys are also handled by the function handling
KeyDown events (outlined above.)
I also built in a way to save the dimensions to an XML file (called Dimensions.xml). So when the teacher opens up an extended Storyline program, this configuration file gets written with the program’s top, left, width, and height settings. If the teacher tweaks the screen settings by way of the arrow keys or the Shift-arrows, these are saved too.
The obvious advantages of saving the screen dimensions are:
- Once set, the screen will open up the same way every time the teacher opens it.
- If other Storyline apps have the same aspect ratio (which they should), a teacher could copy the old Dimensions.xml to the folder of the new app, and it should open the main screen without needing it to be tweaked.
Program Main Menu
My program’s main menu is contained on a drop-down custom menu palette from the top of the screen. If you roll over the thin edge of the bottom of the palette that peaks out at the top of the screen, it will descend so that you can click on these controls (from left to right):
- Slide Index – brings up the Main Menu on the left side
- Lesson Index – shifts program to the first Lesson index page
- Resources – opens up one of many listed PDF files, located in a sub-folder
- Files – opens up a File picker
- Timer – opens up a new Timer control, with a limited amount of controls
- Annotation – brings up an Annotation control
- Minimize (right corner) – minimize the app
- Exit (right corner) – close the app
This extended app never uses any of the controls found on the Storyline player. When you use the SDK, editing and publishing in Flash, you create a new frame.swf file. This file then gets copied over the frame.swf file that is in your “story_content” sub-folder. I believe the frame.swf is the Flash version of the Storyline player. [I could be wrong.]
The Main Menu got moved on to the main screen (by way of the SDK), rather than being in the side panel in the normal Storyline player interface. I made it appear under the Slide Index button, and made it follow the movement of the upper custom menu palette as it moved. I also did some branding color to the menu from within Flash itself.
The Lesson Index is made to appear using extended functionality of the SDK and an XML file (LessonIndex.xml). You can’t directly address a single slide in Storyline at present with a function such as
goToSlide(whichSlide). But that functionality is buried within the SDK.
Okay, so I can address a single slide now. But how can I address the Lesson index slide in particular?
I found that slide information is contained within the compiled file frame.xml within the “story_content” sub-folder. The slides in a Storyline project are kept in
<slidelink> nodes within this XML file. So I found the Lesson index
<slidelink> by its title and position (it is near the top of all the slidelinks.) I copied the
slideid property of the Lesson index slidelink (it is a string of characters within double quotes) and put it in an XML file called LessonIndex.xml. [Hexheads note: you leave off the "_player." front part of the string to get this to work correctly.] I then have the published Articulate SDK read in the value from LessonIndex.xml at run-time. So when a teacher clicks the Lesson Index button, the (custom) exposed function
goToSlide(whichSlide) is given the Lesson Index slide parameter read in at run-time, and the program shows the Lesson Index.
A couple of more notes about the frame.xml file:
- When the file is published, it is written in one continuous horizontal string of characters – which is not easy to read if you like vertically-oriented text. If you are going to work on the file, use a good text editor to do a search-and replace on “><” and put a newline between the less-than and greater-than signs. It won’t affect your file, and it will make it easier to read.
- Notes for slides are also kept in this file. You can edit these notes, as well as the titles of slides, with a decent text editor.
- Also, keep a copy of the frame.xml file handy outside of the publishing folder structure just in case, especially if you heavily edit the file. (It's always best to work from a copy.)
The Notes.exe app is a separate executable file that gets launched when the main application starts up, and uses these notes, and the slide titles, to populate a couple of VB TextBoxes. So, as each slide comes up, its slide title and notes are sent to the Notes app for display. The teacher can see the notes on their laptop screen, while showing the class the main application on the large projection screen.
(This is a partial view of the Notes.exe sub-application.)
The Resources menu, when clicked shows a sub-menu with a number of buttons oriented vertically. This sub-menu is created within the Articulate SDK, and builds the list of buttons dynamically, based upon an XML file called Resources.xml. This XML file keeps links to external PDF files, located in a subfolder called “Resources”, and also keeps labels to these links for the buttons (shorter and easier to read that file names.)
The Files menu opens up a file picker. You can then launch any file in its native format.
The custom Timer was interesting. The spec came down that:
- The timer needed these settings:
- 5 minutes,
- 10 minutes, and
- 15 minutes,
- And of course a Start button.
- The countdown screen needed to show the timer countdown, and have a couple of buttons that would:
- Add an extra minute to the timer
- Close the timer
Here, UI concerns pop up: how much control do we give a user? This philosophical issue is not meant to be covered here in great depth. However, in this case the teacher needs just a little control to do all the things they need to do. Rather than give a teacher unlimited settings for the Timer, we settled on just the three time settings, and these were enough.
The Annotation control also used this philosophy in places.
The drawing control palette gave the user these tools:
- A color picker to choose what color to draw
- A checkbox to turn on and off “transparency” – actually, it was either full on or set to about 15%
- Three radio buttons to choose the width of the line
- And six buttons to choose how to draw:
- Hilite (the default setting – yellow, transparent, thick, and horizontally oriented)
- Box (with the “crawling ants” dimension box showing on mouse down)
- Ellipse (ditto on the “ants”)
- An Erase button
The palette has a minimize, a maximize, and a close button. If you draw on the page, and click the close button, the drawn image will be wiped clean and the control palette will disappear. (There is no Undo – the only undo is Erase.)
This took a bit of time in that I had to create new classes for the SDK to hook on to. In fact, I added many classes to this SDK in order to get it to bend to my will. For those interested, when the Annotate control was engaged:
- I show the control panel in a minimized form. I also make visual a very-lightly shaded drawing surface (movieClip) opver the surface of the slide. I use a combination of the drawing API and a timing movieClip to draw lines and forms on this surface with the mouse.
- When the control palette initially appears, it is in the default Hilite mode. To access the other controls, you need to click the maximize button, which brings up the color picker etc.
- I put in a max/min button system on this control because the palette does take up some bottom screen real estate; a teacher might want to just use the default hilite, or change the setting and get the palette out of the way. Again, I was addressing the UI and the best way to get the most power for the teacher without getting in the way.
Notes on the two SDKS
There is probably a lot more I could discuss on the building of this project & the classes I created.
I initially built this using the 1.0 SDK. We only used this codebase for one project built in Storyline 1.0. When Storyline 2.0 came out, I discovered that my code base didn’t work in the new format. The biggest reason it didn’t work is that Articulate removed the Flash controls, such as variations of the Button class and the ColorPicker, that I had relied upon in the original build.
After considerable study, I ended up rebuilding the 2.0 version of my codebase with the Articulate Storyline 2.0 SDK. And I had to recreate all my buttons and the ColorPicker control from scratch. Luckily, my standard interface buttons, which were built with variations of the SimpleButton class, still worked fine from the 1.0 version. And much of the older code still worked. I had to build radio buttons and check boxes, using the movieClip drawing API (much like the Canvas object in HTML5) to building upon the SimpleButton classes. The ColorPicker class wasn’t as complicated as I had originally thought, once I took a screen shot and saw how the RGB colors were laid out in bit-wise fashion (#000000, #003300, #006600 etc. up to #FFFFFF.)
My Visual Basic code was not affected by this change.
Articulate Storyline SDKs are provided as a way to open up the Powerpoint-on-steroids eLearning program with even more controls and functionality. Because it opens up only one avenue of the program to mess with – the Flash side, it should only be used in cases where the Flash published version is the only version you need. Hopefully, we can have some of our feature requests realized in future versions of this product, and expand its cross-published versatility without having to customize it in one direction only.