I came across the following demo created using Rapt Media (interactive HTML5 video). It is awesome and would love to see if Storyline is capable of doing something very similar.
I have spoken with a few SL developers, and they aren't 100% sure that SL can do exactly the same thing, but could possibly do something similar. SL is still limited.
Hi, Jason -- Thanks so much for sharing, and what a neat example! I'll defer to the community to share their ideas on how something similar could be accomplished, but you may also want to post this question over in our design related forum, Building Better Courses, for additional opportunities for discussion. :)
We'd like to see something very similar to the Rapt Media example done in SL. For example, a video plays and then animated button appear on top of the video at an exact part in a video. It pauses the video and you can click on one of the buttons that are overlaid on top of the video.
We also would like the video to play "inline" on the iPhone vs. appearing in full screen mode. Rapt did this on their site, so thinking there is a way to control the video from appearing in full-screen mode.
I'm not sold that Storyline can do what was shown in the Rapt demo. If someone can figure this out and shoot me a demo, I will be VERY impressed and might even consider hiring you as a consultant! :-)
I don't think the issue would be building it to work, as Jason said, it is stopping the video from going full screen. Some scripting might be required for that. It's easy enough to build the functionality of those video pages. Have to look into the scripting aspect more.
I think this is going to be really difficult on the iPhone I had a read around and found this on the developer site for apple
on iPhone and iPod touch, which are small screen devices, "Video is NOT presented within the Web Page"
and
Currently, Safari optimizes video presentation for the smaller screen on iPhone or iPod touch by playing video using the full screen—video controls appear when the screen is touched, and the video is scaled to fit the screen in portrait or landscape mode. Video is not presented within the webpage. The height and width attributes affect only the space allotted on the webpage, and the controls attribute is ignored. This is true only for Safari on devices with small screens. On Mac OS X, Windows, and iPad, Safari plays video inline, embedded in the webpage.
Inside an App you can disable this but not in Safari itself there may be a way using javascript but this will likely also mean hacking the HTML5 player.
I know I made a suggestion on another thread that you use Screenshots at specific points to look like it is pausing and then display the question and then jump to your next question, I just don't know how seamless this will look.
Right, this sells challenging to get video to play online on an iPhone, however, there HAS to be a way to code this in HTML5, JS, or, etc. If Rapt Media did it, then I'm sure it can be done in SL via coding.
Whoever can figure this out, will truly be a HERO :-) lol
Unfortunately, no way to override that behavior. There is a way to use script libraries to "fake it" in canvas by frame by framing the data into a canvas layer. From what I gather, it's resource intensive and would definitely hit performance on older hardware. Not sure if this is the way that Rapt is doing theirs.
There is a way to add something to the video tag (webkit-playsinline) if you're publishing to a web view. This saves the HTML to a different launch package that doesn't open in Safari by default. Also a pretty short road to "not what I want" in many cases.
Apple has another method they have used to defeat the behavior but it's worse than the mentions above. It involves cranking frames out to JPEG and bringing those in. That's a little silly but it's also something that I've done in the past for some applications with creative applications of Spritespin. Funny way to use these but it did work and I was able to control direction of playback and speed.
Getting any of that to work in Storyline would be a trick;) Unless this method of publishing inline video is something that is added to a future version... Any way you slice it, customization or changing the nature of the output package will be required.
If overlays aren't required, I have had success calling a slide change on video end. This causes the video to close when it's done playing and returns the user to the SL output.
Jason, I am with you on wanting interactive video. The key description of the issue with the iPhone is not that the video goes full screen, but that it plays video in the native player, leaving the browser and all of its overlays and scripting capability behind.
Steve, I came upon this thread while searching on "webkit-playsinline" because we tested an iPhone browser (not a popular one) that explicitly allows video to play inline with the HTML page. When we tried Storyline's player, the native player still launched, which reminded me that the "webkit-playsinline" attribute needs to be set on the video element to avoid leaving the browser context. So, I tried to modify the published output but the DOM is manipulated at run time, meaning that a really intrusive hack would be needed to put the attribute in place. It would be great if Articulate could add this otherwise innocuous (AFAIK) "webkit-playsinline" attribute to the video elements of the output in the event it's found to allow the Storyline HTML5 player to play video inline on iPhone. (Note: any iOS app that sets "allowsInlineMediaPlayback" to "YES" in its UIWebView (or more current WKWebView) should support inline video. More details here.)
BTW, we also are evaluating Rapt Media (prior to seeing mention here in the forums) but have so far found the level of interactivity that we want to achieve unattainable.
To Andre's post, I'll also pass along Fuisz Media (http://fuiszvideo.com/). If you go to that url and try some of their examples on your iPhone, you'll see that they (largely) work and don't invoke the native player. It would be great to at least have "webkit-playsinline" as a settable/toggleable option on each video object (if not as a default) so that we can unleash Storyline's (mobile) interactive video potential.
So is there anyway we can add code to the storyboard published files (JS or HTML5) to keep the iPhone from playing video in it's native player? I would love for a coder to attempt this and see if this can be cracked!
@Steve - you mentioned this might be able to be done, but could be complicated. Would you have time to play around and test this?
If this truly can't be done, then it looks like it will have to be added to Storyline 3. We need to share this with the beta folks! Mike and Tom are already aware of this, so hopefully it will be added :-)
Jason, as an aside. is there a confirmed Storyline 3 Beta? (I would have
hoped we would have been invited given the extensive work we've done to help
Articulate debug and enhance their product the past couple of years.)
Unfortunately, AFAIK and have been able to figure with testing, webkit-playsinline attribute additions to a video element will only prevent full-screen player in a UIWebview on iOS phones. In a plain old browser, it'll still launch in full screen. I don't think this is all fuisz is using to suppress the default behavior.
It does seem like Apple indicates that it should be possible here:
It's not ideal, but if it were an optional flag/toggle, it would still help
our cause (i.e., we'd be willing to write a shell native app that was
basically just a UI/WKWebView that launched our course[s]).
From my limited testing (using Fiddler as a proxy to view the iPhone's network communications), I was able to see that Rapt sends a series of images to the mobile browser. So they have sidestepped the restriction on inline video playback by not playing video in the traditional sense. (That is, video is a series of images, but "traditionally" video has been a self-contained file or byte stream, not a series of distinct files.) The real magic, I think, is in coordinating audio with the rapid display of the sequence of images. I imagine Fuisz works similarly. I would think that Articulate could use a similar technique in their product. Short of that, maybe the question is how to integrate one of the sequence-of-images providers' players into our Storyline projects.
Adding an image sequence player is pretty easy using a Web Object. To integrate with Storyline trigger events, you'll need to get creative. I've had success using Spritespin to bring in image sequences and play them as "video." In my case it was to simulate medical instrumentation. We needed to change the playback rate and affect the repeat. It's relatively simple to setup. However, it's not as size efficient as H.264 single file video.
Wish iOS phones would respect the inline video tag and not assume that all video must play fullscreen. This is the bigger issue... If they'd just respect the workings of the web, we wouldn't have Apple making the same mistakes Microsoft made years ago. MS broke the web and we're still dealing with the consequences ;)
I just came across your discussion because of my Google search. I'm developing a test/proof of concept website for a client and I'm struggling also with the inline video display for an iPhone. For my needs, I just need an iPhone solution. I'm displaying video on iPad and Android as needed. I just found this earlier and was wondering if it's a possible solution.
Wow, prior to today, the only inline iPhone video solution I had come across was a JavaScript-implemented H.264 decoder, which was super processor-intensive (decoding is usually offloaded to special hardware/chips). The one you linked above does work on iPhone and seems super light-weight. Not wanting to be duped into making design and implementation decisions on something that only "seems" to work, I dug deeper.
Not only can I not find anything fishy with "iphone-inline-video," but I found another one that is equally light-weight. This second one does not have minified code, so it is easier to look under the hood. Interestingly, the processor-intensive one I had looked at a year back now seems responsive, though they do not appear open-source.
The common theme is a workaround involving audio and video tags, kept in synch, with the video contents copied/projected onto a canvas element. I will definitely be looking into this more.
Need to dig into the first one. It works pretty well. The second one is a really clever workaround. I worry that playing video in an audio tag might not be a good long-term solution. Still, both of these seem like great short term fixes.
This is awesome! I just noticed this on YouTube the other day on my iPhone. Yay!
We're still on SL2, so will the videos play inline automatically or is there a code we have to add to each slide in SL that has a video to keep it from going full screen on the iOS?
27 Replies
Hi, Jason -- Thanks so much for sharing, and what a neat example! I'll defer to the community to share their ideas on how something similar could be accomplished, but you may also want to post this question over in our design related forum, Building Better Courses, for additional opportunities for discussion. :)
If it is possible, do you have a demo?
We'd like to see something very similar to the Rapt Media example done in SL. For example, a video plays and then animated button appear on top of the video at an exact part in a video. It pauses the video and you can click on one of the buttons that are overlaid on top of the video.
We also would like the video to play "inline" on the iPhone vs. appearing in full screen mode. Rapt did this on their site, so thinking there is a way to control the video from appearing in full-screen mode.
I'm not sold that Storyline can do what was shown in the Rapt demo. If someone can figure this out and shoot me a demo, I will be VERY impressed and might even consider hiring you as a consultant! :-)
I don't think the issue would be building it to work, as Jason said, it is stopping the video from going full screen. Some scripting might be required for that. It's easy enough to build the functionality of those video pages. Have to look into the scripting aspect more.
Example:
http://bit.ly/1ob6InH - still opens in full screen on iphone, otherwise works.
I think this is going to be really difficult on the iPhone I had a read around and found this on the developer site for apple
on iPhone and iPod touch, which are small screen devices, "Video is NOT presented within the Web Page"
and
Currently, Safari optimizes video presentation for the smaller screen on iPhone or iPod touch by playing video using the full screen—video controls appear when the screen is touched, and the video is scaled to fit the screen in portrait or landscape mode. Video is not presented within the webpage. The height and width attributes affect only the space allotted on the webpage, and the controls attribute is ignored. This is true only for Safari on devices with small screens. On Mac OS X, Windows, and iPad, Safari plays video inline, embedded in the webpage.
Inside an App you can disable this but not in Safari itself there may be a way using javascript but this will likely also mean hacking the HTML5 player.
I know I made a suggestion on another thread that you use Screenshots at specific points to look like it is pausing and then display the question and then jump to your next question, I just don't know how seamless this will look.
Thanks everyone for your input!
Right, this sells challenging to get video to play online on an iPhone, however, there HAS to be a way to code this in HTML5, JS, or, etc. If Rapt Media did it, then I'm sure it can be done in SL via coding.
Whoever can figure this out, will truly be a HERO :-) lol
Maybe we start a competition? :-)
Unfortunately, no way to override that behavior. There is a way to use script libraries to "fake it" in canvas by frame by framing the data into a canvas layer. From what I gather, it's resource intensive and would definitely hit performance on older hardware. Not sure if this is the way that Rapt is doing theirs.
There is a way to add something to the video tag (webkit-playsinline) if you're publishing to a web view. This saves the HTML to a different launch package that doesn't open in Safari by default. Also a pretty short road to "not what I want" in many cases.
Apple has another method they have used to defeat the behavior but it's worse than the mentions above. It involves cranking frames out to JPEG and bringing those in. That's a little silly but it's also something that I've done in the past for some applications with creative applications of Spritespin. Funny way to use these but it did work and I was able to control direction of playback and speed.
Getting any of that to work in Storyline would be a trick;) Unless this method of publishing inline video is something that is added to a future version... Any way you slice it, customization or changing the nature of the output package will be required.
If overlays aren't required, I have had success calling a slide change on video end. This causes the video to close when it's done playing and returns the user to the SL output.
Jason, I am with you on wanting interactive video. The key description of the issue with the iPhone is not that the video goes full screen, but that it plays video in the native player, leaving the browser and all of its overlays and scripting capability behind.
Steve, I came upon this thread while searching on "webkit-playsinline" because we tested an iPhone browser (not a popular one) that explicitly allows video to play inline with the HTML page. When we tried Storyline's player, the native player still launched, which reminded me that the "webkit-playsinline" attribute needs to be set on the video element to avoid leaving the browser context. So, I tried to modify the published output but the DOM is manipulated at run time, meaning that a really intrusive hack would be needed to put the attribute in place. It would be great if Articulate could add this otherwise innocuous (AFAIK) "webkit-playsinline" attribute to the video elements of the output in the event it's found to allow the Storyline HTML5 player to play video inline on iPhone. (Note: any iOS app that sets "allowsInlineMediaPlayback" to "YES" in its UIWebView (or more current WKWebView) should support inline video. More details here.)
BTW, we also are evaluating Rapt Media (prior to seeing mention here in the forums) but have so far found the level of interactivity that we want to achieve unattainable.
-Andre
To Andre's post, I'll also pass along Fuisz Media (http://fuiszvideo.com/). If you go to that url and try some of their examples on your iPhone, you'll see that they (largely) work and don't invoke the native player. It would be great to at least have "webkit-playsinline" as a settable/toggleable option on each video object (if not as a default) so that we can unleash Storyline's (mobile) interactive video potential.
I agree with you Andre!
@Tony - thanks for sharing that link!
So is there anyway we can add code to the storyboard published files (JS or HTML5) to keep the iPhone from playing video in it's native player? I would love for a coder to attempt this and see if this can be cracked!
@Steve - you mentioned this might be able to be done, but could be complicated. Would you have time to play around and test this?
If this truly can't be done, then it looks like it will have to be added to Storyline 3. We need to share this with the beta folks! Mike and Tom are already aware of this, so hopefully it will be added :-)
Jason, as an aside. is there a confirmed Storyline 3 Beta? (I would have
hoped we would have been invited given the extensive work we've done to help
Articulate debug and enhance their product the past couple of years.)
This post was removed by the author
Unfortunately, AFAIK and have been able to figure with testing, webkit-playsinline attribute additions to a video element will only prevent full-screen player in a UIWebview on iOS phones. In a plain old browser, it'll still launch in full screen. I don't think this is all fuisz is using to suppress the default behavior.
It does seem like Apple indicates that it should be possible here:
https://developer.apple.com/library/iad/documentation/UserExperience/Conceptual/iAdJSProgGuide/PlayingVideosinAds/PlayingVideosinAds.html
However, I haven't been able to make it work outside of a UIWebview.
It's not ideal, but if it were an optional flag/toggle, it would still help
our cause (i.e., we'd be willing to write a shell native app that was
basically just a UI/WKWebView that launched our course[s]).
Thanks Steve!
So you tried the Apple fix, but it only works in UIWebview?
I really wonder how RaptMedia and Fuisz and doing this? I really hope we can all solve this issue! :-)
Maybe Apple will eventually change this with the iPhone 7.
This works perfectly on my iPhone!! There MUST be a way for us to do this :-)
https://experiences.fuiszmedia.com/lincoln-website/ios.html
Jason,
From my limited testing (using Fiddler as a proxy to view the iPhone's network communications), I was able to see that Rapt sends a series of images to the mobile browser. So they have sidestepped the restriction on inline video playback by not playing video in the traditional sense. (That is, video is a series of images, but "traditionally" video has been a self-contained file or byte stream, not a series of distinct files.) The real magic, I think, is in coordinating audio with the rapid display of the sequence of images. I imagine Fuisz works similarly. I would think that Articulate could use a similar technique in their product. Short of that, maybe the question is how to integrate one of the sequence-of-images providers' players into our Storyline projects.
Adding an image sequence player is pretty easy using a Web Object. To integrate with Storyline trigger events, you'll need to get creative. I've had success using Spritespin to bring in image sequences and play them as "video." In my case it was to simulate medical instrumentation. We needed to change the playback rate and affect the repeat. It's relatively simple to setup. However, it's not as size efficient as H.264 single file video.
Wish iOS phones would respect the inline video tag and not assume that all video must play fullscreen. This is the bigger issue... If they'd just respect the workings of the web, we wouldn't have Apple making the same mistakes Microsoft made years ago. MS broke the web and we're still dealing with the consequences ;)
Interesting. I think several other companies are doing this.
It would be nice to simply get a video to play "inline" on the iPhone :-)
As Steve mentioned, Apple allows you to do this for playing videos in ads.
I just wonder how we could apply their code to a PUB Storyline project...
Hey all,
I just came across your discussion because of my Google search. I'm developing a test/proof of concept website for a client and I'm struggling also with the inline video display for an iPhone. For my needs, I just need an iPhone solution. I'm displaying video on iPad and Android as needed. I just found this earlier and was wondering if it's a possible solution.
I'm not savvy enough with Javascript yet to implement it for my need, but maybe it's a viable solution. I found this link on StackExchange and it led me to Github --> https://github.com/bfred-it/iphone-inline-video/blob/gh-pages/README.md#usage .
Maybe some of you can weigh in on this.
Wow, prior to today, the only inline iPhone video solution I had come across was a JavaScript-implemented H.264 decoder, which was super processor-intensive (decoding is usually offloaded to special hardware/chips). The one you linked above does work on iPhone and seems super light-weight. Not wanting to be duped into making design and implementation decisions on something that only "seems" to work, I dug deeper.
Not only can I not find anything fishy with "iphone-inline-video," but I found another one that is equally light-weight. This second one does not have minified code, so it is easier to look under the hood. Interestingly, the processor-intensive one I had looked at a year back now seems responsive, though they do not appear open-source.
The common theme is a workaround involving audio and video tags, kept in synch, with the video contents copied/projected onto a canvas element. I will definitely be looking into this more.
Need to dig into the first one. It works pretty well. The second one is a really clever workaround. I worry that playing video in an audio tag might not be a good long-term solution. Still, both of these seem like great short term fixes.
Hey all,
Just wanted to update you to let you know that the latest version of iOS 10 now allows for inline video.
https://articulate.com/support/article/Articulate-Storyline-360-Version-History
Mike
Hey Mike,
This is awesome! I just noticed this on YouTube the other day on my iPhone. Yay!
We're still on SL2, so will the videos play inline automatically or is there a code we have to add to each slide in SL that has a video to keep it from going full screen on the iOS?
Hey Jason,
In SL360 all you have to do is publish out your content and the videos will play inline. SL2 isn't supported.
Mike
This post was removed by the author
This discussion is closed. You can start a new discussion or contact Articulate Support.