Storyline under a Microscope: An update from the CTO

Over the past three weeks, the engineering team put Storyline under a microscope. We dug into bugs, dug into our data, and heard a ton of direct feedback to sharpen our interpretation of where things stand and where they need to be.

\n

There were a lot of positive signs that we already knew. For example, nearly a million courses were created with Storyline last year. But to be honest, we found things we didn’t like and were dismayed our heroes had to call us out to look deeper.

\n

As an example of what we found, our error reporting shows us that 3% of application sessions hit an error. Based on some quick math, those errors add up to 3,000+ hours last year of course creator time spent facing errors. That was one example that surprised us. The other was “the state of defects” including the age of bugs, the number of outstanding defects reported by 5 or more customers, and the number of “bug workarounds” the community and our eLearning Heroes team come up with everyday.

\n

As an engineering team, we’ve been very focused on new features, on new releases, and we relied on our QA and support partners to work directly with you all to keep us paying attention to the right things and keep our quality on track. But by being separated from our customers, we were missing the full picture.

\n

We are committed to do the work to earn back your trust. We recognize that in our focus to build new capabilities, we took our eye off the foundations that matter to your success every day.

\n

So here are the top three things we learned and what we plan to do about it.

\n

Quality First: We must prioritize Storyline quality over everything else.

\n

First, we commit to prioritize quality over features. We recognize more features don’t help you succeed when you’re encountering paper cuts on a regular basis. Quality comes first. We’ve still got exciting new features on the horizon but we are doubling down on quality first.

\n

Second, we're changing our test and release cycle. The team operates on a monthly release cycle, with the final week focused on validation. We release a beta build to a dedicated group of customers on a weekly basis so they can test and report any issues to our team. Our Engineering, QA, and Support teams have previously operated in separate yet parallel tracks, but we’ve moved to a partnership where they can lean on each other to learn, test, and focus on customer needs. The next few releases will focus on quality and deepening these partnerships to create a better Storyline.
Third, we will source validation scenarios from the community. We heard from a number of you who say you wait months before upgrading Storyline and do extensive validation in advance. We will source validation scenarios from the community to include directly in our testing.
Fourth, we will invest in Storyline’s foundations. We are doing a deep dive into 64-bit support for Storyline and will report back with a plan. We had taken the stance that adding more capabilities to Storyline was job #1. So we were focused on delivering Storyline updates, along with quality updates, every month instead of disappearing into a cave on a long infrastructure project where the value of the impact is hard to measure until it is done. We are committed to investing in Storyline’s foundations so expect to hear more soon.

\n

Transparency: We need to be more transparent about how we are prioritizing our work.

\n

First priority is to reduce the error rate to below 1%. You can’t build courses, if you keep running into errors. In release 74, we fixed the top 18% of those errors that generated support cases but there is a lot more work to be done so those errors don’t escape our tests. We’ll report back monthly to update how we are doing on this critical metric.
Second we prioritized 35 older issues impacting five or more customers. We reviewed nearly 300 issues open over the life of Storyline. 35 of them are impacting five or more customers that were not at the top of our list. We are prioritizing those now and shipped a number of fixes for those in Release 74. Previously we focused our calories almost exclusively on more recent issues. So we’ve dedicated more engineers to focus on quality and renewed our focus on fixing older issues. As we work through those issues, we will share details.
Third, we are cleaning up our bug database. As a bug ages, finding the issue and fixing it without impacting something else becomes increasingly complex. When we reviewed all open issues, we found issues over 3 years old that affected one customer and had a workaround. We aren’t going to fix those issues. All defects are a problem and we don’t mean to imply we don’t care about quality. We need to focus our team on what impacts the community the most, and report back to those customers transparently about what we won’t fix instead of pretending we might some day.

\n

Communication: We need to communicate proactively instead of reactively.

\n

First, we will commit to monthly updates to the community from our Engineering team. We are working on the format and content to make it valuable, but to start we will commit to providing a “State of Storyline” monthly so you have the opportunity to know and react to what we are focused on. Quality is our number one priority right now and we are looking for measurable improvements that make a difference to you.
Second, our engineering team and I commit to working directly with all of you. Our engineers and engineering leaders have kept a distance from you all by accident. We’ve been focused on building and relied on our partners inside Articulate to be the “voice of our customers” rather than listening directly. So this is a commitment for us to work better together directly. We appreciate the time, energy, and passion you all bring to Storyline and we commit to make that worth it.

\n

Storyline 360 is our flagship product and is how you help your employees, your customers, and your companies to learn and grow everyday. We have a lot of exciting things planned this year but agree that quality must come first. If you’re interested in joining our beta program or sharing your teams’ Storyline validation scenarios, please email beta@articulate.com.

Our team will be updating this thread directly each month to keep you aware of our progress. To share feedback on our process, our plans, and our areas of focus please open a new thread or reach out to the support team so they can connect us directly.

\n

We are committed to more transparency and proactive communication directly with the engineering team because honestly we are only here because of you.

Over the past three weeks, the engineering team put Storyline under a microscope. We dug into bugs, dug into our data, and heard a ton of direct feedback to sharpen our interpretation of where things stand and where they need to be.

\n

There were a lot of positive signs that we already knew. For example, nearly a million courses were created with Storyline last year. But to be honest, we found things we didn’t like and were dismayed our heroes had to call us out to look deeper.

\n

As an example of what we found, our error reporting shows us that 3% of application sessions hit an error. Based on some quick math, those errors add up to 3,000+ hours last year of course creator time spent facing errors. That was one example that surprised us. The other was “the state of defects” including the age of bugs, the number of outstanding defects reported by 5 or more customers, and the number of “bug workarounds” the community and our eLearning Heroes team come up with everyday.

\n

As an engineering team, we’ve been very focused on new features, on new releases, and we relied on our QA and support partners to work directly with you all to keep us paying attention to the right things and keep our quality on track. But by being separated from our customers, we were missing the full picture.

\n

We are committed to do the work to earn back your trust. We recognize that in our focus to build new capabilities, we took our eye off the foundations that matter to your success every day.

\n

So here are the top three things we learned and what we plan to do about it.

\n

Quality First: We must prioritize Storyline quality over everything else.

\n

First, we commit to prioritize quality over features. We recognize more features don’t help you succeed when you’re encountering paper cuts on a regular basis. Quality comes first. We’ve still got exciting new features on the horizon but we are doubling down on quality first.

\n

Second, we're changing our test and release cycle. The team operates on a monthly release cycle, with the final week focused on validation. We release a beta build to a dedicated group of customers on a weekly basis so they can test and report any issues to our team. Our Engineering, QA, and Support teams have previously operated in separate yet parallel tracks, but we’ve moved to a partnership where they can lean on each other to learn, test, and focus on customer needs. The next few releases will focus on quality and deepening these partnerships to create a better Storyline.
Third, we will source validation scenarios from the community. We heard from a number of you who say you wait months before upgrading Storyline and do extensive validation in advance. We will source validation scenarios from the community to include directly in our testing.
Fourth, we will invest in Storyline’s foundations. We are doing a deep dive into 64-bit support for Storyline and will report back with a plan. We had taken the stance that adding more capabilities to Storyline was job #1. So we were focused on delivering Storyline updates, along with quality updates, every month instead of disappearing into a cave on a long infrastructure project where the value of the impact is hard to measure until it is done. We are committed to investing in Storyline’s foundations so expect to hear more soon.

\n

Transparency: We need to be more transparent about how we are prioritizing our work.

\n

First priority is to reduce the error rate to below 1%. You can’t build courses, if you keep running into errors. In release 74, we fixed the top 18% of those errors that generated support cases but there is a lot more work to be done so those errors don’t escape our tests. We’ll report back monthly to update how we are doing on this critical metric.
Second we prioritized 35 older issues impacting five or more customers. We reviewed nearly 300 issues open over the life of Storyline. 35 of them are impacting five or more customers that were not at the top of our list. We are prioritizing those now and shipped a number of fixes for those in Release 74. Previously we focused our calories almost exclusively on more recent issues. So we’ve dedicated more engineers to focus on quality and renewed our focus on fixing older issues. As we work through those issues, we will share details.
Third, we are cleaning up our bug database. As a bug ages, finding the issue and fixing it without impacting something else becomes increasingly complex. When we reviewed all open issues, we found issues over 3 years old that affected one customer and had a workaround. We aren’t going to fix those issues. All defects are a problem and we don’t mean to imply we don’t care about quality. We need to focus our team on what impacts the community the most, and report back to those customers transparently about what we won’t fix instead of pretending we might some day.

\n

Communication: We need to communicate proactively instead of reactively.

\n

First, we will commit to monthly updates to the community from our Engineering team. We are working on the format and content to make it valuable, but to start we will commit to providing a “State of Storyline” monthly so you have the opportunity to know and react to what we are focused on. Quality is our number one priority right now and we are looking for measurable improvements that make a difference to you.
Second, our engineering team and I commit to working directly with all of you. Our engineers and engineering leaders have kept a distance from you all by accident. We’ve been focused on building and relied on our partners inside Articulate to be the “voice of our customers” rather than listening directly. So this is a commitment for us to work better together directly. We appreciate the time, energy, and passion you all bring to Storyline and we commit to make that worth it.

\n

Storyline 360 is our flagship product and is how you help your employees, your customers, and your companies to learn and grow everyday. We have a lot of exciting things planned this year but agree that quality must come first. If you’re interested in joining our beta program or sharing your teams’ Storyline validation scenarios, please email beta@articulate.com.

Our team will be updating this thread directly each month to keep you aware of our progress. To share feedback on our process, our plans, and our areas of focus please open a new thread or reach out to the support team so they can connect us directly.

\n

We are committed to more transparency and proactive communication directly with the engineering team because honestly we are only here because of you.

Welcome to the June 2023 “State of Storyline” update. This month I’d like to touch on our application error rate, customer reported issues, and pre-release quality assurance as well as share our progress on creating a 64-bit version of Storyline 360.

\n

Application Error Rate

\n

In Update 76 of Storyline 360 we addressed 12 of the errors most commonly encountered by users. These errors affected areas such as rendering audio waveforms on the timeline, encoding video, and writing files to disk at publish time.

\n

Update 76 was released at the end of May and we’re currently monitoring our error reports as Storyline 360 users adopt the update. It typically takes 4 weeks for a new Storyline 360 version to be adopted widely enough to gather useful insights from this data, so we’ll share the impact these fixes had on our metrics in next month’s update.

\n

Customer Reported Issues

\n

Over the last month we have continued prioritizing bugs affecting 5 or more customer accounts. When last month’s update was published we were tracking 39 such bugs. Update 76 of Storyline 360 addressed 17 of these affecting areas such as text entry word wrapping behavior, hyperlinks in animated text, and importing tables from PowerPoint presentations.

\n

We also addressed several other bugs that did not meet the criteria of impacting 5 or more customer accounts but were clearly still causing customer pain. These bugs affected areas like the new Background Audio feature, the Storyline 360 window failing to restore after being minimized, and the timeline play/pause/stop buttons becoming disabled unpredictably.

\n

As of this writing we are tracking 23 bugs affecting 5 or more customer accounts and remain committed to continuing this work until they are resolved.

\n

Pre-release Quality Assurance

\n

We’ve continued making changes to our test and release cycles in an effort to improve the quality of each Storyline 360 release. This month I want to highlight a couple of areas that we’ve been focusing on recently:

\n

Release Validation: New versions of Storyline 360 are released on a monthly basis, which means we typically work in 4 week release cycles. In the past, some portion of the Storyline 360 engineering team would spend the last week of each release cycle testing and validating the new version to ensure it’s ready for release. As we’ve taken a critical look at how we approach quality in Storyline it has quickly become clear that having only a portion of the team participate in this release validation effort is not adequate. Starting with Update 76 of Storyline 360 we’ve had the entire team work together on pre-release validation. Having the whole team participate will not only help us catch more issues prior to release, but will also have us working with the application from a customer perspective more frequently. We’ve found that using Storyline in a manner similar to our customers helps us uncover minor bugs and usability issues that don’t tend to get a lot of attention through the normal bug triage and prioritization process.
Test Automation: We employ a suite of automated tests to continuously validate the stability of the Storyline 360 application. Some of these tests only exercise one specific component of Storyline in isolation, while others run the full application and simulate a user performing common operations. These tests have replaced some of the repetitive manual testing that traditionally had to be done by a person. We’ve recently begun an effort to expand this test suite to better cover the Storyline 360 HTML5 Player application. With such a wide variety of devices, operating systems, and web browsers in use today, ensuring that the HTML5 Player continues to work with all of them can be a time consuming process for us to do manually. We are working to leverage modern testing frameworks and platforms to automate as much of this work as possible so that we can focus on more valuable tasks like reviewing customer bug reports, investigating frequently occurring errors, and finding new ways to improve the product.

\n

64-bit Storyline 360

\n

Over the past month we’ve kicked off the initial engineering work on a 64-bit version of Storyline 360. We’ve completed some of the initial infrastructure work to allow us to build and test a 64-bit version of Storyline 360 internally. We’ve also begun addressing some of the 32-bit third party components that will need to be updated, replaced, or removed.

\n

Storyline 360 uses a Chromium-based web browser component for functionality like in-app preview, the 360° image editor, and publishing to video. We’ve successfully created a 64-bit version of this component and are working to integrate it into the 64-bit Storyline build.

\n

We have also identified several different strategies for addressing other third party components that enable functionality like screen recording, spell check, and PowerPoint import. We will begin evaluating and implementing these strategies in the coming weeks.

\n

Looking Forward

\n

To close out this month’s update I’d like to share a couple of additional items:

\n

Storyline Quality Survey: We recently sent out a survey to some of our most prolific Storyline 360 users to gather feedback on the application’s overall quality. As responses to the survey come in we’ve been reviewing them to find common themes and direct our future efforts to improve and expand the product. I want to express our appreciation for anyone who received the survey and took the time to respond. If you were not among the Storyline 360 users who received an invitation to complete the survey but are interested in participating you can find and complete the survey here: Storyline Quality Survey. We will be following up with respondents in the coming weeks to discuss their specific feedback.
Storyline 360 Private Beta: I would once again like to extend an invitation to join our private beta program. The beta version of Storyline 360 is updated on a weekly basis and gets new features and bug fixes before the public version. Participants in this program can report bugs or provide feedback directly to the Storyline 360 engineering team. The beta version can also be installed side-by-side with the public one so you don't have to choose one or the other. If you’re interested in participating please e-mail beta@articulate.com and we’ll get you added to the program!

Welcome to the June 2023 “State of Storyline” update. This month I’d like to touch on our application error rate, customer reported issues, and pre-release quality assurance as well as share our progress on creating a 64-bit version of Storyline 360.

\n

Application Error Rate

\n

In Update 76 of Storyline 360 we addressed 12 of the errors most commonly encountered by users. These errors affected areas such as rendering audio waveforms on the timeline, encoding video, and writing files to disk at publish time.

\n

Update 76 was released at the end of May and we’re currently monitoring our error reports as Storyline 360 users adopt the update. It typically takes 4 weeks for a new Storyline 360 version to be adopted widely enough to gather useful insights from this data, so we’ll share the impact these fixes had on our metrics in next month’s update.

\n

Customer Reported Issues

\n

Over the last month we have continued prioritizing bugs affecting 5 or more customer accounts. When last month’s update was published we were tracking 39 such bugs. Update 76 of Storyline 360 addressed 17 of these affecting areas such as text entry word wrapping behavior, hyperlinks in animated text, and importing tables from PowerPoint presentations.

\n

We also addressed several other bugs that did not meet the criteria of impacting 5 or more customer accounts but were clearly still causing customer pain. These bugs affected areas like the new Background Audio feature, the Storyline 360 window failing to restore after being minimized, and the timeline play/pause/stop buttons becoming disabled unpredictably.

\n

As of this writing we are tracking 23 bugs affecting 5 or more customer accounts and remain committed to continuing this work until they are resolved.

\n

Pre-release Quality Assurance

\n

We’ve continued making changes to our test and release cycles in an effort to improve the quality of each Storyline 360 release. This month I want to highlight a couple of areas that we’ve been focusing on recently:

\n

Release Validation: New versions of Storyline 360 are released on a monthly basis, which means we typically work in 4 week release cycles. In the past, some portion of the Storyline 360 engineering team would spend the last week of each release cycle testing and validating the new version to ensure it’s ready for release. As we’ve taken a critical look at how we approach quality in Storyline it has quickly become clear that having only a portion of the team participate in this release validation effort is not adequate. Starting with Update 76 of Storyline 360 we’ve had the entire team work together on pre-release validation. Having the whole team participate will not only help us catch more issues prior to release, but will also have us working with the application from a customer perspective more frequently. We’ve found that using Storyline in a manner similar to our customers helps us uncover minor bugs and usability issues that don’t tend to get a lot of attention through the normal bug triage and prioritization process.
Test Automation: We employ a suite of automated tests to continuously validate the stability of the Storyline 360 application. Some of these tests only exercise one specific component of Storyline in isolation, while others run the full application and simulate a user performing common operations. These tests have replaced some of the repetitive manual testing that traditionally had to be done by a person. We’ve recently begun an effort to expand this test suite to better cover the Storyline 360 HTML5 Player application. With such a wide variety of devices, operating systems, and web browsers in use today, ensuring that the HTML5 Player continues to work with all of them can be a time consuming process for us to do manually. We are working to leverage modern testing frameworks and platforms to automate as much of this work as possible so that we can focus on more valuable tasks like reviewing customer bug reports, investigating frequently occurring errors, and finding new ways to improve the product.

\n

64-bit Storyline 360

\n

Over the past month we’ve kicked off the initial engineering work on a 64-bit version of Storyline 360. We’ve completed some of the initial infrastructure work to allow us to build and test a 64-bit version of Storyline 360 internally. We’ve also begun addressing some of the 32-bit third party components that will need to be updated, replaced, or removed.

\n

Storyline 360 uses a Chromium-based web browser component for functionality like in-app preview, the 360° image editor, and publishing to video. We’ve successfully created a 64-bit version of this component and are working to integrate it into the 64-bit Storyline build.

\n

We have also identified several different strategies for addressing other third party components that enable functionality like screen recording, spell check, and PowerPoint import. We will begin evaluating and implementing these strategies in the coming weeks.

\n

Looking Forward

\n

To close out this month’s update I’d like to share a couple of additional items:

\n

Storyline Quality Survey: We recently sent out a survey to some of our most prolific Storyline 360 users to gather feedback on the application’s overall quality. As responses to the survey come in we’ve been reviewing them to find common themes and direct our future efforts to improve and expand the product. I want to express our appreciation for anyone who received the survey and took the time to respond. If you were not among the Storyline 360 users who received an invitation to complete the survey but are interested in participating you can find and complete the survey here: Storyline Quality Survey. We will be following up with respondents in the coming weeks to discuss their specific feedback.
Storyline 360 Private Beta: I would once again like to extend an invitation to join our private beta program. The beta version of Storyline 360 is updated on a weekly basis and gets new features and bug fixes before the public version. Participants in this program can report bugs or provide feedback directly to the Storyline 360 engineering team. The beta version can also be installed side-by-side with the public one so you don't have to choose one or the other. If you’re interested in participating please e-mail beta@articulate.com and we’ll get you added to the program!

A Year of Change: Reflecting on 12 months of Storyline development

\n

Since stepping into the CTO role at Articulate in early 2023, I've had the privilege of diving deep into our customer experiences. Back then, it quickly became clear that while there was a lot of love for Storyline, there were valid frustrations around unresolved bugs, lack of transparency, missing 64-bit support, and insufficient communication from our engineering team.

\n

Hearing you loud and clear

\n

About a year ago, your feedback was unmistakable: we needed to listen better. I shared our engineering teams’ commitment to change in a post titled, “Storyline under a Microscope: An update from the CTO”, and I'm thrilled to talk about the strides we've made since.

\n

Our Shift in Focus

\n

Today, our conversations with customers are future-focused, revolving around quality, advanced capabilities, and accessibility. It’s a shift from frustration to anticipation, and while we haven’t solved every issue, the progress is undeniable. So let’s review the key themes from the past year and highlight the progress we made on the journey.

\n

Reflecting on Key Themes: Quality, Transparency, and Communication

\n

Quality became focus number one for our team. In a nutshell, we hit pause on all new features to zero in on fixing existing issues, aligning our teams more closely for better quality control, diving into real customer scenarios to guide our releases, and doubling down on modernizing Storyline, including launching a 64-bit version in beta. In more detail:

\n

Prioritizing Quality Over Features: We took a strategic pause on developing new features for Storyline for six months, dedicating this period solely to enhancing product quality. The entire Storyline Engineering Team shifted their focus to addressing and resolving existing quality issues, ensuring a stronger foundation for our product.

\n

Unifying our Approach for Better Results: Previously, our Engineering, Quality Assurance, and Support teams worked independently on Storyline releases, which sometimes led to siloed efforts. We've since revamped our approach, bringing these teams into a cohesive unit. This collaboration ensures we can identify and address quality issues more efficiently, resulting in faster and more effective solutions.
Learning Directly From User Experiences: In partnership with David Anderson, our Director of Customer Training, we've integrated real-world course-building scenarios into our testing process. These scenarios, reflective of our customers' daily challenges, are now a crucial part of validating each Storyline update before it goes live. This practice has not only improved our issue detection but has also deepened our understanding of what our users truly need.
Modernizing Storyline: A year ago, we reinforced our commitment to Storyline by launching the first 64-bit version, Storyline 360 x64, in a public beta. We're now in the final stages of preparing Storyline 360 x64 for widespread release and are actively working on additional modernization projects to ensure Storyline continues to meet the evolving needs of our users.

\n

Along with our investment in quality, we recognized the need to increase transparency to clarify how we make decisions around what we work on.

\n

Commitment to Transparency

\n

Starting last year we committed to being more transparent about how we approach building and maintaining Storyline. We opened up about our decision-making process, our progress toward reducing unexpected errors, and our plans for addressing long-standing bugs. Our revamped bug triage process ensures we're responsive and focused on what matters most.

\n

Reducing the error rate: When we started this effort in April of 2023, about 3% of Storyline sessions were encountering an unexpected error. We committed to getting this metric under 1%. As of March 1, 2024 1.2% of Storyline sessions are encountering an unexpected error. We’ve worked hard to reduce these errors and we won’t stop until we get under our 1% goal because any error that interrupts your day is like a “paper cut” in your way.

\n

Staying “below five”: Twelve months ago, we discovered we had stopped paying attention to “older bugs”. So we focused our attention on not just recent issues but all bugs that had five customers or more. We have 100s of thousands of users, and set a line in the sand that all bugs must stay “below five customers” otherwise we stop feature development to fix. We cleaned up our backlog of bugs affecting five or more customers and Storyline is much stronger for it.
Revamping our bug triage process: Our bug database was, admittedly, a bit of a mess in April of 2023 and it was difficult to find signal in the noise. The right things were not always getting prioritized. So along with jumping on those older bugs, we revamped our daily bug triage approach to prioritize new bugs in the latest releases. We also started tracking the overall defect rate coming in from support cases to drive it below 10%. These changes helped us jump on issues causing pain quickly so that it didn’t spread.

\n

To reinforce our quality and transparency commitment, we recognized the need to increase communications to ensure your concerns were being heard and addressed.

\n

Emphasis on continuous communication

\n

In summary, monthly updates last year have kept you informed. We’ve increased our engagement with you through direct conversations and expanded our beta program, ensuring your voice is heard and valued.

\n

Monthly updates from Engineering leadership: Storyline’s engineering leader, Jesse Taber, provided monthly “State of Storyline” updates as comments on my original E-Learning Heroes post. In Jesse’s final update in September 2023 after we took Storyline out of Code Red, he committed to continue providing updates on a quarterly basis.

\n

\n
- The first quarterly Storyline quality update is now available.
  As a bonus, Jesse also wrote a separate article all about 64-bit Storyline.
\n

\n

Commitment to working directly with you: We spoke with a lot of customers last year to learn about their experiences using Storyline 360 and have made that a regular practice. The insights from those conversations are invaluable. We also expanded participation in our private beta program by 55%! We love working directly with our private beta customers to gather feedback about new features and hear about issues. If you’re interested in a more direct line to our engineering team, please e-mail beta@articulate.com.

\n

Looking Forward

\n

As we move forward, balancing innovation with quality remains our unwavering commitment. Vanessa Fage, leading our Storyline quality team and beta program, is a testament to our dedication to continuous improvement and open communication.

\n

A Year of Collaboration

\n

This journey has reinforced the value of working directly with you, a lesson we'll carry forward to ensure we never stop listening again. Your trust fuels our progress, and for that, we’re profoundly grateful.

\n

Thank you for inspiring us and for being a pivotal part of Storyline's evolution.

A Year of Change: Reflecting on 12 months of Storyline development

\n

Since stepping into the CTO role at Articulate in early 2023, I've had the privilege of diving deep into our customer experiences. Back then, it quickly became clear that while there was a lot of love for Storyline, there were valid frustrations around unresolved bugs, lack of transparency, missing 64-bit support, and insufficient communication from our engineering team.

\n

Hearing you loud and clear

\n

About a year ago, your feedback was unmistakable: we needed to listen better. I shared our engineering teams’ commitment to change in a post titled, “Storyline under a Microscope: An update from the CTO”, and I'm thrilled to talk about the strides we've made since.

\n

Our Shift in Focus

\n

Today, our conversations with customers are future-focused, revolving around quality, advanced capabilities, and accessibility. It’s a shift from frustration to anticipation, and while we haven’t solved every issue, the progress is undeniable. So let’s review the key themes from the past year and highlight the progress we made on the journey.

\n

Reflecting on Key Themes: Quality, Transparency, and Communication

\n

Quality became focus number one for our team. In a nutshell, we hit pause on all new features to zero in on fixing existing issues, aligning our teams more closely for better quality control, diving into real customer scenarios to guide our releases, and doubling down on modernizing Storyline, including launching a 64-bit version in beta. In more detail:

\n

Prioritizing Quality Over Features: We took a strategic pause on developing new features for Storyline for six months, dedicating this period solely to enhancing product quality. The entire Storyline Engineering Team shifted their focus to addressing and resolving existing quality issues, ensuring a stronger foundation for our product.

\n

Unifying our Approach for Better Results: Previously, our Engineering, Quality Assurance, and Support teams worked independently on Storyline releases, which sometimes led to siloed efforts. We've since revamped our approach, bringing these teams into a cohesive unit. This collaboration ensures we can identify and address quality issues more efficiently, resulting in faster and more effective solutions.
Learning Directly From User Experiences: In partnership with David Anderson, our Director of Customer Training, we've integrated real-world course-building scenarios into our testing process. These scenarios, reflective of our customers' daily challenges, are now a crucial part of validating each Storyline update before it goes live. This practice has not only improved our issue detection but has also deepened our understanding of what our users truly need.
Modernizing Storyline: A year ago, we reinforced our commitment to Storyline by launching the first 64-bit version, Storyline 360 x64, in a public beta. We're now in the final stages of preparing Storyline 360 x64 for widespread release and are actively working on additional modernization projects to ensure Storyline continues to meet the evolving needs of our users.

\n

Along with our investment in quality, we recognized the need to increase transparency to clarify how we make decisions around what we work on.

\n

Commitment to Transparency

\n

Starting last year we committed to being more transparent about how we approach building and maintaining Storyline. We opened up about our decision-making process, our progress toward reducing unexpected errors, and our plans for addressing long-standing bugs. Our revamped bug triage process ensures we're responsive and focused on what matters most.

\n

Reducing the error rate: When we started this effort in April of 2023, about 3% of Storyline sessions were encountering an unexpected error. We committed to getting this metric under 1%. As of March 1, 2024 1.2% of Storyline sessions are encountering an unexpected error. We’ve worked hard to reduce these errors and we won’t stop until we get under our 1% goal because any error that interrupts your day is like a “paper cut” in your way.

\n

Staying “below five”: Twelve months ago, we discovered we had stopped paying attention to “older bugs”. So we focused our attention on not just recent issues but all bugs that had five customers or more. We have 100s of thousands of users, and set a line in the sand that all bugs must stay “below five customers” otherwise we stop feature development to fix. We cleaned up our backlog of bugs affecting five or more customers and Storyline is much stronger for it.
Revamping our bug triage process: Our bug database was, admittedly, a bit of a mess in April of 2023 and it was difficult to find signal in the noise. The right things were not always getting prioritized. So along with jumping on those older bugs, we revamped our daily bug triage approach to prioritize new bugs in the latest releases. We also started tracking the overall defect rate coming in from support cases to drive it below 10%. These changes helped us jump on issues causing pain quickly so that it didn’t spread.

\n

To reinforce our quality and transparency commitment, we recognized the need to increase communications to ensure your concerns were being heard and addressed.

\n

Emphasis on continuous communication

\n

In summary, monthly updates last year have kept you informed. We’ve increased our engagement with you through direct conversations and expanded our beta program, ensuring your voice is heard and valued.

\n

Monthly updates from Engineering leadership: Storyline’s engineering leader, Jesse Taber, provided monthly “State of Storyline” updates as comments on my original E-Learning Heroes post. In Jesse’s final update in September 2023 after we took Storyline out of Code Red, he committed to continue providing updates on a quarterly basis.

\n

\n
- The first quarterly Storyline quality update is now available.
  As a bonus, Jesse also wrote a separate article all about 64-bit Storyline.
\n

\n

Commitment to working directly with you: We spoke with a lot of customers last year to learn about their experiences using Storyline 360 and have made that a regular practice. The insights from those conversations are invaluable. We also expanded participation in our private beta program by 55%! We love working directly with our private beta customers to gather feedback about new features and hear about issues. If you’re interested in a more direct line to our engineering team, please e-mail beta@articulate.com.

\n

Looking Forward

\n

As we move forward, balancing innovation with quality remains our unwavering commitment. Vanessa Fage, leading our Storyline quality team and beta program, is a testament to our dedication to continuous improvement and open communication.

\n

A Year of Collaboration

\n

This journey has reinforced the value of working directly with you, a lesson we'll carry forward to ensure we never stop listening again. Your trust fuels our progress, and for that, we’re profoundly grateful.

\n

Thank you for inspiring us and for being a pivotal part of Storyline's evolution.

Forum Discussion

Storyline under a Microscope: An update from the CTO

Application Error Rate

Customer Reported Issues

Pre-release Quality Assurance

64-bit Storyline 360

Looking Forward

Related Content

Overview of Variables in Storyline

Tables crashing files since latest update (installed today)

Engineering Journal: Storyline Quality Update Q3 2024

Insert Custom Interactions with the Storyline Block in Rise 360

Engineering Journal: Storyline Quality Update Q2 2024

Inheritance of alt text by states in Storyline

Storyline 360: Using Integrated Review 360 Comments

Custom Play/Pause Button in Storyline

BUG: Storyline 'Save' Issues?

Storyline 360: Using the Lost Connectivity Alert

Recent Discussions

Button Layer to cue points doesn't seem to be working

Creating downloadable pdf files in Storyline - an update on earlier methods

Fixed size elements fails ADA Compliance

LEADERBOARD x DASHBOARD

Image upload error in RISE

Learn

Connect

Discover

Community

Company

Trust Center

Application Error Rate

Customer Reported Issues

Pre-release Quality Assurance

64-bit Storyline 360

Looking Forward

Application Error Rate

Customer Reported Issues

Pre-release Quality Assurance

64-bit Storyline 360

Looking Forward