Storyline under a Microscope: An update from the CTO

Over the past three weeks, the engineering team put Storyline under a microscope. We dug into bugs, dug into our data, and heard a ton of direct feedback to sharpen our interpretation of where things stand and where they need to be.

\n

There were a lot of positive signs that we already knew. For example, nearly a million courses were created with Storyline last year. But to be honest, we found things we didn’t like and were dismayed our heroes had to call us out to look deeper.

\n

As an example of what we found, our error reporting shows us that 3% of application sessions hit an error. Based on some quick math, those errors add up to 3,000+ hours last year of course creator time spent facing errors. That was one example that surprised us. The other was “the state of defects” including the age of bugs, the number of outstanding defects reported by 5 or more customers, and the number of “bug workarounds” the community and our eLearning Heroes team come up with everyday.

\n

As an engineering team, we’ve been very focused on new features, on new releases, and we relied on our QA and support partners to work directly with you all to keep us paying attention to the right things and keep our quality on track. But by being separated from our customers, we were missing the full picture.

\n

We are committed to do the work to earn back your trust. We recognize that in our focus to build new capabilities, we took our eye off the foundations that matter to your success every day.

\n

So here are the top three things we learned and what we plan to do about it.

\n

Quality First: We must prioritize Storyline quality over everything else.

\n

First, we commit to prioritize quality over features. We recognize more features don’t help you succeed when you’re encountering paper cuts on a regular basis. Quality comes first. We’ve still got exciting new features on the horizon but we are doubling down on quality first.

\n

Second, we're changing our test and release cycle. The team operates on a monthly release cycle, with the final week focused on validation. We release a beta build to a dedicated group of customers on a weekly basis so they can test and report any issues to our team. Our Engineering, QA, and Support teams have previously operated in separate yet parallel tracks, but we’ve moved to a partnership where they can lean on each other to learn, test, and focus on customer needs. The next few releases will focus on quality and deepening these partnerships to create a better Storyline.
Third, we will source validation scenarios from the community. We heard from a number of you who say you wait months before upgrading Storyline and do extensive validation in advance. We will source validation scenarios from the community to include directly in our testing.
Fourth, we will invest in Storyline’s foundations. We are doing a deep dive into 64-bit support for Storyline and will report back with a plan. We had taken the stance that adding more capabilities to Storyline was job #1. So we were focused on delivering Storyline updates, along with quality updates, every month instead of disappearing into a cave on a long infrastructure project where the value of the impact is hard to measure until it is done. We are committed to investing in Storyline’s foundations so expect to hear more soon.

\n

Transparency: We need to be more transparent about how we are prioritizing our work.

\n

First priority is to reduce the error rate to below 1%. You can’t build courses, if you keep running into errors. In release 74, we fixed the top 18% of those errors that generated support cases but there is a lot more work to be done so those errors don’t escape our tests. We’ll report back monthly to update how we are doing on this critical metric.
Second we prioritized 35 older issues impacting five or more customers. We reviewed nearly 300 issues open over the life of Storyline. 35 of them are impacting five or more customers that were not at the top of our list. We are prioritizing those now and shipped a number of fixes for those in Release 74. Previously we focused our calories almost exclusively on more recent issues. So we’ve dedicated more engineers to focus on quality and renewed our focus on fixing older issues. As we work through those issues, we will share details.
Third, we are cleaning up our bug database. As a bug ages, finding the issue and fixing it without impacting something else becomes increasingly complex. When we reviewed all open issues, we found issues over 3 years old that affected one customer and had a workaround. We aren’t going to fix those issues. All defects are a problem and we don’t mean to imply we don’t care about quality. We need to focus our team on what impacts the community the most, and report back to those customers transparently about what we won’t fix instead of pretending we might some day.

\n

Communication: We need to communicate proactively instead of reactively.

\n

First, we will commit to monthly updates to the community from our Engineering team. We are working on the format and content to make it valuable, but to start we will commit to providing a “State of Storyline” monthly so you have the opportunity to know and react to what we are focused on. Quality is our number one priority right now and we are looking for measurable improvements that make a difference to you.
Second, our engineering team and I commit to working directly with all of you. Our engineers and engineering leaders have kept a distance from you all by accident. We’ve been focused on building and relied on our partners inside Articulate to be the “voice of our customers” rather than listening directly. So this is a commitment for us to work better together directly. We appreciate the time, energy, and passion you all bring to Storyline and we commit to make that worth it.

\n

Storyline 360 is our flagship product and is how you help your employees, your customers, and your companies to learn and grow everyday. We have a lot of exciting things planned this year but agree that quality must come first. If you’re interested in joining our beta program or sharing your teams’ Storyline validation scenarios, please email beta@articulate.com.

Our team will be updating this thread directly each month to keep you aware of our progress. To share feedback on our process, our plans, and our areas of focus please open a new thread or reach out to the support team so they can connect us directly.

\n

We are committed to more transparency and proactive communication directly with the engineering team because honestly we are only here because of you.

Over the past three weeks, the engineering team put Storyline under a microscope. We dug into bugs, dug into our data, and heard a ton of direct feedback to sharpen our interpretation of where things stand and where they need to be.

\n

There were a lot of positive signs that we already knew. For example, nearly a million courses were created with Storyline last year. But to be honest, we found things we didn’t like and were dismayed our heroes had to call us out to look deeper.

\n

As an example of what we found, our error reporting shows us that 3% of application sessions hit an error. Based on some quick math, those errors add up to 3,000+ hours last year of course creator time spent facing errors. That was one example that surprised us. The other was “the state of defects” including the age of bugs, the number of outstanding defects reported by 5 or more customers, and the number of “bug workarounds” the community and our eLearning Heroes team come up with everyday.

\n

As an engineering team, we’ve been very focused on new features, on new releases, and we relied on our QA and support partners to work directly with you all to keep us paying attention to the right things and keep our quality on track. But by being separated from our customers, we were missing the full picture.

\n

We are committed to do the work to earn back your trust. We recognize that in our focus to build new capabilities, we took our eye off the foundations that matter to your success every day.

\n

So here are the top three things we learned and what we plan to do about it.

\n

Quality First: We must prioritize Storyline quality over everything else.

\n

First, we commit to prioritize quality over features. We recognize more features don’t help you succeed when you’re encountering paper cuts on a regular basis. Quality comes first. We’ve still got exciting new features on the horizon but we are doubling down on quality first.

\n

Second, we're changing our test and release cycle. The team operates on a monthly release cycle, with the final week focused on validation. We release a beta build to a dedicated group of customers on a weekly basis so they can test and report any issues to our team. Our Engineering, QA, and Support teams have previously operated in separate yet parallel tracks, but we’ve moved to a partnership where they can lean on each other to learn, test, and focus on customer needs. The next few releases will focus on quality and deepening these partnerships to create a better Storyline.
Third, we will source validation scenarios from the community. We heard from a number of you who say you wait months before upgrading Storyline and do extensive validation in advance. We will source validation scenarios from the community to include directly in our testing.
Fourth, we will invest in Storyline’s foundations. We are doing a deep dive into 64-bit support for Storyline and will report back with a plan. We had taken the stance that adding more capabilities to Storyline was job #1. So we were focused on delivering Storyline updates, along with quality updates, every month instead of disappearing into a cave on a long infrastructure project where the value of the impact is hard to measure until it is done. We are committed to investing in Storyline’s foundations so expect to hear more soon.

\n

Transparency: We need to be more transparent about how we are prioritizing our work.

\n

First priority is to reduce the error rate to below 1%. You can’t build courses, if you keep running into errors. In release 74, we fixed the top 18% of those errors that generated support cases but there is a lot more work to be done so those errors don’t escape our tests. We’ll report back monthly to update how we are doing on this critical metric.
Second we prioritized 35 older issues impacting five or more customers. We reviewed nearly 300 issues open over the life of Storyline. 35 of them are impacting five or more customers that were not at the top of our list. We are prioritizing those now and shipped a number of fixes for those in Release 74. Previously we focused our calories almost exclusively on more recent issues. So we’ve dedicated more engineers to focus on quality and renewed our focus on fixing older issues. As we work through those issues, we will share details.
Third, we are cleaning up our bug database. As a bug ages, finding the issue and fixing it without impacting something else becomes increasingly complex. When we reviewed all open issues, we found issues over 3 years old that affected one customer and had a workaround. We aren’t going to fix those issues. All defects are a problem and we don’t mean to imply we don’t care about quality. We need to focus our team on what impacts the community the most, and report back to those customers transparently about what we won’t fix instead of pretending we might some day.

\n

Communication: We need to communicate proactively instead of reactively.

\n

First, we will commit to monthly updates to the community from our Engineering team. We are working on the format and content to make it valuable, but to start we will commit to providing a “State of Storyline” monthly so you have the opportunity to know and react to what we are focused on. Quality is our number one priority right now and we are looking for measurable improvements that make a difference to you.
Second, our engineering team and I commit to working directly with all of you. Our engineers and engineering leaders have kept a distance from you all by accident. We’ve been focused on building and relied on our partners inside Articulate to be the “voice of our customers” rather than listening directly. So this is a commitment for us to work better together directly. We appreciate the time, energy, and passion you all bring to Storyline and we commit to make that worth it.

\n

Storyline 360 is our flagship product and is how you help your employees, your customers, and your companies to learn and grow everyday. We have a lot of exciting things planned this year but agree that quality must come first. If you’re interested in joining our beta program or sharing your teams’ Storyline validation scenarios, please email beta@articulate.com.

Our team will be updating this thread directly each month to keep you aware of our progress. To share feedback on our process, our plans, and our areas of focus please open a new thread or reach out to the support team so they can connect us directly.

\n

We are committed to more transparency and proactive communication directly with the engineering team because honestly we are only here because of you.

Last month we shared our plans to address quality issues in Storyline 360 and committed to monthly updates in the interest of open, honest, and transparent communication with our customers.

\n

My name is Jesse Taber, and I’m the Engineering Manager for the Storyline 360 Team. I’d like to share our progress in 3 key areas related to improving Storyline 360’s overall quality and stability: application error rate, customer-reported issues, and pre-release quality assurance.

\n

Application Error Rate

\n

You’re working hard on your course, you make a seemingly innocuous change, and suddenly you’re confronted with the “Articulate Storyline Error Report” dialog or even worse, the Storyline application crashes completely and has to be restarted. Not only are these errors an impediment to your work, but they shake your confidence in the application. For these reasons, our primary focus over the past month has been investigating and resolving these errors.

\n

Last month Kerry mentioned that ~3% of Storyline sessions were encountering an error at least once. During the last 3 releases of Storyline 360 ~30% of users were encountering the error dialog at least once per month. Our goal is to drive both of these metrics down under 1%.

\n

In Update 75 of Storyline 360, we addressed 9 of the most common errors that users were experiencing during their Storyline sessions. These errors affected a wide range of Storyline 360 functionality, including manipulating shape states, interacting with the ribbon, and editing or publishing videos. Update 75 was released on April 18, and we are seeing the following improvements to our average error rate metrics:

\n

% of application sessions encountering the error dialog at least once: 2.7% (~0.3% improvement)

\n

% of users encountering the error dialog at least once per month: 22.8% (~7.2% improvement)

\n

While these metrics are promising, we recognize that we have a lot more work to do. We are devoting engineering resources to address these errors until we reach our goal.

\n

On a related note: The Articulate Storyline Error Report dialog allows you to share additional information about the circumstances that led to the error. This goes directly to the Storyline 360 engineering team to use when investigating errors. It does not create a Support case for follow-up, but you can always connect with our Support team here.

\n

Customer Reported Issues

\n

We have been working to clean up our bug database. Our Engineering and Customer Support teams have been working closely together to dive deep on customer-reported issues to be sure that we have the most accurate picture possible of the issues that are impacting the most customers. This effort has led to many issues being de-duplicated and consolidated so that we have a much clearer idea of where to focus our bug-fixing efforts. Prior to this work we had identified 35 issues impacting 5 or more customer accounts based on support case data, but that number rose to 44 as a result of our de-duplication and consolidation.

\n

In Update 75 of Storyline 360 we addressed 5 of these issues affecting 5 or more customers based on support case data, bringing that count down to 39. These issues impacted areas such as video compression, sliders, and performance in the Storyline 360 HTML5 player application.

\n

Additionally, we addressed 5 issues that were reported and discussed by users in public forums such as Twitter or our own E-Learning Heroes forum. These issues impacted areas such as the focus order of objects on a slide, the “print results” action for quizzes, and an annoying bug where the Storyline 360 HTML5 Player would show the Resources tab even if the author had unchecked that option.

\n

Pre-release Quality Assurance

\n

We shared that we were making changes to our test and release cycles with an eye for improving the techniques used to validate quality as well as promote better alignment between the Engineering and Support teams. This effort will be continuously refined, but we have taken the following measures to start:

\n

Establishing a formal Support Liaison role: The Support Liaison role was created to help bridge the gap between Engineering and Support and promote better cooperation between these teams. The Support Liaison works very closely with the Storyline 360 engineering team to ensure that we are hearing the voice of the customer. This role will share customer feedback with us (good, bad, and ugly), bring issues to our attention that might not yet have surfaced through our normal support case data analysis, and bring the support team's perspective on what issues the team should focus on next. The entire engineering team is focused on getting closer to our customers, and the Support Liaison is a big step in that direction.
Sourcing validation scenarios: Storyline 360 is a large, complex application with many features. We know that our customers use it to create interactive, entertaining, informative, and beautiful courses every day, but when we’re heads-down buried in the minutiae of the code on a daily basis, it can be easy to lose sight of the impressive ways our customers use the application every day. When we’re validating a new Storyline 360 release we try to use the product similarly to how our customers do. We’ve tapped into our very talented training team and asked them to define the common usage scenarios in Storyline. We now have a suite of common Storyline 360 course scenarios, such as conditional navigation, tabs interactions, and software simulations, that we use to put the application through its paces prior to releasing it publicly. We’ll continue to look for new validation scenarios from both the customer community and our own Storyline 360 experts to ensure we’re testing things in the right way.

\n

Looking Forward

\n

To wrap up this month’s update, I’d like to share things we’re actively working on:

\n

64-bit Storyline 360: We’re doing a deep dive on what it will take to create a 64-bit version of Storyline 360. Storyline is a large and complex piece of software comprised of many different components, both created internally by Articulate and sourced from third parties. Some of these components were not designed to work outside of a 32-bit process architecture and will need to be updated, replaced, or removed. We have identified the components that represent the largest amounts of effort and highest risk to the success of a 64-bit conversion. Our next step is to evaluate our options on a per-component basis and determine the plan for each. We will begin that effort this month and will share updates on our progress.
Storyline 360 Private Beta: Storyline 360 has a private beta program in which the software is updated weekly. The beta version gets new features and bug fixes before the public version does and serves as a great way for us to gather customer feedback or identify issues before the changes are made widely available. Any bug reports or feedback made through the private beta go directly to the Storyline 360 engineering team for review. Last month we extended an invitation for anyone interested in participating to e-mail beta@articulate.com, and we would like to remind everyone that this invitation still stands. Participating in the private beta helps make the product better and gives you an inside track on what changes are coming in future public releases.

Last month we shared our plans to address quality issues in Storyline 360 and committed to monthly updates in the interest of open, honest, and transparent communication with our customers.

\n

My name is Jesse Taber, and I’m the Engineering Manager for the Storyline 360 Team. I’d like to share our progress in 3 key areas related to improving Storyline 360’s overall quality and stability: application error rate, customer-reported issues, and pre-release quality assurance.

\n

Application Error Rate

\n

You’re working hard on your course, you make a seemingly innocuous change, and suddenly you’re confronted with the “Articulate Storyline Error Report” dialog or even worse, the Storyline application crashes completely and has to be restarted. Not only are these errors an impediment to your work, but they shake your confidence in the application. For these reasons, our primary focus over the past month has been investigating and resolving these errors.

\n

Last month Kerry mentioned that ~3% of Storyline sessions were encountering an error at least once. During the last 3 releases of Storyline 360 ~30% of users were encountering the error dialog at least once per month. Our goal is to drive both of these metrics down under 1%.

\n

In Update 75 of Storyline 360, we addressed 9 of the most common errors that users were experiencing during their Storyline sessions. These errors affected a wide range of Storyline 360 functionality, including manipulating shape states, interacting with the ribbon, and editing or publishing videos. Update 75 was released on April 18, and we are seeing the following improvements to our average error rate metrics:

\n

% of application sessions encountering the error dialog at least once: 2.7% (~0.3% improvement)

\n

% of users encountering the error dialog at least once per month: 22.8% (~7.2% improvement)

\n

While these metrics are promising, we recognize that we have a lot more work to do. We are devoting engineering resources to address these errors until we reach our goal.

\n

On a related note: The Articulate Storyline Error Report dialog allows you to share additional information about the circumstances that led to the error. This goes directly to the Storyline 360 engineering team to use when investigating errors. It does not create a Support case for follow-up, but you can always connect with our Support team here.

\n

Customer Reported Issues

\n

We have been working to clean up our bug database. Our Engineering and Customer Support teams have been working closely together to dive deep on customer-reported issues to be sure that we have the most accurate picture possible of the issues that are impacting the most customers. This effort has led to many issues being de-duplicated and consolidated so that we have a much clearer idea of where to focus our bug-fixing efforts. Prior to this work we had identified 35 issues impacting 5 or more customer accounts based on support case data, but that number rose to 44 as a result of our de-duplication and consolidation.

\n

In Update 75 of Storyline 360 we addressed 5 of these issues affecting 5 or more customers based on support case data, bringing that count down to 39. These issues impacted areas such as video compression, sliders, and performance in the Storyline 360 HTML5 player application.

\n

Additionally, we addressed 5 issues that were reported and discussed by users in public forums such as Twitter or our own E-Learning Heroes forum. These issues impacted areas such as the focus order of objects on a slide, the “print results” action for quizzes, and an annoying bug where the Storyline 360 HTML5 Player would show the Resources tab even if the author had unchecked that option.

\n

Pre-release Quality Assurance

\n

We shared that we were making changes to our test and release cycles with an eye for improving the techniques used to validate quality as well as promote better alignment between the Engineering and Support teams. This effort will be continuously refined, but we have taken the following measures to start:

\n

Establishing a formal Support Liaison role: The Support Liaison role was created to help bridge the gap between Engineering and Support and promote better cooperation between these teams. The Support Liaison works very closely with the Storyline 360 engineering team to ensure that we are hearing the voice of the customer. This role will share customer feedback with us (good, bad, and ugly), bring issues to our attention that might not yet have surfaced through our normal support case data analysis, and bring the support team's perspective on what issues the team should focus on next. The entire engineering team is focused on getting closer to our customers, and the Support Liaison is a big step in that direction.
Sourcing validation scenarios: Storyline 360 is a large, complex application with many features. We know that our customers use it to create interactive, entertaining, informative, and beautiful courses every day, but when we’re heads-down buried in the minutiae of the code on a daily basis, it can be easy to lose sight of the impressive ways our customers use the application every day. When we’re validating a new Storyline 360 release we try to use the product similarly to how our customers do. We’ve tapped into our very talented training team and asked them to define the common usage scenarios in Storyline. We now have a suite of common Storyline 360 course scenarios, such as conditional navigation, tabs interactions, and software simulations, that we use to put the application through its paces prior to releasing it publicly. We’ll continue to look for new validation scenarios from both the customer community and our own Storyline 360 experts to ensure we’re testing things in the right way.

\n

Looking Forward

\n

To wrap up this month’s update, I’d like to share things we’re actively working on:

\n

64-bit Storyline 360: We’re doing a deep dive on what it will take to create a 64-bit version of Storyline 360. Storyline is a large and complex piece of software comprised of many different components, both created internally by Articulate and sourced from third parties. Some of these components were not designed to work outside of a 32-bit process architecture and will need to be updated, replaced, or removed. We have identified the components that represent the largest amounts of effort and highest risk to the success of a 64-bit conversion. Our next step is to evaluate our options on a per-component basis and determine the plan for each. We will begin that effort this month and will share updates on our progress.
Storyline 360 Private Beta: Storyline 360 has a private beta program in which the software is updated weekly. The beta version gets new features and bug fixes before the public version does and serves as a great way for us to gather customer feedback or identify issues before the changes are made widely available. Any bug reports or feedback made through the private beta go directly to the Storyline 360 engineering team for review. Last month we extended an invitation for anyone interested in participating to e-mail beta@articulate.com, and we would like to remind everyone that this invitation still stands. Participating in the private beta helps make the product better and gives you an inside track on what changes are coming in future public releases.

A Year of Change: Reflecting on 12 months of Storyline development

\n

Since stepping into the CTO role at Articulate in early 2023, I've had the privilege of diving deep into our customer experiences. Back then, it quickly became clear that while there was a lot of love for Storyline, there were valid frustrations around unresolved bugs, lack of transparency, missing 64-bit support, and insufficient communication from our engineering team.

\n

Hearing you loud and clear

\n

About a year ago, your feedback was unmistakable: we needed to listen better. I shared our engineering teams’ commitment to change in a post titled, “Storyline under a Microscope: An update from the CTO”, and I'm thrilled to talk about the strides we've made since.

\n

Our Shift in Focus

\n

Today, our conversations with customers are future-focused, revolving around quality, advanced capabilities, and accessibility. It’s a shift from frustration to anticipation, and while we haven’t solved every issue, the progress is undeniable. So let’s review the key themes from the past year and highlight the progress we made on the journey.

\n

Reflecting on Key Themes: Quality, Transparency, and Communication

\n

Quality became focus number one for our team. In a nutshell, we hit pause on all new features to zero in on fixing existing issues, aligning our teams more closely for better quality control, diving into real customer scenarios to guide our releases, and doubling down on modernizing Storyline, including launching a 64-bit version in beta. In more detail:

\n

Prioritizing Quality Over Features: We took a strategic pause on developing new features for Storyline for six months, dedicating this period solely to enhancing product quality. The entire Storyline Engineering Team shifted their focus to addressing and resolving existing quality issues, ensuring a stronger foundation for our product.

\n

Unifying our Approach for Better Results: Previously, our Engineering, Quality Assurance, and Support teams worked independently on Storyline releases, which sometimes led to siloed efforts. We've since revamped our approach, bringing these teams into a cohesive unit. This collaboration ensures we can identify and address quality issues more efficiently, resulting in faster and more effective solutions.
Learning Directly From User Experiences: In partnership with David Anderson, our Director of Customer Training, we've integrated real-world course-building scenarios into our testing process. These scenarios, reflective of our customers' daily challenges, are now a crucial part of validating each Storyline update before it goes live. This practice has not only improved our issue detection but has also deepened our understanding of what our users truly need.
Modernizing Storyline: A year ago, we reinforced our commitment to Storyline by launching the first 64-bit version, Storyline 360 x64, in a public beta. We're now in the final stages of preparing Storyline 360 x64 for widespread release and are actively working on additional modernization projects to ensure Storyline continues to meet the evolving needs of our users.

\n

Along with our investment in quality, we recognized the need to increase transparency to clarify how we make decisions around what we work on.

\n

Commitment to Transparency

\n

Starting last year we committed to being more transparent about how we approach building and maintaining Storyline. We opened up about our decision-making process, our progress toward reducing unexpected errors, and our plans for addressing long-standing bugs. Our revamped bug triage process ensures we're responsive and focused on what matters most.

\n

Reducing the error rate: When we started this effort in April of 2023, about 3% of Storyline sessions were encountering an unexpected error. We committed to getting this metric under 1%. As of March 1, 2024 1.2% of Storyline sessions are encountering an unexpected error. We’ve worked hard to reduce these errors and we won’t stop until we get under our 1% goal because any error that interrupts your day is like a “paper cut” in your way.

\n

Staying “below five”: Twelve months ago, we discovered we had stopped paying attention to “older bugs”. So we focused our attention on not just recent issues but all bugs that had five customers or more. We have 100s of thousands of users, and set a line in the sand that all bugs must stay “below five customers” otherwise we stop feature development to fix. We cleaned up our backlog of bugs affecting five or more customers and Storyline is much stronger for it.
Revamping our bug triage process: Our bug database was, admittedly, a bit of a mess in April of 2023 and it was difficult to find signal in the noise. The right things were not always getting prioritized. So along with jumping on those older bugs, we revamped our daily bug triage approach to prioritize new bugs in the latest releases. We also started tracking the overall defect rate coming in from support cases to drive it below 10%. These changes helped us jump on issues causing pain quickly so that it didn’t spread.

\n

To reinforce our quality and transparency commitment, we recognized the need to increase communications to ensure your concerns were being heard and addressed.

\n

Emphasis on continuous communication

\n

In summary, monthly updates last year have kept you informed. We’ve increased our engagement with you through direct conversations and expanded our beta program, ensuring your voice is heard and valued.

\n

Monthly updates from Engineering leadership: Storyline’s engineering leader, Jesse Taber, provided monthly “State of Storyline” updates as comments on my original E-Learning Heroes post. In Jesse’s final update in September 2023 after we took Storyline out of Code Red, he committed to continue providing updates on a quarterly basis.

\n

\n
- The first quarterly Storyline quality update is now available.
  As a bonus, Jesse also wrote a separate article all about 64-bit Storyline.
\n

\n

Commitment to working directly with you: We spoke with a lot of customers last year to learn about their experiences using Storyline 360 and have made that a regular practice. The insights from those conversations are invaluable. We also expanded participation in our private beta program by 55%! We love working directly with our private beta customers to gather feedback about new features and hear about issues. If you’re interested in a more direct line to our engineering team, please e-mail beta@articulate.com.

\n

Looking Forward

\n

As we move forward, balancing innovation with quality remains our unwavering commitment. Vanessa Fage, leading our Storyline quality team and beta program, is a testament to our dedication to continuous improvement and open communication.

\n

A Year of Collaboration

\n

This journey has reinforced the value of working directly with you, a lesson we'll carry forward to ensure we never stop listening again. Your trust fuels our progress, and for that, we’re profoundly grateful.

\n

Thank you for inspiring us and for being a pivotal part of Storyline's evolution.

A Year of Change: Reflecting on 12 months of Storyline development

\n

Since stepping into the CTO role at Articulate in early 2023, I've had the privilege of diving deep into our customer experiences. Back then, it quickly became clear that while there was a lot of love for Storyline, there were valid frustrations around unresolved bugs, lack of transparency, missing 64-bit support, and insufficient communication from our engineering team.

\n

Hearing you loud and clear

\n

About a year ago, your feedback was unmistakable: we needed to listen better. I shared our engineering teams’ commitment to change in a post titled, “Storyline under a Microscope: An update from the CTO”, and I'm thrilled to talk about the strides we've made since.

\n

Our Shift in Focus

\n

Today, our conversations with customers are future-focused, revolving around quality, advanced capabilities, and accessibility. It’s a shift from frustration to anticipation, and while we haven’t solved every issue, the progress is undeniable. So let’s review the key themes from the past year and highlight the progress we made on the journey.

\n

Reflecting on Key Themes: Quality, Transparency, and Communication

\n

Quality became focus number one for our team. In a nutshell, we hit pause on all new features to zero in on fixing existing issues, aligning our teams more closely for better quality control, diving into real customer scenarios to guide our releases, and doubling down on modernizing Storyline, including launching a 64-bit version in beta. In more detail:

\n

Prioritizing Quality Over Features: We took a strategic pause on developing new features for Storyline for six months, dedicating this period solely to enhancing product quality. The entire Storyline Engineering Team shifted their focus to addressing and resolving existing quality issues, ensuring a stronger foundation for our product.

\n

Unifying our Approach for Better Results: Previously, our Engineering, Quality Assurance, and Support teams worked independently on Storyline releases, which sometimes led to siloed efforts. We've since revamped our approach, bringing these teams into a cohesive unit. This collaboration ensures we can identify and address quality issues more efficiently, resulting in faster and more effective solutions.
Learning Directly From User Experiences: In partnership with David Anderson, our Director of Customer Training, we've integrated real-world course-building scenarios into our testing process. These scenarios, reflective of our customers' daily challenges, are now a crucial part of validating each Storyline update before it goes live. This practice has not only improved our issue detection but has also deepened our understanding of what our users truly need.
Modernizing Storyline: A year ago, we reinforced our commitment to Storyline by launching the first 64-bit version, Storyline 360 x64, in a public beta. We're now in the final stages of preparing Storyline 360 x64 for widespread release and are actively working on additional modernization projects to ensure Storyline continues to meet the evolving needs of our users.

\n

Along with our investment in quality, we recognized the need to increase transparency to clarify how we make decisions around what we work on.

\n

Commitment to Transparency

\n

Starting last year we committed to being more transparent about how we approach building and maintaining Storyline. We opened up about our decision-making process, our progress toward reducing unexpected errors, and our plans for addressing long-standing bugs. Our revamped bug triage process ensures we're responsive and focused on what matters most.

\n

Reducing the error rate: When we started this effort in April of 2023, about 3% of Storyline sessions were encountering an unexpected error. We committed to getting this metric under 1%. As of March 1, 2024 1.2% of Storyline sessions are encountering an unexpected error. We’ve worked hard to reduce these errors and we won’t stop until we get under our 1% goal because any error that interrupts your day is like a “paper cut” in your way.

\n

Staying “below five”: Twelve months ago, we discovered we had stopped paying attention to “older bugs”. So we focused our attention on not just recent issues but all bugs that had five customers or more. We have 100s of thousands of users, and set a line in the sand that all bugs must stay “below five customers” otherwise we stop feature development to fix. We cleaned up our backlog of bugs affecting five or more customers and Storyline is much stronger for it.
Revamping our bug triage process: Our bug database was, admittedly, a bit of a mess in April of 2023 and it was difficult to find signal in the noise. The right things were not always getting prioritized. So along with jumping on those older bugs, we revamped our daily bug triage approach to prioritize new bugs in the latest releases. We also started tracking the overall defect rate coming in from support cases to drive it below 10%. These changes helped us jump on issues causing pain quickly so that it didn’t spread.

\n

To reinforce our quality and transparency commitment, we recognized the need to increase communications to ensure your concerns were being heard and addressed.

\n

Emphasis on continuous communication

\n

In summary, monthly updates last year have kept you informed. We’ve increased our engagement with you through direct conversations and expanded our beta program, ensuring your voice is heard and valued.

\n

Monthly updates from Engineering leadership: Storyline’s engineering leader, Jesse Taber, provided monthly “State of Storyline” updates as comments on my original E-Learning Heroes post. In Jesse’s final update in September 2023 after we took Storyline out of Code Red, he committed to continue providing updates on a quarterly basis.

\n

\n
- The first quarterly Storyline quality update is now available.
  As a bonus, Jesse also wrote a separate article all about 64-bit Storyline.
\n

\n

Commitment to working directly with you: We spoke with a lot of customers last year to learn about their experiences using Storyline 360 and have made that a regular practice. The insights from those conversations are invaluable. We also expanded participation in our private beta program by 55%! We love working directly with our private beta customers to gather feedback about new features and hear about issues. If you’re interested in a more direct line to our engineering team, please e-mail beta@articulate.com.

\n

Looking Forward

\n

As we move forward, balancing innovation with quality remains our unwavering commitment. Vanessa Fage, leading our Storyline quality team and beta program, is a testament to our dedication to continuous improvement and open communication.

\n

A Year of Collaboration

\n

This journey has reinforced the value of working directly with you, a lesson we'll carry forward to ensure we never stop listening again. Your trust fuels our progress, and for that, we’re profoundly grateful.

\n

Thank you for inspiring us and for being a pivotal part of Storyline's evolution.

Forum Discussion

Storyline under a Microscope: An update from the CTO

Application Error Rate

Customer Reported Issues

Pre-release Quality Assurance

Looking Forward

Related Content

Tables crashing files since latest update (installed today)

Engineering Journal: Storyline Quality Update Q3 2024

Engineering Journal: Storyline Quality Update Q2 2024

Inheritance of alt text by states in Storyline

Custom Play/Pause Button in Storyline

Storyline 360: Using the Lost Connectivity Alert

BUG: Storyline 'Save' Issues?

Using Dev Tools with Storyline

Referencing Json files in Storyline 360

Storyline 3: Setting Storyline Options

Recent Discussions

Question Level Quiz Reporting to LMS with Storyline block in Rise 360 course

How to ensure the audience complete all triggers before completing the project?

Can turn off only default glossary button, not customized glossary button, in quizzes...

Rise 360 - Quizzes not behaving on Safari

Lightbox vs. Slide Numbering

Learn

Connect

Discover

Community

Company

Trust Center

Application Error Rate

Customer Reported Issues

Pre-release Quality Assurance

Looking Forward

Application Error Rate

Customer Reported Issues

Pre-release Quality Assurance

Looking Forward