Design and Development metrics/goals
May 20, 2014
I have been asked to set up a goal to reduce our ‘full development’ cycle time (from kick off to module is live) by 10%. Last year, we tracked the amount of total time for each module divided that by the ‘seat’ time of the module to what our average development/design time was. This help to compare modules that are 15 minutes to 1 ½ hours long. I was able to identify in the design process where time was over estimates and reduced the design part of the cycle time.
The challenge I am facing in reducing the ‘full development’ cycle time is from kick-off to storyboard sign-off from the SME & Module requestor/owner. These are usually the same person, but not always. Even though we have an agreed upon milestone timeline, we continue to get off track due to SME response for content and storyboard approval. These are product specific modules, so the pool of SMEs is limited. It is not that they don’t want to help, it more we are the low priority and many travel. Selling product and helping customers is their main job. We were giving them 3 business days to respond to storyboards, but increased it to 5 with smaller amounts of content to review/approve based on the tracking we did over the past year. We went from telling them it would take 4-5 weeks to 5-6 weeks for a module to be developed and live. This put the projected ‘live’ date closer to what actually occurred, but raised the red flag to put a metric/goal in place to get more modules developed.
Does anyone have any advice on:
1. How you track ‘full development’ cycle time and compare it when course range in ‘seat time’?
2. How to keep projects on track when SMEs are unresponsive due to priorities?
3. What do you track for development/design goals
Thanks in advance
6 Replies
Hi Cheryl,
What a worthwile endeavor. Good for you.
Couple of things I've done in the past....
Consider breaking your metrics into separate buckets. For example, report "course design" time and "course development" time separately. This allows your stakeholders to see where the bottleneck(s) are and allows your team raise the bar in areas they control. Then the convo goes something like this.... "Mrs Stakeholder, once we have content approval, it only takes us X to produce, test, and rollout. As you can see by the numbers, the real opportunity is reducting time to approval; mainly with your SME's. Let's discuss some ideas for improving that...."
Carefully consider the impact of tieing full-cycle creation time to course duration. In my experience, rarely (if ever) is this a linear ratio. Rarely does the 30 min modules take "half" the time to create of a 60 min module. Again, this is where spitting your metrics can help you. You will often see that the ratio is indeed -more- linear on the development side, but less so on the design side.
Create an escalation standard for SME approval. Build in just enough slip to allow for days off, long weekend, etc. But then after the first friendly reminder, have a standard "to keep the project on track" reminder you send out. Then if needed a formal "the rollout date is in danger" reminder that has stakeholders CC'd. Again, if you are buttoned down on the development side and can show the time it takes your team for that, the burden clearly shifts to the stakeholders to uphold their part of the timeline.
Finally, consider implementing a SME Charter for what is in (and OUT) of scopre for SME review. For example, SME's should rarely be commenting on how something is presented, and more on what is being presented. You are the ID experts. In a similar vein, SMEs are not experts on look and feel, UI, etc. A written charter can speed things up for all involved and actually make their role less daunting in providing you feedback.
Hope these help and good luck!
I struggle with the ‘out of my control’ part of the schedule as I am at their mercy. One idea I am kicking around is the Lead SME can’t be the Project Owner. That way I have a contact on the business unit side of the project to contact when projects start to slip. I have started to request at least 1 back up SME if the Lead is not available. It has somewhat helped.
Thanks for the words of advice.
Some kind of penalty clause could also help. After all, missed deadlines affect both of you, not just your client.
Alex
Generally we only began tracking "goals" on projects once we had all the materials (or most) that we needed for the project to be considered "in our control." This was when we had it back from the SME and approved upon. The final project we would try to do a week of review.
But honestly - the only way you can stop missing the mark with SMEs is to make that amount of time very significant. Maybe you tell them they have a week to get back to you, but in reality you have budgeted a month for it. That way if they're a couple of days late, you're still way ahead of the curve. A SME has to be just as invested in a project to be able to respond to you in the amount of time you want.
Cheryl - in response to your second question
One thing I do with my clients, is to give them estimates of how long it takes (on average) to review at each stage and we book the dates in the SME calendars up front, so the time is blocked off. This isn't foolproof, of course, but you are right when you say they have other things that are more important to them. I also distinguish between client and SME when it comes to milestones.
You may already be doing this, but if not I thought I'd share it with you in case you don't.
Hope that helps
Holly
I was previously tracking from handoff of completed storyboard/media to module live date. This was changed by my boss, so it’s one of those 'find a way to do this without setting myself up to fail' situations.
I am going to Parato out the delays over the past year then pick the top three to do a counter measure exercise to reduce the occurrence by X%. It shall be interesting J
This discussion is closed. You can start a new discussion or contact Articulate Support.