Last fall’s Voice of the Scout results have been out for some time now. I have looked at it a few times but quickly grew frustrated with trying to figure out what it was saying or what it all meant. The documents I have read are cast in pretty tortured jargon and I don’t have a lot of patience with trying to interpret something that could be written simply.

I’ve always questioned how well this sort of marketing science applies to something like Scouting. Does it address the real issues? Is it an accurate gauge of anything? I actually think there’s a real danger of shifting the focus of our efforts into the wrong things but after learning something about how this works and reading the fall report I think that this may actually help identify and effect things that will improve the program.

After getting some help from Walter Underwood I think I can better understand the message and the methods behind the whole endeavor, so I’ll take a stab at interpreting it.

How it Works
A survey is conducted every six months. A notice is emailed to members divided into seven different audiences over a ten week period (Cub Scouts, Boy Scouts/Ventures, Cub Scout Parents, Boy Scout /Venture Parents, Youth Facing Volunteers, District/Council Volunteers, Chartered Organizations.) I do not know how  these audiences are chosen.

Survey results focus on something called the Net Promoter Score (NPS). It’s worth understanding exactly what NPS is and how it is calculated.

NPS a score based on the  likelihood that members would recommend Scouting to their friends and  family.  Respondents are asked to rate their likelihood of recommending Scouting to others on a scale of 0 to ten. Zero being not at all likely and ten being very likely.

“Promoters”  are those responding with a 9 or 10. “Passively Satisfied” answer 7 or 8, “Detractors” answer 0-6.

The number of Detractors is subtracted from the number of  Promoters resulting in the  Net Promoter Score (NPS).

So what does the NPS score mean? Why does it matter? It turns out that NPS scoring is a common technique companies use to gauge customer loyalty and satisfaction, it’s actually a pretty big deal in the marketing world. But the numerical score is just a part of the story.

Closing the LoopAfter people are asked to score their likelihood of recommending Scouting they are asked another open-ended question; “why?”. The comments ‘close the loop’ they give direct, actionable, feedback that can then be analyzed to identify key issues that effect the way people are perceiving the organization.

This simple question is actually one we ask all the time; “How are we doing?”. If we listen to the replies carefully we may learn something and what we learn can lead us to improve what we are doing.

What Did We learn?
In the fall of 2012 the Voice of  the Scout collected around 68K replies and the executive summary of the results (here’s the full document as a PDF) reported two key findings (called Hot Topics in the summary):

Hot Topic #1: frustration with the delivery of the program at the unit level.

  • Cub Scout Parents described meetings as too boring, not organized and not fun.
  • All audiences cited leadership and leadership support was in need of improvement
  • Boy Scout Parents commented that  they thought the opportunity to be a leader was not what they think it ought to be and questioned  that meetings are a good use of  their son’s time.

Hot Topic #2: Specific reference to the membership standards policy increased in all audiences except the Cub Scouts (youth members, not parents).

  • Conservatively for every comment in support of the policy there were 15 comments that did not support it.
  • A significant number of comments are suggesting that this issue has negatively impacted their perception and/or relationship with Scouting.

The executive summary analyzes these two ‘hot topics’ further, and it’s worth reading through.

**What Happens Next?**At some point the Voice of the Scout will be available at a unit level (so far as I can tell), so we’ll be able to look at a much more localized analysis and hopefully this will help us improve what we are doing.

On the national, council and regional level I am sure that folks are looking at all of this and figuring out what to do next. It’s early days yet, so we’ll see what we see.

Will we learn anything new? Perhaps and perhaps not. Most of the difficulties I hear about are nothing new, just reiterations of ones that have been around for decades. Hot Topic #1 are two oft-repeated complaints – boring meetings and inadequate application of youth leadership. We do know how to fix these things and hopefully we’ll get at it!

For What it’s Worth
Two things prevent the Voice of the Scout from being readily useful to us ‘youth facing volunteers’. First is the results are presented in tortured, jargon-laden language that assumes the reader knows the NPS system. I was able to briefly explain the method in a couple of hundred words – how did I do? While I understand that this is a complex process the tendency to over-explain things in the documents I have read is not making them clearer.

The second problem is accessing the information. It is a real challenge to find things at Scouting.org, and I have all but given up on myscouting.org – it is not what I would call an intuitively designed interface.

I applaud the effort being made to communicate with us all online, and there are valuable resources available but trying to get answers, or trying to find nearly anything is a real drag. Maintaining this blog has shown me that making information and resources available is not as easy as it looks, so I do understand the challenge. It would be nice, though, if things were easier to find and easier to understand once you found them.