What’s the Best Assessment Tool? It Depends…

Assessment: Learning Leads to Excellence

I confess that I don’t know very much about tools. However, I do know enough to recognize that the question, “What tool should I use?” must be asked in some context. Just as you wouldn’t use an Allen wrench to pound in a nail, you wouldn’t want to use a survey instrument that was mismatched for the information that you desire.

There are three assessment tools which lend themselves nicely to congregations and non-profit organizations: questionnaires, interviews and focus groups. These three methods work well because they are simple, easy to learn and enable you to use volunteers who have expertise in these areas. This post is going to review some basics about questionnaires.

If you are looking for more in-depth information about an issue, a questionnaire is not the tool to use. However, questionnaires are a good tool for efficiently gathering basic information from a large number of people at low cost. Today, through the use of inexpensive or free electronic questionnaire services, this method of gathering information has been made incredibly easy. But, there is a science to constructing questionnaires so that the data you get back actually answer the questions about which you want to learn.

Take a moment now to reflect on this question: when you speak with someone either in person or by telephone, how often are you misunderstood? Misunderstandings are frequent but can be corrected when you are in direct conversation with someone. However, when you have distributed a questionnaire to congregants, they will not have the benefit of asking for clarification. That is why it is critical to have someone with experience and expertise help you develop your questions.

Additionally, before you add a question to your questionnaire, ask yourself, “Do I really need to include it—is it germane to the issue?” There is an understandable temptation to ask questions which are not directly related to the assessment topic because you have people’s attention. But, if that temptation gets the better of you, you’ll limit the number of people who respond either because they will start to question your motives or because your have included too many questions.

Another good practice is to test questionnaires before you distribute them. By doing so, you’ll detect any unclear questions and get feedback on the length of the questionnaire. Typically, most volunteers will be willing to spend no more than about 15 minutes on a questionnaire. If they see a questionnaire which has over 20 to 25 questions, you will likely lose them. (This observation is based on my experience.) Aim for a greater response rate by limiting the number of questions. Those who are more invested in your organization, like board members, may be willing to spend more time as they better understand the importance of the effort. Still, you need to explain to people why you are asking for their help in supplying this information, how you hope it will be used and how you will inform them once it is used. In this way, they will be more likely to help you now and in the future. If you guarantee that responses will be confidential, which will increase the number of people who respond, then you must honor that confidentiality!

If you are using an electronic service, make sure that all of the potential respondents have access to a computer and know how to complete an online questionnaire. (Commonly used services are www.zoomerang.com and www.surveymonkey.com. You’ll also find that some newsletters, like Constant Contact, have a questionnaire feature.) Have you accounted for people with special needs? How about the elderly who may not like to use a computer but would respond to a telephone call?

With the right team of professionals and volunteers, you will be gratified by incorporating questionnaires as a part of program planning and congregational engagement into your activities. So—now it’s your turn to ask questions. Please fire away or share your experience with using electronic questionnaires.

Thanks,

Rabbi Hayim Herring

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Reddit
  • StumbleUpon
  • Technorati
  • E-mail this story to a friend!
  • Print this article!
8 Comments

7 Comments

  1. Ellen Flax  •  Oct 9, 2009 @10:06 am

    I’ve found that online surveys can be a great way to get information, from either a small or large group of people. However, I’ve also come across a number of confusing/poorly organized surveys. Here are some online resources:

    For writing a survey:
    http://tinyurl.com/ygvqc2o
    http://tinyurl.com/ypeyjp

    For writing a survey, and info about different online survey companies:
    http://www.techsoup.org/learningcenter/internet/page5048.cfm

    For shuls, I’d suggest finding out if you have a congregant who has experience in this area–it would be a unique volunteer opportunity.

  2. Daniel Alter  •  Oct 9, 2009 @11:26 am

    Hayim
    One issue which you mention is that often those who are more invested in the shul will put in the time to respond. It is important to keep this in mind, since this often skews the results. Those who are less invested often have different perspectives and sometimes are the ones you would like the most to fill out the questionaire.
    Another advantage of questionaires is that it is one more way to make people feel connected to the Shul and that they are being listened to.
    We just sent one out to our Shul, and got some great feedback.

  3. Rabbi Hayim Herring  •  Oct 11, 2009 @7:14 pm

    Ellen–these are great resources-thanks!

  4. Rabbi Hayim Herring  •  Oct 11, 2009 @7:19 pm

    Hi, Dan

    You’ve raised 2 excellent points. I’ll be writing soon about how to present data and an important part of that is putting it in context. Part of the context setting is identifying who responds to questionnaires. And you’re so right about the second point. When a congregation sends a questionnaire to members, it’s already a positive action, because members feel like the congregation is interested in what they have to say. The key is then for the leadership to respond.

    Thanks for commenting and please share your insights again!

  5. Moishe B. Singer  •  Oct 16, 2009 @7:57 am

    I have much experience with surveys, both professionally, as well as within the context of our shul. The biggest error in most questioners is the validity of the questions. Wording questions is a science that is often ignored. How you word the question often leads to the response. The second issue to look at is the type of question and the answer you are looking for. Do you want a free text response? Are you using a scale. Is you scale a 4 point or 5 point scale. If it is 5, how many people choose the neutral (middle choice).
    Furthermore the bigger issue to look at is the self selection of those who reply to your survey. Sending a questioner to a group of shul members is not necessarily a valid sample. Taking an alphabetical list and sending to every 10th member might be. Do you make the survey anonymous or do you ask for names so you can follow up based on responses.
    We have used samples to survey members after events run in the shul as well as to parents of our teen youth programs. We find the people who respond are those who a – have a gripe and b – those who are involved with the running who are very satisfied. That does not really show what the group thinks. For example, our last youth survey had a response rate of 60%, and had a bi-modal distribution of responses, all very positive or all very negative (luckily very few were negative). There was only 1 response in the middle.

  6. Rabbi Hayim Herring  •  Oct 18, 2009 @6:43 pm

    Moshe–thanks for sharing your expertise and experience. While evaluation tools are now readily available, they still require the kind of expertise that you’ve mentioned. The worst thing to do is to realize after a survey that something as central as asking valid questions was neglected. Hayim

  7. Jordan Goodman  •  Nov 4, 2009 @7:58 am

    Shalom All,

    The best assessment tool is the one that actually catalyzes new action i.e.,change!

    Blessings,
    Jordan

1 Trackback

Leave a Reply

Allowed tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>