How confident are you that you know what's really on your employees' or customers' minds? If you have conducted surveys to find out, are you sure that the results are accurate, complete, and useful? My observation, which is based on my extensive experience in designing and implementing workplace surveys, is that employers often end up with results that range from slightly to totally misleading because their surveys are poorly planned, designed, and implemented by well-meaning individuals who do not have the necessary knowledge and skills.
Is it important to you that surveys provide meaningful, accurate, and useful information? If so, before you go to the time and expense of conducting a survey, you may want to consider a number of little-known "secrets" that collectively can make the difference between success and failure. These elements will go a long way toward making your surveys more effective by providing actionable information.
How can you boost the likelihood that your organization's surveys will be effective? First, ensure that the critical success factors are present. They include:
- Management buy-in and active support, not merely lip service
- Validated scales that measure the desired topics
- Management's promise (in advance) to act on the results
- Effective, on-going communication with the target sample or population
- Fulfillment of the promise to take action based on the survey results
Second, make certain that the people who design and implement your surveys have the necessary skills and experience. If you do not have those skills in-house, find a qualified outside person to conduct the survey - and have that person teach the skills to your employees.
Third, address the important yet little-known elements described below before crafting a single question.
Planning the Survey
Be sure you can answer each of these questions clearly before you begin designing the survey:
- Why are you conducting the survey - i.e., what is its purpose? For example, do you want to gather information for specific or general action, or obtain general "climate" information? The answer to this question will help you identify what specific types of information you need to obtain.
- Are the survey's purpose and process clear to everyone involved?
- What topics will be covered - and not covered? How are the included topics related to the survey's purpose?
- What is management willing and able to do with the results? If the answer is "nothing," STOP right there. Because conducting a survey creates expectations that something will happen, doing nothing results in negative consequences due to unmet expectations. It also sharply reduces management's credibility: the next time there is a survey, past participants will be reluctant to respond.
- What level of confidentiality or anonymity are you prepared to offer?
- Do you have the expertise in-house to conduct the survey or should you outsource it? If you use an outsider, make sure the organization's services are a good fit. For example, will the survey be customized for your specific needs or will it be generic?
Technical Aspects and Logistics
Those who are not trained in designing and implementing surveys often overlook important logistical elements, including the following:
- How to maintain the integrity of the process - e.g., promising and maintaining confidentiality, using the results, giving honest feedback to respondents
- Technical aspects - e.g., the survey's length, reading level, form of administration, accessibility (if administered on-line)
- Appropriate sample size - i.e., one that will allow you to obtain reliable results
- Methods for identifying the topics to be covered in the survey - e.g., focus groups, interviews, informal discussions, recent or upcoming changes, management's perceptions
- Timing issues - e.g., when to conduct surveys, how often to conduct them
- Length of survey
- Method of administration - e.g., paper and pencil, online, e-mail
- Importance of conducting a pilot test of the survey
- How to boost response rates
- Importance of having an expert (in-house or from the outside) analyze the data
The quality and reliability of the questions or items on a survey are critical to your ability to elicit useful and accurate information from the respondents. I cannot overemphasize the importance of effective questions. Here are some of the issues that should get your attention:
- The relevance of each item or scale to the survey's purpose - e.g., collect demographic data such as work location, tenure on the job, and job classification only if using them to analyze the results will provide actionable information
- The clarity of the questions from the respondents' point of view
- The precision of words and their unambiguous meanings
- The absence of acronyms and abbreviations, even when you think employees should know them
- The use of scales instead of single items, a practice that generally enables you to obtain much more accurate, useful, and actionable information
- The confidence that the scales you are using have been validated for your intended purpose; do not rely on an off-the-shelf scale whose instructions contain assurances that "This is a valid scale," as it may not be valid for your intended purpose
- The appropriateness of the items - i.e., because all questions are not alike, it is important to choose those that will help you obtain the information you need and avoid those that will confuse respondents
Communication and Action
Communication is a key success factor to an effective survey. Similarly, you must take action once people have taken the time to provide the feedback that you requested. Personally, I will not work with an organization whose managers refuse to commit to taking some kind of action because I don't want to be part of the resultant loss of management credibility. Here are important points related to communication and action:
- For best results, communicate with your target sample before, during, and after the survey
- Decide in advance who will receive the results, and in what format (e.g., highlights, detailed report)
- It's okay not to "fix" everything right now - i.e., management doesn't have to say "yes" to everything respondents want
- Saying "no" or "not now" is fine as long as you explain your reasons for those answers, which can be general rather than specific
- You must be honest
- Identify the appropriate actions based on the results
- Implement the actions
Now that the "secrets" of effective surveys are out, you may want to reconsider your answer to my question regarding the confidence you have that you really know what's on your employees' and customers' minds. Although careful planning of a survey and attention to detail are very time consuming, the payback in terms of accurate information far outweighs your investment. As long as you are going to the trouble of conducting a survey, why not ensure the results are as accurate, as complete, and as useful as possible?
Pat Lynch, Ph.D., is President of Business Alignment Strategies, Inc., a consulting firm that helps clients optimize business results by aligning people, programs, and processes with organizational goals. You may contact Pat or call (562) 985-0333.
Return to Research News Page
Copyright 2010 Business Alignment Strategies. All rights reserved.