Mertens, Research and Evaluation in Education and Psychology, 6e Chapter 6: Survey Methods

Mertens, Research and Evaluation in Education and Psychology, 6e Chapter 6: Survey Methods.

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Design Phase The researcher should begin to articulate the purpose(s) of the survey, state specific objectives, consider the types of information needed, and evaluate design options..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Design Phase Purposes of Surveys Surveys can be used for a variety of purposes: The American Psychological Association (APA, 2017b) conducted a survey to determine how stress affects people's health. The National Institutes of Health conducted a survey to "develop and test the effectiveness of an innovative community health worker intervention to expand access to hearing health care among older adults facing health disparities." The U.S. Department of Education supported two important surveys focused on the experiences of students with disabilities. Using a feminist theoretical framework, Lajoie et al. (2022) used an online survey with follow-up interviews to study mental health researchers' concept of vulnerability in research ethics. A number of national surveys are conducted in the United States on a regular basis that yield useful information. The labor market statistics generated by the Bureau of Labor Statistics and the Bureau of the Census. Basic data about health conditions. Use of health service. Behaviors that affect the risk of illness from the National Health Interview Survey carried out by the Bureau of the Census for the Public Health Service. Information about these surveys is generally available on the internet. A good first step in starting a survey is to write the purpose of the survey in 25v words or less. A carefully worded purpose statement can be used as a basis for the next steps in the development of a survey..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Design Phase Design Considerations The researcher has a choice between simple descriptive, cross-sectional, and longitudinal approaches. The simple descriptive approach is a one-shot survey for the purpose of describing the characteristics of a sample at one point in time. The cross-sectional design involves examining the characteristics of several groups at one point in time. Longitudinal designs survey one group or cohort of subjects at different points in time. Cross-sectional designs have the advantage of collecting information in a shorter time frame. The disadvantage is that the experience of the students who are now in the fifth grade may have been different in the first grade, compared with students who are now in first grade. The advantage of longitudinal research is that it follows the same (or highly similar) respondents over a period of time. The disadvantage is that it takes a long time to do it, and conditions may not be the same for students who are graduating three years later. The students' Individualized Education Programs (IEPs) often define goals for individual growth rather than for progression from grade to grade..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Design Phase Data Collection Choices Technological advances have added options for the conduct of survey research, including email, web-based surveys, video-based surveys, mobile phone surveys, or a combination of these as methods of data collection. The method selected depends on the purpose of the survey, the nature of the data to be collected, cost factors, and the size and characteristics of the sample. Advantages and disadvantages are associated with each approach. For example, mail surveys are good for collecting detailed information in a closed-ended format, the cost is relatively low, and they can allow a respondent to consult records before responding (e.g., checking on the level of expenditure within specific categories). The disadvantages of mail surveys are that the surveyor does not have an opportunity to probe for more in-depth answers or to determine if the respondent understood the questions appropriately..

[Audio] 6-6 Examine the benefits and challenges of the use of technology in an example of survey research. Design Phase Data Collection Choices Surveys and Technology Surveys can be delivered through email, mobile texting, or social media. The advantages of using technologically delivered surveys include convenient access to samples; reduced costs; faster responses; more interactive or tailored formats; quick troubleshooting; automated data collection, scoring, and reporting; and access to larger samples. Response rates can sometimes be enhanced by using multiple modes of contact, combining mail, email, and web-based delivery systems. The use of a web-based survey mode offers opportunities to present information creatively, such as by including video clips in the survey. Multimode designs are becoming more and more popular because of the ease of accessing people via email or other social media methods such as texting. This allows researchers to give a heads-up that a survey is coming or to send reminders to people who have not yet responded..

[Audio] 6-3 Explain response rates and strategies that can be used to increase response rates. Design Phase Response Rate Considerations Researchers have investigated the many factors that may influence people to return mail or web-based surveys or to agree to participate in phone or personal interviews. The AAPOR offers a definition of response rate and various methods to calculate them at their website (www.aapor.org). The response rate is the number of complete interviews with reporting units divided by the number of eligible reporting units in the sample. Response Rate 1 (RR1), or the minimum response rate, is the number of complete interviews divided by the number of interviews plus the number of noninterviews plus all cases of unknown eligibility. I = Complete interview P = Partial interview R = Refusal and break-off NC = Noncontact O = Other UH = Unknown if household/occupied housing unit UO = Unknown, other Calculation of response rates has become quite complicated by the use of multimode survey methods and because of the increased use of online surveys in which the total sampling frame is not known. Researchers in survey research have consistently reported that topic salience and use of incentives have a positive effect on response rates. Factors that influence a decision to respond to surveys include having the time to respond, interest in the survey topic, and perceived benefit to society or self. One recommendation to achieve higher response rates, while keeping the overall costs low, includes the use of mixed-mode strategies, starting with the less costly method first. Other suggestions for increasing response rates: Questionnaire length sometimes has an effect on response rates. Short questionnaires will be returned at a higher rate than long ones unless the respondent endorses the importance of the topic. Monetary and nonmonetary incentives can increase response rates. Sponsorship (through a respected organization or signature by a high-ranking administrator) can yield higher response rates. Good timing is very important, especially with populations that organize their time around a school calendar. Avoid sending the questionnaires at the very beginning or end of a school year or around a holiday period. For mailed surveys, be sure to make it easy to return the questionnaire by providing a stamped, self-addressed envelope (and maybe a pencil). Consider using an attention-getting delivery mode, such as express mail, special delivery, or airmail, and possibly printing your questionnaire on brightly colored paper. Handwritten envelopes seem to attract more attention now than computer-generated address labels. Your follow-ups could be delivered via humorous postcards or telephone calls. A response rate of around 70% has generally been recommended as acceptable. This recommendation is based on the assumption that respondents and nonrespondents are fairly similar..

[Audio] 6-7 Assess survey approaches, specifically in how they address issues of gender, disabilities, racial/ethnic minorities, and children. Design Phase Theoretical Lenses and Diverse Populations Feminist theories: Feminists have expressed different views in terms of survey research; some view it as a rigorous and scientifically sound method that has credibility with many people in the social science community and with the public at large. However, other feminists express deep distrust of survey research and other statistically based forms of research. Feminists' criticisms of quantitative, closed-ended survey research are based on the problem of oversimplifying complex issues by reducing them to a limited number of questions and response options. Literacy levels: If the population of interest has documented low literacy levels, then researchers need to be aware that the use of a self-completed print survey is not an appropriate strategy. In such circumstances, they could consider face-to-face interviews. Diaz et al. (2017) used face-to-face interviews because their participants had low literacy levels. They interviewed Latina mothers who were immigrants in the United States; the researcher recorded the mothers' responses. Children: Surveys conducted with children must follow the same ethical guidelines discussed in Chapter 11 on sampling that all research with children follows. In a review of active parental consent for their children to participate in surveys that dealt with sensitive behaviors (e.g., alcohol use, smoking), Mellor et al.'s (2008) review of possible biases associated with differential rates of parental consent noted that parents were more likely to give consent for their children to participate if the parents did not believe that their children engaged in the behavior. In school-based surveys, researchers can use strategies for parents to either actively or passively give their consent. Teenagers: Researchers experience different challenges when collecting data from teenagers than when working with adults or children. Bassett et al. (2008) describe their frustrations when conducting a survey of families from three racial/ethnic groups in Canada (European Canadian, African Canadian, and Punjabi Canadian). They struggled with strategies to engage the teenagers when the young people had been "volunteered" by their parents as well as how to get the teens to express themselves beyond a monosyllabic response. Disability perspectives: Combinations of methods can result in better response rates. For example, the NLTS2 survey of youth with disabilities used three different methods (Wagner et al., 2007). If parents said their children could respond to questions, the interviews were conducted by telephone (computer-assisted telephone interviewing—CATI). If the parents said their children could answer for themselves but not by telephone, then the data were collected by a mailed questionnaire. If parents indicated their children could not answer questions through either telephone or printed questionnaire, then the interviewer asked the parents to answer those questions..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Sampling Plan Identification of the Respondents When you conduct a survey, you should inquire about who has access to the kind of information you need. Your choice of the source of information will depend on the following factors: Who has access to the information? You may need to inquire to find out who would have access to the kind of information that you need. The characteristics of the people who have experienced the event in terms of age or disabling conditions. The type of information needed can help determine the best source of that information. In longitudinal studies, the researcher must also decide which type of group will be followed over the years. Trend analysis involves studying the same general population but not the same individuals. Cohort analysis involves studying the same specific population but not necessarily the exact same individuals. Panel analysis involves following the exact same individuals over the time period of the study..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Sampling Plan Population Definition Once the general nature of the respondents has been identified, it is time for the researcher to become more specific about the information sources. The conceptual definition of the population must be translated into operational terms..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Sampling Plan Sampling Decisions Sampling procedures include probability sampling, and purposeful sampling. If a probability sampling procedure is used, researchers need to specify to whom the results will be generalized. If a purposeful sampling procedure is used, the researcher needs to provide sufficient details about the people in the study to communicate to the reader their important characteristics. Serious bias can occur in the interpretation of results from a survey that does not make explicit its sampling strategy and the characteristics of the respondents. When using a probability sampling approach, the researcher needs to specify a sampling frame—that is, a list of the people who have a chance to be selected. A sample can be representative only of the population included in the sampling frame. Three sources of error in sampling for surveys require attention: coverage, nonresponse, and sampling. Coverage errors arise in two circumstances: (a) People who should be in the sampling frame are not there, or (b) people who are truly ineligible are in the sampling frame. A nonresponse error occurs when someone refuses to be interviewed or to complete the questionnaire or cannot be reached. A sampling error occurs because each sample drawn from a population is somewhat different from any other sample that could be drawn..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Sampling Plan Sampling Decisions Sampling Racial/Ethnic Minority Populations Sampling strategies with racial/ethnic minority populations can guide the researcher in decisions about survey research with this population. The National Opinion Research Center (NORC) recommends using a soundly constructed national, cross-sectional survey to address potential underrepresentation of Black and Latino respondents. Members of racial/ethnic minority groups are generally less willing to participate in research studies for a variety of reasons, some of which are rooted in abuse by researchers, ongoing concerns about discrimination, and distrust of institutions..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Sampling Plan Sampling Decisions Random Digit Dialing Random digit dialing (RDD) is a technique that researchers use in selecting samples for telephone surveys. It involves identifying areas to be sampled (at random) and their corresponding area codes and exchanges (the first three digits of the phone number). It is possible to identify which exchanges are used primarily for residential telephones and to focus the sampling on that subgroup. Researchers can combine landline and cell phone coverage to ensure that a wider population is sampled. Response rates to telephone surveys have fallen off precipitously over the last 10 years, making RDD a less-effective strategy to collect this type of data..

[Audio] 6-2 Construct different types of questions, such as demographic, sensitive and nonthreatening behavioral, knowledge, and attitude questions based on the examples in this chapter. Designing the Survey Instrument Researchers should review the literature before undertaking the task of designing a survey instrument. Specific suggestions related to different types of questions: Outline the various topics you want to include in your survey. Another alternative to devising the topics yourself for your survey is to use a strategy called the Delphi Technique: Assemble a pool of specialists and highly invested potential consumers of information about the topic in question. Using an open-ended questionnaire, ask respondents to propose a few essential statements that represent the critical needs or issues in the topic area. This may require defining such terms as critical, needs, and the field. Gather the responses and eliminate overlap or redundancies in the topics suggested. Reformat the topics into a clear and consistent survey form. Return the reformatted list to each expert participant for review. Ask participants to rate each topic on a seven-point scale ranging from 1 (very critical) to 7 (of little importance). Calculate the mean and standard deviation for each item; report these summary statistics back to the respondents. Use their responses to develop a final survey form that is sent to the intended target population. Explain to yourself why you are asking each question. Decide on the degree of structure that is most appropriate. Generally, avoid psychologically sensitive questions. Clarity is paramount. Short items are generally preferable to long items. Negative wording should be avoided. Avoid items that ask about more than one idea. Avoid jargon and big words. Avoid biased or leading questions. Emphasize critical words by using italics or underlining or bold letters..

[Audio] 6-2 Construct different types of questions, such as demographic, sensitive and nonthreatening behavioral, knowledge, and attitude questions based on the examples in this chapter. Designing the Survey Instrument Formatting the Questionnaire A few hints about the physical appearance of your questionnaire: a. Make it attractive. b. Organize and lay out the questions so that they are easy to answer. Be sure to number the items and the pages. Put the name and address of the person to whom the questionnaire is to be returned at the beginning and end of the questionnaire, even if a self-addressed envelope is included. Include brief, clear instructions. Use examples before any item that might be confusing. Organize the questions in a logical sequence. Begin with a few interesting and nonthreatening items. Do not put the most important items at the end of a long questionnaire. Avoid using the words questionnaire or checklist on your forms. Give sufficient thought to the format of the survey instrument in terms of visual layout because even small variations in the visual presentation can change how people respond to items compared response patterns for items that only gave the anchors (1 = completely satisfied; 5 = not at all satisfied) with responses that required the respondent to use the same scale but to write the number in a box. Researchers also have to be aware of the effect of format on responses to web-based surveys and check to be sure that they are functional for mobile phones, computers, and tablets..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Pilot Testing the Questionnaire Pilot testing your questionnaire means that you try it out with a small sample similar to your intended group of respondents. Follow these steps in the pilot test: Select a pilot sample that is similar to your population. In formatting the questionnaire, you may want to modify it a bit if necessary to allow room for comments to be written on the pilot version. Instruct your pilot respondents that you are interested in their reactions to the process and questions and encourage them to note any ambiguities or response options that are not included. Follow the procedures for administration that you plan to use in your study. When the data from the pilot survey are collected, read the comments, check the responses item by item, do a brief analysis, add, change, or delete any questions as needed, follow instructions regarding establishing reliability and validity as found in Chapter 12..

[Audio] 6-5 Describe how language differences between researchers and participants can be addressed. Language Differences Between the Researcher and the Respondent When the language of the researcher is different from that of the respondents, the usual method of constructing the instrument is to rely heavily on collaboration with colleagues who speak the respondents' language. variations of back translation: Translation and back translation Translation, back translation, committee review, and pretesting Two forward translations, reconciliation of the two versions, back translation, committee review, and psychometric testing Translation, expert committee, back translation, and pilot testing several times Translation, two back translations, reconciliation of the two versions, review of new version through an additional back translation, revisions, and finalization of the translated version All of these approaches rely on having someone who is proficient in the original language and the target language. Bilingual people are used to provide the translations and the back translations. Linguistic researchers have demonstrated that the idea of equivalence of words and concepts is tricky and that it is dangerous to assume that a literal translation will accomplish the goal of obtaining meaningful data from participants. Literal translations can impact the readability and comprehension of the survey. Colina et al. (2017) suggest an alternative to the back-translation approach to a more functionalist approach that takes into account the audience, purpose, and social conditions in which the translation will be used. Colina et al. (2017) illustrate two approaches to using this functionalist approach: one for already translated instruments and one for documents that need a new translation. As survey questions are developed, researchers should be aware of the contrast in questions that represent a "blame-the-victim" theory versus a "transformative" theory in the communities they plan to survey. Blame-the-victim questions place the responsibility for a problem on those who suffer from the problem and not on the social conditions that underlie the problem..

[Audio] 6-2 Construct different types of questions, such as demographic, sensitive and nonthreatening behavioral, knowledge, and attitude questions based on the examples in this chapter. Special Types of Questions Demographic Questions The demographic part of the questionnaire is commonly labeled "Background Information" and asks about the personal characteristics of the respondents. Some questions seem to serve researchers fairly well. Other background characteristics are a bit more controversial or difficult to identify precisely. Suggestions for handling race identification: Add a "multiracial" category Add an "other" category Provide an open-ended question to probe for information on race and ethnicity Add some categories not traditionally included, such as Native American, Native Hawaiian, and Middle Easterner Use the U.S. Census Bureau's 2020 categories for race and ethnicity The collection of demographic data about disabilities can be aided by knowledge of the categories as they are defined in the federal Individuals with Disabilities Education Act: Intellectual disabilities Hearing impairments (including deafness) Speech or language impairments Visual impairments (including blindness) Serious emotional disturbance Orthopedic impairments Autism Traumatic brain injury Other health impairments Specific learning disabilities A useful source of information is the U.S. Department of Education's website dedicated to IDEA (https://sites.ed.gov/idea/)..

[Audio] 6-2 Construct different types of questions, such as demographic, sensitive and nonthreatening behavioral, knowledge, and attitude questions based on the examples in this chapter. Special Types of Questions Nonthreatening Behavioral Questions Nonthreatening behavioral questions ask people about behaviors that are typically performed and easily talked about. Thoughts to keep in mind: Aided recall may help. Especially with behavioral questions, it might be helpful to put a time period in the question. This is called "bounded recall" and could be worded "In the last year . . ." or "In the last week . . ." It is best to avoid asking about "usual" behavior because this is ambiguous. If you want detailed information about low-salience behavior, a diary approach might be useful. Specify whose behavior you are interested in..

[Audio] 6-2 Construct different types of questions, such as demographic, sensitive and nonthreatening behavioral, knowledge, and attitude questions based on the examples in this chapter. Special Types of Questions Sensitive Behavioral Questions Any questions that potentially elicit a defensive reaction in the respondent fit this category. A few hints adapted from Patton (2014) about asking such questions: Open-ended questions are usually better than closed-ended questions on frequency of socially undesirable behaviors. In contrast to the general rule that was presented earlier about length of questions, longer sensitive questions are generally better than shorter ones. Use words that are familiar to respondents. Other ways of allowing respondents to indicate their answers can increase their honesty. The wording of the question can suggest to the respondents that it is all right for them to reveal some aspect of their behavior that might not be socially desirable. "Many people have taken a drink . . ." "Such behaviors occur with different frequencies . . ." "Some doctors recommend drinking a glass of wine . . ." "Did you ever happen to . . ." "Many people complain that the tax forms are just too complicated . . ." Survey researchers who are asking about sensitive topics may find that using multiple measures will enhance the reliability and validity of their results..

[Audio] 6-2 Construct different types of questions, such as demographic, sensitive and nonthreatening behavioral, knowledge, and attitude questions based on the examples in this chapter. Special Types of Questions Knowledge Questions Knowledge questions are typified by those found on tests in school. These questions ask how much a person knows about a particular subject. Henry (1996) recommends using knowledge questions in surveys to understand people's knowledge and sentiments about public policies and programs. A few suggestions for writing and using knowledge questions in surveys: Use a knowledge question as a screen before you ask an attitude question. Use an appropriate level of difficulty. When possible, reduce the level of threat of knowledge questions by asking, "Do you happen to know . . . ?" No one likes to look stupid. One technique that political scientists have used is the name recognition list—just to see if people are reporting opinions about someone they really know or someone they are pretending to know. Usually, mail surveys are not appropriate for knowledge questions, because the person could go look up the answer..

[Audio] 6-2 Construct different types of questions, such as demographic, sensitive and nonthreatening behavioral, knowledge, and attitude questions based on the examples in this chapter. Special Types of Questions Attitude Questions In attitude questions, make sure that the attitude object is clearly specified. A few suggestions for writing and using attitude questions: Consider asking about the three components of attitudes—for example, affective (like vs. dislike: How does the person feel about this?), cognitive (knowledge: What does the person know about this?), and action (What is the person willing to do about this?). Assess attitude strength (e.g., How much do you like or dislike . . . ?). Avoid questions that include more than one concept (e.g., Would you vote for me and peace or my opponent and war?). Generally, use bipolar questions rather than unipolar ones (e.g., Are you in favor of or opposed to . . . ? Are you satisfied or dissatisfied with . . . ?). Start with the answer to the bipolar question, then move on to measure the degree of positive or negative attitude. Do not use more than five points on the rating scale, unless you can give the person a visual aid with the options presented on it. Ranking should be limited to no more than five alternatives and should be done only when the individual can see or remember all the alternatives. More complex ratings can be achieved by employing card-sorting procedures..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Letter of Transmittal A letter of transmittal can be used as a cover letter for mail surveys or as an introductory letter for a phone, web-based, email, or personal interview survey. The most important function that a letter of transmittal can serve is to give the respondent a good reason to respond to your survey. Some hints related to establishing motivation for responding to a questionnaire: Appeal to authority. Appeal to self-interest Appeal to professional interests Appeal to altruism Appeal to curiosity Appeal to greed Appeal to a sense of connection with you Other things to keep in mind when writing a transmittal letter should be to specify the time frame in which the study is taking place..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Conducting the Survey The following steps summarize the process of conducting the survey itself: Be sure to send out an advance communication in the form of a letter or email or text. Enclose the questionnaire with the transmittal letter for mail surveys. Supervise the data collection. Send a follow-up to nonrespondents. You could follow this process: Send another letter and questionnaire a few days after the time limit to nonrespondents. Send a second follow-up Call a select sample of nonrespondents Control processing errors. Enter the data into the database of your choice. Clean up the data before you begin analysis..

[Audio] 6-4 Explain specific issues related to mailed surveys, phone interviews, and web-based surveys. Conducting the Survey Notes Specific to Phone Interviews It is usually a good idea to send an advance letter. Provide a brief explanation of your purpose, who you are, and what your expectations are. Make sure that you are talking to the right person! Once you are sure that you are talking to the right person, make sure that this is a good time to talk. Try to keep the phone time to a short duration..

[Audio] 6-4 Explain specific issues related to mailed surveys, phone interviews, and web-based surveys. Conducting the Survey Notes Specific to Phone Interviews On the telephone, it is sometimes best to use somewhat structured questions. Establish rapport and move quickly. Make your first questions fairly simple and nonthreatening. Allow "don't know" as an option because of lack of knowledge..

[Audio] 6-4 Explain specific issues related to mailed surveys, phone interviews, and web-based surveys. Conducting the Survey Notes Specific to Phone Interviews Use an appropriate tone of voice. Speak at an appropriate speed Keep a log of calls made and their outcomes and date and time your notes. Before conducting the survey, be sure to rehearse. Set hour-by-hour goals You can record a phone call, but you must inform the respondent that you are doing so..

[Audio] 6-4 Explain specific issues related to mailed surveys, phone interviews, and web-based surveys. Conducting the Survey Notes Specific to Phone Interviews After the Interview Allow at least twice as much time as you spent interviewing to go over the notes as soon as possible. Fill in gaps in your notes before you move on to the next task..

[Audio] 6-4 Explain specific issues related to mailed surveys, phone interviews, and web-based surveys. Conducting the Survey Notes Specific to Web-Based Surveys Internet access is growing. In North America, 93% of the population has access to the net; however, in Africa the percentage is only 43%. Web-based surveys offer advantages because of their ability to accommodate such strategies as branching, editing, tailoring, and randomization, as well as avoiding interviewer effects and reducing costs. Recorded voices can be used to present questions, and the technology can accurately record answers in formats that facilitate analysis. Web-based surveys have been used successfully to investigate several sensitive topics and to engage communities that might wish to maintain confidentiality about their identities. The capacity to creatively design the survey may provide useful ways to adapt the survey to meet the restrictions that might be encountered in surveying a person with specific disabilities. There has been an increase in the number of resources available for constructing, administering, and analyzing the surveys. There is a growing number of software development systems designed specifically for web-based surveying..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Data Analysis and Troubleshooting Your Survey Instrument Some ideas for you to use to troubleshoot your survey instrument: What are your response rates? If it is an online survey, have you checked to see what happens when the respondent clicks on the link? If you have conducted a follow-up of nonrespondents (which you should do!), you can do a comparison between respondents and nonrespondents. You can also do comparisons between those who responded on the first wave, second wave, and so on..

[Audio] 6-1 List the steps for conducting a survey research study, including the design phase, the sampling plan, designing the questionnaire, conducting the survey, and troubleshooting the questionnaire. Data Analysis and Troubleshooting Your Survey Instrument Some ideas for you to use to troubleshoot your survey instrument: How much data is missing? Is there a pattern to the items that are left unanswered? Sometimes it is helpful, with quantitative data particularly, to take a clean copy of the questionnaire and fill in raw numbers for frequency counts, percentages, and measures of central tendency where appropriate. Look for interesting findings that suggest extremes, trends, or patterns in the data. Is it possible to do cross-tabulations for subgroup analyses? Does this reveal anything about the quality of the survey as perceived by people with different characteristics? Display the data using tables and graphs..

[Audio] 6-8 Find survey research studies and critically analyze them using questions for critical analysis presented at the end of this chapter. Questions for Critically Analyzing Survey Research Questions to consider. Examine the wording of the questions. Could the way questions are worded cause bias because they are leading? Could self-report result in bias in this study? Were any other response-pattern biases evident, such as question order effects, response order effects, or social desirability, acquiescence, recency, or primacy effects? What was the response rate? Was a follow-up done with nonrespondents? How did the respondents compare with the nonrespondents? Who answered the questions? Was it the person who experienced the phenomenon in question? Was it a proxy? How adequate were the proxies? If interviews were used, were interviewers trained? What method was used to record the answers? Was it possible or desirable to "blind" the interviewers to an "experimental" condition? How did the surveyors handle differences between themselves and respondents in terms of gender, race or ethnicity, socioeconomic status, or disability? What consideration was given to interviewer effects? If the survey instrument was translated into another language, what type of translation process was used? What kind of assurance do you have that the two forms were conceptually equivalent and culturally appropriate? How was accommodation made for language differences based on country of origin, geographic region, and education level of the respondents? What sampling strategy was used? Was it appropriate for reaching adequate numbers of underrepresented groups (such as ethnic minorities or low-incidence disability groups)? Was the survey descriptive, cross-sectional, or longitudinal? How did this design feature influence the interpretation of the results?.