         |
 |
|
Questionnaires and Surveys: A quick and dirty overview
Purposes: To rapidly determine
how a relatively large group of people respond to predetermined
issues and focused questions. Unlike interviews which
allow the researcher to probe a relatively small number of
respondents in depth, surveys are less flexible and lack the
ability to ask the follow up question. Surveys are
often a second step after pilot interviews or open-ended surveys
have been completed. Content: may be on issues of
fact or as a way to assess attitudes or opinions. Surveys about
factual issues are easier to construct. Opinions are the
expressions of deeper, guiding attitudes. Attitudes are
convictions, fears, inclinations. Attitudes are
expressed when “triggered” by some object or event. (e.g.,
response to the Katrina fiasco). It’s relatively
easy to assess opinions by using a simple questions and a rating
scale. Attitudes
require sets of 5-20 related questions with which respondent has
to agree or disagree. Survey Formats and getting people to
respond: People need to see it as worthwhile –
altruistic or serving some need of their own.
Surveys may be
1.
mailed,
2.
distributed electronically via email or forum,
3.
given out in classes, or
4.
conducted verbally (an interview) either in person
or by telephone.. Writing clear, concise questions that
relate to the issues of interest is the most difficult part of
using survey methods. Question formats:
·
open ended. No fixed answer.
Open ended questions are very
difficult to evaluate and should be used sparingly once the
pilot stage has passed and categories of answers have been
identified.
·
Close-ended: Have a fixed set of answers to
respond to. May be factual or attitudinal. These are quick and easy to
evaluate. However, providing answer categories can
introduce bias by giving people responses they may not have
thought of, or, by not including a person’s choice the person
can become negatively influenced toward the survey. Close-Ended Questions:
Response Formats 1. Fixed choice e.g., three ways to structure
responses to closed ended questions. Which do you prefer?
Problems? How many time per week do you meet each science class on average?
Circle the best answer
Check the best answer
Circle the number of the best answer 2 or 3 ___ 2 or 3 1. 2
or 3 4 ___ 4 2. 4 5 ___ 5 3. 5 other ___ other 4.
other 2. Rating scales
These are used to measure intensity of response on some set of
ordered categories
Strongly disagree disagree
agree
strongly agree
no opinion I like ice cream
__
__
__
__
__ Joe is doing a good job
1
2
3
4
0
Too little
About right
Too much The amount of
reading is
I--------------------------------------I--------------------------------I 3. Ranking schemes
Respondent
must prioritize a set of options. Important that
directions be clear to eliminate multiple scores of any number. e.g.
Please tell what is most important to you in a teacher.
Rank the most important item “1,” the next most important item
“2,” and so on. Do not use a ranking more than once.
Use only whole numbers. _____ Being
prepared _____ Being
fair _____ Knowing how
to motivate students _____ Knowing
subject matter _____ Good
classroom management Criteria
for writing close-ended questions.
Response categories must be:
exhaustive, mutually
exclusive, clearly
defined in terms the respondent is likely to understand
examples
1. Not exhaustive
Sciences courses taken
Chemistry _____
Physics _____
Biology _____ Number of
different preps per day
_____ 1-3
_____ 3-5
_____ 5-7
How many science
classrooms are in your school?
___ none
___
1-3
___
4-6
___
more than 6
What kind of science furnishings are in science classrooms at
your school?
_____ tables
_____ desks
_____ lab benches
_____ stools
_____ chairs
_____ storage cabinets
_____ lockers Sequencing
the questions: Use the inverted funnel model. Start with narrow questions,
followed by broader ones. Go from facts to
generalizations, from specifics to abstract ideas. The first questions: 1 Are the ones
most often responded to favorably. They also often shape
the respondents attitude to the survey and subsequent responses.
2. Should put
the responder at ease because they are easy to answer and don’t
touch on sensitive issues. 3. Are
usually closed ended For example.........
1.
What grade were you in when you first encountered science
in school?
2.
What were the science lessons like?
a.
Did you use a text?
b.
Did you have a lab?
c.
Did you do investigations?
d.
Did you write papers?
e.
other?
3.
In general, how effective do you think your early science
experiences were?
4.
In your opinion, what would a very good first science
experience be like? For every question or group of questions
with similar response types, you need to write a clear set of
instructions.
They can be simple. “Circle the one choice that best
reflects your opinion”
Or, they can be complex. See the rank ordering example
above. If your questionnaire is long, you may need
to introduce different sections of the questionnaire with
separate instructions. If you are going to need parental
permission or are going to mail the survey, you need a cover
letter telling the purpose of the survey. Otherwise, the
survey needs a little introduction at the beginning that helps
to make the likelihood of people responding higher. These
guidelines are similar to information to be given before
conducting interviews.
1.
Identify who is doing the survey
2.
Tell why the survey is being done. There may be a
conflict between research needs and ethics here.
3.
Tell why it is important that they respond to the survey
(how they benefit)
4.
Confidentiality and how it will be maintained (if
appropriate). This is
especially a problem in matrix formatted questions.
E.g., Don’t you
think the best teachers ……?
e.g., You wouldn’t say this text was too bad, would you?
e.g., Most people say that…., what do you think?
e.g., issues like
starvation,
Food should be sent to starving people in Bosnia
SA A
D
SD N
5.
Threatening questions: These touch on
anxiety arousing topics like sex, drugs, religion, gambling.
People tend to underreport their activities. Can
increase by: Using a long explanatory introduction.
Avoiding short questions
Using open-ended instead of closed questions
6.
Double-barreled questions: These are two in one.
e.g. Would you say most science is inquiry based and fun? If both elements are important, then sort
out the questions and ask them separately. Or, you could have people rate them.
Which of the two elements of a science program is most
important?
1.
that it be inquiry based
2.
That it be fun Reference: David Hopkins, A Teacher’s Guide to
Classroom Research, Philadelphia: Open University Press,
1985 |
