- There are at least two ways to conduct surveys
on the
- Internet: using email and using forms on the WWW. Before discussing
those two methods let me make a methodological acknowledgment. There is no way
either an email or a WWW survey could meet acceptable standards for random
sampling of subjects, with the possible exception of a survey dealing with email
or WWW usage, and even then a kilo or two of salt would be necessary to
interpret findings. "Internauts" are still a minuscule minority and a vastly
unrepresentative minority at that. Anyone intent upon skewing an Internet
survey could easily do so, and the unintended misfirings of "newbies" would skew
findings even more. Nevertheless, there is still a role for email and WWW
surveys in conducting research.
- If studies based on unrepresentative samples
were excluded
- from social science research, whole sections of library
shelves would begin to look like supermarkets in the former Soviet Union. Such
rigorous standards would require that we exclude all surveys relying upon
college student subject pools. While that prospect has more than just a little
appeal, when viewed in context, and when not relied upon exclusively, student
subject pools serve a useful purpose: they allow us to pre-test survey
instruments, gain preliminary assessments of the utility of our hypotheses, and
last, but not least, tap a cheap, captive, compliant, resource---our own students.
- While all the same criticisms and more can be
applied to
- Internet surveys, there are several advantages to Internet
surveys that make them potentially even more useful than student surveys.
Internet surveys can 1) be cheaper (no paper, no stamps, no interviewer or
survey proctor salaries, and no phone charges--yes, our home institutions pay
access charges for the Internet, but they do so whether we use email and the Web
or not); 2) include a much wider range of subjects in terms of age,
occupation, geography, and education; 3) be collected from a much larger
number of people; and 4) be coordinated across a much wider range of
institutions much more easily than traditional student surveys. Used in
context, therefore, there is a role for Internet surveys in social science
research.
- Email Surveys
- There is really nothing fundamentally different in conducting
- an email survey as compared to the standard mail survey. Although I know of no studies of this issue (doesn't mean there aren't any), my prediction is that you will find a higher response rate to email surveys than is the case with mail surveys since the comparative costs of responding are less. The delete button is a bit closer than the waste paper basket, but the reply option takes much less effort than a trip to the post office.
- To increase the likelihood that a subject will respond to an
- email survey, the survey must make it easy for the subject to answer each item, and the items should be kept to the absolute bare minimum. Provide space between each question for the response, and keep all the introductory material brief and to the point. Otherwise, everything that applies to mail surveys applies to email surveys. The "trick" to email surveys is finding acceptable ways to get your survey to potential subjects.
- In the realm of email, nothing is more annoying
than spamming.
- Stepping over the line between an email survey and a spam is easy to do.
Under no circumstances should you post an email survey to a list without first
consulting the list owner and gaining permission. Likewise, it is probably best
to check with your academic computing staff about their policy concerning the
use of student or faculty email lists for a survey. To broaden your survey,
check with colleagues who might be willing to forward your survey to a list on
their campus. You could certainly use a professional email list such as
POL-PSYCH list to solicit collaborators in such a project. A useful way to
find suitable lists to search for possible collaborators is discussed in Lesson
6 of Crispin's Internet roadmap.[9]
- Assuming, then, you gain permission to post to a
list or use
- your own and colleagues' academic-based mail lists, be sure that
the "reply to" address is your own address, NOT the list address and NOT the
person who may have forwarded the survey for you. Also be sure that in the
survey itself you tell the subjects to use the "reply to" option, not the reply
to sender option, if there is one in their email application.
- WWW Surveys
- Spamming is not a problem with WWW surveys since a
- person must voluntarily arrive at your website in order to encounter the survey and need do nothing but move on to the next site or back to the last site to avoid the survey. The problem with Web surveys is getting people to your website, then getting them to go to the trouble of answering the survey. Unfortunately, it is highly unlikely that you can make the survey attractive enough, well-known enough, and brief enough to generate much of a response, and therefore it is unlikely the WWW will prove particularly useful as a survey instrument.
- There are ways, however, to get more traffic
running over a
- survey and to get some value from a Web survey. One way to
increase traffic is place the survey on many sites. It will take effort, but as
more and more colleagues find their way onto the Web, we can begin to dedicate
space to "shared research". A section of our personal webpages could be set
aside for collaborative surveys or experiments. It will be impossible to
completely eliminate the possibility of a single person taking the survey more
than once, but there are techniques to prevent casual returnees from submitting
another response. Committed ballot stuffers can never be completely barred and
since many Web citizens operate from shared machines, existing security methods
will bar at least some people who might want to respond from responding. The
limits of Web surveys should therefore never be underestimated.
- Another way to increase traffic over a Web survey is to pay to
- put the survey on a heavily used commercial site. Many sites get tens of thousands of "hits" per day and even if less than one percent bother to check the survey and respond, that could result in a rather large amount of data, albeit not data of the highest quality.
- Assuming, then, that you still want to try to
put a survey on
- the Web, how is it done? Putting a survey onto the Web
involves the same techniques as putting an exam on the Web or providing response
forms for a Web experiment. In the teaching section below, I have provided a
template that can be used to create any variety of survey forms. These
templates are by no means comprehensive, nor definitive, but they will get you
on your way with little effort after you have mastered the very basic elements
of writing HTML. For information on constructing a survey, go to "Online Test
Taking", in section III, part C.
- The Web has far more potential as an
experimental resource
- than it has as a support for survey instruments.
Still, great caution must be taken interpreting results since most of the
same subject pool problems exist. Since valid experimental results can be
produced with far fewer numbers of subjects than is the case with surveys, the
WWW can be used in creative ways in controlled circumstances to produce far more
diverse and representative samples than is often the case in a university
setting. With the help of collaborators at other institutions it would be
possible to expose several small subject pools to simultaneous experimental
manipulations over the WWW.
- For example, it is not difficult to imagine conducting an
- experiment dealing with a candidate's physical appearance using exactly
the same stimuli under comparable circumstances on three different continents
simultaneously. Small subject pools could be gathered at several locations in
Great Britain, the U.S. and New Zealand and directed to a Web site with the
experimental material. Half the subjects at each location would be given one
url, and half another. Written campaign material would be identical, the only
differences being party attributions according to location, but the accompanying
graphic material could be manipulated such that candidates independently judged
as more "attractive", or more "charismatic" would be shown to half the subjects
as "conservative" candidates and to the other half as "liberal" candidates. The
graphic material could be either photographs embedded in Webpages, or short
video clips also embedded in Webpages. Subjects would then answer questions on
the Webpage and post the responses to the investigator.
- Naturally, such an experiment could be done
without the
- WWW. Using the WWW, however, has several advantages. First,
such a Web experiment would be far less costly than producing and shipping video
and graphic material to several sites. Second, the conditions of exposure at
each setting can be better controlled since the same Web site would transmit the
material and instructions. Third, the collection of data from far flung sites
is virtually instantaneous. Fourth, the subject pools can be far more
culturally diverse than the general experimental subject pool. While none of
these advantages will revolutionize the conduct of social science inquiry, the
advantages are sufficient to warrant development of methods for conducting
experiments on the WWW. For the details of placing such material on the Website, go
to "Online Test Taking", in section III, part C.
- By far the most important ways to use the Internet for
- research purposes is to gather information, consult primary sources, and share databases. In this section of the workshop we will go through the basics of searching the Internet and visit several sites with databases.
Go to Section I: Internet Basics
Go to Section III: Teaching
Back to Table of Contents