Sex, Drugs and Public Hangings
A series by Spiralbound.net on social deviance and punishment in the United States and Europe
No research project is without its limitations, and even before the original research strategy had to be changed this one was no exception. In the end, the limitations of this study can best be broken down to, and discussed in three separate sections: the the survey recipients, the newsgroups used to contact European subjects, and the survey itself. Allow me now to take a few moments to discuss each of these separately, before moving on to analyze the the data.
Ideally, I would have been able to obtain a totally random, reasonably large sample from each of the four countries. As I hinted at above, this may have been possible in the United States, but without either traveling to Europe, or having a very dedicated contact in each country, doing this in across the Atlantic would be nearly impossible. Thus, in order to be a t all practical, I would have to take what I could get in terms of survey respondents.
In the United States, I can be certain that those who took the survey were either randomly selected Plymouth State College students, or college Information Technology Department staff. The first, and most obvious limitation of this group is the fact that they are associated with a college. Colleges and universities tend to be, almost by definition, more liberal than the median of American society, and as such, these subjects were no doubt more likely to answer in favor of social programs designed to aid deviants than a true cross-section of the American public. Also, we must consider the demographics of those associated with colleges and universities. This group is likely to be younger, more educated, and wealthier than the average, randomly selected person, which is almost certain to account for an increased tendency to answer in support of social programs. Secondly, the survey was only made available to those who used e-mail. While it is difficult to find a student or college staff member who does not use this technology, they certainly do exist, and as a result, the less tech-savvy would have been eliminated as a research subject. Finally, the issue of geography needs to be addressed. The survey was only distributed to those living, in the North Eastern United States. This is important to remember because this region (aside from New Hampshire) tends to be slightly more liberal than the rest of the United States, and many people at Plymouth State College originally come from surrounding states.
Overseas, some of the same problems were faced such as the non computer / Internet users being eliminated as a subject, but the most important limitations come from the way the respondents were contacted.
Since the bulk of my European respondents found the survey listed on one of the three Usenet newsgroups listed in the table above, it is absolutely critical that the limitations of this method be addressed. Newsgroups are organized in terms of subject, and as such a person subscribes to a newsgroup only if that person is interested in reading the discussions therein. Knowing this, it becomes clear that I have reached only a very limited audience. I chose the newsgroups I did, not because of their subject matter, but rather because of their large readership, and since the only requirement for completing the survey was that the person be from either Germany or England (I had, at this point, given up on France), I wished to reach as many people as possible. Thus, the most apparent limitation of news groups is the narrow scope of their readership. Since I posted to only three groups, it is easy to see that I did not reach a very diverse audience.
Another thing that is important to know about newsgroups is that they can be read by anyone, regardless of nationality. Just because a group is part of the “UK” or “DK” domain does not mean that someone from another country is eliminated from reading it, so there is the risk that a person who completed the German survey, for instance, was not, in fact German. In an attempt to control for this, I kept an eye on my UNIX server’s web access log, and ensured that the number of British or German surveys matched up with the number of British or German connections. Even with this control, however, I can only confirm that the page was accessed from a given country, and I have no way of verifying that the individual taking it was actually British or German. I had to depend on the honor system for this.
One final note about newsgroups: it is important to note that Usenet readers can, and do, become VERY hostile should they run across a posting which they don’t like. Survey postings on the newsgroups are very unpopular, and are thought of by most to be inappropriate. Knowing this. I realized that it would become necessary to keep a careful eye on the returns to ensure that no malicious subject would intentionally corrupt my data. Aside from receiving a few e-mail messages asking why I had not included certain questions, I only ended up having to throw out two responses. The first, clearly malicious, stated among other things that there should be a death penalty for “crosposting twits on newsgroups”, and that the state should provide free healthcare for people who have suffered mental trauma caused by “crosposting twits on newsgroups.” The second throw away was simply a religious rant.
While not perfect, I do feel that the actual construction of the survey was good, as it yielded some very useful results. I was approached by several people throughout the study who stated that it was “fun” to take, and that they had enjoyed putting in their own “two cents”. There are, however, three important ways I believe the survey could be improved. First, as I stated above, I believe the option for subjects to qualify a “No” answer should have been added. Secondly, I believe there should have been a mechanism built into the PHP database front end to ensure that the same person did not take the survey twice. Finally, I would have liked to employ a system by which I could verify that the person taking the survey was actually a citizen of the the country he or she claimed. Unfortunately, this would mean collecting personal data which may have prevented the study from passing the Human Subjects Committee.