Bebell, D., O’Dwyer, L., M., Russell, M., & Hoffmann, T. (2010). Concerns, considerations,
and new ideas for data collection and research in educational technology studies.
Journal of Research on Technology in Education, 43(1), 29-52.
The authors of this paper describe three major errors made by many researchers in the educational technology field. The first is the broad use of the term “technology usage.” Because this term can mean anything from using technology to plan instruction to answering parent emails, it is important that the researcher be more specific in what type of technology usage they are wanting to describe. Another flaw often seen in these types of studies is the way surveys are used to ask participants to describe their frequency of use. Creating general statements like once or twice a year, several times a month, or several times a week tend to give unreliable data points. Finally, the authors point out that using state tests to measure what students learned through technology is not an accurate measure because too many skills are measured within each test. Even when the researcher can break the test into specific skill sets by question, there are not enough questions in each skill set to provide an accurate depiction of how or if the technology was effective in teaching a skill.
Each of the errors described in this article are well documented by the authors. They provide specific examples of educational technology research which was ineffective because of one of these errors in research methodology.The examples provided allow the reader to understand why certain research methods which are commonly used are unreliable. In addition to explaining how the errors could provide ineffective data, the authors discuss how they have been able to avoid such problems and gather data that is more reliable by altering the traditional methods. These changes have given them the ability to better illustrate the effect technology has on education.
Educational technology researchers benefit from the explanations and suggestions that are provided in this article. While the suggestions made by the authors are useful, the best use for this article is to help educational technology researchers (especially those new to the field) to realize the importance of examining and reexamining their research methodologies when they are conducting a study of the impact technology has on learning.