CONCEPT MAPPING ASSESSMENT:

Concept Map Assessment of Classroom Learning:

Reliability, Validity, and Logistical Practicality

JOURNAL OF RESEARCH IN SCIENCE TEACHING            VOL. 36, NO. 4,      PP. 475–492 (1999)

            In 2003 the National Institutes of Health (NIH) initiated a new push to emphasize interdisciplinary collaboration.  The initiative was brought on by the realization that findings from increasingly nuanced research programs which, while very useful, were not being translated across specializations.  This new paradigm seeks to fund research that incorporates creative interdisciplinary approaches. One program which has emerged as a result of the initiative is Human Pharmacogenomic Epidemiology (HPE).  The goal of the research in the paper is to design an assessment rubric to explore the understandings of the various participants in these types of projects as to how, and whether, the new interdisciplinary approaches will work. The authors used Concept Mapping as the analysis tool. One main goal was to identify how individuals cross correlate concepts associated with their particular field of interest with those of the other potential members of the team.

            One major challenge for the researchers was to come up with a method by which they could consistently and reliably evaluate the commonalities and differences among the very professionally diverse members of the group. Previous studies of these types of conceptual relationships have relied on ethnographic or purely observational data. In contrast to these very time and labor intensive methodologies, the authors of this paper were determined to develop a  “…mixed-method study that did not rely on observation, could be repeated periodically with only moderate effort, and would produce useful data that documented how actors understood existing relationships and commonalities among HPE participants.”

             The subjects of the research were 31 members of the HPE team including bioethicists, bioinformaticians, biostatisticians, epidemiologists, geneticists, and pharmacologists. Methods involved a two step process, first, an e-mailed survey and second a follow-up voluntary personal interview, at which each respondent was asked to draw and discuss 2 concept maps.  One map was to depict the respondents understanding of current relationships between the various participating disciplines and the second map was to outline how the participant felt those relationships would change under a successful interdisciplinary program.

            Each concept map was scored twice; once as a “relationships matrix” and then as “verbal descriptors”.

            The authors conclude that the use of concept mapping “…was a particularly successful research method applied to this group.”