Keywords: design, language, community of practice, multi-method
Characterizing Design Learning Through the Use of Language: A Mixed-methods Study of Engineering
Designers
This analysis, that utilizes data from part of the Academic Pathways Study (APS) of the Center for the Advancement
of Engineering Education (CAEE), found that as a result of taking a course in engineering design and/or studying
engineering for four years, students acquire engineering design language that is common to a larger community of
practice as well as situated in their own programs and institutions of higher learning. The study also suggests
that engineering design language shapes the knowledge that students have about engineering design. Finally, students
did not always put this knowledge into practice, suggesting the need for educational improvements, as well as research
to better define the kinds of knowledge necessary to bridge the gap.
| ...although most students are able to select from a list what kind of information would be important
to have while designing a playground, many of them do not actually seek it in an open response exercise.
|
Research and Practice Implications
Using multiple methods to gather and analyze data is necessary to paint a more comprehensive picture of complex
phenomena like student learning and development. By analyzing the experiences of students from a number of different
perspectives, we are able to broadly consider wide-ranging theories like sociocultural learning theory and adaptive
expertise. These findings show that students do acquire engineering design language through engagement with engineering
curricula, programs, and institutions. The select and generate lenses (discussed below) can be used to assess both
students and curricula/programs/institutions by addressing the question: Are our students learning what we intend for
them to learn? Using tools like these, researchers and practitioners can measure the extent to which language is
being acknowledged and translated into thought by their students, as well as the effectiveness of their courses and
programs in conveying local and global priorities for engineering design. Assessment tools should continue to be
developed to measure the acquisition and use of engineering design language. To demonstrate the effects of design
pedagogy on students’ design learning, future research should be conducted at institutions offering different kinds
of design education (e.g., institutions with a focus on engineering design versus those with a focus on engineering
analysis; institutions with strong experiential learning components; institutions with an interdisciplinary approach
to design, etc.).
The demonstrate lens (discussed below) is itself multi-faceted. The closed ended survey questions revealed
that by their fourth year, students know about a certain aspect of engineering design (information gathering). The
two previous lab-based design task studies revealed that fourth-year students did not always internalize this
knowledge in a way that allowed them to employ it when doing design. Further research should address this gap by
attempting to clarify the tacit skills and knowledge that are required in engineering design.
Methods and Background
The research team used a multi-method approach to inquiry. The studies included in the paper employ three different
methods for researching and evaluating the extent to which student designers acquire the language of engineering design
and develop an ability to use this language in solving engineering design problems. These methods are closed-ended
survey questions, open-ended design scenarios, and lab-based engineering design problems. These methods allow us to
examine engineering design knowledge through three lenses: select, generate, and demonstrate:
- Select: As part of a survey on their perceptions and experiences of engineering education and practice, respondents
are asked to select the six most important activities from a previously developed list of engineering design activities. The purpose of this data collection method is to provide a lens on how respondents prioritize design activities.
- Generate: As part of a classroom assessment exercise, respondents are given an engineering design scenario and
are asked to describe their plans to solve the design problem. The purpose of this data collection method is to
provide a lens on respondents’ engineering design knowledge as reflected in their appropriate use of engineering design
language to describe a specific engineering design process.
- Demonstrate: This lens is multi-method in itself, as it consists of a comparison of a new data set with two
previously published analyses. For the new data set, respondents select from a pre-existing list in response to a
survey question about a specific design task. These data are compared to studies in which respondents were asked to
“think aloud” while designing a similar specific object or system in a lab-based setting. The purpose of this
multi-method analysis is to provide a lens on the extent to which respondents’ selection of design language in a
prioritizing task matches their approach in solving an actual design problem.
What We Found
The APS students exhibited an increased understanding of engineering design as mediated by engineering design language.
In comparison to their first year in engineering education, the students’ engineering design activity selections in
their fourth year more closely matched those of experts. Furthermore, taken as a whole, students’ design activity
priorities appeared to become more engineering-specific after they had nearly completed their engineering education.
At the same time, there was variability across student groups at different APS institutions, which may reflect
qualitatively different engineering cultures at those institutions. While commonalities are expected among
institutions all working to produce engineers for a common national and global stage, we also should expect
differences in more locally situated communities of practice that emphasize certain components of the engineering
design process.
Using the classroom-based assessment method, we found that students generated engineering design language
appropriately in response to a design scenario, as they had been taught in a freshman level introductory design
course. Furthermore, this growth in engineering design knowledge as mediated by language re-oriented these students
to a new way of explaining engineering design. Not only did students’ language become more engineering design-specific,
but the focus of their narratives shifted from an immediate solution orientation to a design process orientation.
We gained insight into students’ design abilities by comparing results from a question in the survey and the
lab-based design exercise that both provided a way for students to demonstrate an understanding of engineering
design through a problem focused on a playground design. We found that after four years of engineering education,
students had acquired the language of the information gathering design activity, and also had knowledge of the
importance of certain kinds of information to the extent that their answers became more aligned with experts’
information gathering behavior.
Despite fourth-year APS students’ recognition of the importance of budget and safety information as reflected in
their responses to the survey question, the actual information gathering behavior of a different sample of fourth-year
students did not follow suit. Less than half the fourth-year students in this sample asked for budget and safety
information during the lab-based playground design, even though the vast majority of APS students recognized the
importance of budget and safety information when prompted to select from the pre-existing list. These findings
suggest that although most students are able to select from a list what kind of information would be important to
have while designing a playground, many of them do not actually seek it in an open response exercise. Two possible
interpretations come to mind. First, our assessment tools, based on students’ abilities to express their knowledge
of engineering design via appropriate language, are limited in their efficacy. Second, we have not fully explored
how to access the internal speech that student designers develop through engineering education.
In sum, these methods yielded a multi-faceted picture of design language, knowledge and practice. We were able to
examine language choice with the select lens. The generate lens provided information about choosing and
appropriately using language to express design knowledge. Finally, the demonstrate lens offered insight into the
application of engineering design knowledge.
Citation: Atman, Cindy, Deborah Kilgore, and Ann McKenna. 2008. Characterizing Design Learning Through the Use
of Language: A Mixed-methods Study of Engineering Designers. Journal of Engineering Education, July 2008,
Vol. 97(3): 309-326.
The full paper can be viewed on the ASEE (JEE) website.
For a printable pdf of this research brief, click here.
Brief created November 2009
Back to Research Briefs