Keywords: impact, engineering education, research
“Scholarship of Impact” Framework in Engineering Education Research:
Learnings from the Institute
for Scholarship on Engineering Education
The Institute for Scholarship on Engineering Education (ISEE) program is one element of the NSF-sponsored
Center for the Advancement of Engineering Education (CAEE). Its primary goal is to build a community of engineering
education scholars who can think and work across disciplines with an ultimate aim of improving the engineering
student experience.
Three year-long institutes were hosted by three CAEE campuses each with a different theme and scholarship focus:
University of Washington, 2004-2005, Classroom as Lab; Stanford University, 2005-2006, Campus as Lab; and Howard
University, 2006-2007, Nation as Lab.
| Impact studies all start with a concern, work towards a goal, and vary in terms
of whether or not good information exists out there to use.
|
Each Institute cycle involved adapting and improving the model from the year before, recruiting Scholars,
hosting them at a week-long Summer Summit kick-off, academic year activities to support Scholars conducting
their studies, and a culminating event. Each Institute cycle examined scholarship from a different angle. This
paper focuses on the “Scholarship of Impact” framework from the 2005 Stanford Institute.
Defining Scholarship of Impact
Definitions for a “Scholarship of Impact” emerged during the 2005 Summer Summit at Stanford University.
Participants and facilitators were given the task of defining impact and what would constitute an
impact study.
The ISEE team broadly defines impact as the measurement or evidence of change. Along with evaluation,
research helps to define measures of impact that can be used to assess program changes (e.g., whether program goals
were met). In turn, evaluation activities may inspire new theories and definitions of program impact. These examples
illustrate the linked relationship between research and practice. Practical problems prompt and define research
problems, and research problems seek to find answers to resolve practical problems. As the scope and complexity
of the problem increases, measures of “impact potential” – the capacity to cause changes – become important for
breaking down seemingly intractable goals into more realizable intermediate goals.
Therefore, in the context of engineering education research, “impact studies” are distinguished from other kinds
of research studies. While most studies seek to have theoretical and practical significance, impact studies seek
to close the gap between research and practical problems.
What Makes a Good Impact Study?
The Stanford Scholars reached consensus on some features of good impact studies:
- Intentional (with an explicit point of view)
- Collaborative (includes diverse stakeholders)
- Articulated at the start of the project
- Made up of intermediate goals (with intermediate metrics of success)
- Involve gathering information about the problem (drawing on existing information and collecting
new information)
- Involve identifying impact pathways (those who care, who can facilitate or block progress)
- Meet the needs of the users (stakeholders)
- Involve telling a compelling story for different audiences
Impact studies all start with a concern, work towards a goal, and vary in terms of whether or not
good information exists out there to use.
Key questions to consider were defined and compiled in a "Scholarship of Impact" study plan worksheet used by
Scholars. Through this process, big goals of each individual’s impact study were broken down into framing
first steps, describing information needed, and identifying who was important in the
impact pathway.
An Example Impact Study
A group at Stanford University has been examining the early undergraduate engineering experience as part of a
shared 2005 Institute project with faculty, staff from the local Center for Teaching and Learning, and student
Scholars. The group worked closely with the office of the Vice Provost for Undergraduate Education and the School
of Engineering to investigate the early engineering education experience at Stanford. This involved examining
the role that introductory engineering courses have in shaping the students' experience (for exploring engineering,
providing exposure or breadth to various engineering fields or subspecialties, or providing depth in one
disciplinary area).
Data was collected on the School of Engineering introductory classes about student enrollment (who takes these
courses and when do they take them), from academic transcripts (with collaboration with the university registrar
and assistance from the CAEE Academic Pathways Study) as well as through faculty focus groups, benchmarking of
other institutions, etc.
Stakeholders (students, faculty, departments, university administrators and staff) were engaged early and often.
Their particular responsibilities for the undergraduate student were mapped and change agents were specifically
targeted among the university’s administration and faculty. Change opportunities at Stanford were identified that
could positively impact student learning quality and satisfaction with the early engineering experience of
undergraduate engineering majors.
This example of “Scholarship of Impact” was enthusiastically supported by the Dean of Students in the School
of Engineering and presented to a wider audience of deans, administrators, and curriculum committees. The research
is spurring a review of the role of introductory engineering courses by the Undergraduate Engineering Council
and a reform of this part of the curriculum is expected in the near term. This work is an example of the
“Scholarship of Impact” framework in practice and illustrates what may be possible with this approach. It is
hoped that this framework will be of interest and transferable to other engineering education research projects.
Authors: Micah Lande, Robin Adams, Helen Chen, Becky Currano, and Larry Leifer
Source: Proceedings of the 37th Frontiers in Engineering Conference 2007
For a printable pdf of this research brief, click here.
Brief created November 2007
Back to Research Briefs