What is evidence centered design?
Evidence centered design (ECD) is a framework — complete with its own terminology — to “make explicit, and to provide tools for, building assessment arguments that help both designing new assessments and understanding familiar ones” (Mislevy and Riconscente, 2006). It has been led by Robert Mislevy, and its major contributors have included Russell Almond, Geneva Haertel, Janice Lukas, Michelle Risconte and Linda Steinberg,
ECD simultaneously has offered very little to the world of assessment development and almost taken it over. It has been both incredibly humble and incredibly ambitious. Unquestionably, it has been amazingly successful.
ECD is the inspiration for the RTD Project.
ECD’s Ambition and Success
ECD offered a unified framework and structure for viewing and understanding the steps in assessment design and development — and perhaps even stepped into the operational phase of the life of tests. It explained how these steps and processes fit together.
ECD offered reasoning for why these steps were important and how each contributed to the inferences that are the real goal of testing.
ECD offered a unified and consistent vocabulary — in part by repeatedly terming things as types of “models” – that professionals could use to communicate with each other.
ECD offered a template for what assessment design and development was, how it worked and how it was done.
Today, most assessment development projects use some form or elements of that ECD framework, and if they don’t they are quite explicit about it. Even some of those who do use ECD processes say that they do not.
ECD is so successful and ambitious that it is difficult to listen to assessment developers talk about their processes without hearing ECD ideas — even if they do not mention ECD or use any of its terminology. Two decades in, ECD is everywhere.
ECD’s Humility
ECD actually offered rather little for assessment designers and/or developers that was new. It did not tell anyone that they were doing things wrong or that they should change how they do their work, in their part of the larger assessment design and development process.
Instead, ECD surveyed existing practices and put them together. In offering a framework that explained the connections and reasoning between elements of assessment design and development, it validated and supported so many existing practices. ECD did not try to reinvent the wheel, nor did it attempt to replace any elements of the existing machines and structures.
Yes, ECD did pick and choose. Yes, ECD has highlighted some portions of the larger process more than others and leaned into certain kinds of thinking more than others. But ECD did not attempt to revolutionize practice.
Rather, it simply offered a framework to help people to better understand the larger processes of test design and development.
ECD as Inspiration for the RTD Project
As we learned about ECD, we noticed a particular hole; ECD says almost nothing about item development. It mentions task models, but it took us years to understand what task models were because we kept looking to find more than a couple of sentences about them.
Evidence centered design is why we call the second phase of the life of a test the design phase, and why we split it from the production phase as we do (and why we split it from the commissioning phase as we do).
ECD highlighted for us how little attention is paid to item development. ECD could not explain item development by looking to the literature to sort through practices and organize them because there simply was so little in the literature. We saw that to fill that gap, we would have to mine practice from our experience and from consulting with practicing content development professionals.
What do you need to know about ECD?
The most relevant and important parts of ECD depend on your relationship to assessment and/or your role in assessment development. Our explanation of assessment development, in the form of a draft chapter from our forthcoming RTD book, can be downloaded from the sidebar to the left.