Empirical methods for evaluating maps: Illustrations and results

Empirical methods for evaluating maps: Illustrations and results


Date
Event
Annual meeting of the National Council on Measurement in Education
Location
Toronto, Canada

This presentation is part of a coordinated session, Beyond Learning Progressions: Maps as Assessment Architecture.

Learning progressions (LPs) are commonly used in educational assessments to identify interim steps on a pathway toward a grade-level target. LPs describe typical expected pathways, but may not represent the multiple pathways by which students develop knowledge in a domain. Another type of cognitive model, the learning map, is better suited to describing heterogeneous pathways that support learning for all students including those with the most significant cognitive disabilities. This session ties together four presentations on different facets of a project involving the creation and use of maps as cognitive learning models to support the design of large-scale assessments. The first presentation illustrates how an assessment’s theory of action and validity argument are grounded in the maps as models of the content domains. The second presentation describes the map creation process, including intentional design decisions and the application of universal design for learning principles. The third presentation describes the iterative design process and the use of stakeholder evaluation processes to evaluate the maps for content and accessibility. The fourth presentation describes empirical methods for map validation. The session ends with discussion of lessons learned and future directions, and commentary from a national expert in cognitive learning models and large-scale assessment.