Ace actionable feedback with standards-level assessment and measurement
by Dr. Laine Bradshaw, PhD, Vice President of Classroom Solutions, Pearson
What are standards-level assessments and measurements and why are they important?
This summer I attended a tennis camp — yes, an adult tennis camp! At the end of the program, I got a certificate and a report with feedback on my backhand swing, my forehand swing, my net game, and my agility. As an assessment professional, I saw parallels between the information I received and two types of educational assessments.
For example:
A certificate that shows I completed and passed a tennis camp’s level 4 requirements is like a standards-based assessment where the assessment questions are written to measure or be aligned to academic standards, but the result is about the degree of proficiency at the overall grade level. By knowing my overall level, I understood how I generally compared to other players.
A report that gives feedback on each tennis skill is like what I call a standards-level assessment: an assessment where the measurement supports reporting categories that are the state academic standards. This level of detail was helpful for me because it gave me feedback specific to each skill, letting me know exactly what I needed to work on.
The distinction between standards-based and standards-level is a matter of precision of measurement. What does that mean and why does that matter? Let’s take a look.
What’s not a standards-level assessment?
Summative assessments
Typical end-of-year state assessments, or summative assessments, are not standards-level assessments, but rather standards-based assessments that include a sample of items aligned to the state standards.
Summative assessments often provide:
An overall score for a given course (e.g., sixth grade math score or fourth grade ELA score)
Domain-level reporting categories.
For example, this sixth-grade math individual student report has an overall score of 638 and has these 5 domain reporting categories: Ratios and Proportions, The Number System, Expressions and Equations, Geometry, and Statistics and Probability.
Summative Assessment Sample Report
Summative assessments have validity criteria to indicate the overall scores for a course mean what we think they mean and reliability criteria to show the overall scores provided are sufficiently stable. To achieve this, summative assessments use items that are aligned to the standards, and the assessment design includes enough items to adequately represent the collection of standards to produce an overall measure, while taking into consideration the length of the assessment. To produce these overall scores for students in a course, summative assessments use course-level measurement or psychometric scoring methods that are aligned to this design. Sometimes, domain-level measurement and designs are also utilized to provide domain-level results that aim to meet acceptable validity criteria; other times, as shown above, descriptive information like points earned is provided without using measurement techniques. Summative assessments do not use standards-level measurement, nor do they produce reliable standards-level results.
Interim assessments
Interim assessment results are like summative assessment results, with three primary exceptions:
- An interim assessment’s overall score may more generally reflect a subject rather than a specific course or grade level in a subject (e.g., a math score versus a sixth-grade math score), if the interim is designed for off-grade level measurement.
- Interim assessments are often given three times through a school year — during the fall, winter, and spring — so they produce a summative-like overall score for each administration, as shown in the sample report below. The multiple time points allow for insight into the change in the overall score throughout the year.
Summative Assessment Sample Report
- Interim assessments are typically not governed by the same professional guidelines that state end-of-year assessments are governed by. So, unlike the sample interim report shown above, interim reports may use some reporting categories that are not supported by measurement — such as providing a standard/skill mastery or performance level — that a state assessment would not provide because there are too few items for each standard or skill to provide a reliable result.
Like summative assessments, interim assessments are overall-level ability assessments and sometimes domain-level assessments, but not standards-level assessments because they are not designed to give a trustworthy score for individual standards.
So, where can I find standards-level assessments?
This was the pickle a few Georgia districts were in when I started working with them in 2017. They wanted to start using a competency-based education approach where the pace of the student’s instruction was based on which standards they’d learned and which ones they hadn’t yet learned. This means they also needed standards-based grading with strong evidence indicating if the student had learned the standard or not, so they knew when to move on.
They didn’t have assessments that would provide that level of information with scores they could trust. They had summative and interim assessments designed to give that overall score and domain scores. And they had teacher-made assessments, but these didn’t meet the quality standards they needed.
I designed Navvy to fill this gap. Navvy is a standards-level assessment system designed to report whether a student has either learned the standard or not, with a diagnosis supported by the same professional guidelines as state end-of-year assessments. The standards-level assessments require a lot more content and assessment designs (thousands!) than standards-based assessments because we aim to generate more specific information. But getting the standard-level detail of reporting that is trustworthy and meaningful for a student, teacher, caregiver, or educational leader to act on is worth the effort.
Summative Assessment Sample Report
Why does it matter if it’s standards-based and not standards-level?
The distinction dictates what you want to measure and act on.
Standards-based assessments ensure the assessment items are aligned to the academic standards the state defines as what students should learn. This alignment is important for certain uses and decisions such as comparing the performance of schools as part of accountability ratings or determining the specific grade levels and subjects that may need more system-level support for educators. But this alignment doesn’t imply measurement of individual standards. If the standards are being measured together — as with summative and interim assessments — results about individual standards shouldn’t be acted upon; that could quickly lead to misguiding instruction and misusing resources because the results aren’t reliable at the standard-level grain size.
Standards-level assessments are also aligned to the academic standards and indicate what is being measured are the individual standards. When designed following professional measurement guidelines, these results provide useful, trusted feedback for students, teachers, leaders, and families to take standards-level action after the assessment, like celebrating the specific standards the student has learned or addressing the specific standards they haven’t yet learned.
Let’s use my tennis camp report as an example! An overall result of “Level 4 tennis game” tells me something different than the skill-based feedback like “excelling at my forehand” and “needing to work on my net game”. And “needing to work on my net game” is different than “needing to work on my grip and my angles at the net”. The latter is more specific and more actionable feedback. Teachers, students, and families often need more specific feedback to guide the next steps in student learning. Like my tennis game, overall feedback helps me know where I stand amongst others, but it is the specific feedback that helps me improve.
Standards-level assessment surpasses standards-based assessment because it provides precise measures of individual academic standards. While the latter aligns with academic standards, the former delivers actionable measures of each standard; this precision is essential for guiding personalized learning.
Keep an eye out for more blogs about types of assessments, their quality, and what it all means!