Getting access to quality data visualizations does not seem to be a priority for most state education agencies even though they are very important to support the development of both district and school improvement plans. In California, the Data Quest system visualizes data in a very bureaucratic and user-unfriendly fashion. And mere mortals should never try and download and use the state data research files as they are comma-delimited and not easily decipherable. The state of California has now developed an Accountability Dashboard to supposedly make the use of data easier for districts and schools. Really?
Unfortunately, at least for the Academic Indicator, the state education agency developed a convoluted and inappropriate process for assigning colors for academic performance in ELA and Math on the state SBA tests by redistributing individual scale scores based on the low end of the Level 3 scale score band – Meets Standards.
The system works like this. If the low end of the scale score band for Level 3 is say 100 scale score points and student A scores 90 scale score points on the Math SBA for their grade level, and student B scores 110 scale score points, and student C scores 115 scale score points, the system subtracts each student scale score from 100 arriving at derived scores of -10, +10, and +15. Tallying these values gives you +15 and then dividing by 3 students gives you +5. This value is slightly positive but would receive a green color or a HIGH performance on the Academic Indicator.
This is a flawed process because it is merely a redistribution of “excess” scale score points and does not give you a true reflection of actual student performance. It masks the fact that Student A is not scoring within the proficient band by redistributing scale score points from students B and C to student A. Unfortunately, students B and C will probably not be able to follow Student A to college to help remediate issues he may be having with college level math!
Interestingly, the state does not include 11th grade students in this process and builds a second indicator for 11th graders called College and Career Readiness. Could it be that there are not enough extra scale score points to spread around for 11lth graders?
So with this system, my school district – San Jose Unified School District reported the rosy picture that they were a green school district for the state academic performance indicator on their state mandated Local Control Accountability Plan (LCAP). I then built a Python Program to visualize Math Performance on the Math SBA for all grades and for all subgroups for 3 years producing a heat-mapped one page visualization. I found that fewer than 1/2 of San Jose Unified students at most grade levels (3-8) meet standards and only about 1/3 of 11th graders meet standards in Math. By reporting a status of Green on Academic Performance for Math, San Jose Unified masks an actual poor performance on the Math SBA.
So rather than helping school districts visualize a comprehensive, open, and transparent performance on state tests as an academic accountability indicator, the state of California obfuscates performance using a “Spreading the Scale Score Points” gambit. Do no harm first! Of course the school districts and schools love this system because of its simplicity and ability to mask actual poor performance. The state loves it because it artificially supports their LCAP accountability system as “getting” student results. The adults are saved with this system while the students, parents, and community wait for a true comprehensive picture of student performance!
If you would like to see your school and or district data visualized in an easy to understand and interpret fashion, please let me know by visiting the the We’ d Love to Hear From You page on our Web site! We would be glad to help!