Blog

Let the Games Begin

Let the Games Begin
Dr. Bill Conrad
Article Submitted to San Jose Inside
February 5th, 2018

Gather round my fellow citizens to hear the amazing tale of your faithful servant in his quest so reminiscent of the ancient travails of the magnificent and great Greek King, Odysseus.  As you recall, the story begins with my many consultations with students, parents, and community members across this great nation.  We sought to answer the question: What do our children want from their schools?  The children eloquently communicated that the they wanted their teachers to listen to them, to respect them, and to give them some voice in their learning. They also wanted their teachers to be experts in their content knowledge and teaching skills in order to prepare them for their future college and career aspirations.

Many of you reported to me that you were baffled by the way in which school districts reported student performance on state tests that are designed to measure the degree to which our children are on track to be college and career ready.  The data presented to you within the LCAPs were minimal, cherry picked, and aligned with a state accountability gambit of spreading the scale score points around.  You also reported that the LCAPs themselves enveloped you in the fog of education with eclectic and bureaucratic solutions to address poorly defined student needs.

Thus, upon hearing your pleas for relief, I girded my loins and travelled to Mount Sacramento to do battle with the 32-headed Hydra of Santa Clara County School Districts’ data.  The battle that ensued was ferocious and engaged all of my computational and physical skills to overcome this voracious beast.  I even had to consult the oracle of Python in order to develop a computer code that would visualize the data for each school district in ways that were truly open, transparent, and interpretable.  The magnificent beauty and radiance of these data displays can be found at a web site called The SIP Big Picture (http://sipbigpicture.com).

Some of you have reported to me that you have visited the site and have been enthralled by the elegance of these one-page displays and how they have helped you truly understand and interpret not only the overall performance of your school district by grade and by year but also the performance of a plethora of subgroups over time.  I am overjoyed with your responses.  Yet some have reported to me that after viewing the displays for a short time, they have become afflicted by the pernicious MERO effect (My Eyes Role Over).  You beseech me: What are we to do?  Fear not as I have consulted the Muse Madonna who exhorts us all to have Fun!

In the spirit of fun, I have resurrected the idea of the Academic Olympics in conjunction with the upcoming Winter Olympics in Seoul this February.  I reorganized the data in such a way that you can now view Academic Olympic Events on my web site.  The first event that has been recently posted is the 11th grade Overall Math Performance on the State Math Tests.  Each event will have a compulsory component: the overall percentage of students who meet or exceed standards on the State Math Test in 2017. The second component is the free style event that will be the 3-year improvement in performance on the State Math Test.

Congratulations to the Fremont Union High School District who achieved a 77% Meets or Exceeds percentage for the 2017 State Math Test and won a Gold Medal for overall Math Performance. Congratulations also to the Gilroy Unified School District who won a Gold Medal for demonstrating an 8 percentage-point 3-year improvement on the State Math Test from 24% of 11th grade students Meeting or Exceeding Standards in 2015 to 32% Meeting or Exceeding Standards in 2017.

Please go to the web site (http://sipbigpicture.com) where you can see if your school district received a medal for this event.  You can also view the rankings of all of the school districts in Santa Clara County in case your school district did not medal. Have some fun!  I will continue to add events daily in conjunction with the real Winter Olympics in Seoul so return to the sit often!

As you might imagine, the State, the County, and the School Districts are averse to the idea of a competition. The sources of power would prefer that you continue to remain within the fog of education feeding upon the miniscule morsels of selected data that they provide to you.  We must break out of these artificial bureaucratic chains of restraint and demand a full accounting of student performance. In fact, we should also demand a full accounting of the professional practices of teachers, principals, and district administrators from a system view and not just a special case view.

Being of advanced age, I could use some support in my ongoing quest to visualize data to meet your voracious needs.  Please use the comments section on the web site to let me know if I should pursue the many headed Charter school hydra for Santa Clara County or if I should pursue student performance in English Language Arts.  I am also available to support you in representing the student results through the Academic Olympics or the actual Data Visualizations to your parent groups or even to your School Board.  Also, please let me know if you could spare a few drachmas to help me continue my quest as I occasionally need the sustenance from a Gyro at Nick’s Greek Restaurant in San Jose to pursue the Quest.

In conclusion, let us not become lost in the darkness of poorly conceived accountability, but let us instead light our Olympic torches and together ignite the Olympic Flame of openness and transparency to the success of all of our students in achieving their college and career dreams.

 

 

Fake Accountability
Dr. Bill Conrad
January 9th, 2018

It is truly surreal to see the Federal Department of Education under the mendacious Trump administration call out the California State Board of Education for significant and accurate flaws in its State’s Accountability Plan for K-12 schools.   The primary responsibility of our school districts and schools is to ensure that all students are academically prepared for college and career when they graduate from the system.  The new State Accountability Dashboard only contributes to the fog of education.

The state adopted rigorous Common Core State Standards in ELA and Math that identify what students should know and be able to do at grades 3-8 and 11 to be on track for college and career by the end of 12th grade.  The state also participated in the development and implementation of a high-quality assessment called the Smarter Balanced Assessment (SBA) to gauge whether or not students meet or exceed these standards.  The state crafted an academic indicator for its Accountability Dashboard that is more about protecting the adults in the education system than it about clearly holding school districts and schools accountable for student academic performance.

Rather than link school and district accountability to the percentage of student overall and subgroup performance at these bands, the state devised a system that obfuscates actual student performance. The state academic indicator gambit works like this.  Students achieve scale score points on the ELA and Math state tests. The scale score is assigned a proficiency level based on where it falls within four proficiency bands.  Students are expected to score at the Meets or Exceeds proficiency bands.   The system subtracts individual student scale scores from the low-end scale score in the Meets Standards proficiency band.  These subtracted values are then added and divided by the number of scores to produce an Accountability value.  A color and ranking is then assigned to the derived value.

For ease of explanation, let’s say that the range for Meets Standards on the 5th grade Math test is between 100 and 150 Scale score points.  Imagine that Student A scores 90 points (Below Proficiency); Student B scores 110 points; and Student C scores 115 points.  The state system would then subtract each student score from the low value of 100 scale score points) for the Meets Standards proficiency level resulting in derived values of -10, +10, and +15.  The system then adds these values and divides by 3 resulting in a derived value of +5.  This +5 value would receive a green color or highly proficient on the state accountability system.

This system masks the performance of low performing students by redistributing “excess” scale score points of high performing students.  Unfortunately, Students B and C will probably not be able to accompany Student A to college to provide their excess test scoring capacities.  Additionally, the state does not include the 11th graders in this system presumably because there are not enough scale scores to spread around!  Rather than an innovation as David Plank suggests in the article, the state system is a subterfuge designed to protect the adults in the education system rather than the students, parents, and community.

In their Local School Accountability (LCAP) report, San Jose Unified School District (SJUSD)reports that overall academic progress as Green or High.  I wrote a Python script that comprehensively and clearly visualizes San Jose Unified School District performance on the state Math test by grade and by year as well as by year and by grade.  Please take a look at this report.  As you can easily see from the overall performance, fewer than ½ of all students meet or exceed Math standards and slightly more than 1/3 of eleventh graders meet or exceed Math standards.  The state allows SJUD the ability to claim a High Academic Performance. For Students with Disabilities, SJUSD reports an orange color (Medium) with no values.  But my table shows a continuous drop in the Math performance of Students with Disabilities across grades with only 5% of 11th grade Students with Disabilities Meeting or Exceeding Standards in 2016-17, an increase of 1 percentage point from the previous year.

Obviously, there is a big disconnect between the color-coded state academic indicator and actual student performance on the state Math Test.  The State Academic Dashboard is truly Fake Accountability and as our mendacious president might say: So, Sad!  There are real consequences as well.  The community thinks that student academic performance is satisfactory when it really is not!  The result of course is that almost 40% of California students enrolling into the California State College system require remediation.  The parents and students want and expect that their schools and school districts prepare all students for college and career. Parents and students have filed a lawsuit against the state to seek redress to their very legitimate grievance.  Yet the state continues to contribute to the fog of education by building an accountability system that is contrived to make the adults look good at all costs.

Enough is enough.  If you would like to see comprehensive pictures of how your school district performs in Math on the state test, please visit http://sipbigpicture.com.  Share these data visualizations with your District and School leaders.  Ask your leaders to set up posters of these one-page data visualizations within the District and schools.  Use the data visualizations to advocate for the improvement in curriculum, professional practices, and assessments. Please send me a comment on my web site if you want me to build a visualization tor your school or district.  Let’s get active. Let’s truly hold our educational leaders accountable with real data.

Strength in Numbers

Article submitted to San Jose Inside

The Strength in Numbers motto adopted by the Golden State Warriors Basketball team beautifully represents the organization at multiple levels.  The most obvious manifestation of this motto can be found immediately upon opening their App or visiting their Web site where they proudly post their current Win/Loss record along with an abundance of other team statistics.  Wouldn’t it be great if our school districts, schools, and charters in Santa Clara County proudly pronounced their own Strength in Numbers by posting their own comprehensive “Win/Loss” records as evidenced by the percentage of their students who meet or exceed Standards on the State Math and ELA tests?

Sadly, when we visit our School District, County, or State web sites, we often do not see how the children are performing academically.  I have had the opportunity to work with school districts across the country for over 45 years.  Most recently, I have engaged in conducting surveys and focus group meetings with students to find out what is important to them as we initiate District Strategic Planning processes in school districts in Washington State and the State of New York.  For the most part students want their teachers to treat them well and to respect them.  Most importantly, though, they want their teachers to prepare them for their future college and career goals. As we subsequently engage with parents and community members, we find great concordance with the student aspirations.

It is my belief that students, parents, community members, teachers, and administrators should be able to access and interpret comprehensive and complete data visualizations of student academic performance. The interpretation of these data visualizations should lead to the development of student goals for the district and schools that will ultimately support most students in achieving their dreams of being successful in college and career.  To this end, I have begun to build data visualizations of student academic performance for English Language Arts and Mathematics at the following Web site: http://sipbigpicture.com.  This will be a Herculean task but hopefully not a Sisyphean one! Please visit the site, get some data, and cheer me on!

At this site, you will find a Math Data visualization for San Jose Unified School District (SJUSD) among others.  You will find two heat mapped tables for student performance.  The first table provides a comprehensive view of student performance on the State Math test for 3 years. The table is organized by grade and by year.  The color continuum goes from red (low performance) to green (high performance).  The State Math Test is aligned with the Common Core State Standards in Math which identify what students should know and be able to do in Math in order to be on track for college and career.  Viewing the table, you can begin to answer the following fundamental questions:

  • Is there overall improvement in grade-level student performance over time?
  • Is there subgroup improvement in grade level student performance over time?

The colored heat maps make it easy to see patterns and to develop findings for these key questions.  Overall, student performance is low (yellow) with most grades consistently scoring under 50% meeting or exceeding Math standards.  There has been some modest improvement at each grade level over 3 years. (with the exception of 11th grade) The Asian subgroup demonstrates consistent high (green) performance over time while the English Learner subgroup demonstrates low (red) performance over time.

The table on the second page shows student performance by year by grade level.  Viewing this table, you can begin to answer the following questions.

  • Is there overall grade-level improvement in student performance within a given year?
  • Is there subgroup grade level improvement in student performance within a given year?

Within any year, there is no improvement in Math performance as you go from 3rd grade to 11th grade in any given year. In fact, there is a drop in performance every year between student Math performance in 3rd grade compared with student Math performance in 11th grade. (Yellow to Red). The Asian subgroup demonstrates consistently high performance (green) across grade levels within any of the three years while Students with Disabilities demonstrate consistently low (red) performance across grade levels within a given year.

You can see that using the colored heat maps. it is easy to discover patterns and findings of student performance.  Wouldn’t it be nice to be able to access data visualizations of this kind for your school district and school?  Regrettably, neither the County nor the State offer any real help in producing easy to interpret data visualizations.  In fact, the State has been complicit in obfuscating student performance by developing an Academic Indicator Dashboard that contributes to the fog of education.

The state system works like this. If the low end of the scale score band for Level 3 (Meeting Standards) is say 100 scale score points and student A scores 90 scale score points on the Math State Test for their grade level, and student B scores 110 scale score points, and student C scores 115 scale score points, the system subtracts each student scale score from 100 arriving at derived values of -10, +10, and +15. Tallying these values gives you +15 and then dividing by 3 students gives you +5. This value is slightly positive but would receive a green color or a HIGH performance on the Academic Indicator.

This is a flawed process because it is merely a redistribution of “excess” scale score points and does not give you a true reflection of actual student performance. It masks the fact that Student A is not scoring within the proficient band by redistributing scale score points from students B and C to student A. Unfortunately, students B and C will probably not be able to follow Student A to college to help remediate issues he may be having with college level math!  This would be like the NBA allowing teams to take excess points from their Wins and then distributing them to their losing scores in order to bolster their Win/Loss records!

Interestingly, the state does not include 11th grade students in this process and builds a second indicator for 11th graders called College and Career. Could it be that there is not enough extra scale score points to spread around for 11lth graders?

SJUSD reported the positive picture that they were a green school district for the state academic performance indicator on their state mandated Local Control Accountability Plan (LCAP).  From the previous data visualizations and interpretations, you can see that SJUSD is not yet a highly proficient school district as it relates to student Math performance.

So rather than helping school districts visualize a comprehensive, open, and transparent performance on state tests as an academic accountability indicator, the state of California obfuscates performance using a “Spreading the Scale Score Points” gambit. Of course, the school districts and schools love this system because of its simplicity and ability to mask actual poor performance. The state loves it because it artificially supports their LCAP system as “getting” student results. The adults are saved with this system while the students, parents, and community wait for a true comprehensive picture of student performance

If you are interested in viewing a comprehensive and easily interpretable data visualization of your school or district’s student academic performance in Math, please visit my web site at http://sipbigpicture.com. If you do not see your school or district data visualization, please send me a note via the web site and I will hasten the development of your report.

We need to support our school districts in becoming more like sports team such as the amazing Golden State Warriors who have well-defined goals, metrics to gauge success, and a theory of action to achieve those results.  Let’s help our school districts follow a similar path to success! A little competition is a good idea!

I look forward to the opportunity to continue to share my over 45 years of insights about teaching and learning in the future for San Jose Inside.

 

 

State Data Visualization Support?

 

Getting access to quality data visualizations does not seem to be a priority for most state education agencies even though they are very important to support the development of both district and school improvement plans.  In California, the Data Quest system visualizes data in a very bureaucratic and user-unfriendly fashion.  And mere mortals should never try and download and use the state data research files as they are comma-delimited and not easily decipherable.   The state of California has now developed an Accountability Dashboard to supposedly make the use of data easier for districts and schools.  Really?

Unfortunately, at least for the Academic Indicator, the state education agency developed a convoluted and inappropriate process for assigning colors for academic performance in ELA and Math on the state SBA tests by redistributing individual scale scores based on the low end of the Level 3 scale score band – Meets Standards.

The system works like this. If the low end of the scale score band for Level 3 is say 100 scale score points and student A scores 90 scale score points on the Math SBA for their grade level, and student B scores 110 scale score points, and student C scores 115 scale score points, the system subtracts each student scale score from 100 arriving at derived scores of -10, +10, and +15. Tallying these values gives you +15 and then dividing by 3 students gives you +5. This value is slightly positive but would receive a green color or a HIGH performance on the Academic Indicator.

This is a flawed process because it is merely a redistribution of “excess” scale score points and does not give you a true reflection of actual student performance. It masks the fact that Student A is not scoring within the proficient band by redistributing scale score points from students B and C to student A. Unfortunately, students B and C will probably not be able to follow Student A to college to help remediate issues he may be having with college level math!

Interestingly, the state does not include 11th grade students in this process and builds a second indicator for 11th graders called College and Career Readiness. Could it be that there are not enough extra scale score points to spread around for 11lth graders?

So with this system, my school district – San Jose Unified School District reported the rosy picture that they were a green school district for the state academic performance indicator on their state mandated Local Control Accountability Plan (LCAP). I then built a Python Program to visualize Math Performance on the Math SBA for all grades and for all subgroups for 3 years producing a heat-mapped one page visualization. I found that fewer than 1/2 of San Jose Unified students at most grade levels (3-8) meet standards and only about 1/3 of 11th graders meet standards in Math. By reporting a status of Green on Academic Performance for Math, San Jose Unified masks an actual poor performance on the Math SBA.

So rather than helping school districts visualize a comprehensive, open, and transparent performance on state tests as an academic accountability indicator, the state of California obfuscates performance using a “Spreading the Scale Score Points” gambit. Do no harm first! Of course the school districts and schools love this system because of its simplicity and ability to mask actual poor performance. The state loves it because it artificially supports their LCAP accountability system as “getting” student results. The adults are saved with this system while the students, parents, and community wait for a true comprehensive picture of student performance!

If you would like to see your school and or district data visualized in an easy to understand and interpret fashion, please let me know by visiting the the We’ d Love to Hear From You page on our Web site!  We would be glad to help!

 

A New Conception for SIP Development

Many school improvement plans are built using templates provided by the state or from the school district.  These traditional plans look and feel very bureaucratic and they primarily serve the purpose of fulfilling a state or district requirement.  They often include a plethora of actions that are often disconnected from each other.  They generally do not have a theory of action that brings coherence to the school improvement system. They are often written in educational jargon that is not easy for educators to interpret let alone parents and community members.

I propose that we rethink the idea of School Improvement Plans.  We should approach them from the perspective of truly aligned action plans that link the improvement of student outcomes through the improvement of professional practices.   By identifying and aligning teaching, principal, and District administrator practices, we will be able to act in concert to improve student outcomes.

The process begins with the identification of student goals.  Traditional school improvement plans often conflate means and ends. For example, these plans will often include goals linked to parent engagement.  Yet parent engagement is a just a means to ultimately improve student outcomes. The core business of schools is to improve student outcomes so all School Improvement goals should always address improvement in student performance.

The process of identifying student goals begins with a comprehensive review of student outcome data.  Teams who are building the school improvement plan Big Picture should not have to wade through reams of tables and graphs to identify those trends in student performance that should be celebrated and those that should be addressed.  Most state web sites do a poor job of comprehensively visualizing multiple years of student data for multiple subgroups so that trends can be easily seen.  We are beginning to build a repository of data visualization templates that you may find useful in identifying trends in student performance.  Please see a link to a sample visualization of student data from my home district – San Jose Unified School District.

A second key element of the establishing the big picture for the School Improvement Plan is to identify specific practices that will support the achievement of student goals.  These practices begin with teaching practices. Traditional school improvement plans will often times only offer a general improvement in overall teaching practices or will relegate improvement in student outcomes to such things as after school support.  I suggest that it is better to identify specific teaching practices many of which can be found in the book Visible Learning by John Hattie. The systematic use of research-based teaching practices is the most important element in improving student outcomes.

Often times teaching practices can be found embedded within educational strategies that are currently underway within a school or school district. For example, the teaching practice of formative assessment with descriptive feedback can often times be embedded within a strategies such as those promoted by the Assessment Training Institute or similar organizations.  It is important to call out these strategies within the SIP Big Picture in order to demonstrate an alignment between programs and strategies with specific teaching practices.  We intend to include data bases of research-based professional practices soon.

Professional development and collaboration is another essential element to the Big Picture for School Improvement.  Professional development and coaching need to be clearly identified and aligned with both the strategies as well as the professional practices.  The opportunity for teacher teams to collaborate is also an essential element of the SIP Big Picture.

What gets measured gets done. School teams should identify those metrics that will let them know that student outcomes are being achieved. Often times these metrics already exist but need to be identified within the big picture. If there is a goal to ensure that students are reading by the end of 3rd grade, it might be good to include a metric in the Big Picture that measures student on track fluency in grades 1-3.  Additionally, it is important to measure the actual implementation of professional practices.

Once, the big picture is completed, it will provide a wonderful opportunity for teacher teams to discuss both vertical and horizontal alignment amongst all of the key elements. The Big Picture should be able to tell a coherent story about school improvement aligned to specific student goals.  The Big Picture can also provide a beautiful and comprehensible communication resource for parents and the community as an addition to the web site or as large posters placed within the school.

Of course, the Big Picture of the School Improvement plan will require monitoring and continuous improvement. Identifying specific tasks aligned to the educational strategies, professional development and collaboration, as well as the metrics is essential.  We will provide tools to help develop and monitor these key tasks.

I  have developed  a template and a sample of the Big Picture for School Improvement.  We look forward to working with you on making your school improvement come alive through the Big Picture approach.  I would love your feedback about this post.  Also, feel free to contact me for more information and support.