Rolling out a district-wide assessment solution is no small task. Williamson County Schools in Tennessee has gone through the process several times, so we have learned the hard way what to do and what not to do. Here are a few suggestions to find the right assessment solution for your district and avoid pitfalls along the way.
1. Decide how the test items will be used and who will build them.
As part of our district assessment plan, we wanted to build a bank of test items aligned to our local curriculum. The item bank would be used for district formative assessments, and it would be available to all teachers for classroom use. We decided that teachers would write the items, but district specialists would vet them before placing the items in the bank. We felt this was a good balance for getting teacher buy-in through ownership, while still having quality control.
One year before putting the assessment platform in place, we began collecting items for the item bank. Specialists salvaged some items from our former platforms, but most of the work involved having teachers create new items. Since we had no idea which platform we would use, we simply collected item content through shared documents.
2. Determine what you need versus what you want.
As the district assessment analyst, I was charged with finding an assessment platform. A simple search online or a stroll through the vendor hall at a conference reveals so many options that it can be overwhelming. I first needed a short list of potential vendors. So, while considering our district processes and expectations, I had to determine what we needed versus what we just wanted. Here's what I learned:
Decide how assessments will be administered. Since our state assessments were moving online, we committed to computer-only testing. Eliminating paper testing simplified my search because I did not have to consider options like scanners, cameras, or the printing of tests. Choosing an online versus paper option also played into the cost factor. Aside from the additional funds needed for peripheral equipment and paper, paper scanning was often an add-on to the subscription fees.
Figure out which item types are most important. Assessment platform options range from having only multiple-choice items to offering dozens of different item types. Deciding which ones are most important allows for more focus when vetting platforms, and more time can be spent asking about specific item interaction details.
Determine where will the items come from. Some platforms advertise the ability to create customized tests, but only allow you to choose pre-existing items from a bank. Others allow the creation of new items through a variety of methods. Our district plan included a need to create local items but to also have the availability of third-party items. Both took additional consideration when vetting potential partners.
As for item banks, most vendors either have their own or subscribe to companies specializing in item banks. If you decide to go down this road, consider any additional subscription costs and check the specs, too. Some solutions advertise item variety in all subject areas, but in reality, that variety is limited to English language arts and math.
If you plan on creating your own items, set up a demonstration to review that in detail. Item editors vary greatly, and a platform that supports one item type for testing may not support the creation of the same type. So, I had three requests I insisted on being demonstrated: How can you create passage-dependent items? How do you insert and edit images within items? How do you include mathematical expressions? If the demonstration seemed overly cumbersome, I knew the item writing process would be a significant hurdle in the implementation.
Of course, there were many considerations beyond these areas. Will it work on student devices? Will it integrate with our SIS? How easy is it to create tests? While important, these rarely distinguished one platform from another since most were on a fairly level playing field in these areas. By determining our key areas of interest, we were able to whittle down the competitor pool and choose the solution that met all of our needs and expectations.
In January 2017, we committed to implementing the PM Assessment platform and contracted to have limited access to the item writing components initially. This gave our specialists six months to begin entering vetted items into the item bank before the start of the next school year. This was a huge advantage because it allowed everyone time to learn best practices for creating some of the newer interaction types.
3. Create a process for the platform rollout.
To roll out the assessment platform to our schools, we launched a multi-step process. We trained two teacher leaders from each school on the administration of online tests, and they trained their staff. We then administered a district-wide practice assessment to all students in all core content areas. This allowed us to identify and correct technical issues, and it gave students an opportunity to interact with the newer item types. We used the “fake” data from the practice assessment when we trained two teacher leaders from each school on reading the item analysis reports available in the platform. In turn, they trained the staff at their schools.
Then it was time for the moment of truth — the administration of our first district-wide benchmark assessment to all students. Our teachers successfully administered over 122,000 assessments in just two weeks. Everything went smoothly, which meant that our rollout plan worked!
Our next step is to train teachers to build tests for classroom use and create items directly in the assessment platform. Given how well our first steps have gone, I am optimistic that what lies ahead will go well, too.
Kevin Deck has spent 13 years teaching high school mathematics in Cobb County, Ga., and Williamson County, Tenn. He is currently the assessment analyst for Williamson County Schools, responsible for a variety of district and state assessments for its 44 schools.
Tech Tips is a weekly column in SmartBrief on EdTech. Have a tech tip to share? Contact us at firstname.lastname@example.org
Like this article? Sign up for SmartBrief on EdTech to get news like this in your inbox, or check out all of SmartBrief’s education newsletters, covering career and technical education, educational leadership, math education and more.