New York State kicked off a week of public voting machine certification testing today. The state is testing two systems, the Dominion ImageCast scanner/Ballot Marking Device and the ES&S DS200 scanner with the Automark BMD. The test plans seem to be quite comprehensive, and are designed to provide a full end to end test, from creating ballot programs on the Election Management system, through printing, marking and scanning of ballots, to the final compiled tallied reports back in the EMS system.

The first day saw the machine software validated by hash checking, a technique which compares an unique identifier generated from the binary files of the authorized software against one generated from the software installed on the machine. After hash checks were completed, two teams began creating the ballots using the EMS system.

Each team consisted of one Board of Elections staffer doing the actual ballot programming, paired with a SysTest staffer who read off each test step from a spreadsheet, checking it off when completed. Given the step by step approach (Create a ballot position, enter the candidate name, format the entry, and so on) this is quite time consuming. But it’s the right way to do it. Good software Quality Assurance teams use test plans which are extremely explicit, documenting each individual step in excruciating detail, and requiring the testers to follow it to the letter, over and over. This is how testing needs to be done, and it’s good to see the State Board taking a very detailed approach here.

Laying out the ballots took the rest of the day and will likely take the full day on Tuesday (there are two different layouts being created, one portrait orientation and one in landscape orientation, both allowed under NY’s election law). After the ballot files are created, the PDF files will be sent to one of two print shops, who’ll print them overnight and return them for marking the next day to create the final ‘test decks’. After the test decks have been created, they’ll be run through the scanners, but that is probably not going to happen until Thursday.

The Board has also set up an area where several machines are setup and available for the public to try out. Vendor representatives from Dominion and ES&S are also attending, but not taking part in the tests.

There’s still a whole week to go, and this first part is really just the groundwork for what follows. But my first impression is that the State Board, and their independent consultant NYSTEC, have done a good job designing and planning for the tests, and are off to a good start carrying them out.

I’ve posted some pictures from today’s testing in the Facebook photo gallery.