SLI Compliance was selected by a State to provide independent testing of voting systems for certification, decertification, and recertification.

The State Board of Elections (SBOE) issued a Request for Proposals (RFP) to obtain the services of an Independent Voting Systems Testing Authority (ITA) and selected SLI Compliance to provide examination and testing of voting systems for use in the State’s elections. Before any voting system may be purchased or used in the state, the SBOE must certify that such system(s) meets the requirements of the State’s election laws and regulations as well as the Federal Voluntary Voting System Guidelines (VVSG).

Scope of Project:

SLI Compliance’s Voting System Test Lab (VSTL) team provided an examination and compliance testing against the entire VVSG version 1.0 as well as with the State Election Law and Regulations, which exceed, and in some cases, replace the federal requirements.

The examination utilized procedures and processes as required by the EAC accreditation program plus the additional requirements specified by the SBOE and required by State Election Laws and Regulations. The scope of services included examination and testing for the State Board’s certification, decertification, and recertification of voting systems.

SLI Compliance, as required, adhered to all EAC requirements regarding the EAC Laboratory Accreditation Program and the Voting System Testing and Certification Program. The SBOE representatives were permitted to be present at lab facilities, observe voting system testing, review documentation, and examine lab conditions and practices.

Deliverables:

Project Kick-Off Meeting and Initial Project Management Deliverables

SLI Compliance facilitated a project kick-off meeting, at the discretion of the SBOE, with key project stakeholders to provide an overview of the project scope and initial schedule for services, introduce the project team and outline project start-up procedures. The project kick-off meeting was also used to discuss the project timeline.  SLI Compliance provided the following:

  • Organizational Chart
  • Schedule and Work Plan
  • Quality Assurance Plan
  • Change Control Plan
  • Communications Plan
  • Issue Resolution Plan

Bi-Weekly Project Status Meetings

SLI Compliance conducted regularly scheduled Project Status Meetings with the SBOE and the voting system vendors to review the status of the project.

Weekly Written Project Status Report

A weekly status report was presented and included progress made, and issues requiring resolution.

Project Steering Committee Meetings

SLI Compliance key personnel participated as needed in scheduled Project Steering Committee Meetings with the SBOE key personnel.

Project Issue Tracking

SLI Compliance tracked and managed project issues using an issue-tracking tool and developed an issue resolution process.

Risk Management Plan

SLI Compliance developed a risk management plan identifying potential project risks and reported any potential risks that could impact the overall project.

Testing Requirements Confirmation Matrix

SLI Compliance VSTL team conducted testing against the VVSG requirements as well as the state requirements in the State Election Law and Regulations. SLI Compliance developed a proposed requirements matrix which was reviewed by the SBOE. The SBOE and SLI Compliance conducted a series of meetings to discuss and jointly finalize the requirements confirmation matrix documentation, which was approved by the SBOE for use in the test plan development.

Evaluation of Prior Work

SLI Compliance completed an evaluation of prior certification testing documentation submitted by a voting system vendor.  Existing test plans, results, and other relevant documentation were used wherever possible to avoid duplication of effort. Documents in scope for this review included but were not limited to Functional and Security Master Test Plans, Environmental test plans, usability tests, detailed individual vendor specific test plans, test results, and anomaly reports. SLI Compliance developed and presented an evaluation report that documented the results of the evaluation of prior testing and included recommendations for re-use.

Review of Technical Data Packages (TDPs)

The TDPs provided by the voting system vendors were reviewed for content to ensure that they included all documentation required by the regulatory requirements. The TDP contents were also reviewed and reported on individually by SLI Compliance to ensure that the content provided was of such quality that it could be utilized to achieve the desired results of the package.

Final Master Test Plan

SLI Compliance developed a Master Test Plan that encompassed all forms of testing required to satisfy all areas of testing required by the VVSG and the State requirements. Specifically, the Master Test Plan followed the testing process and phases outlined in VVSG 1.0. The plan included a tabular reference to all the requirements to be tested against, the type of testing (Functional, Security, Environmental, Source code (Functional and Security), etc.) and included a threat analysis to further define testing requirements at the voting system test plan level. The high-level methodology for each type of testing was also included.

Voting System Specific Test Plans

SLI Compliance developed individual test plans by voting system and were used as a script for testing all components of the voting system. Voting system-specific test plans included:

  • Functional testing
  • Functional security testing
  • Source code testing
  • Security source code testing.

Since the Master Test Plan can be considered the “what is to be tested,” the individual test plans are the “how it will be tested” as outlined in the National Institute of Standards & Technology (NIST) SP-800-53A Section 3.2. The plans are detailed and comprehensive enough to ensure that the tests themselves are repeatable with consistent results from different testers. All test plans were reviewed and approved by the SBOE prior to finalization and implementation.

Perform Testing as Outlined in Test Plans

SLI Compliance performed all tests outlined in the master and individual test plans based on the schedule defined in the master project plan. All test results were reviewed and approved by the SBOE prior to finalization.

Voting System Individual Test Reports

SLI Compliance prepared test reports for all stages of testing and presented them to the SBOE at predetermined dates.

Final Test Reports

The final report was a compilation of the outcome of all the individual test reports indicating a pass/fail for each requirement broken down by each type of test performed against the requirement. The final reports were presented to the SBOE to be used as part of the certification process by the SBOE. The report also included possible “Compensating Controls” for those required tests that a system failed and comments on any non-required test that the system failed. All test results were reviewed and approved by the SBOE prior to finalization.

 

What we did

  • Voting System Certification Testing and Examination

  • I want to thank you again for SLI's independent assessment of Broomfield's election procedures and subsequent development of updated election procedures. SLI's work assisted the City Council greatly in understanding the strengths and weaknesses of existing systems and helped restore confidence in these systems for future elections.

    James L. Becklenberg

    Former Assistant City and County Manager

  • It is with sincere gratitude and appreciation that we unconditionally recommend SLI. Our effort to obtain Meaningful Use Stage 2 Certification for our community health software platform required significant interpretation of the federal guidelines that SLI provided in plain English, on an as needed, and always timely basis. The staff is outstanding and their attitude is nothing less than grace under pressure. To keep it all together, with client after client struggling to interpret often vague federal guidelines, is laudable. The bottom line is that we had a choice, and we went with a recommendation for SLI based on a couple interviews with folks that preceded us, and it turned out to be the best possible decision.

    Milton Allione

    President

  • I sought SLI’s assistance in providing information to me on testing practices for the Uniform Voting System (UVS) that the State of Colorado is seeking. I also worked with them as they developed a report for the Colorado Voter Access and Modernized Elections Commission on an assessment of voting system technology in Colorado. The results provided by SLI were valuable and helped the state make key decisions regarding Colorado's UVS strategy. As our project moves forward, we expect to call upon SLI to test one or more proposed systems to meet Colorado Voting System Certification Standards.

    Al Davidson

    Former Program Manager

  • The SLI team, in my opinion, was outstanding. I truly appreciated this process and all the guidance they provided. Not only did this team well represent themselves and the expertise they possess, but also well represented the SLI organization. I would highly recommend SLI to anyone...

    Norman Joseph

    Vice President, Product Management

  • We are so gratified that our cumulative efforts have yielded the much-needed results. I must hasten to add that the work of the entire SLI team has lent so much credibility to this entire electoral process. For this alone, you have the gratitude of not just this member of the Technical Evaluation Committee, but more likely that of an entire nation.

    Tim Diaz de Rivera

    Director General, National Computing Center