CET+-+Pilots



toc

1.0 Pilots Workgroup Overview
To date, many S&I Framework Initiatives have included a Pilots Workgroup (see S&I Framework Functions) throughout the lifecycle. The Pilots Workgroup (WG) organizes and guides the deployment of Pilot projects that will test the suitability of the Initiative specifications, standards, and technology in real-world settings. The types of members that have participated in Pilots Workgroups include EHR vendors, provider groups, HIT Integrators, etc. The Pilots WG will form Pilot Teams to deploy instances of the Initiative use case and establish the feasibility of implementing the Initiative’s outputs in real-world, production-grade environments.

The Pilots WG and its Pilot Teams will leverage the work done by the antecedent S&I Framework Initiative WGs and S&I Functions (e.g., Use Cases and Functional Requirements, Standards, Harmonization, Reference Implementations). A Pilots WG targets the scope and proof of technology to ensure that the Initiative's functional requirements are implementable for electronic health information data exchange and deployment-ready for release to the open community for adoption in production environments.

The Pilots WG provides guidance and support for helping to steer organizations toward becoming committed Pilot Team Members. The Pilots WG is responsible for establishing the necessary guidelines for pilot proposals and acceptance criteria. Using Success Metrics, the Pilots WG/Pilot Teams are expected to submit a Pilot Feedback Report to ONC upon completion of the Pilots WG project, as well as participate in the preparation of the Initiative Evaluation Phase Lessons Learned document (see http://wiki.siframework.org/S%26I+Initiative+Overview%2C+Phases+and+Outputs).

2.1 Pilots Workgroup Lead
Every S&I Initiative Workgroup is headed by a Workgroup Lead (occasionally two Co-Leads). The Workgroup Lead is a community participant, and is typically vetted by the Initiative Coordinator. The Pilots WG Lead works with ONC Support Staff (contractors) to organize the Workgroup, edit the Workgroup Wiki pages, recruit S&I committed members to serve on the Workgroup, and run the Workgroup meetings. This individual is the chief point of contact between the Pilots WG and ONC management.

2.1 Pilots WG Action Plan
One of the first tasks that have been proven to be useful is the preparation of a Pilots WG Action Plan, which provides a high-level overview of the WG strategy and processes. The Action Plan can be prepared as a PowerPoint brief (see example [|Pilots Action Plan]), which can then be shared with Stakeholders. Components of the Action Plan include:
 * Pilot Plan
 * High-Level Overview of the S&I Initiative that is being piloted
 * Outreach and Commitment Strategy
 * Pilot Operations
 * Implementation
 * Minimal Components
 * Sample Pilot
 * System Functions
 * Proposed Timeline
 * Challenges
 * Current Status
 * References

2.2 Pilot Participant Outreach Strategy and Approach
Once an S&I Initiative forms a Pilots WG for the Initiative, the WG will want to recruit committed members to act as Pilot Team members. A [|Pilot Outreach Activities document] may be used to plan the approach and strategy in pursuit of Pilot Team participation. Prospective Pilots WG members can be conveniently tracked in this [|Workgroup Participants Template].

Some important tips for conducting Pilot participant outreach are as follows:
 * Target a diverse community in order to create the appropriate mix of users.
 * Divide the population into appropriate segments such as, Market Share, Piloting Experience, Groups, Communities, Networks etc., and Pilots WG Committed Members
 * Use the best suited outreach methodology (as determined by the Pilots WG) for reaching the participants
 * Broadcast (E-mail, blog, website, social networking etc.)
 * Follow up (personal contact to extend the outreach)
 * Support (providing all necessary material and answers to questions
 * Prepare reference material to be shared with the prospects.
 * S&I Framework Wiki – Collaboration site for Workgroup meetings and all supporting documents for each initiative. Examples of two Pilots WG wiki pages can be found here and here.
 * One-pager - summary on the pilot goals, expectations and next steps ([|example])
 * Pilots WG Action Plan ([|example])
 * Pilots Schedule/Project Plan ([|MS Project Example] /[|MS Excel Example])
 * Value Statements – consisting of marketing material providing value propositions for stakeholders (Value Statement examples here and here)
 * Broadcast by E-mail Blast (used SendBlaster2 with tracking reports) ([|example])
 * Follow up by Phone or E-mail (selective focus groups)
 * Investigate other tools that can be used for outreach (e.g., social media)
 * Partnerships – provide leads to possible partnerships to fill in the gaps for a complete ecosystem

3.0 Integrating Pilots WG members into the S&I Framework
Once the Initiative's Pilots WG has been established, the members of the Pilots WG will want to acquaint prospective Pilots with the mission of the Pilot Teams, and how the Initiative fits in with the overall S&I Framework mission.


 * Introductory Meetings**
 * Set up introductory meetings with interested parties (who usually represent organizations such as EHR vendors, provider groups, HIT Integrators, etc) to discuss goals of the Pilots WG, expectations, contact information and next steps.


 * Roles within the Pilots Workgroup**
 * Members of a Pilots WG should align themselves with the role they intend to play, and post this chosen role with their contact information on the WG wiki page.
 * A list of Pilot WG roles:
 * **Role** || **Responsibilities** ||
 * **Analyst** || Participate in the development and review of detailed design documentation for the Pilots. ||
 * **Developer** || Actively participate in the development and peer review of code for the Pilots. Coding may involve the modification and adaptation of existing tools and software packages, or development of original code required to realize necessary functions in the architecture. ||
 * **Domain** **SME** || A Subject Matter Expert (SME) in the Initiative’s technology, use cases, requirements, standards, etc. The Domain SMEs should be familiar with the S&I Framework Clinical Element Data Dictionary (CEDD). ||
 * **Pilot** || Organization is willing to serve as a Pilot site and as a Pilot Team member. ||
 * **Test Data Provider** || Able to provide test data for pilots. ||
 * **Pilot Tester** || Able to help plan or conduct pilot testing. ||
 * **ONC Contractor** || Contractor to ONC as part of the S&I Framework. ||
 * **Tools SME** || Provide the group with expertise on a particular existing tool set which might be leveraged in achieving the goals of the Reference Implementation. Please indicate the tool set in which you are expert. ||
 * **Communications POC** || Serve the Pilots Workgroup as the Communications Point of Contact (POC). ||

Some applicable material that can aid the Pilot Team members in understanding the Pilots WG expectations are:
 * Supporting Documentation**
 * **Pilot Guide**
 * Each Initiative's Pilots WG should prepare a Pilots Guide that details the goals, objectives, scope, requirements, actors and their roles and responsibilities, and an ecosystem, including configuration and workflow, etc. Examples of actual Pilots Guides are [|here] and [|here]
 * **Pilot Charter/Brief template**
 * The Pilot Team Charter/Pilots Team Brief template (see this example) contains relevant sections to understand the pilot objectives, design, scope, use cases and a pilot timeline including milestones. The Pilot Team Brief can serve as a basis for a more detailed project plan.
 * After the Pilot WG member adds an entry into the “Proposed Pilot Team Projects” table, the ONC Support team creates a Pilot Brief page for the Pilot Team to complete. The Pilot Team Brief (also sometimes referred to as a Pilot Team Charter), will describe all aspects of the Initiative Pilot Team. After the Pilot Team template is added, an entry will be shown in the "Proposed Pilot Teams Up for Consensus" table. The Workgroup members accept or reject the proposed Pilot Team project. The accepted Pilot Team’s information is placed in the Accepted Pilot Team Projects table.
 * **Pilot Configuration Sample**
 * An S&I Initiative's Pilot WG should prepare a sample template of a Pilot Team Configuration document, which briefly outlines (1) Actors Involved, (2) Minimum Configuration, (3) Sample Pilot Diagram, and (4) Pilot Project Roles. Examples from actual S&I Initiatives are [|here] and [|here].
 * **General Project Plan**
 * A general project plan that will guide the Pilots Workgroup members to understand the projected milestones and timeline that the Pilot Teams will encounter, and that can be used to track progress toward the ultimate goal of Pilot deployments. An example of a Project Plan from an actual S&I Initiative is [|here].

4.0 S&I Pilots Communication
Efficient communication between the Pilots WG, Pilot Teams, ONC, and other S&I stakeholders and participants is essential to the success of the Pilots work efforts (see the [|Pilots Communication Plan]).

4.1 Communications Point of Contact (POC)
It is recommended to appoint one of the Pilots WG members to the role of Communications Point of Contact (POC) for the Workgroup. The Pilots WG is encouraged to maintain a Communication Plan document (an example of one is [|here]).

Some key responsibilities for the Pilots WG POC are:
 * Monitor comments and questions wiki page.
 * Escalate issues and risks to proper teams as needed and communicate any updates back to the Pilots WG.
 * Ensure the proper S&I Team members and Pilot Team members are notified (consider use of email notifications).
 * Conduct a weekly follow-up email and/or meeting to obtain the status of outstanding questions.
 * Escalate questions to ONC as needed.

4.2 Pilot Feedback


S&I Initiative Pilot Teams must provide ONC with feedback and updates on their progress. In the interests of efficient and timely communication, the S&I Pilots Workgroups will design and implement formal feedback channels for communication with ONC, as well as with their fellow Workgroups.
 * Pilot Team Feedback to ONC**

The Pilots Ecosystem diagram is a table that summarizes the roles and resources that each of the Pilot Teams are currently supporting. The Ecosystem is a dynamic document where one can track the progress as the Pilots assemble their teams and define their goals and expectations. Examples of actual Pilot Ecosystems can be seen here and here.

At the completion of their pilot, each Pilot Team will present a Pilot Feedback Report to ONC (see above).

The Pilots WG Communications POC will update the S&I Metrics database monthly, by adding pilot data statistics to a spreadsheet on Google Docs. These metrics provide ONC Management with pertinent information involving the activities of the workgroup and level of pilot participation.
 * Pilot Workgroup Feedback to ONC**

Each Pilot Team should document their success criteria at the time of Pilot Team formation. It is the responsibility of all Pilot Teams to provide feedback on a regular basis to the Pilots WG, which will in turn keep ONC Management apprised. Once a Pilot Team has deployed, it shall provide quantitative metrics weekly to its Pilots WG.
 * Individual Pilot Success Criteria and Feedback**

An Initiative may identify metrics that are unique to the Initiative to help quantify its success. These metrics are not necessarily related specifically to the success of the pilots, and instead, are formulated to identify and capture vital information in relation to the Pilot Teams. In the past, workgroups devoted to formulating these Initiative level Success Metrics have been formed (examples here and here).
 * Initiative Level Success Metrics**

4.3 Pilot Feedback to other S&I Teams

 * Initiative Comments and Questions wiki page**
 * Each S&I Initiative should create and maintain on the S&I Wiki a "Comments and Questions" wikipage, similar to the example page here. Alternative Tracker tools like MS Excel, Redmine, or Jira can be used instead of, or in addition to, the S&I Wiki, if desired.
 * The data table on the Initiative Comments and Questions wiki page should include fields like:
 * priority, status, category (e.g., User Story, IG), reporter, owner, description, resolution, date opened, date closed
 * If this goes on the wiki, the Pilots WG Communications POC will have the option of receiving an e-mail notification when a comment or question is added to the wiki page.