Query+Health+Technical+Approach+Risk+Assessment+v2.o

include component="page" wikiName="siframework" page="Query Health Header" toc  The purpose of this page is to serve as a forum for discussing the risks associated with the Query Health Technical Approach. This discussion is based on the Technical Approach Primer that was presented to the Operations Workgroup on February 2nd, 2012. This page will examine each step of the Technical Approach and provide a place for the community to document any risks, issues, or concerns.

** Technical Approach- General Concerns **

The Query Lifecycle


What about Query Encryption? ||  || What about quality checks? Audits of queries? Audits of data being sent in response to queries? || None of what we are building should be based on "buddy system;" Trust should be outlined, not assumed or implied; there should be rules of the road. People travelling the road should live by the rules; User Design side of this... drafting some benchmarking language (Paulo Machado). If there is a benchmark, and the timing is not met, is there a penalty for no response? Is a query invalidated after a certain amount of time? This links back to acknowledgement; Acknowledgement of request, receipt etc. There is a consequence if timing is not met. Framing expectations: 1. You want to know they got it? 2. You want to know how they understand it? 3. When will they get back to you?
 * < **Name** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||
 * < Rob McClure ||< This cannot be implemented from the word 'go.' There is a significant amount of start up time before queries can become automated and easy. ||<  ||<   ||<   ||
 * < Steve Beller, PhD ||< I have had similar concerns to Rob from the very beginning, and my angst about this technical approach has been confirmed over these past few months. The approach seems excessively complex technologically and, from a researcher’s or clinician’s perspective, it appears to overlook the real purpose for doing health queries that focus on quality improvement and public health protection. We ought to be establishing the standards and specifications based on implementations that demonstrate their ability to deliver the desired payloads and generate the desired results (such as calculating the requisite numerators and denominators). It seems we did it backwards by establishing standards prior to demonstrating their feasibility. Even worse, everyone is required to adopt those unproven standards instead of seeking innovative alternatives. ||<  ||<   ||< From a software engineering perspective, complexity and component parts along with the links between parts gives rise to failure. For example: if you have two nodes connected together and each node includes a software stack and a link with the following availabilities: node= 95% and the link= 99.99%. The combined availability of this node-to-node pair is 0.95 X 0.95 X 0.9999, or we could anticipate an availability of 90%. Hence, given 24X7 query operational requirements and an anticipated yearly operational period of 61,320 hours, one would calculate that the failure period in this scenario could approach 6,132 hours. Increase the number of components to the aforementioned twelve presented in Step 3, given that each component exhibits an availability of 0.9999, and we could anticipate a failure period of 6,967 hours. I find this unacceptable. If the query health end user works a maximum of 2,000 hours per year at 100% efficiency, and their work time is mostly between 8 AM and 6 PM, the numbers imply that the query health solution presented might easily frustrate the end user because the system could be unavailable when the end user needed it. The failure hours per year amortized over the work hours implies 3.5 hours of failure per day. Further, the query health solution could become very expensive because this level of complexity may require significant redundancy to compensate for the failure modes. If the end user, (a) was a state department of health attempting to isolate an outbreak of H1N1, and (b) they needed the immunization records and GIS information along with an information payload to identify the at risk population, and(c) the incident response team needed the information immediately, I wouldn’t want this team to depend on a query health solution/system that exhibited this type of failure potential. ||
 * Support Team ||  ||   || Should the Technical Approach utilize RBAC (Role Based Access Contols)? Is it necessary for de-identified, aggregate data? ||   ||
 * OWG ||  ||   || Data Encryption is a must for data "at rest" and data "in motion";
 * Support Team ||  ||   ||   || Should there be a log of who submits queries, when they were submitted etc.?
 * Thompson Boyd ||  ||   ||   ||   ||
 * Paulo Machado/Thompson Boyd ||  ||   ||   || Expectation around the turn-around time for query results; establishing a formal business relationship; standards around timing response; simple query- 1 day; complex query- 1 week; benchmark the time. (This is also related to TRUST)

Setting up measures to validate success. Success Metrics?
 * 100% on time delivery date?
 * 75% objective etc.?

How about ROI? ||

Click to Return to Top

Step 1: Create Query
** Occurs within the Department of Health **
 * The New York State Department of Health must create a query using one of three possible ‘Query Builders’ (hQuery, i2b2, or HQMF)
 * These builders were chosen so that requestors do not have to be technical experts to create a query. Instead, they can create queries using common clinical language in a user-friendly interface.

What technical expertise do requestors lack that prevent them from creating a query? There are ways in which they could simply ask for a few criteria that define the basic target population (such as age range, gender, and the value sets defining the requisite diagnoses, procedures, meds, tests, encounters, etc.). The DMS of each responder’s EHR could then generate the necessary XML data files according to the CCD standards required by Stage 1 MU. Those files could then be compiled into a simple database file containing the required payload, which is then shipped to the requestor for aggregation and analysis, ||<  ||<   ||<   ||~   || Click to Return to Top
 * < **Name** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * < Natalie Menser ||<  ||<   ||< Required Authentication to ask a query ||<   ||~   ||
 * < Steve Beller, PhD ||< I assume "common clinical language" means standardized terminologies for interoperability. If so, I agree we need such standards, or at least we should have robust value sets that incorporate all responders’ “local terminology standards” with a requestor’s “global terminology standard” in order to enable sensible aggregation. There are various ways to do this.

Step 2: Query is sent to PopMedNet Portal
** Occurs within the Department of Health **
 * The query is sent in the format of Modified HQMF* to the PopMedNet Portal. PopMedNet has a user-friendly web-based portal, where queries are submitted.
 * *HQMF is a standard for representing health quality measures as an electronic document. The Query Health initiative is making modifications to the standard to suit the needs of Query Health.


 * < **Name** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * < Rob McClure ||< Queries must be sent in a format that is understandable for those orgs that do not want to or cannot implement automation and have to run the queries manually. ||<  ||<   ||<   ||~   ||
 * < Steve Beller, PhD ||< Concur with Rob ||<  ||<   ||<   ||~   ||

Click to Return to Top

Step 3: PMN Portal sends Query to PMN Data Client
** Occurs within Beth Israel Hospital **
 * The PopMedNetPortal sends the query, via Query Envelope, to the PopMedNet Data Client.
 * The Query Envelope is designed to be query and content agnostic and ensure enforcement of privacy and security requirements.
 * The PopMedNet Data Client is the counterpart of the PopMedNet Portal where queries are received by (Beth Israel Hospital).

focus on re-use. Alignment with Data Segmentation intiaitve. ||<  ||< Under Query Syntax (part of Query Envelope), recommendation to include frequency of query and something around the duration or lifespan of a query. ||~  ||
 * < **Name** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * < Jeff Brown ||<  ||<   ||<   ||< Meta Data Suggestions:
 * Restrictions on use/other limitations
 * Warnings, agreements, requirements for security (currently the software allows free text notes to be returned with all queries)
 * Requesting Authority
 * Intended Use of Data
 * Designation if query meets definition of research
 * Preparatory to Research
 * Public Health Surveillance ||~  ||
 * < Thompson Boyd ||<  ||< UEL Data Provenance- HITSC Considerations;

Click to Return to Top

Step 4: PMN Data Client Sends Query to Procedural Translator

 * Occurs within Beth Israel Hospital **


 * PMN Data Client sends the Query to the Procedural Translator which creates specific directions on how to run the query. The Procedural Translator takes the message in HQMF (declarative language) and translates it to JavaScript or SQL(procedural languages).

Note: In this step, Responding Organizations, like Beth Israel, have the right to reject any query.

Also, did the query data source actually get the query result. . Essentially these are forms of acknowledgement. ||<  ||< The Translator is key to consistency, high quality, data, and maintaining data integrity; it is really a critical element of the process, especially when scaled up to population level. Lots of details at the translator level. ||< The use of some sort of Data Dictionary will be critical here to make sure translations are consistent ||~  ||
 * < **Name** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * < Thompson Boyd ||< Recommendation to include some form of confirmation of query results receipt; this will make sure that the Data Source knows that the query requestor received and read the results.
 * < Bobby Lee ||<  ||<   ||<   ||< Software released under an open source license; accepting the license, accepting at your own risk ||~   ||

Click to Return to Top

Step 5: Procedural Translator sends Query to Execution Engines

 * Occurs within Beth Israel Hospital **


 * The Procedural Translator sends the query to the Execution Engines.
 * They query is now in JavaScript or SQL (procedural languages).
 * The Execution Engines run the queries while matching the concepts (e.g. “diabetes” and/or codes (e.g. an ICD 9 code) to the Data Sources using the CIM(CEDD).
 * Clinical Information Model (CIM)/Clinical Elements Data Dictionary (CEDD) is a dictionary that is an ‘overlay’ on the Data Source. It is a translation tool.
 * Example: A query may ask for the number of people taking a particular medication. The system will use the CIM/CEDD as a reference to identify the number of patients who have that medication listed in their ‘active medications list.’
 * <span style="display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">All of this occurs behind Beth Israel’s firewall, thus maintaining patient privacy and security of data.

Scott Wenstein Bobby Lee ||<  ||< How do we protect behavior health data like substance abuse? 1 - Create a safeguard so that the parts of your database that contain behavioral health data cannot be queried. AND/OR 2- Prevent queries that ask about behavioral health data from ever being created. (See Step 1). ||<  ||<   ||~   ||
 * < **Name** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * < Rob McClure ||< Initial concept mapping for new participants will take significant time and effort ||<  ||<   ||<   ||~   ||
 * < Question: Erin Fitzsimmons Response:
 * Steve Beller, PhD ||  || It seems to me that the EHRs will have to contain flags, at the individual data element or data category level, which identify each patient's privacy preferences. The flags would have to mask data that the patient does not want to send. Hopefully, if data de-identification is found to adequately protect QH PHI, this won't be a very big problem because people will tend to trust the process. ||   ||   ||   ||
 * Bobby Lee ||  ||   ||   || Version control; translator version alignment with the data dictionary version; helps to maintain the integrity of the query request ||   ||

Click to Return to Top

Step 6: Execution Engines Send result to QRDA Translator

 * Occurs within Beth Israel Hospital **

>> *QRDA Category III was selected because it allows for returning aggregate or summary level results.
 * <span style="display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">The Execution Engines take the results, which are still in JavaScript or SQL and send them to the QRDA Translator, to be changed to QRDA
 * <span style="display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">QRDA is a document format that will provide a standard structure with which to report results.
 * <span style="display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">Remember: The Department of Health does not necessarily understand the coding language of Beth Israel Hospital (JavaScrip or SQL).
 * <span style="display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">Therefore results must be translated in a standard way before they can be sent back to the Department of Health.
 * <span style="display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">The standard selected by the Technical Workgroup for expressing query results is QRDA Category III*.


 * < **Name** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * < Stephen Beller, PhD ||< I don’t understand how QRDA helps improve the meaning and value of the query results. And Category III is too limiting for many purposes because patient-level data is lost. So, for example, the requestor is unable to examine particular data elements about the patients in the numerator, thereby making them unable to determine why they are that way. ||<  ||<   ||<   ||~   ||

Click to Return to Top

Step 7: QRDA Translator sends results back to PMN Data Client

 * Occurs within Beth Israel Hospital **


 * <span style="display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">The QRDA Translator sends the results back to the PopMedNet Data Client, which routes them back to the PopMedNet Portal.
 * <span style="display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">The PopMedNet Data Client is the last stop at Beth Israel before the results are sent back to the Department of Health.

This is the last stop before information is returned; may need final sign-off and authorization. ||< May want to examine results for any specific HIPAA violations ||<  ||<   ||~   ||
 * < **Name** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * < Erik Pupo ||< Results Review prior to being sent back to the query requestor?

Click to Return to Top

Step 8: PMN Data Client sends results back to PMN Portal
<span style="color: #7f7f7f; display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">** Query Transported from Beth Israel Hospital to Department of Health **
 * <span style="color: #000000; display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">The PopMedNet Data Client sends the results, now in QRDA, back to the Department of Health through the PopMedNet Portal using the Query Envelope.
 * <span style="color: #000000; display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">The Query results are now in the hands of the Department of Health.
 * <span style="color: #000000; display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">Reminder: The Query Envelope is designed to be query and content agnostic and ensure enforcement of privacy and security requirements.


 * < **Name** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * < Linda Martino ||<  ||<   ||< Does the Query Envelope act as a substitute for encryption? Is there any risk of security breaches due to human error? ||<   ||~   ||
 * < Stephen Beller, PhD ||<  ||<   ||< I recommend that all transmissions be encrypted in transit and at rest, and that a DIRECT QH implementation be made available. ||<   ||~   ||
 * Bobby Lee || This is automatic as of now; groups may want to have this be manual/review. Something to consider. ||  ||   || Data Recall; "Please destroy data once you receive the notice of recall;" important to have a mechanism of some sort. ||   ||
 * Thompson Boyd ||  ||   || Recommendation for encryption of data in transit. (Data should also be encrypted at rest?) || Level of TRUST ||   ||

Click to Return to Top

Step 9: PMN Portal sends the results to the results viewer
<span style="color: #7f7f7f; display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">** Occurs within the Department of Health **
 * <span style="color: #080808; display: block; font-family: arial; font-size: 12px; text-align: left; vertical-align: baseline;">The PopMedNet Portal now sends the results to the Results Viewer, a specific interface designed for reviewing query results. Now an individual at the Department of Health can review the results.

If this feature exists or planned for, all security/privacy/policy must be re-applied again for the related query as though it was done from scratch. There should not be any inherent "trust" based on the original query. ||  ||
 * < **Name** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * < Rob McClure/Susan Campbell ||<  ||<   ||<   ||< One needs to be able to view the questions that were asked along with the query results; otherwise one will not understand the ratio that is sent back; this will provide a way to ensure that the data you get back is accurate and matches the query. ||~   ||
 * < Thompson Boyd ||<  ||<   ||<   ||< Strongly encourage that this be in QRDA Cat III; Cat III is preferable; allows for some rara Cat II situations. ||~   ||
 * Bobby Lee ||  ||   ||   || Is there a mechanism to submit a related query based on the response? Something like, I want to expand the criteria (e.g. "Male" --> "Male" & "Female") or get more data points or more detailed results.

Click to Return to Top

HQMF - General Concerns

 * < **Name** || **Standard** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * <  || **HQMF** ||<   ||<   ||<   ||<   ||~   ||
 * <  || **HQMF** ||<   ||<   ||<   ||<   ||~   ||

Click to Return to Top

QRDA - General Concerns

 * < **Name** || **Standard** ||< **Policy** ||< **Privacy** ||< **Security** ||< **Other** ||~  ||
 * < Rob McClure || QRDA ||< QRDA Category II indicates that patient level data may or may not be used. Query Health should be more restrictive in saying that patient level data will NEVER be used. ||<  ||<   ||<   ||~   ||
 * < Stephen Beller, PhD || QRDA ||< I contend that, depending on the purpose for the query, patient level data SHOULD be used with the constraint that patient identifiers NOT be included. When it comes to the continuous quality improvement of healthcare processes/methods/guidelines, having only a percentage figure (numerator over denominator) from many different data sources is of minimal use because that single number doesn’t enable drill-down into the details of what caused (or is associated with) those results. I suppose those numbers could be useful for comparing the relative performance of providers of certain metrics, if that’s all we want to do. ||<  ||<   ||<   ||~   ||

Click to Return to Top

Reference Documents

 * **Document Name** || **Description** || **Version** ||
 * [|OWG Technical Approach Primer] ||  ||   ||
 * [|HQMF_QRDA_] ||  ||   ||

Click to Return to Top

include component="page" wikiName="siframework" page="space.template.inc_contentleft_end"