Patient de-identification Process API - Implementation Template
Developer guide
This document provides information generated from the DataWeave scripts included in the project, such as function definitions, variable definitions, and data mapping tables.
Module | Description |
---|---|
createDeIdentifiedResources | This module defines functions that are used to generate De-Identified resources based on original resource, rules, and De-Identified Resource IDs. This module require changes if any new fields need to be included in a specific resource. |
fhirTools | This module defines a number of common functions that are used to generate De-Identified values for the fields of a resource based on original value and Rule. Introducing new rules may require modifications to the functions. |
export-config-request-to-db-mapping | Create an Export Configuration |
export-config-get-response-mapping | Returns Transaction Details along with Export Configuration details for a specific Export Configuration |
createDeIdentifiedResources
This module defines functions that are used to generate De-Identified resources
based on original resource, rules, and De-Identified Resource IDs.
This module require changes if any new fields need to be included in a specific
resource.
Source:
./src/main/resources/dwl/createDeIdentifiedResources.dwl
Variables
var observationResourceType
Variable to hold the Resource Types that has Observation, used for filtering references
var patientResourceType
Variable to hold the Resource Types that has Patient, used for filtering references
var supportedResourceTypes
Variable to hold the supported Resource Types, used for filtering references
Functions
fun getPatientBirthDate (item, format)
This function takes a patient object that need to be De-Identified and validates the age to produced a De-Identified date of birth.
param
item
Patient object that need to be De-Identified.
paramformat
date format to be used to produce De-Identified date of birth.
return datetime which is the De-Identified date of birth of the patient.
fun getDeIdentifiedContainedResources (containedArray, muleIds, rules, resourceTypes, zipCodeToPopulation)
This function takes an array of resources to De-Identify and returns an array of De-Identified resources based on rules,supported resource types and De-Identified resource ids. This function is mainly used to De-Identify the resources in the 'contained' field of the top level resource.
param
containedArray
an array of resources that are to be De-Identified from the contained field.
parammuleIds
an object that has IDs of resources as a key and De-Identified IDs as value.
paramrules
an array of rules to apply to De-Identify the resource.
paramresourceTypeArray
an array of resource types that are to be retained if reference exists.
return resource Array with De-Identified objects.
fun createDeIdentifiedPatient (data, muleIds, rules, zipCodeToPopulation, metaTag)
This function takes a Patient Resource Object to De-Identify and returns the De-Identified resources based on rules and De-Identified resource ids provided. Only a specific set of fields are retained in the De-Identified object.
param
data
Patient resource object that is to be De-Identified.
parammuleIds
an object that has IDs of resources as a key and De-Identified IDs as value.
paramrules
an array of rules to apply to De-Identify the resource.
paramzipCodeToPopulation
an array of Zip Codes and population to De-Identify Zip Codes based on rules.
parammetaTag
a tag applied to search De-Identified resources based on export configuration id.
Field Name | Type(s) | Min | Max | Default retain in the output | User determined | Options presented to user | Comments |
---|---|---|---|---|---|---|---|
Patient.id | id | 0 | 1 | YES | NO | The application will generate a unique ID for each FHIR resource instance and substitute the source system resource identifier with it. Both the source system and the application-generated resource instance identifiers will be stored in a data store | |
Patient.implicitRules | uri | 0 | 1 | NO | NO | If implicitRules are detected; the application will not process the record and log that it was not processed | |
Patient.contained | Resource | 0 | * | YES | NO | Retained if the reference to the contained resource is retained | |
Patient.modifierExtension | Extension | 0 | * | NO | NO | If modifierExtensions are detected; the application will not process the record and log that it was not processed | |
Patient.active | boolean | 0 | 1 | YES | NO | Actual value will be returned | |
Patient.gender | code | 0 | 1 | YES | NO | Actual value will be returned | |
Patient.birthDate | date | 0 | 1 | NO | YES | Keep the birthdate in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM 4. YYYY-MM-DD | If the patient age is 89 or older then default to 90 years and set the year accordingly |
Patient.deceased[x] | boolean dateTime | 0 | 1 | NO | YES | Keep the deceased date in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM 4. YYYY-MM-DD 5. YYYY-MM-DDThh:mm:ss+zz:zz | |
Patient.address | Address | 0 | * | NO | YES | Keep the address or geographic identifiers in output? 1. Do not retain 2. Truncate to the first 3 characters of zip code 3. Truncate to the first 5 characters of zip code 4. Do not truncate | If postal code is not found or population is less than 20K include default capability to modify output to 000 or 00000 or 00000-0000 |
Patient.maritalStatus | CodeableConcept | 0 | 1 | NO | YES | Keep the marital status in output? 1. Do not retain 2. Yes | |
Patient.multipleBirth[x] | boolean integer | 0 | 1 | NO | YES | Keep multiple birth information in output? 1. Do not retain 2. Convert to boolean if numeric 3. Yes | |
Patient.communication.language | CodeableConcept | 1 | 1 | NO | YES | Keep patient language in output? 1. Do not retain 2. Yes |
fun createDeIdentifiedCondition (data, muleIds, rules, zipCodeToPopulation, metaTag)
This function takes a Condition Resource Object to De-Identify and returns the De-Identified resources based on rules and De-Identified resource ids provided. Only a specific set of fields are retained in the De-Identified object.
param
data
Condition resource object that is to be De-Identified.
parammuleIds
an object that has IDs of resources as a key and De-Identified IDs as value.
paramrules
an array of rules to apply to De-Identify the resource.
paramzipCodeToPopulation
an array of Zip Codes and population to De-Identify Zip Codes based on rules.
parammetaTag
a tag applied to search De-Identified resources based on export configuration id.
return Condition resource object with De-Identified fields based on rules.
Field Name | Type(s) | Min | Max | Default retain in the output | User determined | Options presented to user | Comments |
---|---|---|---|---|---|---|---|
Condition.id | id | 0 | 1 | YES | NO | The application will generate a unique ID for each FHIR resource instance and substitute the source system resource identifier with it. Both the source system and the application-generated resource instance identifiers will be stored in a data store | |
Condition.implicitRules | uri | 0 | 1 | NO | NO | If Application detects implicitRules; it will not process the record and log that it was not processed | |
Condition.contained | Resource | 0 | * | YES | NO | Retained if the reference to the contained resource is retained | |
Condition.modifierExtension | Extension | 0 | * | NO | NO | If modifierExtensions are detected; the application will not process the record and log that it was not processed | |
Condition.clinicalStatus | CodeableConcept | 1 | 1 | YES | NO | CodeableConcept.text is removed | |
Condition.verificationStatus | CodeableConcept | 0 | 1 | YES | NO | CodeableConcept.text is removed | |
Condition.category | CodeableConcept | 0 | * | YES | NO | CodeableConcept.text is removed | |
Condition.severity | CodeableConcept | 0 | 1 | YES | NO | CodeableConcept.text is removed | |
Condition.code | CodeableConcept | 0 | 1 | NO | YES | Filter by condition codes 1. Include all conditions 2. Filter by selected codes | CodeableConcept.text is removed; If the condition code filter yields no results then do not process the condition resource instance itself. Assumptions: If the source record does not include system and code then filter the condition resource instance. |
Condition.bodySite | CodeableConcept | 0 | * | YES | NO | CodeableConcept.text is removed | |
Condition.bodyStructure | Reference(BodyStructure) | 0 | 1 | NO | YES | Keep the Body Structure details in output? 1. Do not retain 2. Retain | |
Condition.subject | Reference(Patient or Group) | 1 | 1 | YES | NO | Only need to support Patient and not Group | |
Condition.onset[x] | dateTime | 0 | 1 | NO | YES | Keep the condition onset date in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM 4. YYYY-MM-DD 5. YYYY-MM-DDThh:mm:ss+zz:zz | |
Condition.onset[x] | Age | 0 | 1 | YES | NO | Actual value will be returned | |
Condition.onset[x] | Period | 0 | 1 | NO | YES | Keep the condition onset period in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM | |
Condition.onset[x] | Range | 0 | 1 | NO | YES | Keep the condition onset range in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM | |
Condition.abatement[x] | dateTime | 0 | 1 | NO | YES | Keep the condition abatement date in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM 4. YYYY-MM-DD 5. YYYY-MM-DDThh:mm:ss+zz:zz | |
Condition.abatement[x] | Age | 0 | 1 | YES | NO | Actual value will be returned | |
Condition.abatement[x] | Period | 0 | 1 | NO | YES | Keep the condition abatement period in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM | |
Condition.abatement[x] | Range | 0 | 1 | NO | YES | Keep the condition abatement range in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM | |
Condition.recordedDate | dateTime | 0 | 1 | NO | YES | Keep the condition recorded date in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM 4. YYYY-MM-DD 5. YYYY-MM-DDThh:mm:ss+zz:zz | |
Condition.stage | BackboneElement | 0 | * | YES | NO | Actual value will be returned | |
Condition.stage.summary | CodeableConcept | 0 | 1 | YES | NO | CodeableConcept.text is removed | |
Condition.stage.type | CodeableConcept | 0 | 1 | YES | NO | CodeableConcept.text is removed |
fun createDeIdentifiedObservation (data, muleIds, rules, zipCodeToPopulation, metaTag)
This function takes a Observation Resource Object to De-Identify and returns the De-Identified resources based on rules and De-Identified resource ids provided. Only a specific set of fields are retained in the De-Identified object.
param
data
Observation resource object that is to be De-Identified.
parammuleIds
an object that has IDs of resources as a key and De-Identified IDs as value.
paramrules
an array of rules to apply to De-Identify the resource.
paramzipCodeToPopulation
an array of Zip Codes and population to De-Identify Zip Codes based on rules.
parammetaTag
a tag applied to search De-Identified resources based on export configuration id.
return Observation resource object with De-Identified fields based on rules.
Field Name | Type(s) | Min | Max | Default retain in the output | User determined | Options presented to user | Comments |
---|---|---|---|---|---|---|---|
Observation.id | id | 0 | 1 | YES | NO | The application will generate a unique ID for each FHIR resource instance and substitute the source system resource identifier with it. Both the source system and the application-generated resource instance identifiers will be stored in a data store | |
Observation.implicitRules | uri | 0 | 1 | NO | YES | If implicitRules are detected; the application will not process the record and log that it was not processed | |
Observation.contained | Resource | 0 | * | YES | NO | Retained if the reference to the contained resource is retained | |
Observation.modifierExtension | Extension | 0 | * | NO | YES | If modifierExtensions are detected; the application will not process the record and log that it was not processed | |
Observation.status | code | 1 | 1 | YES | NO | Actual value will be returned | |
Observation.category | CodeableConcept | 0 | * | YES | NO | CodeableConcept.text is removed | |
Observation.code | CodeableConcept | 1 | 1 | YES | NO | CodeableConcept.text is removed | |
Observation.subject | Reference(Patient or Group or Device or Location or Organization or Procedure or Practitioner or Medication or Substance or BiologicallyDerivedProduct or NutritionProduct) | 0 | 1 | YES | NO | Retains the value when the Reference is a Patient | |
Observation.focus | Reference(Any) | 0 | * | YES | NO | Retains if Reference is to a supported resource (Patient or Condition or BodyStructure) | |
Observation.effective[x] | dateTime | 0 | 1 | NO | YES | Keep the observation effective date in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM 4. YYYY-MM-DD 5. YYYY-MM-DDThh:mm:ss+zz:zz | |
Observation.effective[x] | Period | 0 | 1 | NO | YES | Keep the observation effective period in output? Options: 1. Do not retain 2. YYYY 3. YYYY-MM | |
Observation.effective[x] | instant | 0 | 1 | NO | YES | Keep the observation effective instant in output? Options: 1. Do not retain 2. Retain | |
Observation.issued | instant | 0 | 1 | NO | YES | Keep the observation issued instant in output? Options: 1. Do not retain 2. Retain | |
Observation.value[x] | Quantity CodeableConcept string boolean integer Range Ratio SampledData time dateTime Period Attachment Reference(MolecularSequence) | 0 | 1 | YES | NO | Actual value will be returned | |
Observation.dataAbsentReason | CodeableConcept | 0 | 1 | YES | NO | CodeableConcept.text is removed | |
Observation.interpretation | CodeableConcept | 0 | * | YES | NO | CodeableConcept.text is removed | |
Observation.bodySite | CodeableConcept | 0 | 1 | YES | NO | CodeableConcept.text is removed | |
Observation.bodyStructure | Reference(BodyStructure) | 0 | 1 | YES | NO | Actual value will be returned | |
Observation.method | CodeableConcept | 0 | 1 | YES | NO | CodeableConcept.text is removed | |
Observation.referenceRange | BackboneElement | 0 | * | YES | NO | Actual value will be returned | |
Observation.referenceRange.low | Quantity {SimpleQuantity} | 0 | 1 | YES | NO | Actual value will be returned | |
Observation.referenceRange.high | Quantity {SimpleQuantity} | 0 | 1 | YES | NO | Actual value will be returned | |
Observation.referenceRange.normalValue | CodeableConcept | 0 | 1 | YES | NO | CodeableConcept.text is removed | |
Observation.referenceRange.type | CodeableConcept | 0 | 1 | YES | NO | CodeableConcept.text is removed | |
Observation.referenceRange.appliesTo | CodeableConcept | 0 | * | YES | NO | CodeableConcept.text is removed | |
Observation.referenceRange.age | Range | 0 | 1 | YES | NO | Actual value will be returned | |
Observation.hasMember | Reference(Observation or QuestionnaireResponse or MolecularSequence) | 0 | * | YES | NO | Retains if a Reference is to a supported resource (currently just Observation) | |
Observation.derivedFrom | Reference(DocumentReference or ImagingStudy or ImagingSelection or QuestionnaireResponse or Observation or MolecularSequence or GenomicStudy) | 0 | * | YES | NO | Retains if a Reference is to a supported resource (currently just Observation) | |
Observation.component | BackboneElement | 0 | * | YES | NO | Actual value will be returned | |
Observation.component.code | CodeableConcept | 1 | 1 | YES | NO | CodeableConcept.text is removed |
fun getBodyStructure (bodyStructure, muleIds, rules, metaTag)
This function takes a BodyStructure Resource Object to De-Identify and returns the De-Identified resources based on rules and De-Identified resource ids provided. Fields in Body Structure vary from FHIR R4 and R5 versions and both scenarios are handled. Only a specific set of fields are retained in the De-Identified object.
param
bodyStructure
BodyStructure resource object that is to be De-Identified.
parammuleIds
an object that has IDs of resources as a key and De-Identified IDs as value.
paramrules
an array of rules to apply to De-Identify the resource.
paramzipCodeToPopulation
an array of Zip Codes and population to De-Identify Zip Codes based on rules.
parammetaTag
a tag applied to search De-Identified resources based on export configuration id.
return BodyStructure resource object with De-Identified fields based on rules.
Field Name | Type(s) | Min | Max | Default retain in the output | User determined | Options presented to user | Comments |
---|---|---|---|---|---|---|---|
BodyStructure.id | id | 0 | 1 | YES | NO | The application will generate a unique ID for each FHIR resource instance and substitute the source system resource identifier with it. Both the source system and the application-generated resource instance identifiers will be stored in a data store | |
BodyStructure.implicitRules | uri | 0 | 1 | NO | NO | If implicitRules are detected; the application will not process the record and log that it was not processed | |
BodyStructure.modifierExtension | Extension | 0 | * | NO | NO | If modifierExtensions are detected; the application will not process the record and log that it was not processed | |
BodyStructure.active | boolean | 0 | 1 | YES | NO | Actual value will be returned | |
BodyStructure.includedStructure | BackboneElement | 1 | * | YES | NO | Actual value will be returned | |
BodyStructure.includedStructure.structure | CodeableConcept | 1 | 1 | YES | NO | Actual value will be returned | |
BodyStructure.includedStructure.laterality | CodeableConcept | 0 | 1 | YES | NO | Actual value will be returned | |
BodyStructure.patient | Reference(Patient) | 1 | 1 | YES | NO | Actual value will be returned |
fun getIncludedStructure (includedStructure)
This function takes a includedStructure field of BodyStructure Resource Object to De-Identify and returns the De-Identified structure and laterality fields.
param
includedStructure
includedStructure object of BodyStructure resource object that is to be De-Identified.
return includedStructure resource object with De-Identified fields.
fhirTools
This module defines a number of common functions that are used to generate
De-Identified values for the fields of a resource based on original value and Rule.
Introducing new rules may require modifications to the functions.
Source:
./src/main/resources/dwl/fhirTools.dwl
Functions
fun getRuleValue (rules, name)
This function takes an array of rules and name of the rule to filter the value to apply.
param
rules
an array of rules defined as a name and value object.
paramname
filters the value based on the matched name from the rules.
return The value of the corresponding rule that matched the name.
fun getDeIdentifiedCodeableConcept (data)
This function takes an array or object of CodeadbaleConcept and returns a De-Identified CodeadbaleConcept.
param
data
an array or object of CodeadbaleConcept to De-Identify.
return CodeadbaleConcept array or object that has been De-Identified.
fun getDeIdentifiedCodeableConceptFiltered (data, format)
This function takes an array of CodeadbaleConcept along with a string of of filters (system|code) csv to and returns a filtered De-Identified CodeadbaleConcept that matches the filters.
param
data
an array or object of CodeadbaleConcept to De-Identify.
paramformat
string data in format of (system|code) csv to match and retain.
return CodeadbaleConcept array that has been De-Identified.
fun getDeIdentifiedDateTime (dateTime, format)
This function takes date and time along with a string to format as De-Identified date and time.
param
dateTime
Date and Time to De-Identify.
paramformat
string format of date and time to retain.
return dateTime De-Identified date and time based on format.
fun getDeIdentifiedPostalCode (postalCode, format, zipCodeToPopulation)
This function takes postal code along with a string to format as De-Identified postal code.
param
postalCode
postal code to De-Identify.
paramformat
string format of postal code to retain.
paramzipCodeToPopulation
an array of Zip Codes and population to De-Identify Zip Codes based on rules.
return postalCode De-Identified postal code based on format.
fun getDeIdentifiedMaritalStatus (maritalStatus, format)
This function takes marital status along with a string to format as De-Identified marital status.
param
maritalStatus
marital status to De-Identify.
paramformat
string format of date and time to retain.
return maritalStatus De-Identified marital status on format.
fun getDeIdentifiedBoolean (bool, format)
This function takes a boolean along with a string to format as De-Identified boolean status.
param
bool
boolean to De-Identify.
paramformat
string format of date and time to retain.
return boolean De-Identified boolean on format.
fun filterNoImplictRulesandModifiedExtension (data)
This function takes an array of resources and ignores any resource that has implicitRules or modifiedExtension fields. It returns an array of filtered resources.
param
data
an array of resources.
return data an array of filtered resources.
fun getDeIdentifiedReferencesArray (referencesArray, resourceTypeArray, muleIds)
This function takes an array of references that needs to be De-Identified along with an array of resource types to retain and an object that holds resource IDs that are De-Identified. The De-Identified ids are used for mapping references. If there is no De-Identified id for a given resource, the reference is null and will be excluded from the list.
param
referencesArray
an array of references that are to be De-Identified.
paramresourceTypeArray
an array of resource types that are to be retained if reference exists.
parammuleIds
an object that has IDs of resources as a key and De-Identified IDs as value.
return references array that is De-Identified.
fun getDeIdentifiedReferenceObject (referenceObj, resourceTypeArray, muleIds)
This function takes a reference object that needs to be De-Identified along with an array of resource types to retain and an object that holds resource IDs that are De-Identified. The De-Identified ids are used for mapping references. If there is no De-Identified id for a given resource, the reference is not returned.
param
referencesArray
an array of references that are to be De-Identified.
paramresourceTypeArray
an array of resource types that are to be retained if reference exists.
parammuleIds
an object that has IDs of resources as a key and De-Identified IDs as value.
return references object that is De-Identified.
export-config-request-to-db-mapping
Create an Export Configuration
Source:
./src/main/resources/dwl/export-config-request-to-db-mapping.dwl
Mapping Tables
Create an Export Configuration. Mapping from API Field to DB field to create an Export Configuration
API Field | DB Field Name | Description |
---|---|---|
name | NAME | Name of the Export Configuration |
description | DESCRIPTION | Description about Export Configuration |
resources | RESOURCES | Resources that need to be exported from EHR. |
fhirGroupId | FHIR_GROUP_ID | Group Id of EHR Bulk Export |
earliestRecordDate | EARLIEST_RECORD_DATE | Data to be retrieved from EHR Bulk export APIS using sinceDate |
frequency | FREQUENCY | Frequency when the Job should run based on Export Configuration |
dayOfWeek | DAY_OF_WEEK | Day of the week when the Job should run based on Export Configuration |
weekOfMonth | WEEK_OF_MONTH | Week of the month when the Job should run based on Export Configuration |
status | STATUS | Status of the Job based on the Export Configuration. |
rules | RULES | JSON Array that hold the ruleName and rule Values for Export Configuration |
isDeleted | IS_DELETED | Soft deletion of Export Configuration |
export-config-get-response-mapping
Returns Transaction Details along with Export Configuration details for a specific Export Configuration
Source:
./src/main/resources/dwl/export-config-get-response-mapping.dwl
Mapping Tables
Returns Transaction Details along with Export Configuration details for a specific Export Configuration
API Field | DB Field Name | Description |
---|---|---|
id | ID | Unique Auto Generated Id of the Export Configuration |
name | NAME | Name of the Export Configuration |
description | DESCRIPTION | Description about Export Configuration |
resources | RESOURCES | Resources that need to be exported from EHR. |
fhirGroupId | FHIR_GROUP_ID | Group Id of EHR Bulk Export |
earliestRecordDate | EARLIEST_RECORD_DATE | Data to be retrieved from EHR Bulk export APIS using sinceDate |
lastRunEndTime | LAST_RUN | Last Successful Job Run based on Export Configuration |
frequency | FREQUENCY | Frequency when the Job should run based on Export Configuration |
dayOfWeek | DAY_OF_WEEK | Day of the week when the Job should run based on Export Configuration |
weekOfMonth | WEEK_OF_MONTH | Week of the month when the Job should run based on Export Configuration |
status | STATUS | Status of the Job based on the Export Configuration. |
rules | RULES | JSON Array that hold the ruleName and rule Values for Export Configuration |
isDeleted | IS_DELETED | Soft deletion of Export Configuration |
createdDateTime | CREATED_DATE | Date when the Export Configuration has been created |
runHistory | JOB_END | Only last 10 records with status as completed will be returned |
initialSucessfulRunTime | JOB_END | Mapped with first record that has completed status for the Export Configuration Id |