[[Pathology|Back]]

Version 2 of the i2b2 pathology data integration load will also use SSIS, but will address the following issues with version 1.

1. Address and document the correct procedure for dealing with duplicates, by getting Paul Smalley to define which items are true duplicates and may be discarded and identifying a means of distinguish between duplicates where both should be loaded into i2b2.
1. Improve the error handling and reporting.
1. Move the SSIS packages onto the BRICCS SQL Server so that we can control their running and updating.
1. Be configurable so that the source and destination databases can be changed more easily.

= Implementation

Version 2 of the i2b2 pathology data integration load is written as the SQL Server Integration Services (SSIS) packages i2b2ImportPathologyObservations.dtsx and i2b2BuildPathologyOntoloy.dtsx.  The mapping, ontology and logging data is stored in the i2b2ClinDataIntegration database on the UHLSQLBRICCSDB\UHLBRICCSDB server.

== i2b2ClinDataIntegration Database

The database contains the following tables used by the two pathology integration packages:

||= Table =||= Description =||
||  PathologyHierarchy  || The hierarchy nodes used to build the tree structure in the pathology ontology ||
||  PathologyCodes  || The relevant LOINC codes used in the pathology ontology, linked to their place in the hierarchy ||
||  PathologyMapping  || The mapping of LOINC codes to iLab codes.  Each LOINC code can be mapped to more than one iLab code, but each iLab Code can only be mapped to one LOINC code. ||
||  ETLHistory  || Log file showing when each job was last run for each destination database. ||
||  Failed_Observation_Fact  || Log file showing Observation_Fact records that could not be loaded into the Observation_Fact table, when they were loaded and what the error message was. ||