Changes between Version 15 and Version 16 of LEGACY - i2b2 - The algorithms applied to importing a PDO

Timestamp:

12/29/11 20:59:37 (13 years ago)

Author:

jeff.lusted

Comment:

--

LEGACY - i2b2 - The algorithms applied to importing a PDO

@@ -1 +1 @@
 == Importing a PDO - the algorithms examined in detail ==
 
-This is work in progress.
+This is work in progress based upon i2b2 version 1.5.5
 
 The context of this discussion is what happens to a PDO file when you ask the CRC loader to process the file using one of i2b2's web services. There are so many things happening here that throwing some light on the detail is important to understanding the process, and - by reflection - on how to format a PDO suitable for importing in the first place.
@@ -79 +79 @@
 
 One thing to note in the above is the relative independence of the concept_set: it depends upon nothing, and nothing depends upon it. What role does the concept_set play - or can play - in a PDO upload is an interesting question. From the dependencies above, it seems you could use the upload facility just to upload concepts to the concept dimension: a PDO with nothing but concepts in it! 
+
+=== Basic Algorithm ===
+
+Each set has the same overall algorithm applied to it, covered by five separate transactions:
+
+ 1. Set up a status message ("PROCESSING").
+ 1. Create a temporary table.
+ 1. Load the temporary table with data from the relevant PDO set.
+ 1. Merge the temporary table with the real table.
+ 1. Issue a status message ("FINISHED" or "WARNING")
+
+Remember that each of the above is a separate transaction. Each step cannot be undone once committed, although presumably step/txn 4 is the most critical.
+If anything goes wrong and a transaction has begun but has not been committed, then it is rolled back.
+