Onyx to CiviCRM

There are three options available to get BRICCS data from Onyx into CiviCRM:

1. Load from an Onyx extract.
2. Load directly from the Onyx database.
3. Load from the data already loaded into i2b2.

The 2nd option is the most desirable, if possible.

BRICCS Study Enrollment (Case) Type in CiviCRM

I have created a new briccs Drupal module to contain code specific for the BRICCS study. The module creates the BRICCS case type and all custom values specific for the study when it is enabled, and deletes them when it is disabled. The module WILL also contain the cron hook that loads the participants and their data from the Onyx database.

Data Requirements

The following data will be copied from the Onyx database to CiviCRM (the question name within Onyx is shown in brackets):

• Date and Time of Interview {Interview.start_date}
• Partiticant Number (Participant.barcode)
• Interview Status (Interview.status) Values of
  • IN_PROGRESS
  • CLOSED
  • CANCELLED
  • COMPLETED
• Title (participant_attribute_value.pat_title)
• First Name (Participant.firstName)
• Surname (Participant.lastName)
• Address
  • participant_attribute_value.pat_address1
  • participant_attribute_value.pat_address2
  • participant_attribute_value.pat_address3
  • participant_attribute_value.pat_address4
  • participant_attribute_value.pat_postcode
• Date of Birth (Participant.birthDate)
• Gender (Participant.gender)
  • FEMALE or MALE
• Participant NHS Number {participant_attribute_value.pat_nhsnumber}
• Participant UHL System Number {Participant.enrollmentId}
• Consent Values
  • Understands the request for consent (question_answer.question_name="consent_q1")
  • Consents to donate blood and urine (question_answer.question_name="consent_q2")
  • Consents to entry in the BRICCS database (question_answer.question_name="consent_q3")
  • Consents to possible further contact from BRU (question_answer.question_name="consent_q4")
  • Understands the rules for withdrawal (question_answer.question_name="consent_q5")

Additional Requirements

• All interviews will be extracted, regardless of their status.
• Since interviews can be updated if they have not been completed, the procedure will have to cope with updates as well as inserts.
• Since the cron jobs are run in a user's request, make sure that only a few participants are processed for each request.
  • It is possible to have hidden custom fields within CiviCRM by ​marking the Custom Group as inactive. This could be used to store a date last updated or a hash value to cut down the number of interviews that need to be processed every time.

Query

SELECT
      p.id AS ParticipantID
    , p.barcode AS StudyID
    , p.first_name
    , p.last_name
    , p.birth_date
    , p.gender
    , p.enrollment_id AS UhlSystemNumber
    , consent_q1.value AS consent_q1
    , consent_q1.value AS consent_q2
    , consent_q1.value AS consent_q3
    , consent_q1.value AS consent_q4
    , consent_q1.value AS consent_q5
FROM participant p
LEFT JOIN (
    SELECT
          qp.participant_id
        , ca.category_name AS value
    FROM question_answer qa
    JOIN category_answer ca ON ca.question_answer_id = qa.id
    JOIN questionnaire_participant qp ON qp.id = qa.questionnaire_participant_id
         AND qp.questionnaire_name = 'ManualConsentQuestionnaire'
    WHERE qa.question_name = 'consent_q1'
    ) consent_q1 ON consent_q1.participant_id = p.id
LEFT JOIN (
    SELECT
          qp.participant_id
        , ca.category_name AS value
    FROM question_answer qa
    JOIN category_answer ca ON ca.question_answer_id = qa.id
    JOIN questionnaire_participant qp ON qp.id = qa.questionnaire_participant_id
         AND qp.questionnaire_name = 'ManualConsentQuestionnaire'
    WHERE qa.question_name = 'consent_q2'
    ) consent_q2 ON consent_q2.participant_id = p.id
LEFT JOIN (
    SELECT
          qp.participant_id
        , ca.category_name AS value
    FROM question_answer qa
    JOIN category_answer ca ON ca.question_answer_id = qa.id
    JOIN questionnaire_participant qp ON qp.id = qa.questionnaire_participant_id
         AND qp.questionnaire_name = 'ManualConsentQuestionnaire'
    WHERE qa.question_name = 'consent_q3'
    ) consent_q3 ON consent_q3.participant_id = p.id
LEFT JOIN (
    SELECT
          qp.participant_id
        , ca.category_name AS value
    FROM question_answer qa
    JOIN category_answer ca ON ca.question_answer_id = qa.id
    JOIN questionnaire_participant qp ON qp.id = qa.questionnaire_participant_id
         AND qp.questionnaire_name = 'ManualConsentQuestionnaire'
    WHERE qa.question_name = 'consent_q4'
    ) consent_q4 ON consent_q4.participant_id = p.id
LEFT JOIN (
    SELECT
          qp.participant_id
        , ca.category_name AS value
    FROM question_answer qa
    JOIN category_answer ca ON ca.question_answer_id = qa.id
    JOIN questionnaire_participant qp ON qp.id = qa.questionnaire_participant_id
         AND qp.questionnaire_name = 'ManualConsentQuestionnaire'
    WHERE qa.question_name = 'consent_q5'
    ) consent_q5 ON consent_q5.participant_id = p.id
;

Try a different approach:

SELECT
      p.id AS ParticipantID
    , p.barcode AS StudyID
    , p.first_name
    , p.last_name
    , p.birth_date
    , p.gender
    , p.enrollment_id AS UhlSystemNumber
    , consent.q1
    , consent.q2
    , consent.q3
    , consent.q4
    , consent.q5
FROM participant p
LEFT JOIN (
    SELECT
          qp.participant_id
        , GROUP_CONCAT(if(qa.question_name = 'consent_q1', category_name, NULL)) AS q1 
        , GROUP_CONCAT(if(qa.question_name = 'consent_q1', category_name, NULL)) AS q2
        , GROUP_CONCAT(if(qa.question_name = 'consent_q1', category_name, NULL)) AS q3
        , GROUP_CONCAT(if(qa.question_name = 'consent_q1', category_name, NULL)) AS q4
        , GROUP_CONCAT(if(qa.question_name = 'consent_q1', category_name, NULL)) AS q5
    FROM question_answer qa
    JOIN category_answer ca ON ca.question_answer_id = qa.id
    JOIN questionnaire_participant qp ON qp.id = qa.questionnaire_participant_id
         AND qp.questionnaire_name = 'ManualConsentQuestionnaire'
    GROUP BY qp.participant_id
    ) consent ON consent.participant_id = p.id
;