Configuring Logs

Karnak offers the possibility of injecting your own log configuration.

The logging system used by Karnak is logback. For more details on logback, it provides a manual

Variables

Some variables are available and can be used in the logback file.

• issuerOfPatientID are the issuer of patient ID before the deidentification
• PatientID are the patient ID before the deidentification
• SOPInstanceUID are the SOPInstanceUID before the deidentification
• DeidentifySOPInstanceUID are the SOPInstanceUID after the deidentification
• SeriesInstanceUID are the SeriesInstanceUID before the deidentification
• DeidentifySeriesInstanceUID are the SeriesInstanceUID after the deidentification
• ProjectName are the project name used for the deidentification
• ProfileName are the profile name used for the deidentification
• ProfileCodenames are a list of concatenated profile items that has been used for the deidentification

Inject the logback configuration file

By using docker

You must set the environment variable LOGBACK_CONFIGURATION_FILE with the path of Logback file configuration, it will override the default log file.

For example, if you create your own configuration with the file name my-logback.xml at the root of the docker-compose.yml.

You must create a volume that will copy the file in the Karnak docker and define the environment variable LOGBACK_CONFIGURATION_FILE with the path of the file in the docker container.

services:
  karnak:
    container_name: karnak
    image: osirixfoundation/karnak:v0.9.7
    volumes:
      - ./my-logback.yml:/logs/my-logback.xml
    environment:
      LOGBACK_CONFIGURATION_FILE: /logs/my-logback.xml

By using JAVA

If you use directly the Karnak jar, you must add the following parameter at the startup:

-Dlogging.config=my-logback.xml

Default logback file

The default logback file (see below) has two modes.

Dev Mode

• The variable ENVIRONMENT must be set to DEV to be activate.
• Log everything at the INFO level except for the package org.karnak and org.weasis, they are at the DEBUG level.

Production Mode

• Always activate, except if the variable ENVIRONMENT is set to DEV.
• Logs everything at the WARN level
• Writes all.log file, it contains:
  • Every WARN level log
  • org.weasis INFO level log
  • org.karnak INFO level log, excepted clinical log
• Writes clinical.log, it contains:
  • Every log with the marker CLINICAL which concerns the deidentification

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <property name="LOGS" value="logs" />
    <!--
    DEV LOGS
    Condition to verify if "ENVIRONMENT" is "DEV"
    -->
    <if condition='property("ENVIRONMENT").contains("DEV")'>
        <then>
            <!--
            How the logs will be displayed in the standard output (console)
            SOPInstanceUID, issuerOfPatientID and PatientID are variables to associate the current DICOM with the log
            -->
            <appender name="DEV_OUT" class="ch.qos.logback.core.ConsoleAppender">
                <encoder>
                    <Pattern>%black(%d{ISO8601})  %highlight(%-5level) %marker %highlight(%X{SOPInstanceUID}) %highlight(%X{issuerOfPatientID}) %highlight(%X{PatientID}) [%yellow(%t)] %yellow(%C{1.}): %msg%n%throwable</Pattern>
                </encoder>
            </appender>
            <!--
            Log everything at the INFO level except for the package org.karnak and org.weasis, they are at the DEBUG level.
            -->
            <root level="info">
                <appender-ref ref="DEV_OUT" />
            </root>
            <logger name="org.weasis" level="debug" />
            <logger name="org.karnak" level="debug" />
        </then>
        <!--
        PRODUCTION LOGS
        -->
        <else>
            <!--
            How the warning logs will be displayed in the standard output (console)
            SOPInstanceUID, issuerOfPatientID and PatientID are variables to associate the current DICOM with the log
            -->
            <appender name="WARNING_OUT" class="ch.qos.logback.core.ConsoleAppender">
                <!--
                Log only at WARN level
                -->
                <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
                    <level>WARN</level>
                </filter>
                <encoder>
                    <Pattern>%black(%d{ISO8601})  %highlight(%-5level) %marker %highlight(%X{SOPInstanceUID}) %highlight(%X{issuerOfPatientID}) %highlight(%X{PatientID}) [%yellow(%t)] %yellow(%C{1.}): %msg%n%throwable </Pattern>
                </encoder>
            </appender>
            <!--
            Write all.log file in the file system.
            -->
            <appender name="ALL_LOGS" class="ch.qos.logback.core.rolling.RollingFileAppender">
                <file>${LOGS}/all/all.log</file>
                <!--
                Window rolling policy defined by the variable KARNAK_LOGS_MIN_INDEX or KARNAK_LOGS_MAX_INDEX
                By default the min is 1 and the max is 10
                -->
                <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
                    <fileNamePattern>${LOGS}/all/all_%i.log</fileNamePattern>
                    <minIndex>${KARNAK_LOGS_MIN_INDEX:-1}</minIndex>
                    <maxIndex>${KARNAK_LOGS_MAX_INDEX:-10}</maxIndex>
                </rollingPolicy>
                <!--
                Size rolling policy defined by the variable KARNAK_LOGS_MAX_FILE_SIZE
                By default the max file size is 50MB
                -->
                <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
                    <maxFileSize>${KARNAK_LOGS_MAX_FILE_SIZE:-50MB}</maxFileSize>
                </triggeringPolicy>
                <!--
                Filter not to write logs with the clinical marker
                -->
                <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
                    <evaluator class="ch.qos.logback.classic.boolex.OnMarkerEvaluator">
                        <marker>CLINICAL</marker>
                    </evaluator>
                    <onMismatch>NEUTRAL</onMismatch>
                    <onMatch>DENY</onMatch>
                </filter>
                <encoder>
                    <pattern>%d %-5level %m%n</pattern>
                </encoder>
            </appender>
            <!--
            Write clinical.log file in the file system.
            Will contains only the logs with the CLINICAL marker. The logs with CLINICAL marker concerns the information about the deidentification.
            Variables used for the CLINICAL log:
            SOPInstanceUID, DeidentifySOPInstanceUID, SeriesInstanceUID, DeidentifySeriesInstanceUID, ProjectName, ProfileName, ProfileCodenames
			-->
            <appender name="CLINICAL_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
                <file>${LOGS}/Clinical/clinical.log</file>
                <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
                    <fileNamePattern>${LOGS}/Clinical/clinical_%i.log</fileNamePattern>
                    <minIndex>${KARNAK_CLINICAL_LOGS_MIN_INDEX:-1}</minIndex>
                    <maxIndex>${KARNAK_CLINICAL_LOGS_MAX_INDEX:-10}</maxIndex>
                </rollingPolicy>

                <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
                    <maxFileSize>${KARNAK_CLINICAL_LOGS_MAX_FILE_SIZE:-50MB}</maxFileSize>
                </triggeringPolicy>

                <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
                    <evaluator class="ch.qos.logback.classic.boolex.OnMarkerEvaluator">
                        <marker>CLINICAL</marker>
                    </evaluator>
                    <onMismatch>DENY</onMismatch>
                    <onMatch>NEUTRAL</onMatch>
                </filter>
                <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
                    <Pattern>%d SOPInstanceUID_OLD=%X{SOPInstanceUID} SOPInstanceUID_NEW=%X{DeidentifySOPInstanceUID} SeriesInstanceUID_OLD=%X{SeriesInstanceUID} SeriesInstanceUID_NEW=%X{DeidentifySeriesInstanceUID} ProjectName=%X{ProjectName} ProfileName=%X{ProfileName} ProfileCodenames=%X{ProfileCodenames}</Pattern>
                </encoder>
            </appender>

            <!--
            Log everything at the WARN level except for the package org.karnak and org.weasis, they are at the INFO level.
            The INFO logs won't appear in the standard output, because they will be filtered by the WARNING_OUT appender
            -->
            <root level="warn">
                <appender-ref ref="ALL_LOGS" />
                <appender-ref ref="CLINICAL_FILE" />
                <appender-ref ref="WARNING_OUT" />
            </root>
            <logger name="org.weasis" level="info" />
            <logger name="org.karnak" level="info" />
        </else>
    </if>
</configuration>

Karnak as service

Create docker-compose service

Example of systemd service configuration with a docker-compose.yml file in the folder /opt/karnak (If it’s another directory you have to adapt the script).

By default, Docker needs root privileges to be executed.

Manage Docker as a non-root user

For more details, the following commands are inspired by the official Docker documentation.

1. Create the docker group.

    sudo groupadd docker

2. Add your user to the dockergroup.

   sudo usermod -aG docker $USER

3. Activate the changes to groups.

   newgrp docker

4. Verify that you can run docker commands without sudo.

   docker run hello-world

Specify User in the service

In the [Service] section of the karnak.service (see below), it’s possible to specify the user who will run the service.

User=root

Create the service

Instructions:

• Go to /etc/systemd/system
• Create the file ( eg: $ sudo touch karnak.service )
• Copy and paste the config below (eg: $ sudo nano karnak.service):

# /etc/systemd/system/karnak.service 

#########################
#    KARNAK             #
#    SERVICE            #	
##########################

[Unit]
Description=Docker Compose KARNAK Service
Requires=docker.service
After=docker.service network.target

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/opt/karnak
ExecStart=/usr/local/bin/docker compose up -d
ExecStop=/usr/local/bin/docker compose down
TimeoutStartSec=0

[Install]
WantedBy=multi-user.target

Test the service:

• $ systemctl start karnak.service
• $ systemctl status karnak.service
• $ systemctl enable karnak.service

Profile Structure

In the Karnak user interface, the Profiles page can be accessed using the menu bar on the left. It displays the list of existing profiles and offers to import new profiles.

A profile file is one or a list of profile elements that are defined for a group of DICOM attributes and with a particular action. During de-identification or tag morphing, Karnak will apply the profile elements to the applicable DICOM attributes. Only one profile element can be applied to a DICOM attribute. The profile elements are applied in the order defined in the yaml file and, therefore, the first applicable profile element will modify the value of a DICOM attribute. If other profile elements were applicable to that specific tag, they won’t be applied since it has already been modified.

Currently, the profile must be a yaml file (MIME-TYPE: application/x-yaml) and respect the definition as below.

Profile metadata

All these metadata are optional, but for a better user experience we recommend defining at least the name and the version. They will be used to identify and select your profile in a Project.

• name - The name of your profile

• version - The version of your profile

• minimumKarnakVersion - The version of Karnak when the profile has been imported

• defaultIssuerOfPatientID - Default value in case the IssuerOfPatientID value is not available in DICOM file, it is used to build the patient’s pseudonym when applying de-identification

• profileElements - The list of profile elements, the elements are applied accordingly to their position in the list

Profile element

A profile element is defined as below in the yaml file.

• name - The name of your profile element

• codename - The codename represents the type of profile element. The available types of profiles elements and their codename are described in details in this documentation

• condition - A boolean condition that defines some requirements to apply this profile element

• action - The type of action that will be applied

• option - Required for certain types of profile elements, contains a single value

• arguments - Required for certain types of profile elements, contains a list of key-value pairs

• tags - List of tags or pattern that identifies the DICOM attributes this profile should be applied to

• excludedTags - List of tags or pattern that identifies the DICOM attributes this profile should not be applied to. These attributes can then be modified by another profile element if applicable

Tag

DICOM Tags can be defined in different formats: (0010,0010); 0010,0010; 00100010;

A tag pattern represent a group of tags and can be defined as follows: e.g. (0010,XXXX) represent all the tags of group 0010. The pattern (XXXX,XXXX) targets all the DICOM attributes.

Condition

A condition can be added to any type of profile element. It contains an expression that will be evaluated for each tag the profile element is applied to.

The syntax and usage of these conditions is detailed in the Conditions page.

Validation

The content of the yaml file is validated upon import. If the structure or parameters are not defined correctly, detailed errors will be displayed to the user.

Please refer to the Profiles page for more information.

Basic Dicom Profile

The Basic DICOM Profile is defined by DICOM to remove all the attributes that could contain Individually Identifying Information (III) about the patient or other individuals or organizations associated with the data. The details of this profile element can be found in the DICOM Standard.

Further details on this profile element and its implementation in Karnak can be found in the How does de-identification work? page.

We strongly recommend including this profile as basis for de-identification.

This profile element can be included in the profile definition by referencing its codename:

- name: "DICOM basic profile"
  codename: "basic.dicom.profile"

Example of a complete and valid profile yaml file that applies only the Basic DICOM Profile:

name: "Dicom Basic Profile"
version: "1.0"
minimumKarnakVersion: "0.9.2"
defaultIssuerOfPatientID:
profileElements:
  - name: "DICOM basic profile"
    codename: "basic.dicom.profile"

Actions on tags

Actions on specific tags

This profile element applies an action on a tag or a group of tags defined by the user. Its codename is action.on.specific.tags.

This profile element requires the following parameters:

• name: description of the action applied
• codename: action.on.specific.tags
• action: Remove (X) / Keep (K)
• tags: list of tags the action should be applied to

This profile element can have these optional parameters:

• condition: optional, defines a condition to evaluate if this profile element should be applied to this DICOM instance
• excludedTags: list of tags that will be ignored by this action

In this example, all the tags starting with 0028 will be removed excepted (0028,1199) which will be kept.

- name: "Remove tags"
  codename: "action.on.specific.tags"
  action: "X"
  tags:
    - "(0028,xxxx)"
  excludedTags:
    - "(0028,1199)"

- name: "Keep tags 0028,1199"
  codename: "action.on.specific.tags"
  action: "K"
  tags:
    - "0028,1199"

Actions on private tags

This profile element applies an action on a private tag or a group of private tags defined by the user. This action won’t be applied in case the tag is not private.

Its codename is action.on.privatetags.

This profile element requires the following parameters:

• name: description of the action applied
• codename: action.on.privatetags
• action: Remove (X) / Keep (K)

This profile can have these optional parameters:

• condition: optional, defines a condition to evaluate if this profile element should be applied to this DICOM instance
• tags: list of tags the action should be applied to. If not specified, the action is applied to all private tags
• excludedTags: list of tags that will be ignored by this action

In this example, all tags starting with 0009 will be kept and all the other private tags will be removed.

- name: "Keep private tags starting with 0009"
  codename: "action.on.privatetags"
  action: "K"
  tags:
    - "(0009,xxxx)"

- name: "Remove all private tags"
  codename: "action.on.privatetags"
  action: "X"

Add new tags

This profile element adds a tag if it is not already present in the instance. This action will be ignored if the tag is already present in the instance.

Its codename is action.add.tag.

This profile element requires the following parameters:

• name: description of the action applied
• codename: action.add.tag
• arguments:
  • value: required value to set the tag’s value to
• tags: must contain exactly one tag, the one to add

This profile can have these optional parameters:

• condition: optional, defines a condition to evaluate if this profile element should be applied to this DICOM instance

Regarding the application of profile elements, if the tag is not initially present in the instance, then it will be added by this action. Further actions that match this tag won’t be applied. If the tag was already present in the instance, then the add action is ignored and a further action can be applied to that tag. It is only possible to add tags at the root level of the DICOM instance. Adding elements in a sequence is not supported.

The attribute must exist in the DICOM Standard, otherwise the profile validation will fail and cannot be added. The VR is retrieved from the Standard as well. If this profile element is applied on a SOP class that does not contain this tag, it won’t be added to prevent corrupting the DICOM instance, and a warning will be generated in the logs

This feature is especially useful when applying masks to non-compliant SOPs by using the attribute Burned In Annotation. Please refer to the Cleaning Data Pixel Exceptions page for an exhaustive example.

In this example, we add the optional tag Recognizable Visual Features (0028,0302).

- name: "Add Recognizable Visual Features tag"
  codename: "action.add.tag"
  arguments:
    value: "YES"
    vr: "CS"
  tags:
    - "(0028,0302)"

Add new private tags

This profile element adds a private tag if it is not already present in the instance. This action will be ignored if the tag is already present in the instance or if there is a collision with a different Private Creator ID.

Its codename is action.add.private.tag.

This profile element requires the following parameters:

• name: description of the action applied
• codename: action.add.private.tag
• arguments:
  • value: required value to set the tag’s value to
  • vr: VR of the tag
  • privateCreator: optional value of the PrivateCreatorID, see below
• tags: must contain exactly one tag, the one to add

This profile can have these optional parameters:

• condition: optional, defines a condition to evaluate if this profile element should be applied to this DICOM instance

PrivateCreator value

The privateCreator argument is optional. It is recommended to specify it for coherence purposes.

In case the PrivateCreatorID tag does not exist for the private tag to be added, the privateCreator argument must be specified, and it will be created at the same time.

In case the PrivateCreatorID tag exists, if the privateCreator argument is not specified, the tag will be added as a part of it. It the privateCreator argument is specified, the existing PrivateCreatorID tag value will be matched against the privateCreator argument. If it matches, the new tag is added. If it doesn’t, there is a PrivateCreatorID collision and the tag won’t be added to prevent incoherent data in the DICOM instance. A warning will be generated in the logs.

The other specificities of the Add action are the same as above, for non-private tags.

The details about the management of tags and element numbers in private tags are detailed in the DICOM Standard.

In this example, we add a private tag containing the value “sample-project” linked to the PrivateCreatorID “KARNAK-PRIVATE”. The PrivateCreatorID tag (0057,0010) is created with the value “KARNAK-PRIVATE” if it is not present in the DICOM instance. If it exists already and contains the value “KARNAK-PRIVATE”, the (0057,1000) tag will be created.

If the PrivateCreatorID tag (0057,0010) exists already but contains a value different from “KARNAK-PRIVATE”, the tag (0057,1000) won’t be added since it would result in linking it to the wrong PrivateCreatorID.

- name: "Add Private Tag"
  codename: "action.add.private.tag"
  arguments:
    value: "sample-project"
    vr: "LO"
    privateCreator: "KARNAK-PRIVATE"
  tags:
    - "(0057,1000)"

Example of a complete and valid profile yaml file that applies the following profile elements in order:

• Remove the tags that match (0008,00XX) and (0010,00XX) except for (0008,0008) and (0008,0013)
• Keep specifically the previously excluded tags (so that they won’t be removed by a profile element applied afterward)
• Remove all the private tags
• Apply the Basic DICOM Profile to all the tags that did not undergo a modification previously

name: "De-identification profile"
version: "1.0"
minimumKarnakVersion: "0.9.2"
defaultIssuerOfPatientID:
profileElements:
  - name: "Remove tags"
    codename: "action.on.specific.tags"
    action: "X"
    tags:
      - "(0008,00XX)"
      - "0010,00XX"
    excludedTags:
      - "0008,0008"
      - "0008,0013"

  - name: "Keep tags"
    codename: "action.on.specific.tags"
    action: "K"
    tags:
      - "0008,0008"
      - "0008,0013"

  - name: "Remove all private tags"
    codename: "action.on.privatetags"
    action: "X"

  - name: "DICOM basic profile"
    codename: "basic.dicom.profile"

This example keeps a Study Description tag if that as contain ‘R2D2’ value, keep the Philips PET private group and apply the basic DICOM profile.

The tag patterns (0073,xx00) and (7053,xx09) are defined in Philips PET Private Group by DICOM.

name: "Profile Example"
version: "1.0"
minimumKarnakVersion: "0.9.7"
profileElements:
  - name: "Keep StudyDescription tag according to a condition"
    codename: "action.on.specific.tags"
    condition: "tagValueContains(#Tag.StudyDescription, 'R2D2')"
    action: "K"
    tags:
      - "(0008,1030)"

  - name: "Keep Philips PET Private Group"
    codename: "action.on.privatetags"
    action: "K"
    tags:
      - "(7053,xx00)"
      - "(7053,xx09)"

  - name: "DICOM basic profile"
    codename: "basic.dicom.profile"

Actions on dates

This profile element applies a specific action on a tag or a group of tags containing a date. This action can only be applied to the following Value Representations: Age String (AS), Date (DA), Date Time (DT) and Time (TM).

This profile element requires the following parameters:

• name: description of the action applied
• codename: action.on.dates
• option: the action to be applied, detailed below
• arguments: additional parameters depending on the chosen option

This profile can have these optional parameters:

• condition: optional, defines a condition to evaluate if this profile element should be applied to this DICOM instance
• tags: list of tags the action should be applied to. If not specified, the profile element will be applied to all the tags that have a AS, DA, DT or TM VR in the instance
• excludedTags: list of tags that will be ignored by this action

The parameter option can have one of the following values:

• shift
• shift_range
• shift_by_tag
• date_format

Shift option

The shift option applies a shift to a date according to the following required arguments:

• seconds: integer representing the number of seconds the shift operation should apply
• days: integer representing the number of days the shift operation should apply

In the case of a shift action applied to an Age String (AS) Value Representation, the seconds and days will be added to the existing value.

In case of a shift action applied to a Date (DA), Date Time (DT) or Time (TM) Value Representation, the seconds and days will be subtracted to the existing value.

In this example, all the tags starting with 0010 and that have a AS, DA, DT or TM VR will be shifted by 30 seconds and 10 days.

- name: "Shift Date"
  codename: "action.on.dates"
  arguments:
    seconds: 30
    days: 10
  option: "shift"
  tags:
    - "0010,XXXX"

Shift Range option

The shift range option applies a random shift to a date parametrized by ranges defined by the user as follows:

• max_seconds (required): integer representing the upper bound of the number of seconds the shift operation should apply
• max_days (required): integer representing the upper bound of the number of days the shift operation should apply
• min_seconds (optional): integer representing the lower bound of the number of seconds the shift operation should apply, the default value is 0 is not specified
• min_days (optional): integer representing the lower bound of the number of days the shift operation should apply, the default value is 0 is not specified

The random operation is deterministic and reproducible based on the patient and the project. For more details about the usage of randomization in Karnak, please refer to the Karnak Project.

In this example, all the tags starting with 0008,002 and that have a AS, DA, DT or TM VR will be shifted randomly in a range of 0 to 60 seconds and 50 to 100 days.

  - name: "Shift Range Date"
    codename: "action.on.dates"
    arguments:
      max_seconds: 60
      min_days: 50
      max_days: 100
    option: "shift_range"
    tags:
      - "0008,002X"

Date Format option

The date format option applies a partial deletion to a date depending on the specified option:

• day: this option will delete the day information and replace it with the first day of the month
• month_day: this option will delete the day and month information and replace it with the first month and first day of the month

This action can only be applied to the following Value Representations: Date (DA) and Date Time (DT).

day example

In this example, all the tags starting with 0008,003 and that have a DA or DT VR will have the day information removed.

  - name: "Date format"
    codename: "action.on.dates"
    arguments:
      remove: "day"
    option: "date_format"
    tags:
      - "0008,003X"

For example, if the value contained in the tag is 20230512, the output value will be 20230501.

month_day example

In this example, all the tags starting with 0008,003 and that have a DA or DT VR will have the day and month information removed.

  - name: "Date format"
    codename: "action.on.dates"
    arguments:
      remove: "month_day"
    option: "date_format"
    tags:
      - "0008,003X"

For example, if the value contained in the tag is 20230512, the output value will be 20230101.

Shift By Tag option

The shift by tag option applies a shift to a date according to a value contained in a specified DICOM tag. The arguments are detailed below:

• seconds_tag: tag that contains the number of seconds the shift operation should apply
• days_tag: tag that contains the number of days the shift operation should apply

Both arguments are not required, but at least one of them must be specified in order to apply the action.

In this example, all the tags starting with 0010 and that have a AS, DA, DT or TM VR will be shifted by the number of days stored in the private tag (0015,0011).

- name: "Shift Date By Tag"
  codename: "action.on.dates"
  arguments:
    days_tag: "(0015,0011)"
  option: "shift_by_tag"
  tags:
    - "0010,XXXX"

Example of a complete and valid profile yaml file that applies the following profile elements in order:

• Shift the tags that match (0008,003X) and (0008,0012) except for (0008,0030) and (0008,0032) and that have a AS, DA, DT or TM VR will be shifted randomly in a range of 0 to 60 seconds and 10 to 50 days.
• Remove the day and month information contained in the tags (0008,0023) and (0008,0021) and that have a DA or DT VR
• Shift the tags that match (0010,XXXX) except for (0010,0010) and that have a AS, DA, DT or TM VR will be shifted by 30 seconds and 10 days.
• Apply the Basic DICOM Profile to all the tags that did not undergo a modification previously

name: "De-identification profile"
version: "1.0"
minimumKarnakVersion: "0.9.2"
defaultIssuerOfPatientID:
profileElements:
  - name: "Shift Range Date with arguments"
    codename: "action.on.dates"
    arguments:
      max_seconds: 60
      min_days: 10
      max_days: 50
    option: "shift_range"
    tags:
      - "0008,0012"
      - "0008,003X"
    excludedTags:
      - "0008,0030"
      - "0008,0032"
      
  - name: "Date Format"
    codename: "action.on.dates"
    arguments:
      remove: "month_day"
    option: "format_date"
    tags:
      - "0008,0023"
      - "0008,0021"

  - name: "Shift Date with arguments"
    codename: "action.on.dates"
    arguments:
      seconds: 30
      days: 10
    option: "shift"
    tags:
      - "0010,XXXX"
    excludedTags:
      - "0010,1010"

  - name: "DICOM basic profile"
    codename: "basic.dicom.profile"

Expressions

Actions on specific tags

This profile element applies an expression to a tag or a group of tags defined by the user. An expression is based on Spring Expression Language (SpEL) and returns a value or an action according to a certain condition. Its codename is expression.on.tags.

This profile element requires the following parameters:

• name: description of the action applied
• codename: expression.on.tags
• arguments: definition of the expression
• tags: list of tags the action should be applied to

This profile can have these optional parameters:

• condition: optional, defines a condition to evaluate if this profile element should be applied to this DICOM instance
• excludedTags: list of tags that will be ignored by this action

The expression is contained in the expr argument. The expression will be executed and can either return an action that will be executed, or a null value that will do nothing and move on the next profile element.

Some custom variables are defined in the context of the expression, they contain information relative to the current attribute the profile element is being applied to:

• tag contains the current attribute tag, for example (0010,0010)
• vr contains the current attribute VR, for example PN
• stringValue contains the current element value, for example ‘John^Doe’

Some constants are also defined to improve readability and reduce errors.

• #Tag is a constant that can be used to retrieve any tag integer value in the DICOM standard. For example, #Tag.PatientBirthDate corresponds to (0010,0030).
• #VR is a constant that can be used to retrieve any VR value in the DICOM standard. For example, #VR.LO corresponds to the Long String type.

Some utility functions are defined to work with the tags:

• getString(int tag) returns the value of the given tag in the DICOM instance, returns null if the tag is not present

• tagIsPresent(int tag) returns true if the tag is present in the DICOM instance, false otherwise

The actions are defined as functions, they are used to set the action and its parameters for the profile element.

The possible actions are:

• ReplaceNull(): Sets to empty the current tag value
• Replace(String dummyValue): Replaces the current tag value
• Remove(): Removes the tag
• Keep(): Keeps the tag unchanged
• UID(): Replaces the current tag value with a newly generated UID and sets the tag’s VR to UI
• Add(int tagToAdd, int vr, String value): Adds a tag to the DICOM instance
• ComputePatientAge(): Replaces the current tag value with a computed value of the patient’s age at the time of the exam

Examples

If the current tag corresponds to the Patient Name attribute and its value is ‘John’, its content will be replaced by the value of the Institution Name attribute, otherwise the tag is kept unchanged. This example is applied to all the attributes of the DICOM instance, and returns either a Keep or Replace action for every tag, implying that the following profile elements, if any, will be ignored since all the attributes already had an action applied to them.

  - name: "Expression"
    codename: "expression.on.tags"
    arguments:
      expr: "stringValue == 'John' and tag == #Tag.PatientName? Replace(getString(#Tag.InstitutionName)) : Keep()"
    tags: 
      - "(xxxx,xxxx)"

If the current tag value is UNDEFINED, it will be kept as is (and it will implicitly block any further potential modifications applied by other profile elements), otherwise the tag is removed. This expression is applied to 2 tags, (0010,0010) and (0010,0212).

  - name: "Expression"
    codename: "expression.on.tags"
    arguments:
      expr: "stringValue == 'UNDEFINED'? Keep() : Remove()"
    tags: 
      - "(0010,0010)" #PatientName
      - "(0010,0212)" #StrainDescription

Replacement of the Study Description tag value with a concatenation of the Institution Name and the Station Name of the DICOM instance.

- name: "Expression"
  codename: "expression.on.tags"
  arguments:
    expr: "Replace(getString(#Tag.InstitutionName) + '-' + getString(#Tag.StationName))"
  tags: 
    - "(0008,1030)" #StudyDescription

Computation of the patient’s age at the time of the exam.

- name: "Expression"
  codename: "expression.on.tags"
  arguments:
    expr: "ComputePatientAge()"
  tags: 
    - "(0010,1010)" #PatientAge

Image modifications

Defacing

This profile applies defacing to the image data of the DICOM instance.

This profile can only be applied to images in the Axial orientation of the following SOP:

• 1.2.840.10008.5.1.4.1.1.2 - CT Image Storage
• 1.2.840.10008.5.1.4.1.1.2 - Enhanced CT Image Storage

This profile element requires the following parameters:

• name: description of the action applied
• codename: clean.recognizable.visual.features
• condition: optional, defines a condition to evaluate if this profile element should be applied to this DICOM instance

Pixel Data Cleaning

This profile applies a mask defined by the user on the DICOM instance pixel data to remove identifying information burned in the image.

The details on the masks definition can be found below.

This profile is applied only on the following SOP:

• 1.2.840.10008.5.1.4.1.1.6.1 - Ultrasound Image Storage
• 1.2.840.10008.5.1.4.1.1.7.1 - Multiframe Single Bit Secondary Capture Image Storage
• 1.2.840.10008.5.1.4.1.1.7.2 - Multiframe Grayscale Byte Secondary Capture Image Storage
• 1.2.840.10008.5.1.4.1.1.7.3 - Multiframe Grayscale Word Secondary Capture Image Storage
• 1.2.840.10008.5.1.4.1.1.7.4 - Multiframe True Color Secondary Capture Image Storage
• 1.2.840.10008.5.1.4.1.1.3.1 - Ultrasound Multiframe Image Storage
• 1.2.840.10008.5.1.4.1.1.77.1.1 - VL Endoscopic Image Storage

Or if the tag value Burned In Annotation (0028,0301) is “YES”

This profile element requires the following parameters:

• name: description of the action applied
• codename: clean.pixel.data
• condition: optional, defines a condition to evaluate if this profile element should be applied to this DICOM instance

The condition parameter can be used to exclude the images coming from a specific machine for example.

profileElements:
  - name: "Clean pixel data"
    codename: "clean.pixel.data"
    condition: "!tagValueContains(#Tag.StationName,'ICT256')"

Masks Definition

The mask definition requires the following parameters:

• stationName: source station name that is matched against the attribute Station Name in the DICOM instance. It allows the mask to be specific depending on the station that generated the image. The value can also be set to * to match any station.
• color: color of the mask in hexadecimal
• rectangles: defines the list of rectangles to apply to mask identifying information

The mask definition can have these optional parameters:

• imageWidth: mask specific to an image of a given width in pixels, this value will be matched against the value of the Columns attribute in the DICOM instance
• imageHeight: mask specific to an image of a given height in pixels, this value will be matched against the value of the Rows attribute in the DICOM instance.

The selection of the mask based on the image size requires both attributes to be set, height and width. The definition of the width or height solely is not supported.

A rectangle is defined by the following required parameters:

• x: x coordinate of the upper left corner of the rectangle
• y: y coordinate of the upper left corner of the rectangle
• width: width of the rectangle
• height: height of the rectangle

The upper left corner of the image corresponds to the coordinates (0,0).

The schema below illustrate the definition of a rectangle having the following parameters (25, 75, 150, 50).

The example below shows how to define a default mask (stationName: *), a mask specific to the R2D2 station and a more specific mask applied only depending on the image size.

Depending on the instance image size and station name, the following actions will be performed:

• The instance Rows, Columns and Station Name attributes are retrieved.
• These values are matched against the masks defined in the masks list. If an exact match is found using the imageWidth, imageHeight and stationName values, this mask is used for the cleaning pixel action. In this example, if the instance contains the value 1024 in the Rows and Columns attribute and “R2D2” in the station name attribute, the third mask will be selected.
• If no match is found, the image size attributes are removed and a match is performed only on the station name attribute. In this example, if the instance contains “R2D2” in the Station Name attribute, the second mask will be selected without any regards for the image size of the instance.
• If no match is found, the default mask will be selected, here the first one.

masks:
  - stationName: "*"
    color: "ffff00"
    rectangles:
      - "25 75 150 50"
  - stationName: "R2D2"
    color: "00ff00"
    rectangles:
      - "25 25 150 50"
      - "350 15 150 50"
  - stationName: "R2D2"
    imageWidth: 1024
    imageHeight: 1024
    color: "00ffff"
    rectangles:
      - "50 25 100 100"

Pixel Data Cleaning Exceptions

In some cases, often based on the manufacturer and equipment, pixel data can contain embedded identifying information. Below is an exhaustive example illustrating how to apply cleaning pixel profile element depending on the station that produced the image, applicable to any DICOM modality.

name: "Clean pixel data"
version: "1.0"
minimumKarnakVersion: "0.9.2"
defaultIssuerOfPatientID:
profileElements:
  - name: "Add tag BurnedInAnnotation if does not exist"
    codename: "action.add.tag"
    condition: "tagValueContains(#Tag.StationName, 'ICT256') && !tagIsPresent(#Tag.BurnedInAnnotation)"
    arguments:
      value: "YES"
      vr: "CS"
    tags:
      - "(0028,0301)"

  - name: "Set BurnedInAnnotation to YES"
    codename: "expression.on.tags"
    condition: "tagValueContains(#Tag.StationName, 'ICT256')"
    arguments:
      expr: "Replace('YES')"
    tags:
      - "(0028,0301)"

  - name: "Clean pixel data"
    codename: "clean.pixel.data"

  - name: "DICOM basic profile"
    codename: "basic.dicom.profile"

masks:
  - stationName: "*"
    color: "ffff00"
    rectangles:
      - "25 75 150 50"
  - stationName: "ICT256"
    color: "00ff00"
    rectangles:
      - "25 25 150 50"
      - "350 15 150 50"

Conditions

A condition is an expression evaluated in a certain context and that returns a boolean value (true or false).

Some constants are also defined to improve readability and reduce errors.

• #Tag is a constant that can be used to retrieve any tag integer value in the DICOM standard. For example, #Tag.PatientBirthDate corresponds to (0010,0030).
• #VR is a constant that can be used to retrieve any VR value in the DICOM standard. For example, #VR.LO corresponds to the Long String type.

Utility functions are available to define the conditions and are detailed below.

tagValueIsPresent(int tag, String value) or tagValueIsPresent(String tag, String value)

This function will retrieve the tag value of the DICOM and check if the value parameter is the same as the tag value.

# Check if the study description is equals to "755523-st222-GE"
tagValueIsPresent(#Tag.StudyDescription, "755523-st222-GE")
tagValueIsPresent("0008,1030", "755523-st222-GE")

# Check if the study description is not equals to "755523-st222-GE"
!tagValueIsPresent(#Tag.StudyDescription, "755523-st222-GE")
!tagValueIsPresent("0008,1030", "755523-st222-GE")

tagValueContains(int tag, String value) or tagValueContains(String tag, String value)

This function will retrieve the tag value of the DICOM and check if the value parameter appears in the tag value.

# Check if the study description contains "st222"
tagValueContains(#Tag.StudyDescription, "st222")
tagValueContains("0008,1030", "st222")

# Check if the study description does not contain "st222"
!tagValueContains(#Tag.StudyDescription, "st222")
!tagValueContains("0008,1030", "st222")

tagValueBeginsWith(int tag, String value) or tagValueBeginsWith(String tag, String value)

This function will retrieve the tag value of the DICOM and check if the tag value begins with the parameter value

# Check if the study description begins with "755523"
tagValueBeginsWith(#Tag.StudyDescription, "755523")
tagValueBeginsWith("0008,1030", "755523")

# Check if the study description does not begin with "755523"
!tagValueBeginsWith(#Tag.StudyDescription, "755523")
!tagValueBeginsWith("0008,1030", "755523")

tagValueEndsWith(int tag, String value) or tagValueEndsWith(String tag, String value)

This function will retrieve the tag value of the DICOM and check if the tag value ends with the parameter value

# Check if the study description ends with "GE"
tagValueEndsWith(#Tag.StudyDescription, "GE")
tagValueEndsWith("0008,1030", "GE")

# Check if the study description does not end with "GE"
!tagValueEndsWith(#Tag.StudyDescription, "GE")
!tagValueEndsWith("0008,1030", "GE")

tagIsPresent(int tag) or tagIsPresent(String tag)

This function will check if the tag is present in the DICOM instance.

# Check if the tag study description is present in the DICOM file
tagIsPresent(#Tag.StudyDescription)
tagIsPresent("0008,1030")

# Check if the tag study description is not present in the DICOM file
!tagIsPresent(#Tag.StudyDescription)
!tagIsPresent("0008,1030")

Multiple conditions can be combined using logical operators.

&& corresponds to the AND logical operator

|| corresponds to the OR logical operator

# Check if the tag study description ends with "GE" and if the station name is "CT1234"
tagValueEndsWith(#Tag.StudyDescription, "GE") && tagValueContains(#Tag.StationName, "CT1234")

# Check if the tag study description ends with "GE" or if the station name is "CT1234"
tagValueEndsWith(#Tag.StudyDescription, "GE") || tagValueContains(#Tag.StationName, "CT1234")

How does de-identification work?

Karnak is a gateway for sending DICOM files to one or multiple Application Entity Title (AET). Karnak offers the possibility to configure multiple destinations for an AET. These destinations can communicate using the DICOM or DICOM WEB protocol.

A destination contains several configurations for the DICOM endpoint, the credentials, and is also linked to a project.

A project defines the de-idenfication or tag morphing method and a secret that will be used to generate deterministic random values like UIDs or shift date arguments.

Basic Profile

The reference profile for de-identifying DICOM objects is provided by the DICOM standard. This profile defines an exhaustive list of DICOM tags and their related action to allow the de-identification of the instance.

In order to properly de-identify the sensitive data, five different actions are defined in the standard:

• D – Replace with a dummy value
• Z – Set to null
• X – Remove
• K – Keep
• U – Replace with a new UID

The Basic Profile defines one or more actions to be applied to a list of tags.

The DICOM’s type is often dependent on the Information Object Definition (IOD) of the instance. To avoid DICOM corruption, multiple actions can be defined for a tag, ensuring that a destructive action like REMOVE won’t be applied on a Type 1 or Type 2 attribute.

• Z/D – Z unless D is required to maintain IOD conformance (Type 2 versus Type 1)
• X/Z – X unless Z is required to maintain IOD conformance (Type 3 versus Type 2)
• X/D – X unless Z is required to maintain IOD conformance (Type 3 versus Type 1)
• X/Z/D – X unless Z or D is required to maintain IOD conformance (Type 3 versus Type 2 versus Type 1)
• X/Z/U* – X unless Z or replacement of contained instance UIDs (U) is required to maintain IOD conformance (Type 3 versus Type 2 versus Type 1 sequences containing UID references)

Karnak loads the SOPs and attributes as specified in the DICOM Standard. Based on the tag’s type in the current instance, the proper action is set and applied. If the tag cannot be identified in the SOP or its type cannot be inferred, the strictest action will be applied (U/D > Z > X).

Below is a concrete illustration of the action applied in case of multiple actions defined in the Basic Profile:

• Z/D, X/D, X/Z/D → apply action D
• X/Z → apply action Z
• X/Z/U, X/Z/U* → apply action U

Action D, without a dummy value

The action D replaces the tag value with a dummy one. This value must be consistent with the Value Representation (VR) of the tag.

Karnak will use a default value based on the VR in this case, as defined below:

• AE, CS, LO, LT, PN, SH, ST, UN, UT, UC, UR → “UNKNOWN”
• DS, IS → “0”
• AS, DA, DT, TM → a date is generated using shiftRange(), as explained in the Shift Date section
• UI → a new UID is generated using the Action U

The shiftRange() action will return a random value between a given maximum days and seconds. By default, maximum days is set to 365 and maximum seconds is set to 86400.

The following VRs FL, FD, SL, SS, UL, US are of type Binary. By default, Karnak will set to null the value of this VR.

Action U, Generate a new UID

For each U action, Karnak will hash the input value. A one-way function is created to ensure that it is not possible to revert to the original UID. This function will hash the input UID and generate a new UID from the hashed input.

Context

It’s possible for a DICOM study to be de-identified several times and in different ways, potentially implying the use of multiple hashing and one-way functions. Karnak ensures deterministic generation of UIDs in order to maintain the quality and usability of the data.

To achieve that behavior, a project must be created and associated to the destination that requires de-identification. A project defines a de-idenfication method and a secret, either generated randomly or imported by the user. The project’s secret will be used as key for the HMAC.

Project secret

The secret is 16 bytes long and randomly defined when the project is created.

A user can upload his own secret, but it must be 16 bytes long. It can be uploaded as a String in hexadecimal format.

Hash function

The algorithm used for hashing is the “Message Authentication Code” (MAC). Karnak uses the MAC, not as message authentication, but as a one-way function. Below is a definition from the JAVA Mac class used in Karnak:

« A MAC provides a way to check the integrity of information transmitted over or stored in an unreliable medium, based on a secret key. Typically, message authentication codes are used between two parties that share a secret key in order to validate information transmitted between these parties.

A MAC mechanism that is based on cryptographic hash functions is referred to as HMAC. HMAC can be used with any cryptographic hash function, e.g., SHA256 or SHA384, in combination with a secret shared key. HMAC is specified in RFC 2104. »

For each use of the HMAC, it uses the SHA256 hash function combined with the project’s secret.

Generate UID

« What DICOM calls a “UID” is referred to in the ISO OSI world as an Object Identifier (OID) » [1]

To generate a new DICOM UID, Karnak will create an OID beginning with “2.25”, that is an OID encoded UUID.

« The value after “2.25.” is the straight decimal encoding of the UUID as an integer. It MUST be a direct decimal encoding of the single integer, all 128 bits. » [2]

The generated UUID will use the first 16 bytes (128 bits) from the hash value. The UUID is a type 4 with a variant 1. See the pseudocode below to ensure the type and the variant are correct in the UUID:

// Version
uuid[6] &= 0x0F
uuid[6] |= 0x40

// Variant
uuid[8] &= 0x3F
uuid[8] != 0x80

The hashed value will be converted in a positive decimal number and appended to the OID root separated by a dot. See the example below:

OID_ROOT = “2.25”
uuid = OID_ROOT + “.” + HashedValue[0:16].toPositiveDecimal()

Shift Date, Generate a random date

Karnak implements a randomized date shifting action. This shift must be identical based on the project and the patient for data consistency. For example, if a random shift is made for the birthdate of the patient “José Santos”, it must be the same for each instance associated to “José Santos”, even if the instance is loaded later.

The random shift date action will use the HMAC defined above and a range of days or seconds defined by the user. If the minimum isn’t specified, it defaults to 0.

The patientID, along with the project’s secret, will be passed to the HMAC, ensuring data consistency by patient.

The code below illustrates how a random value is generated within a given minimum (inclusive) and maximum (exclusive) range.

scaleHash(PatientID, scaledMin, scaledMax):
    patientHashed = hmac.hash(PatientID)
    scale = scaledMax - scaledMin

    shift = (patientHashed[0:6].toPositiveDecimal() * scale) + scaledMin

Pseudonym

This chapter details the problems linked to the PatientID generation for different de-identification methods.

A patient can participate in several studies using different de-identification methods. Depending on the project or the clinical research, the de-identification profile can keep some of the patient’s metadata. A pseudonym and a patientID are generated and affected to the patient in order to identify him in the context of the project.

Most of the patient’s identifying information is contained in the Patient Module.

Below is illustrated a case where some patient’s data could be leaked. During the de-identification, the patient is associated with a pseudonym provided by an external service or mapping table. In this example, a patient’s study falls in the scope of two different projects in Karnak. The first de-identification removes the patient birthdate (3. Apply Project 1) and the second keeps it (5. Apply Project 2). If the patient pseudonym is used as patient identification in the Patient Module, and the data is reconciled between the two study, the birthdate will be leaked.

Below is illustrated an alternative way of handling pseudonyms during the de-identification.

In this example, the patient is still associated with a pseudonym provided by an external service or mapping table. But Karnak will generate a PatientID based on this pseudonym and other characteristics linked to the project specifically. This ID will be used to identify the patient in the context of the project. In case of different de-identification methods in different projects, the patients cannot be reconciled and data won’t be leaked.

PatientID generation

Karnak generates a PatientID to solve the problem explained previously. The PatientID is generated using the HMAC function defined in the project, see chapter Action U, Generate a new UID for more details.

The de-identified patientID is generated as follows :

• the patient’s pseudonym is retrieved from an external service or a mapping table
• the pseudonym is hashed using the hmac function and the project’s secret, making it unique and deterministic in the context of the project
• the patientID is set to the first 16 bytes of the hashed pseudonym

The pseudonym will be used as the patient’s name if no other action has been defined during de-identification.

Keep the correspondence between pseudonym and patient

It is possible to retrieve the patient information once de-identified.

The patientID is generated from the pseudonym and the project’s secret using the hmac function, making it irreversible. The pseudonym is stored in the attribute Clinical Trial Subject ID (0012,0040). Using the mapping table or the service that provided the pseudonym, it is possible to retrieve the original patient identity.

« The Clinical Trial Subject ID (0012,0040) identifies the subject within the investigational protocol specified by Clinical Trial Protocol ID (0012,0020). » [3]

Attributes added by Karnak

Some attributes are automatically set by Karnak during de-identification in the following modules.

SOP Common

The following attributes are set in the SOP Common module during de-identification:

• Instance Creation Time (0008,0013), is set to the time the SOP instance was created. The value representation used is TM (HHMMSS.FFFFFF).
• Instance Creation Date (0008,0012), is set to the date the SOP instance was created. The value representation used is DA (YYYYMMDD).

Patient Module

The following attributes are set in the SOP Common module during de-identification:

• Patient ID (0010,0020), is set to the hashed pseudonym, see PatientID Generation for more details
• Patient Name (0010,0010) is set to the pseudonym if no other action is applied to that tag during de-identification
• Patient Identity Removed (0012,0062) is set to YES
• De-identification Method (0012,0063) is set to the concatenated profile element codenames applied to the instance in order of application

The profile element codenames are concatenated and separated by -. For example, a profile composed of the profile elements action.on.specific.tags and basic.dicom.profile will appear as action.on.specific.tags-basic.dicom.profile.

Clinical Trial Subject Module

The following attributes are set in the Clinical Trial Subject Module during de-identification:

• Clinical Trial Sponsor Name (0012,0010) is set to the project name
• Clinical Trial Protocol ID (0012,0020) is set to the profile codename (concatenated profile elements’ codename)
• Clinical Trial Protocol Name (0012,0021 is set to null
• Clinical Trial Site ID (0012,0030) is set to null
• Clinical Trial Site Name (0012,0031) is set to null
• Clinical Trial Subject ID (0012,0040) is set to the patient’s pseudonym

Gateway

Forward Node

This page lists all the forward nodes configured in Karnak and allows to edit, create and delete nodes. The forward node contains the configuration of the DICOM node’s Application Entity.

1. Creation of a forward node

After clicking on the New Forward Node button, a form will appear as illustrated below.

To create a new forward node, the “Forward AETitle” input must be filled, then click on the “Add” button. Once added, it will appear in the list of forward nodes and will be selected. The Forward AETitle must be unique.

2. Forward node list

This list displays all the forward nodes created in the Karnak instance. A forward node can be selected to view and manage its details on the right panel.

3. Forward node parameters

The value of the forward AETitle and its description can be modified after creation on the detailed view of the forward node. To save your changes, click on the “Save” button.

4. Forward node sources or destinations

In the forward node’s details, source control can be configured, checking if the received DICOM is provided from a known source. Destinations can also be created and configured to distribute received DICOM instances. These destinations can communicate with the DICOM or DICOM WEB protocol.

4.1 Navigation

Depending on the selected tab, the list of destinations or sources associated to this forward node will be displayed.

4.2 Filtering

The destination’s list filter is applied to the destination description.

The source’s list filter is applied to the AE title, hostname and description source.

4.3 List

All the destinations or sources associated to the forward node are displayed here.

Click on an element of the list will open its detailed view, also allowing its edition.

4.4 Actions

The action buttons depend on the current tab selected.

In the Destinations view, two actions are displayed, corresponding to the protocol associated to the destination being created. The available protocols are either DICOM or DICOM WEB. The Destination creation is detailed in the section Destinations.

In the Sources view, a button for creating a new Source is displayed. The Source creation is detailed in the Sources page.

5. Forward node actions

Three actions are available:

• Save: saves the changes made on the forward node parameters
• Delete: deletes the selected forward node
• Cancel: reverts the changes made on the forward node parameters

Profiles

This page lists all the profiles configured in Karnak and allows to edit, create and delete them. A profile contains a definition of actions that should be applied to a DICOM instance before being sent.

The “Dicom Basic Profile” is present by default and cannot be deleted. This profile is detailed in the section How does de-identification work?. This profile cannot be deleted or edited, except for the value of default issuer of patient ID.

1. Profile drag and drop area

A profile can only be created by importing a YAML file using the top right component of the Profiles view. The file can be selected by clicking the “Upload File” button or drag and dropped there. It will be loaded and analyzed. If there are no errors, a new profile will automatically be created with the content of the file, and added to the list of profiles.

2. Profile list

All the profiles available are listed here. A profile can evolve and be present in the list under the same name but with a different version.

By selecting a profile in the list, its details are displayed on the right.

3. Profile details

This area contains the details of a profile.

If the profile is selected in the list, its details will appear on the right panel.

Some information can be edited such as the name, version and minimum required version of Karnak, by clicking on the pen icon.

The field’s value can be edited, the changes are saved by clicking on the check mark button and reverted by clicking on the cross button.

The rest of the profile cannot be modified.

The entire profile can be downloaded as a YAML file by clicking on the button on the right or deleted by clicking on the button on the left.

If a YAML file was imported, this view will display the details of the newly created profile, if no errors occurred during the import and the creation was successful.

If some errors occurred during the profile creation, the profile won’t be created and added to the profiles’ list. The errors will be displayed in the right panel with details about the source of the error(s). An example is shown below.

Projects

This page lists all the projects configured in Karnak and allows to edit, create and delete them. A project is linked to a profile and contains a secret used for de-identification.

1. Create a project

To create a new project, the project name field and the profile select box must be filled, then click on the button “Add”. The project will be added to the list of projects and its details displayed in the right panel.

2. Project list

All the projects available are listed here. By selecting a project in the list, its details are displayed on the right.

3. Project details

In the details view, the project name and de-identification profile can be modified. The changes must then be saved using the “Update” button.

4. Project secret

The project’s secret is at the core of Karnak, more specifically for the de-identification process. It is a 32-character hexadecimal value.

The project’s secret is automatically generated when the project is created. The date and time of the secret’s creation are appended at the end of the secret in the details view.

A new secret can be generated by clicking the “Generate Secret” button. If new secrets are generated, previous secrets are saved in the database. Previous versions can be selected to be used.

A destination is associated with a project regarding the de-identification, as explained in the Destination configuration

In order to generate new values, as well as pseudonymize certain patient information, a hash function is used with the secret as seed to have a project-wise unique yet determinist generated value. When the secret is changed, it must be taken into consideration that this determinist mechanism will be broken. If a DICOM instance is de-identified twice using the same project, same secret and same profile, the resulting de-identified instances will be the equal. If the secret is changed, then the same DICOM instance de-identified twice, using the same project, same profile but a new secret will result in different de-identified instances.

Details about the algorithm and UID generation using the project’s secret can be found here.

5. Action buttons

Any change done on the project’s details must be persisted using the “Update” button.

The selected project can be deleted by clicking on the “Remove” button. If the project is associated to a destination, an error message will be displayed.

External Pseudonym

In this page, pseudonyms can be created or imported, and will then be used during de-identification by Karnak. The de-identification process and how the pseudonym is used is detailed in the Pseudonym chapter.

The activation of de-identification is done in the Destination configuration.

The created or imported pseudonyms in this page are stored in a cache that will be cleared after a maximum of 7 days. The pseudonyms saved will also be lost in case Karnak is restarted.

1. Choose a project

External pseudonyms are linked to a project. It allows Karnak to handle properly the case where a patient is participating in multiple clinical studies, as well as potential collisions.

2. Upload a CSV file

A CSV file containing external pseudonyms can be uploaded by clicking the “Upload File” button or drag and dropped there. It launches the import process.

A pop-up is displayed asking for the separator of the CSV file. By default, the value is set to “,”.

After clicking on the “Open CSV” button, a grid is displayed containing the CSV file data.

2.1 From line

This field defines the index of the first line of the CSV file to import. It allows to skip a number of lines at the beginning of the file, especially if it contains headers.

2.2 Columns assignments

In this view, the CSV columns are associated with the corresponding pseudonym attributes. All the fields are mandatory, except the Issuer of patient ID.

2.3 Upload CSV

The “Upload CSV” button will start importing the CSV data. It performs data validation on the data, such as Patient ID or Pseudonym duplication.

3. Add a new patient

It’s also possible to add an external pseudonym manually. All the fields are mandatory, except the Issuer of patient ID.

Clicking on “Add patient” will add the entered data into the external pseudonyms table.

4. Delete all patients

This button will delete all the external pseudonyms stored in the cache but linked to the selected project. The other pseudonyms linked to other projects won’t be impacted.

A popup will be displayed before the deletion is effectively done, asking for the confirmation of the user.

5. Pseudonym edition

Once stored in the cache, it is possible to edit the patient fields by clicking on the “Edit” button on the external pseudonym row that should be modified. The external pseudonym can also be deleted by clicking in the “Delete” button on the external pseudonym row that should be removed.

Multiple rows can be selected by checking the boxes on the left of each row. These rows can be deleted by clicking on the “Delete selected patients” at the top of the external pseudonyms table. Again, a confirmation is asked before the deletion is performed.

Pseudonym mapping

The pseudonym mapping is used to retrieve the mapping of pseudonyms added in the External pseudonym view.

1. Search field

In order to retrieve the original value, enter the pseudonym and click on the “Find” button. The search is case-sensitive.

2. Results

If a correspondence is found, it is displayed below. “[External]” means that the match was found in the external pseudonym table followed by the associated project. The original patient data is displayed as well.

If that pseudonym has no correspondence, an error message will be displayed as illustrated below.

Monitoring

Monitoring view is used to follow transfers in progress or recent transfers.

Currently limited to 150000 transfers, it is possible to browse the different pages via the navigation bar at the bottom center of the view

An automatic cleaning occurs on the monitoring tables, if the limit is exceeded.

The view displays the most recent transfers and is ordered with most recent first.

Filters are available to easily browse or find a transfer.

It is possible to export the list of transfers via the export functionality in csv format depending on the filters selected.

By clicking on a transfer, its details are displayed containing the de-identified and original information.

1. Filters

Filters allow to easily search and browse transfers. It is possible to filter by:

• Date/Time of transfers
• Study UID (original or de-identified)
• Serie UID (original or de-identified)
• Sop Instance UID (original or de-identified)
• Status of the transfer (not sent, sent, all, excluded and error)

The status of a transfer is defined as follows :

• if the instance was successfully sent, it appears with the label “Sent” in green
• if the instance was not sent because of the SOP filter, the destination condition or the use of ExcludeInstance() in the profile, it appears in orange with a label corresponding to the reason why it was excluded
• if the instance was not sent because an unexpected error occurred, it will appear in red with a label corresponding to the error that occurred

2. Browsing

It is possible to go through the different pages of the list of transfers by using the navigation bar.

3. Refresh

By clicking on the refresh button, the list of transfers is updated with last transfers.

4. Export

Export settings allow to customize the csv before exporting it. It is possible to change the csv delimiter and the quote character of the file.

Once customized, the export is launched by clicking on the export button.

Export will use the filters selected in the monitoring view and export only transfers matching the filters’ criteria.

DICOM tools

This page provides an overview of the utility tools available in the DICOM Tools module, designed to assist with connectivity, query, and status checks for DICOM servers. These tools enable users to:

• Test the availability of a DICOM node using DICOM Echo
• Query a DICOM Worklist and display the DICOM result
• Group and monitor DICOM nodes using DICOM Echo and WADO protocols

The different modules are accessed through the tabs at the top of the page.

Those functionalities will be presented in details below.

DICOM Echo tab

This tool tests the DICOM communication between 2 AETs.

The field “Calling AETitle” defines the identity of the calling DICOM entity. The field “Called AETitle” defines the identity of the DICOM entity to test. The field “Called Hostname” and “Called Port” define the hostname and port of the DICOM node to test.

All the fields are mandatory to execute the ECHO test.

Clicking on the button “Echo” will launch the connectivity test and DICOM Echo command. Below is an example of a successful test.

In this example, the “Called AETitle” value is purposely wrong, we can see that the network connectivity test is successful but not the DICOM Echo, displaying the reason of the failure.

Select Node Utility

The Select Node popup can be displayed by clicking on the “Select Node” button.

The type of DICOM Node is set in the “Dicom Nodes Type” field. The list of values in the “Dicom Node” field is then updated according to the selected node type. The values can be filtered in this field by typing directly a string that will be matched against the AET, hostname, port or description.

Clicking on the “Select” button will automatically fill the Called AETitle, Called Hostname and Called Port fields with the values of the selected DICOM node.

DICOM Worklist tab

This tool retrieves the content of a DICOM Worklist and tests their behavior and connectivity.

Worklist Configuration

The field “Calling AETitle” defines the identity of the calling DICOM entity. The field “Worklist AET” defines the identity of the worklist to test. The field “Worklist Hostname” and “Worklist Port” define the hostname and port of the worklist to test.

All the fields are mandatory to retrieve the worklist content.

Worklist Query

Additionally, some fields are defined to filter the results returned by the worklist. These fields are optional. If none are filled, no filters are applied and all the elements of the worklist are retrieved.

Query Results

Clicking on the button “Select Worklist” will launch the test and data retrieval. The retrieved worklist data is displayed in a table, that can be sorted by column, by clicking on its header.

Select Worklist Utility

The Select Worklist popup can be displayed by clicking on the “Select Worklist” button.

A list of available worklist will be displayed and a node can be selected. The worklists can be filtered in this field by typing directly a string that will be matched against the AET, hostname, port or description.

Clicking on the “Select” button will automatically fill the Worklist AET, Worklist Hostname and Worklist Port fields with the values of the selected DICOM worklist.

Monitor tab

This tool checks the status of DICOM nodes using both DICOM and WADO protocols.

DICOM ECHO

A DICOM node must be selected in the list, and the button “Check” can be clicked to launch the DICOM Echo test for that node. Below is an example of a successful DICOM Echo test.

WADO

A DICOM node must be selected in the list, and the button “Check” can be clicked to launch the WADO test for that node. Below is an example of a successful WADO test.

Kheops

Create a destination album

To create a Kheops album as a destination, the following values must be set:

• Protocol: STOW
• DICOM endpoint: /api/studies

To create the album destination, please refer to the official documentation. of Kheops.

Once the album is created, please follow these steps to configure the album as a destination in Karnak.

1. Create a new token

2. Give WRITE permission to the token and set the expiration date

3. Copy the authentication token value to be used in the header of your Karnak destination

The creation and configuration of a STOW Destination in Karnak is detailed here.

Switching in different Kheops albums

When a destination points to a Kheops album, the data can be propagated to underlying albums.

This is useful when a cohort of studies is sent to a research group for example, without sharing all the album studies.

Studies cannot be shared between different Kheops instances, one destination must be configured in Karnak per Kheops instance.

The purpose of this functionality is to take advantage of the Kheops API to propagate the data to different places without having to create new destinations in Karnak. At the same time, the data is split according to rules defined in Kheops to prevent data leakage and allow the authorized persons to access only the relevant data and not all the main album.

The following diagram illustrates the behavior of data being processed through Karnak and sent to multiple Kheops albums.

graph LR;
  A(DICOM Data) --> B[Karnak]
  
  subgraph Kheops
    D[Main Album] --> E[Album X]
    D --> F[Album Y]
  end
  
  B --> D

First, a DICOM instance is received by Karnak. After processing it, it sends the instance to the main Kheops album. Depending on existing rules and conditions, the instance will also be shared to the album X and Y.

Create a switching Kheops album

To share a DICOM instance in different Kheops albums, the following fields must be filled and validated by clicking on Add button.

The condition field defines a condition to enable sharing an instance to a specific album if it is evaluated to true.

The conditions syntax and usage is detailed in the Conditions page.