Versions Compared

Old Version 4

changes.mady.by.user DamienD

Saved on

compared with

New Version 5

changes.mady.by.user DamienD

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

...

  1. Add a file system to the NEARLINE storage group
    1. Open a web browser, and navigate to the JMX console. e.g. http://localhost:8080/jmx-console
    2. Locate the "dcm4chee“dcm4chee.archive" archive” section, and click on "group“group=NEARLINE_STORAGE,service=FileSystemMgt"FileSystemMgt”
    3. Scroll down to the "List “List of MBean Operations" Operations” section and find the "addRWFileSystem“addRWFileSystem()" ” operation. Enter in a path for nearline storage. This will not actually be used, since we are going to store the files in S3, but we need to configure something here so that the system knows we are using the NEARLINE storage. I have entered: "tar“tar:/storage/nearline"nearline”. Note the tar prefix. This tells dcm4chee that all of the files going to this storage group will be tarred up.
    4. Click Invoke to create add the file system record into the database.
  2. Configure the FileCopy service
    The FileCopy service is responsible for physically copying files to your nearline storage. This is where we configure our particular plugin.
    1. Specify a value for the DestinationFileSystem. This value should equal the value you specified for your nearline storage file system so that dcm4chee knows that this FileCopy service is associated with that file system configuration. e.g. "tar“tar:/storage/nearline"nearline”
    2. Specify a value for HSMModulServicename. This should be the JMX ObjectName of our S3 plugin module, and enables it for use within this service when storing and retrieving files. Enter: "dcm4chee“dcm4chee.archive:service=FileCopyHSMModule,type=S3"S3”
    3. Leave the FileStatus set to TO_ARCHIVE. This will be the status of files stored in S3. When the SyncFileStatus service runs and verifies that these files are stored properly, it will change the status to ARCHIVED.
    4. Click Apply Changes.
  3. Configure the TarRetriever service
    This service is responsible for fetching and extracting tar files from the nearline storage during retrieve requests.
    1. Specify a value for HSMModulServicename. This should be the JMX ObjectName of our S3 plugin module, and enables it for use within this service when storing and retrieving files. Enter: "dcm4chee“dcm4chee.archive:service=FileCopyHSMModule,type=S3"S3”
    2. Click Apply Changes.
  4. Configure the S3 HSMModule (service=FileCopyHSMModule,type=S3)
    Now we are ready to configure the S3 integration. Here are the main things to configure here:
    1. Amazon S3 bucket name
    2. Amazon AWS Access Key
    3. Amazon AWS Secret Key (this is write only, and you will not see a value after clicking Apply Changes)
    4. The Outgoing and Incoming directories in this configuration are temporary storage areas that are used for tarring and untarring files.
    5. Click Apply Changes.
  5. Configure the SyncFileStatus service
    This service will run periodically and verify the files that have been stored to S3. It will fetch the tar files and ensure that the correct files are contained within. Once it verifies the files, it will update the file status in the database to ARCHIVED.
    1. Specify a value for the MonitoredFileSystem. This value should equal the value you specified for your nearline storage file system so that dcm4chee knows that this service is associated with that file system configuration. e.g. "tar“tar:/storage/nearline"nearline”
    2. Specify a value for HSMModulServicename. This should be the JMX ObjectName of our S3 plugin module, and enables it for use within this service when fetching tar files. Enter: "dcm4chee“dcm4chee.archive:service=FileCopyHSMModule,type=S3"S3”
    3. Specify a TaskInterval. It is set to NEVER by default, so you you should set it to a proper interval that is good for your workflow, preferably not during peak business hours.

...

In the database, the files should have a changed file status, and should reflect their tar path as shown in this screenshot:

If you can't can’t read it in the picture, the filepaths look like this: 2011/8/3/16/745ABFED/CF024730-323397.tar!CF024730/000004A0

Note the "tar" “tar” designator in the path. This tells the system that the file is contained within a tar file.

Setting up Retention Rules to Remove Studies from Online

We probably don't don’t want two copies of the study forever, so lets set up some rules now so that studies are deleted from the ONLINE storage group after a period of time. This will leave the remaining copy on S3.

...

  1. Configure deletion of ONLINE studies
    1. Open a web browser, and navigate to the JMX console. e.g. http://localhost:8080/jmx-console
    2. Locate the "dcm4chee“dcm4chee.archive" archive” section, and click on "group“group=ONLINE_STORAGE,service=FileSystemMgt"FileSystemMgt”
    3. Set DeleteStudyIfNotAccessedFor = your retention period (52w or whatever your SLA requires)
    4. Set DeleteStudyOnlyIfStorageNotCommited = false
    5. Set DeleteStudyOnlyIfCopyOnMedia = false
    6. Set DeleteStudyOnlyIfCopyOnReadOnlyFileSystem = false
    7. Set ScheduleStudiesForDeletionInterval = a reasonable time interval for the system to check the database and schedule deletion jobs.
    8. Set DeleteStudyOnlyIfCopyOnFileSystemOfFileSystemGroup = NEARLINE_STORAGE
    9. Set DeleteStudyOnlyIfCopyArchived = true (only delete studies that have been verified by the SyncFileStatus service. If you don't don’t care about that or are not running that service, you can set this false.)
    10. Click Apply Changes

At this point, dcm4chee will look for studies in ONLINE that meet these criteria and schedule them for deletion. After they are deleted, and the only copy is on S3, a retrieve request will trigger a fetch from Amazon. The tar file(s) will be fetched, images extracted and sent to the destination.

That's That’s it!