Error Description: Filesystems on ONLINE_STORAGE Filesystem Group run out of space, even deleter criteria are specified and studies could be deleted.

Check basic configuration:

{table}
{table-row}
{table-cell}Open ONLINE_STORAGE FilesystemMgt service in jmx-console ( [link:http://<host>:8080/jmx-console|http://localhost:8080/jmx-console/HtmlAdaptor?action=inspectMBean&name=dcm4chee.archive%3Aservice%3DFileSystemMgt%2Cgroup%3DONLINE_STORAGE] )
{table-cell}
{table-cell}
{info}Links on this page will only work when dcm4chee is on localhost\!
{info}
{table-cell}
{table-row}
{table}

Either ScheduleStudiesForDeletionInterval not NEVER and/or ScheduleStudiesForDeletionOnSeriesStored = true.

DeleterThresholds must not be NONE and have an adequate value.

Current treshold value (dependent on daytime) can be seen in CurrentDeleterThreshold

DeleteStudyOnlyIfNotAccessedFor attribute is responsible for how long a study have to be ONLINE.
The value is dependent on online disk space and expected data volume.

If the value is to high (e.g. 365d and disk capacity is just for 180 days) no studies can be deleted!

Which criterias are enabled? (all enabled criteria must be fulfilled)
Are all basic conditions for the criterias configured correctly?
(e.g. DeleteStudyOnlyIfExternalRetrieveable: Forward Service, Storage Commitment)

DeleteStudyOnlyIfStorageNotCommited
DeleteStudyOnlyIfExternalRetrieveable
DeleteStudyOnlyIfCopyOnMedia
DeleteStudyOnlyIfCopyOnFileSystemOfFileSystemGroup
DeleteStudyOnlyIfCopyArchived
DeleteStudyOnlyIfCopyOnReadOnlyFileSystem

Check server.log

Ensure that 'org.dcm4chex.archive.mbean' and/or 'org.dcm4chex.archive.mbean.FileSystemMgt2Service' has at least INFO level
org.dcm4chex.archive
org.dcm4chex.archive.mbean
org.dcm4chex.archive.mbean.FileSystemMgt2Service

A 'Operation completed successfully without a return value.' result message means, that no logger level is set for this category and the level of the parent will be applied!

Check ScheduleStudiesForDeletionInterval attribute in ONLINE_STORAGE FilesystemMgt service to be not NEVER.

The Deleter is disabled when DeleteStudyIfNotAccessedFor is NEVER and DeleterThresholds is NONE!

Most likely the UsableDiskSpace is NOT below CurrentDeleterThreshold.
Set DeleterThreshold according to your needs.

Check if studies are ready to delete:

By enabling DEBUG logger level for 'org.jboss.ejb.plugins.cmp' (log4j service in jmx-console) you can find the queries of the deleter in server.log,

Set NUM_DAYS to corresponding value of DeleteStudyOnlyIfNotAccessedFor (one hour can be expressed as 1/24)

Simple query: (select of all FS groups!)

SELECT * FROM study_on_fs WHERE access_time < (sysdate - <NUM_DAYS>) AND rownum < 10;

Query of studies stored on ONLINE_STORAGE Filesystems:

SELECT * FROM study_on_fs, filesystem WHERE filesystem.pk = study_on_fs.filesystem_fk
AND filesystem.fs_group_id = 'ONLINE_STORAGE'
AND access_time > (sysdate - <NUM_DAYS>) AND rownum < 10;

As used by deleter:

SELECT * FROM STUDY_ON_FS t0_sof, SERIES t2_s, FILESYSTEM t1_sof_fileSystem, STUDY t7_sof_study
WHERE ((t1_sof_fileSystem.fs_group_id = 'ONLINE_STORAGE' AND t1_sof_fileSystem.fs_status IN (0, 1)
AND t2_s.series_status = 0 AND t0_sof.filesystem_fk=t1_sof_fileSystem.pk))
AND t7_sof_study.pk=t2_s.study_fk AND t0_sof.study_fk=t7_sof_study.pk AND t0_sof.access_time < (sysdate - <NUM_DAYS>);

Result of last statement is empty?

  • Check configuration values
  • Simply all studies are to young to die!
  • STUDY_ON_FS entries are missing!
    This may happen if a study is scheduled for deletion but not deleted (Deletion and all subsequent retries failed or DeleteOrders are removed from deleter queue (e.g. removeAllMessages() ) 
    SELECT DISTINCT study.pk, files.filesystem_fk FROM study, series, instance, files 
WHERE study.pk=series.study_fk AND series.pk=instance.series_fk AND 
instance.pk=files.instance_fk AND NOT EXISTS(SELECT study_on_fs.pk FROM study_on_fs WHERE study_on_fs.study_fk=study.pk);
    In this case the missing STUDY_ON_FS entries must be added manually: 
    INSERT INTO study_on_fs (PK, STUDY_FK, FILESYSTEM_FK, ACCESS_TIME) VALUES
(<PK>,<STUDY_PK>,<FILESYSTEM_FK>,(sysdate - <NUM_DAYS>));
    <PK>...new study_on_fs.pk using auto numbering! e.g. Oracle: study_on_fs_pk_seq.nextval

In general all combinations of deleter criteria can be configured and act in conjunction (logical AND).
There is one exception: DeleteStudyOnlyIfStorageNotCommited and DeleteStudyOnlyIfExternalRetrieveable are mutually exclusive!

Study External retrieveable:

  • Check configuration: Forward service, StorageCommitment activated (RequestStorageCommitFromAETitles attribute in QueryRetrieveScp service).
  • Look for Exceptions, check C_MOVE, C_STORE and N_ACTION if all worked as expected.
  • Check in DB:
    All Instances of study must be external retrieveable:

    DeleteStudyOnlyIfExternalRetrieveable=true

    SELECT COUNT(i.pk) FROM instance i, series, study
WHERE i.series_fk = series.pk AND study_fk = study.pk AND series.study_fk = <STUDY_PK> AND i.ext_retr_aet IS NOT NULL;
    must be equal to 
    SELECT num_instances FROM study WHERE pk = <STUDY_PK>;

Copy of Study available on Filesystemgroup:

  • Check configuration: Copy service, (HSM if archived status is needed).
  • Look for Exceptions
  • Check in DB:

    DeleteStudyOnlyIfCopyOnFileSystemOfFileSystemGroup=NEARLINE_STORAGE

    SELECT count(DISTINCT t0_i.pk) FROM instance t0_i, files t3_f, series t5_i_series, study t1_i_series_study,
filesystem t2_f_fileSystem WHERE (((t1_i_series_study.pk=<STUDY_PK>) AND t2_f_fileSystem.fs_group_id = 'NEARLINE_STORAGE'
AND t0_i.series_fk=t5_i_series.pk AND t5_i_series.study_fk=t1_i_series_study.pk
AND t3_f.filesystem_fk=t2_f_fileSystem.pk)) AND t0_i.pk=t3_f.instance_fk;

    DeleteStudyOnlyIfCopyOnFileSystemOfFileSystemGroup=NEARLINE_STORAGE
    DeleteStudyOnlyIfCopyArchived=true

    SELECT count(DISTINCT t0_i.pk) FROM instance t0_i, files t3_f, series t5_i_series, study t1_i_series_study,
filesystem t2_f_fileSystem WHERE (((t1_i_series_study.pk=<STUDY_PK>) AND t2_f_fileSystem.fs_group_id = 'NEARLINE_STORAGE'
AND t3_f.file_status = 2 AND t0_i.series_fk=t5_i_series.pk AND t5_i_series.study_fk=t1_i_series_study.pk
AND t3_f.filesystem_fk=t2_f_fileSystem.pk)) AND t0_i.pk=t3_f.instance_fk;

    DeleteStudyOnlyIfCopyOnFileSystemOfFileSystemGroup=READONLY_STORAGE
    DeleteStudyOnlyIfCopyOnReadOnlyFileSystem=true

    SELECT count(DISTINCT t0_i.pk) FROM instance t0_i, files t4_f, series t3_i_series, study t1_i_series_study, filesystem t2_f_fileSystem 
WHERE (((t1_i_series_study.pk=<STUDY_PK>) AND t2_f_fileSystem.fs_group_id = 'READONLY_STORAGE' AND t2_f_fileSystem.fs_status = 2 
AND t0_i.series_fk=t3_i_series.pk AND t3_i_series.study_fk=t1_i_series_study.pk AND t4_f.filesystem_fk=t2_f_fileSystem.pk)) 
AND t0_i.pk=t4_f.instance_fk
    must be equal to 
    SELECT num_instances FROM study WHERE pk = <STUDY_PK>;

Copy of Study available on Media (CD / DVD / ...):

  • Check configuration: Media Creation Managment Service (MCMScu) .
  • Look for Exceptions, check C-STORE, N_CREATE and N_ACTION communication with Media Creation SCP if all worked as expected.
  • Check in DB:
    All Instances of the study must be also available on Media with media status COMPLETED(4):

    DeleteStudyOnlyIfCopyOnMedia=true

    SELECT count( t0_i.pk) FROM instance t0_i, series t3_i_series, study t1_i_series_study, media t2_i_media 
WHERE (t1_i_series_study.pk = <STUDY_PK> AND t2_i_media.media_status = 4 
AND t0_i.series_fk=t3_i_series.pk AND t3_i_series.study_fk=t1_i_series_study.pk 
AND t0_i.media_fk=t2_i_media.pk)
    must be equal to 
    SELECT num_instances FROM study WHERE pk = <STUDY_PK>;