Guide to the SpectralRTI_Toolkit for ImageJ

Contents

  1. What is Spectral RTI and do I need it? (LINK)
    1. Cost/benefit of adding RTI (LINK)
    2. Cost/benefit of adding spectral (LINK)
    3. How does the integration work? (LINK)
  2. Capture hardware (LINK)
    1. Spectral imaging hardware (LINK)
    2. Fixed dome for RTI capture (LINK)
    3. Swinging arc for RTI capture (LINK)
    4. Hand-held flash for RTI capture (LINK)
  3. Capture procedures (LINK)
    1. Light position capture and calibration (LINK)
    2. Hemisphere captures for RTI (LINK)
    3. Preprocessing typically done at capture (LINK)
  4. Processing with the SpectralRTI_Toolkit
    1. Software installation (LINK)
    2. Managing input data (LINK)
    3. Running for the first time (LINK)
    4. Select tasks (LINK)
    5. Adjust brightness (LINK)
    6. Static raking option (LINK)
    7. Options for Extended Spectrum (LINK)
    8. Identify region of interest (for Extended Spectrum and Pseudocolor) (LINK)
    9. Options for PCA Pseudocolor (LINK)
    10. Options for custom processes (LINK)
    11. Light position data (LINK)
    12. Additional prompts (LINK)
    13. Tips for frequent use (LINK)
  5. Make the images accessible to users (LINK)
    1. RTI files (LINK)
    2. WebRTI (LINK)
    3. Tiled and layered Jpeg2000 files for IIIF (LINK)
  6. Help make Spectral RTI better (LINK)
  7. Glossary (LINK)

Software installation

All the software required for Spectral RTI processing is freely available. Status of operating system compatibility is as follows:

Fiji distribution of ImageJ

ImageJ is an open-source image processing suite originated by the National Institutes of Health. It is well suited to scientific and medical imaging applications.

Fiji (Fiji Is Just Imagej) is a distribution of ImageJ that includes additional plugins. Although the extra plugins required for the SpectralRTI_Toolkit can be downloaded separately, we recommend starting with the Fiji distribution.

ImageJ and Fiji require Java, which can be downloaded together with the installer if it is not already installed on your system.

Resources:

SpectralRTI_Toolkit

The SpectralRTI_Toolkit is currently available as a macro for ImageJ. The Saint Louis University Center for Digital Humanities is currently developing the Toolkit into a plugin written in Java which should reduce memory requirements and increase speed.

Download the following files from https://github.com/thanneken/SpectralRTI_Toolkit

  1. SpectralRTI_Toolkit.ijm and copy to the macros folder under ImageJ or Fiji
  2. ijp-toolkit_bin_2.1.0.zip and unzip to the plugins folder under ImageJ or Fiji (required for all options)
  3. bij_plugin.zip and unzip to the plugins folder under ImageJ or Fiji (required for Principal Component Analysis, which is required for Extended Spectrum and PCA Pseudocolor)
  4. For ImageJ but not Fiji, Stack_Manipulation-2.0.1.zip and unzip to the plugins folder under ImageJ (required for Extended Spectrum and PCA Pseudocolor)

RTIBuilder

The RTIBuilder distributed by Cultural Heritage Imaging is the best way to create an LP file (unless you already have one) and also includes the HSHfitter required for creating RTI files.

Windows and MacOS installers are available from http://culturalheritageimaging.org/What_We_Offer/Downloads/Process/index.html. For Linux, download and extract either version (e.g., 7z x RTIbuilder_2_0_2.dmg). The application can be run from a terminal shell by changing to the directory that contains RTIbuilder.jar and entering the command java -jar RTIbuilder.jar

The SpectralRTI_Toolkit will prompt when it is time to run RTIBuilder to create an LP file. See below, "Light position data" (LINK)

Optional Jpeg2000 encoding for IIIF Image servers

The Toolkit will give the option of creating "Static Raking" images. "Static" contrasts with dynamically relightable RTI images. "Raking" means any angle of light among the hemisphere captures. Utilizing this option avoids lossy jpeg encoding (currently required for RTI) and creates files ready for IIIF Image repositories (tiled and layered). To utilize this feature it is necessary to download a jpeg2000 encoder. The best available is Kakadu. The executables for Windows, MacOS, and Linux can be downloaded from http://kakadusoftware.com/downloads/.

Optional WebRTI

To prepare RTI files for WebRTI you will need webGLRTIMaker.exe from http://vcg.isti.cnr.it/rti/webviewer.php.

Managing input data

Data captured by the Jubilees Palimpsest Project is available online for testing the Toolkit prior to application on your own data. The file https://raw.githubusercontent.com/thanneken/SpectralRTI_Toolkit/master/SpectralRTI_Dataset.bash can be run directly from a bash terminal in Linux or Windows. Running on MacOS may require replacing wget with curl, or you could download the files listed manually.

Folder structure

It is strictly necessary that no spaces appear anywhere in the file paths.

The Toolkit assumes that all input files are in the tiff format. Some adaptation to the Toolkit may be necessary to use other formats supported by ImageJ.

The Toolkit is indifferent to filenames but requires precise folder names to identify components. Each image set (e.g., page) requires a project folder (e.g. "Figurine" in the sample data set) and several subfolders, as follows:

  1. AccurateColor (required for accurate color images, strongly recommended)
  2. Captures-Fluorescence-NoGamma (can be used for PCA Pseudocolor, optional)
  3. Captures-Hemisphere-Gamma (required)
  4. Captures-Narrowband-NoGamma (required for Extended Spectrum, can be used for PCA Pseudocolor, recommended)
  5. Captures-Transmissive-Gamma (if more than one file is in the folder the user will be prompted to select one, optional)

Rotation

There is not a good way to rotate RTI files after they are created so data should be rotated at this point, if necessary.

Running for the first time

  1. Start ImageJ
  2. Plugins > Macros > Install..., then find and select SpectralRTI_Toolkit.ijm (see "Tips for frequent use" for how to add the Toolkit to Startup Macros)
  3. Plugins > Macros > Spectral RTI [n1] (or press 1 on the keyboard number pad)

Consult Preferences
The following settings are remembered from the configuration file or a previous run. Edit or clear as desired.

This will be empty on the first run and subsequently will allow you to review settings that are not frequently changed.

Choose a directory

Locate the project directory folder that contains the sub-folders Captures-Hemisphere-Gamma, etc.

Select tasks

Select Tasks
Select the tasks you would like to complete

Adjust brightness

Adjust brightness of hemisphere captures?
Adjust brightness of hemisphere captures?

Note: this dialog needs to be rewritten for greater clarity

The hemisphere captures use the same exposure settings for all light positions, which is valuable for rendering RTI images, but leaves the static raking images too dark.

Apply adjustment to which output images?

Select area
Draw a rectangle containing the brightest white and darkest black desired then press OK (hint: use a large area including spectralon and the object, excluding glare)

Note: rewrite dialog to read "hint: exclude glare but include the darkest black and whitest white, typically on the color checker chart"

This option appears if "normalizing each image to a selected area" was chosen on the previous dialog.

Go back and document interaction if "multiplying all images by a fixed value" was selected.

Static raking option

Select light positions
Select light positions for static raking images

This option appears if "Static Raking" was selected under "Select Tasks" previously.

The prompt will list the images found in the Captures-Hemisphere-Gamma directory. Typically one might want low angles from four directions, minimally impacted by shadow from the copy stand. When using the MegaVision Arc for RTI the positions A03, A13, G05, and G13 are good choices.

Options for Extended Spectrum

Assign Narrowband Captures
Assign each narrowband capture to the visible range of R, G, B, or none

This option appears if "Extended Spectrum" was selected under "Select Tasks" previously.

The prompt will list the images found in the Captures-Narrowband-NoGamma directory. The default options should work well if the filenames sort from shortest to longest wavelength. Roughly a third of the images with the shortest wavelengths should be assigned to the B (blue) category, the middle wavelengths to the G (green) category, and the longest to R (red) category. Use "none" if that image should not be used for Extended Spectrum processing.

Identify region of interest (for Extended Spectrum and Pseudocolor)

Select area
Draw a rectangle containing the colors of interest for PCA (hint: limit to object or smaller)

Situations and opinions differ with respect to how small the rectangle should be (e.g., should it include overtext), but it is at least clear that it should exclude non-artifact distractions such as the color checker chart.

Options for PCA Pseudocolor

Select sources for Pseudocolor
Pseudocolor images require two sources images (typically principal component images).
Method:

Options for custom processes

Choose a source for Custom Process

SpectralRTI_Toolkit can also take chrominance information from any stack of one, two, or three layers. The image should be in a subdirectory under the project folder with any name (e.g., CustomColor1).

Light position data

Select ROI
Draw a rectangle loosely around a reflective hemisphere and press Ok

The sequence is counter-intuitive here as a result of efforts to front-load interaction. Also, change dialog to "Select Area" to avoid jargon (Region of Interest). Also, capitalize OK for consistency.

This is the beginning of the process to create a Light Position (lp) file. Make sure the rectangle includes the entire reflective sphere.

Use RTI Builder to Create LP File
Please use RTI Builder to create an LP file based on the reflective hemisphere detail images in [Project Directory\LightPositionData\
Press cancel to discontinue Spectral RTI Toolkit or Ok to continue with other tasks after the lp file has been created.

Leave this window alone and start RTIBuilder (see above, Software installation)

RTIbuilder
Project Name
Operation Sequence

Give the project any name (e.g., shinysphere) and select Highlight Based (HSH Fitter) for the operation sequence. Click Start New Project

Click Open Folder and navigate to your Project directory and select the folder LightPositionData. Be sure to select that directory and not the parent directory or the jpeg-exports folder within it.

If the capture went well each image should have a highlight (glare spot) on it, but you have the option of deleting failed shots here. Then press Next.

Again, draw a rectangle around the sphere. Click Add Area then Detect Spheres

The software tries to identify the center and radius of the sphere. Check by clicking on different images, especially "Median." If it is necessary to change the radius or center be sure to click Set New Center before Next.

Click Highlight detection and a red cross will appear where the software detected the highlight. Click Next

If the field "HSH Fitter Location" is not already filled, click Find and navigate to the file HSHfitter. Take note of the location, as you will need it later.

Click Execute. The software may return "Fitting Completed" or an error message, but it does not matter as long as the LP file was created. Close RTIbuilder and return to ImageJ.

At the prompt "Use RTI Builder to Create LP File" click OK. The processing will run for several minutes, depending on options selected.

Locate Fitter

Locate Preferred RTI Fitter or cmd file for batch processing

Note: needs testing on MacOS

The straightforward option is to direct the prompt to the path RTIbuilder_v2_0_2\Fitters\HSHfitter\hshfitter.exe. See "Use deferredbatch.cmd" under "Tips for frequent use" below for the advantages of deferring HSH fitting.

This option does not change frequently so is remembered and set to default the next time the Toolkit runs.

Locate webGLRTIMaker.exe

If not already downloaded at the installation stage (above), download webGLRTIMaker.exe from http://vcg.isti.cnr.it/rti/webviewer.php.

Locate Jpeg2000 encoder

Locate kdu_compress or ojp_compress

If not already downloaded at the installation stage (above), download from http://kakadusoftware.com/downloads/, install, and point the Toolkit to the kdu_compress executable.

There seems to be a bug such that it keeps asking for the kdu_compress executable for each image. This is annoying but only applies to the first run. The selection will be remembered on subsequent runs of the Toolkit.

More options for PCA Pseudocolor

After some processing, the next questions will pick up from the selection made for PCA Pseudocolor (skip at Select Tasks, Generate and select using defaults, Generate and manually select two, or Open pregenerated images).

Select area Delete slices from the stack until two remain
(Hint: Image > Stacks > Delete Slice)
Enhance contrast as desired
Then press Ok

Change prompt heading from "Select area" to "Reduce stack." Also, enhance contrast should be mentioned before delete.

This results from the choice "Generate and manually select two"

The other paths are more straightforward but should be documented nonetheless.

The area of interest for PCA selected previously should be automatically selected. Feel free to modify the area.

If the images are too dark are light (which is likely) use Process > Enhance Contrast and check Process all slices.

The number of slices in the stack is equal to the number of narrowband captures. The last few are typically all black. Delete those, then delete the least helpful slices until two remain. Hint: navigate through slices with the , and . keys, which could also be thought of as < and > without shift.

Tips for frequent use

Include SpectralRTI_Toolkit.ijm in Startup Macros

TBA, see documentation in Fiji and ImageJ

Use deferredbatch.cmd

Let HSHfitter and WebGLRtiMaker run in batches overnight.

You can save waiting by deferring the time-consuming HSH fitting process. This could also be helpful if you need or want to use multiple computers or operating systems. First, create a file named deferredbatch.cmd. Run the Toolkit and clear the selection for preferred fitter. When prompted to identify the fitter, point to the cmd file. Now the toolkit will add commands to the batch file rather than running the fitter. When you want to start the batch, execute the cmd file. If the HSH Fitter and WebGLRTIMaker (if desired) are not in your path you could add them in the operating system or as an initial command in the cmd file.

Reuse LP files

When using a handheld flash a new LP file must be generated for each object (but not each color process). The beauty of a fixed dome or stable arc is that the LP file can be created once from a calibration sequence and reused.

Edit the LP file created from the calibration sequence and replace the file path up to the position identifier with the word "canonical." For example, the first few lines of a reusable LP file might read:

56
canonical-A01_043.jpg 0.17889495 -0.9825294 0.05130836
canonical-A03_044.jpg 0.46035585 -0.8594868 0.22215976
canonical-A05_045.jpg 0.7653712 -0.56276625 0.31225163
canonical-A07_046.jpg 0.90808934 -0.28366238 0.3080737
canonical-A09_047.jpg 0.9675771 0.09901111 0.23236032

If some images are rotated (but the capture apparatus remains fixed) LP files appropriate for other rotations can be derived by switching columns and inverting values, as appropriate.

x90° = yy90° = -x
x180° = -xy180° = -y
x270° = -yy270° = x

For any project directory with the same light positions, simply create a sub-directory LightPositionData and copy the canonical lp file into the directory.

As of July 2017, the canonical (reusable) lp files must be created with external tools such as text editors with regular expression search and replace. This would be a simple feature to add to the SpectralRTI_Toolkit. The place to do it would be right after accepting that RTI Builder has run. Relying on metadata to store the needed rotation is too prone to error. The regular expressions for files created by PhotoShoot are:
s/^.+(-[A-G][0-1][0-9]_[0-9]+\.jpg) (.+) (.+) (.+)$/canonical\1 \2 \3 \4/ (no rotation)
s/^.+(-[A-G][0-1][0-9]_[0-9]+\.jpg) (.+) (.+) (.+)$/canonical\1 \3 -\2 \4/ (rotation 90°)
s/^.+(-[A-G][0-1][0-9]_[0-9]+\.jpg) (.+) (.+) (.+)$/canonical\1 -\2 -\3 \4/ (rotation 180°)
s/^.+(-[A-G][0-1][0-9]_[0-9]+\.jpg) (.+) (.+) (.+)$/canonical\1 -\3 \2 \4/ (rotation 270°)
s/--//g (all rotations)