- In pxitclbrainregister and pxitclmouseregister:
- Image-intensity based methods via the Registration/Overlay Tool accessible from the General submenu graphical user interface.
- Point-based methods via the RPM Registration Control, similarly accessible from the General submenu graphical user interface (Point Based Registration)
- Registration Command Line/Batch Mode Utilities -- see the command line utilities page for more details.
The Registration Tools
Fig. 1 The pxitclbrainregister Menu Bar. In the pxitclbrainregister program, this menu contains options that direct the many of the software's registration, transfer, and multisubject operations.
The idea behind the two viewer style of display is to show the relationship between two images, as defined by the transformation between the two. Thus, when the "x-hairs" box is checked in both viewers, if you move the crosshairs in any view style in either viewer, the crosshairs in the other viewer move as well, to reflect the corresponding point in its image, according to the current transformation matrix that has been loaded, as described here. (Unchecking the "x-hairs" box in either viewer disables the movement of the crosshairs in that viewer, allowing you to lock them in place, while you navigate in the other viewer. This will be useful in manually defining crude initial transformations; more on this below.) The main menu of these applications has four submenus
- Viewers -- opens either the reference or transform viewer, both of which are orthogonal viewers by default but provides access to the Simple Viewer (an instance of the Mosaic Viewer) which can display multiple identically oriented slices. See the viewers page for more details on individual viewer functionality. Both the Reference and the Transform Viewer also have the full complement of Image Processing and Segmentation functionality.
- Transfer -- commands for moving images from the reference to the transform viewer and vice versa or to swap them.
- Registration -- provides access to the Registration/Overlay tool and the Point Based Registration controls.
- Multisubject -- a dedicated Multisubject control that has functionality for generating composite functional maps. (The pxitcldualmultisubject app has a second multisubject control that facilitates group comparisons). There is no currently multisubject control in the mouseregister application.
A note on registration and transformation conventions
A great deal of confusion can arise if care is not taken to
note which image defines the space of the
registration, and which image is being transformed by it. The image in
the Reference Viewer defines the space, and the image in the Transform
Viewer is "registered TO it". Thus, "a transformation is calculated
FROM the image in the Reference Viewer TO the image in the Transform
Viewer". This transformation is saved in a matrix, which, when applied
to the Transformation Viewer image, will place it into the space of the
Reference Viewer image. These terminologies become important when
building trees with the DataTree
Tool which acts as a master control over and above
the pxitclbrainregister application.
The Registration/Overlay Control
Loading and Saving
Transformations
In the menu bar shown in Fig. 1, choose
(Registration | Transformation) to bring up the
Registration/Overlay Tool. The Registration/Overlay Tool window
contains the tools needed to register and compare a pair of
images. All transformation and registration operations use the
Reference/Transform convention described in the box
above. Transformations are generated FROM the Reference image TO
the Transform image, and when loading a transformation, you should
remember this scheme. The top section of the Transformation window in
Fig. 2, above, is a simple file load/save mechanism which allows you
to save the transformation that is currently defined between the
reference image and the transformation image. The 4x4 matrix below the
filename reflects the transformation, if it's linear. The Load
button brings up a file selection dialog box that lets you load a
previously saved transformation file of types Matrix (*.matr),
Grid/Combo (*.grd), PolySimilarity Transforms (*.psim),
Thin Plate Spline Transforms (*.tps), and Old Matrix Files
(*.mat). (Note: This last type, *.mat, is the same format as
*.matr, but with a different suffix. This has been phased out due to
the obvious potential confusion with MATLAB .mat files, which are not
used by BioImage Suite) When you load a transformation, the info
box showing the transformation matrix changes to reflect the
parameters of the transformation. If the loaded file contains a linear
transformation matrix, this is shown; if the loaded file is a Grid or
Combo file, specifying a non-linear transformation (*.grd),
then the box shows the Grid Dimensions and Spacing, as well as the the
linear transform matrix dimensions for Combo transformations.
Fig. 2 The Transformation Control Window. This window, a tab in the Registration/Overlay Tool, contains functions related to the loading, saving, and verification of transformations, as well as reslicing and stitching functions. It can be accessed directly from the (Registration | Transformation) menu command in the pxitclbrainregister menu bar.
Transformation List
The listbox at the left side of the "Transformation" tool space is a holding space for multiple transformations, which can be loaded into memory simultaneously, but applied one at a time to the images in the viewers. When a new transformation is calculated, it will appear here. You can manually add a transformation (using the Add button), or delete a transformation (using the Delete button) to/from memory.Clear
Hitting the Clear button removes whatever transformation is currently selected in the Transformation List, and replaces it with the identity transform. This is reflected in the matrix display.Inverse
With a transformation in memory (i.e. selected in the Transformation List), hitting the Inverse button computes the inverse transformation, and replaces the current transformation with it.Manual Transformation Control
You can use this tool to manually change translation, rotation, and scaling factors in all three dimensions. The tool can be accessed by clicking the Manual button underneath the matrix display in the transformation control window (See Fig 2) which will bring up the small manual control popup (Fig 3) containing input fields for the above parameters in each dimension. The fields will contain the current values for these parameters, which you can edit freely. The X- Y- and Z- shift parameters are in the native scale of the reference image; the rotation parameters are in degrees; the scaling parameters are percentages. After inputting the desired values, hit the Set Xformation button to compute a 4x4 linear transformation matrix from your values, and send it to the transformation control. The transformation matrix will update to show your changes. This will not affect the images, however. To apply the transformation, you must hit the Copy Results to Image in the transformation control window. Equivalently, and as a shortcut, you can hit the Set & Apply button in the manual transformation control popup, which will directly apply your changes to the transformation and apply the result to the image.
The Extract! button lets you grab the shift, rotation, and scale parameters from the transformation matrix currently loaded in the transformation control. This way, you can load a transformation as described above, and alter it manually. To reset all values to their defaults (0 translation, 0 rotation, 100% scale), press the Clear Values button.
Auto Crosshairs: Sometimes it is easier to define a translation by placing cursors than by estimating numbers. The Auto Crosshairs button gives you this capability. If you can locate the same point in both of your images, you can use it to direct the manual setting of the translation parameter. In the viewers, simply place the cross hairs over the point in one image, uncheck the "x-hairs" checkbox in that viewer to lock them in place at that point, and place the crosshairs over the same point in the other image. Then, back in the manual transformation control popup, click the Auto Cross Hairs button, and the X- Y- and Z- translation parameters will be updated to specify a translation that maps the two cross hair locations together (i.e. maps the crosshair point in the Transformation Viewer onto the point in the Reference Viewer. This technique is often easier than guessing at translation parameters and then navigating in the image to confirm them.
The manual transformation control popup is most often used as a rough alignment tool, as a preprocessing step to automatic registration. Besides reducing subsequent registration processing time, it also makes the registration more reliable, since corresponding regions are in closer relative proximity, increasing the likelihood of their overlap, and helping the registration algorithm to converge properly. It is strongly advised that you make an attempt to line your images up using the manual control if they are in radically different orientations or scales.
Reslicing
Once you have registered two images, you can put the resulting transformation information to work by "Reslicing". Reslicing takes the loaded transformation matrix or grid and applies it to the Transformation Image, so that it "looks like" the Reference image. That is, it places the Transformation Image into the space of the Reference Image. This means that the Transformation Image gets diced up, analyzed, and rebuilt using voxels that are the same size as those of the Reference Image, and occupy an equally sized and shaped image volume. This process is wholly dependent on the transformation matrix that links the two images. To do it, simply hit the Reslice! button in the transformation control window. The image in the transformation viewer will be resliced into the space of the image in the reference viewer, and the output will be displayed (saved in the "Results" display of the transformation viewer).
Fig. 4 The Reslice Options Section
in the Transformation Control
This section contains the functions for reslicing images into other
images' spaces, and various options that relate to this operation.
(Note: The reslicing functionality has been incorporated into the Data Tree manager tool, which you may find
more intuitive to use, especially with multiple images.
Reslice Options
There are a few options that apply to the Reslicing function, which you can control in the "Reslice Options" section of the transformation control:Interpolation
Choose the method for calculating values between those values that coincide in spatial location with the original values. Since the new image comprises differently sized voxels that occupy a new overall space, some method of interpolationg must be employed. You may choose Nearest Neighbor, Linear, or Cubic.Wrapping
If the "Wrap" checkbox is checked, the image will be allowed to wrap around the volume (i.e. anything that falls off the left side will appear back on the right side, etc.).Computing Measures of Registration Quality
The other three buttons in the "Reslice Options" section are functions that compute measures of quality after a registration has been computed. Thus, after a transformation is loaded in the transformation control (either by clicking the Load button, or by having just completed a registration), these functions will yield a number of different numerical parameters that describe your registration. All operations are comparisons between two images.Similarity
Clicking the Compute Similarity! button brings up a console window that contains values for a number of measures of registration quality: Joint Entropy, Correlation, Normalized Mutual Information, Sum of Squared Differences, and Difference Entropy. These provide a quick diagnostic of how reliable the transformation loaded is.ROI Stats
ROI stats give information about mean and standard deviations within discrete regions defined by a Region Of Interest image, which is a mask image that is registered to the reference image. Thus, you should load your ROI definition image into the Transform Viewer, your reference image in the Reference Viewer, and register them. Then click the Compute ROI stats button to get statistical measures out for each region in the ROI definition image.Overlap
The Compute Overlap button thresholds both loaded images at 50% of their maximum value, and then computes the percentage of overlap between the resulting regions. This is very useful with binary valued images, but can also be a good diagnostic of registration quality in other images.Stitching
The stitching function allows you to combine two adjacent images, provided that you are able to register them first. Thus you need a region of overlap between the two images. The result is a single image. The x, y, and z radio buttons let you choose which dimension to stitch in. The "Flip" checkbutton option lets you toggle which image comes first (i.e. lies at the top, left, or front, depending on the dimension of stitching). The "Pad" value is editable, and lets you specify how much to grow the image by before stitching (since you need to make the image bigger to fit both pieces into it).Additional Procedures
This section contains a few extra functions that are associated with reslicing, and may be useful for evaluating the quality of registration operations. Often, the quickest way to check your registration is to create a blended or checkerboard image, and be sure that structures that cross image boundaries remain reasonably continuous. All of these operation apply the current transformation loaded into the transformation control.Checkerboard
Creating a checkerboard does just this: creates a combined image, in which alternating cube-shaped sections are contributed by each of the two images registered together. The "Spacing" option lets you control the size of the checkerboard spaces. The "Normalize Images" box should usually be left checked, since it normalizes the intensity scales of the two images, so that one set of checkerboard spaces does not appear much much brighter than the other.Blended Images
A blended image has contributions from both images, but they are complementary at every point. The effect is that both images become semi-transparent and get merged. Adjusting the "Opacity" slider button lets you adjust which image dominates in the result. A value of 0.30 means that the image is contributed to 30% by the image in the transformation viewer and 70% by the image in the reference viewer. A value of 0 would result in an image identical to that in the reference viewer. Check the "Auto Update" box to have the blended image result update as you move the opacity slider. A. 
B.
Fig. 5 Combining Images as a test
of Registration Quality.
The combined images provide a quick diagnostic of the quality of the
registration between two images. Part A shows a
checkerboard image, created by merging alternating cubic sections of
two images. Areas discrepancy can be easily picked out, and areas that
line up are also readily apparent (Trace the contours of the corpus
callosum, for example: They are fairly continuous, even as you cross
from one square in one image to the next in the other image).
Part B shows a blended image, created from a whole
brain MRI and a scout slice MRI that has been resliced into the whole
brain space. The image comprises 90% scout slice image and 10% whole
brain image (the scout slice has no data outside the narrow band seen,
thus outside this region only the whole brain image is visible). Again,
by looking at the interface between the two regions, continuities and
discrepancies are easily seen.
Masking
Masking is the process of creating a binary image, i.e. an image that contains only two distinct values. Thus separate regions are created: those which have the value zero, and those which have the non-zero value. The Mask Image! button thresholds the image in the Transformation viewer at 50% of its maximum value, and then sets any voxels above the threshold to 100. This creates distinct regions in the image, creating a mask that can be used for region of interest definitions. The "Dilation of Mask" option allows you to specify how much to grow the mask image by after thresholding.Visualize Transformation (Jacobian)
This tab has two parts. The top part can be used to compute and overlay the Jacoabian of the transformation. The overlay part is similar the the Functional Overlay section -- the user is refered to this for more details. Along the bottom of the Compute Jacobian frame there are 5 buttons with the following functionality:- Compute Jacobian -- this computes the determinant of the Jacobian (|J-1|*100) at each voxel, which is a measure of local contraction/expansion with respect to the reference image.
- Compute Tensor -- this computes the full 3x3 tensor.
- Compute Strains -- this computes the principal strains which are the eigenvalues of the symmetric component of the Jacobian matrix.
- Display Jacobian -- displays the Jacobian in the Transform Viewer
- Overlay -- creates an overlay of the Jacobian on the current reference image -- much like the functional overlay tool below.
These options are invoked under Registration|Visualize Transformation or the Jacobian button in the Registration/Overlay tool.
More later.
Functional Overlay
The functional overlay control enables the overlaying of functional images onto anatomical images, to enable the combined display of structure and function. The anatomical image in this case is the Image (i.e. not results) of the Reference Viewer and the functional image is the Image of the Transform Viewer -- the later is first resliced to match the anatomical image using the current transformation.
The basic principle used for the overlay is that the users sets a threshold for what constitutes significant function using the Low Threshold slider and then saturates the functional data at the level set by the High Threshold slider.
Consider, for example, the case where the functional map has range -3000 to 3000 the anatomical image has range 0 to 256 and the F1 colormap is used. The F1 Colormap maps the anatomical image in the range 0 to 55, the negative functional data (if selected) in the range 56-59 (56 is the most significant) and the positive functional data in the range 60-63 (with 63 being the most significant). If the thresholds are set to 2000 and 2500 respectively, then the output of the overlay tool will be:
- The anatomical image if functional image < 2000 and functional image > -2000 (i.e. insignificant activations), or if the anatomical image intensity (scaled in the range 0..55) is less than Inten Threshold. This last step masks spurious activations outside the brain or in the ventricles.
- Otherwise
- If function is positive and over 2000 then 2000->60, 2500 -> 63 and anything over 2500 will saturate to 63.
- If function is negative and under 2000 then -2000->59,-2500->56 and anything less than -2500 will saturate to -56.
The Clustering slider performs a cluster filter operation on the functional image. Only clusters greater than the selected size in voxels will be overlaid on the anatomy.
If the input functional image is four-dimensional, i.e. there are multiple frames/tasks in it, the user can select to overlay only one frame by enabling the Single Component checkbox. The frame (component) is selected using the Select Component slider.
If all goes well the overlay is created using the CreateOverlay button. The Reslice button can be used to reslice and display the functional image only, if this is desired.
Image Compare
The Image Compare control can be used to either perform a straight image addition, subtraction and %change operations or compute a two-group t-test. It has four image controls labeled Mean1, Mean2, Standard Deviation 1 and Standard Deviation 2, where the images to be used for the comparison must be loaded.
There are three standard operations that only utilize the mean images namely:
- Mean1+Mean2
- Mean1-Mean2
- 100*(Mean1-Mean2)/Mean2
For the t-test computation, the standard deviation images and the size of the two groups must be also defined. The size of the two groups can be specified in the textboxes labeled N1 and N2 respecitvely at the bottom of the control. Invoking the Compute Tmap button will result in the computation of a voxelwise t-test between the groups represented by their mean and standard deviation images. The output image, which is equal to the t-score * 1000 is displayed in the Transform Viewer. Conversion of these t-scores to p-values can be perfomed using the T-distribution table tool that is accessible under the Help menu of the main pxitclbrainregister application.
Linear Registration (Intensity Based)
Fig. 8 The Linear Registration Controls. Above: the most common options under the Simple tab. Below: advanced options.
To perform a linear registration, first load the reference image in the "Reference" Viewer and then the target image in the "Transform Viewer". The resulting transformation will map coordinates FROM the reference TO the target, and when it is appled for image reslicing will move the target image to the space of the reference -- the overall goal of all registration being to move the target to look like the reference.
There are two sets of controls. In the Simple Controls (Figure 6 top), the user specifies the resolution (as a multiple of the native resolution of the reference image), as well as whether to automatically save and overwrite the resulting transformation. If the "Use Current" transformation checkbutton is enabled then the currently selected transformation in the Transformation List is used to initialize the registration parameters. If not then the transformation is initialized by mapping the "centers" of the two images. Three default operations are provided:
Rigid -- compute a full 3D rigid mapping
Rigid2D -- compute a 2D rigid mapping
Affine -- compute a 12-parameter affine mapping
Under the Advnaced tab (Figure 8 bottom) the user can set additional parameters such as the similarity metric (default=Normalized Mutual Information), the optimization method etc. Key parameters are the number of levels (for multiresolution optimization = 3) and the number of steps (different step sizes for computing the gradient etc. = 1). The "Old Style" optimization method is that used in the original paper by Studholme et al. If this is selected it is advisable to set the number of steps to 4 (default = 1).
Nonrigid Registration (Intensity Based)
Fig. 9 The Non-Linear Registration Controls. Above: the most common options under the Simple tab. Below: advanced options.
Non-Linear Registration allows for additional degrees of freedom and permits the more accurate mapping of images from different subjects into a common space. The methodology in this control derives from the work of Papademetris et al. (MICCAI 2004) which in turn is closely related to work by Rueckert et al. (TMI 1999). The computed transformation is parameterized in terms of a tensor b-spline grid with uniform control point spacing (e.g. 20.0 mm in the example on the left).
To perform a nonlinear registration, just as in the case of the linear registration above, first load the reference image in the "Reference" Viewer and then the target image in the "Transform Viewer". Again, similarly to the linear registration case, there are two sets of controls.
In the Simple Controls (Figure y top), the user specifies the resolution (as a multiple of the native resolution of the reference image), the control point spacing and whether to automatically save and overwrite the resulting transformation. The registration is invoked by pressing the "Compute Linear + Non Linear Registration" which first computes an affine mapping followed by a nonrigid mapping. The resulting transformation when saved in the .grd file format includes both the affine and the non-linear components.
Under the Advnaced tab (Figure y bottom) the user can set additional parameters such as the similarity metric (default=Normalized Mutual Information), the optimization method etc. Perhaps the most important of these parameters is the smoothness, setting this to a higher value ensures a more rigid mapping at the expense of less accuracy in the registration. Here the registration can be invoked in two ways. The first is as a straight non-linear registration using the "Compute Non Linear Registration" button -- in this case either the current transformation is used for initialization or the linear component of the registration is assumed to be the identity matrix. The second is as combo Linear+Non Linear Registration in the same way as for the simple controls.
Distortion Correction (Single Axis Distortion)
Fig. 10 The Distortion Correction Controls. Above: the most common options under the Simple tab. Below: advanced options.
The Distortion Correction functionality is essentially a constrained form of the non-linear registration in which the warping is assumed to be only in one direction. This reflects the case of distortion in echoplanar magnetic resonace acquisitions -- such as those used for DTI and fMRI. The distortion direction is assumed to be along the "Y-axis" of the image (which is typical of the phase encode direction), if this is not the case for the specific images, the axis can be change using the Phase-encode direction drop menu under the advanced controls (Figure 8 bottom).
Most of the functionality of the distortion correction algorithm is closely related to the nonrigid registration methods described immediately above. There are two key differences:
(a) The resolution can be set to be lower i.e. 2.0 or 4.0 since typically the reference image (anatomical) has voxel size of ~ 1mm and the target (echoplanar) has voxel size ~ 4 mm, in which case running the algorithm at 1 or 1.5 mm resolution is overkill. (b) if the Echo-planar image used is a spin-echo image with no signal loss the adjust drop menu can be used to enable the signal-loss conservation constraint (originally from the work of Studholme et al. TMI 2001).
Point Based Registration
Methods for nonrigid registration based on point features are described in the Point-Based Registration page.Batch Mode Registration
Command line versions of the registration programs, suitable for batch mode usage, are described in the command line utilities page.Appendix: What is Registration?
Image registration is the process of calculating realigning and warping factors that transform one image into the space of another. This allows for comparisons of corresponding regions of different images, as well as the creation of overlays. This function is important since acquisitions via different methods (MRI, SPECT, CT, etc.) result in differently sized, spaced, and oriented images. Furthermore, the comparison of images from different subjects, or a single subject before and after a morphological change, requires realignment of corresponding structures in order to extract meaningful information. The BioImage Suite software provides a variety of user-directed functions to perform these calculations, allowing you to register images in the manner most efficient and relevant to the information you are trying to obtain.Transformation Types
A transformation is the result of image registration; it is simply some function that maps a point in one image to a point in another. There are many different types of functions that can be used to specify this mapping, with varying degrees of associated flexibility, reliability and computational load.Rigid
Rigid transformations are used for the registration of images from the same patient that have no distortion, and need to be realigned to the same orientation for meaningful comparisons to be made. A rigid transformation comprises 3 rotations and 3 translations. Therefore, it is a linear operation, and can be fully expressed in a 4x4 matrix. This type of registration is most often the fastest to compute.
Affine
Affine transformations are a broader class of linear transformations than rigid transformations, in that they include parameters for stretches and shears as well as rotations and translations. Nonetheless, they can still be represented by a 4x4 matrix, and are relatively quick to compute. Thus, affine transformations are typically used as a crude approximation to nonrigid transformations, either for rough estimates of location, or as a preliminary step.
Piecewise Affine
A piecewise affine transformation is one step closer to a nonrigid transformation, in that the image is sectioned into discrete regions, and a different affine transformation is applied to each region.
Nonrigid or Elastic
Nonrigid transformations are used for registrations between images of different subjects, or images with distortions or actual physical differences (images of a patients brain before and after an operation, for example). These transformations are non-linear, and thus have no matrix representation. In fact, they have a great many different parameterizations, the results of which get saved in a large grid file.
References
1. J. Michael Fitzpatrick, Derek L. G. Hill, Calvin R. Maurer Jr. "Chapter 8: Image Registration" Handbook of Medical Imaging Vol. 2: Medical Image Processing and Analysis Ed. Milan Sonka, J. Micheal Fitzpatrick. Washington: SPIE Press, 2000. 447-513.