Next: 8. The Interactive Segmentation Up: 2 B. Anatomical Image Previous: 2 B. Anatomical Image Contents

Subsections

7. The Image Processing and Histogram Tools

7.1 Introduction

Image Manipulation or Image Processing is the process via which we take one image as an input, perform some operation on it and then output a new image as an output. For our purposes an image is a large three-dimensional matrix of numbers where each cell in the matrix (or voxel element or voxel) also has ``size'' corresponding to the voxel size. There are a number of different categories of procedures - the following is an attempt at a crude taxonomy:

Intensity Manipulation - this type of operations take an image and processes each voxel separately based on simply its intensity. A simple example of this is image thresholding.
Regional Intensity Manipulation - this type of operations replaces the intensity at a voxel by some function of the intensities of a (mostly) small neighborhood of voxels around this. An example of this type of operation is Gaussian smoothing.
Image Resampling/Resizing - here the goal is to generate an image whose dimensions are different from the input image. For example, in image resampling our goal might be to produce an image of resolution 2x2x2 mm from an input image of resolution 1x1x1 mm.

Most BioImage Suite applications include an image processing option on their menu bars. This enables access to two tools, the Histogram Control and the Image Processing Utility Control which are described in this handout.

7.1.0.0.1 Image Type - Not All Images are created equal:

While in memory, each image is stored as a large array of numbers. These arrays have a `type' such as integer/float etc. The type of the image imposes some restrictions as to what operations can be done to it successfully. For example if an image is of type `short' it can only store integer values in the range -32768:32767. Consider the following two examples:

Example 1: Our ``short'' image has minimum intensity 0 and maximum intensity 4000. We cannot multiply this by a factor of 10 (e.g. under Math Operations) as the upper value will exceed 32767 (10x4000=40000!) and will result in weird and incorrect results.

Example 2: Our ``short'' image has minimum intensity 0 and maximum intensity 1 - this could be generated by a thresholding operation etc. This image can not be usefully smoothed with a Gaussian filter as the desired output would have values like 0.5 which can not be stored in a short image - as this can store only integers and decimals are ignored.

The most common types are:

Unsigned Char or Uchar - one byte per pixel, integers only, range 0:255.
Signed Char or char - one byte per pixel, integers only, range -128:127.
Unsigned Short or Ushort - two bytes per pixel, integers only, range 0:65535.
Singed Short or Short - two bytes per pixel, integers only, range -32768:32767.
Float - four bytes per pixel unlimited range, can store decimal points with a finite precision.
Double - eight bytes per pixel, same as float but with ``double'' precision.

7.2 ``Image'' and ``Results''

All BioImage Suite viewer applications store at least two images in memory at the same time, labeled as ``Image'' and ``Results''. This can be accessed under the Display menu. When an operation is performed on an image, the results are saved in the ``Results'' image of the viewer. This allows you to revert back to the original image, which still resides in ``Image''. In order to perform further operations on a result image, you must choose (Display | Copy Results to Image) - see Figure 7.1 (left) . This will overwrite ``Image'' with ``Results'' display. This process can be undone, uncommitting any changes that have been made and reverting back to the image in memory before copying results, by using the (Display | Undo Copy Results) command. The top two commands in the ``Display'' menu, ``Image'' and ``Results'' simply select which image is shown. Choosing one does not delete the other - the other image is still there. It can be accessed by switching back. It is important to note that most of the operations provided by the Image Processing toolbox and other analysis tools take the ``Image'' as an input and send their output to the ``Results''. In order to work with them further, the results must be copied to ``Image''.

**Figure 7.1:** **Left:** The Display menu. **Right:** The Image Processing Menu.

**Figure:** The Histogram Control Window (left), the Image Processing Control Window (right).

7.3 Histogram Control

The Image Histogram control (shown in Figure 7.2) allows you to see how intensity values are distributed in your image. It can be accessed by choosing the (Image Processing | Histogram Control) menu option in the viewer window (Figure 1 - Right image). This brings up the histogram control window, which shows the intensities in the image distributed among a set of discrete bins.

7.3.0.0.1 Intensity Range:

You can edit the range of intensities displayed in the image (and thus, displayed in the histogram control). Simply edit the values in the ``Range'' fields, and click Update . Hitting Reset resets the range to show the entire range of intensity bins in the image.

7.3.0.0.2 Normalize

This performs a specialized intensity normalization function - this functionality is mostly obsolete now, see the Image Processing/Piecewise Map section for a replacement.

7.3.0.0.3 Histogram Equalize:

Histogram Equalization attempts to flatten the histogram (i.e., distribute intensities evenly across a range), thus providing good contrast in the image. Clicking the Histogram Eq button redistributes the intensities such that all bins below the grey cursor become zero, and all bins above the white cursor saturate to the value at the white cursor. The intensity distribution for the region between the cursors is flattened.

7.3.0.0.4 High Res

If the ``High Res'' checkbox is disabled, disabled the histogram is obtained by sampling the image at a lower resolution to make the process faster (a reduction of a factor of 4 in each dimension). The shape of the histogram is approximately correct but the bin counts are roughly a factor of 64 off).

7.3.0.0.5 Show Max

The ``Show Max'' checkbox allows the user to toggle the functionality of showing or hiding the 'tallest' bin. In some cases, the tallest bin is the background and ends up obscuring the interesting part. Figure 7.3 shows an example of the ``Show Max'' checkbox being checked for the same histogram as Figure 7.2.

**Figure 7.3:** The ``Show Max'' box is checked which shows the tallest bin. In this case, the tallest bin is the background. Figure 7.2 shows a histogram for the same image with the Show Max checkbox unchecked.

7.3.0.0.6 Save

The ``Save'' button saves the histogram as a text file.

Mouse Control in the Histogram Control Window

Left button: Use the left mouse button to click on and drag either the grey or white bar to the desired location. Also, clicking and holding with the left button anywhere on the histogram changes the top display bar to show that bin and its value.

Middle button: Clicking on the histogram with the middle button places the grey bar at the clicked location.

Right button: The right mouse button places the white bar. Right-click at any location in the histogram control window to make the white bar jump there.

7.4 The Image Processing Control

This is a complex multi-tab control which encompasses a variety of image processing functionality. This control, which is shown in Figure 7.2 allows the user to quickly navigate to the various image processing functions available using the tabs on the left. These match the choices in the Image Processing menu. At the bottom of this window are a few buttons that commit the results of the operations to the image:

Copy Results to Image: This button is equivalent to the (Display | Copy Results to Image) menu command (see Displaying and Working with Results). You must click it after any operation performed in the Image Processing Utility window if you want the results to be permanent and able to be operated on by subsequent processing steps. Undo Copy Results: Undoes the copying of the ``Results'' to ``Image'' - uncommits results. Unload Results - Removes all images from memory (from this control) to save memory. Close - Closes the Image Processing Utility window. We describe each of the tabs next.

7.4.1 The ``Smooth'' Tab

Gaussian Smoothing: Image smoothing operations typically replace the intensity value at a voxel with a new value that is a function of the intensity of this voxel and its neighboring voxels. The most common operation which is ``Gaussian smoothing'' replaces the intensity at a voxel with a weighted sum of it and its neighbors where the weighting is defined using a Gaussian function (or kernel). The degree of the smoothing is directly dependent on the shape of this kernel, which in turn is function of the standard deviation or ( $\sigma$ ) used.

The value of sigma is often specified either in voxels (which is what is used in the BioImage Suite GUI), or using the so-called ``Full-Width at Half-Maximum'' (FWHM) specification. As stated in a Wikipedia page ``A full width at half maximum (FWHM) is an expression of the extent of a function, given by the difference between the two extreme values of the independent variable at which the dependent variable is equal to half of its maximum value.'' Given a FWHM of w we can obtain the equivalent Gaussian $\sigma$ using the relationship: $\sigma= w / 2.355$ . Additional scaling may be needed if one is in and the other in voxels.

In BioImage suite one can perform Gaussian smoothing by (i) setting the filter size $\sigma$ in voxels and pressing either ``Smooth 2D'' for 2D smoothing or ``Smooth 3D'' for full 3D smoothing of the image. Use ``Smooth 3D'' unless you have a really good reason not to.

7.4.1.0.1 Diffusion and Diffusion Gradient Smoothing

Gaussian smoothing blurs across edges. Nonlinear smoothing techniques were developed to avoid this. The algorithms available in BioImage Suite (based on original work by Perona and Malik) attempt to perform smoothing only within relatively homogeneous areas - and suspend smoothing if the intensities (or intensity gradients) of two adjacent pixels differ by an amount greater than the threshold.

Diffuse This algorithm takes into account the raw intensities of pixels in evaluating the threshold requirement for smoothing. Thus, only the Thr field and Factor field must be specified. Click the Diffuse button to make the magic happen. DiffuseG To use the Diffusion Gradient smoothing algorithm, set the Grad Thr field along with the Factor and click the DiffuseG button. This will only smooth across pixels that do not have an intensity gradient difference greater than the threshold.

7.4.1.0.2 Median Filtering:

Median filtering operates by a similar mechanism to Gaussian smoothing, but instead of using an average of the surrounding voxels to generate the new smoothed voxel value, it uses the median value of all the pixels in the median window radius. Thus, to use the Median Filter tool, simply set the window size (``Median Window Radius'') and click either Median Filter 2D to compute medians using only in-plane voxels, or Median Filter 3D to compute medians using a 3-dimensional window that includes voxels in adjacent slices. Median filtering is useful for removing what is known as ''salt and pepper'' noise which is uncorrelated one/of noise specs on the image.

7.4.1.0.3 Voxel Grouping:

Voxel Grouping combines groups of voxels together, yielding a larger voxel with a single value. This simulates a lower acquisition resolution for the image, and is useful for comparing high-res images with lower-res images. The ``In Plane'' option sets the number of pixels to group into a single pixel within a slice, while the ``Slice'' option dictates the number of slices across which grouping will be performed. After setting these values, click Voxel Group! to do the grouping.

7.4.2 Edge Detection

The Edge detection tab has facilities for computing edgemaps of the image. We use a simplified form (single-scale) of the famous Canny edge detector. In essence this first computes the gradient of the image at a certain scale (set by the sigma variable) and additionally performs non-maximal suppression to thin the edges.

7.4.3 Cropping and Reorienting

7.4.3.0.1 Reorienting an Image: The Axes:

Reorienting an image is, essentially, choosing along which pair of axes the planes of the slices will lie. Thus, in BioImage Suite, if you define your image as ``Axial'', the ij plane will be in the axial slices. Likewise, choosing ``Coronal'' defines the coronal plane as the ij plane. This applies to sagittal as well. Thus, if your Image has the i and j axes (red and green lines) in the coronal plane, and you would like them to be in the axial plane, simply go to the ``Reorient/Crop'' tab or choose the (Image Processing | Reorient/Crop) choice from the menu, and click the ``Axial'' radio button. The image axes will be redefined.

7.4.3.0.2 Relabel:

Clicking the Relabel! button does not affect the image, except to alter the image header so that the views are labeled correctly. Thus, if you have an image in which, for whatever reason, the view that is actually coronal is labeled as ``Axial'' in the viewer, simply change the label settings using this feature.

7.4.3.0.3 Cropping an Image:

Cropping an image comprises removal of peripheral pixels along any of the three axes. This reduces the size of the image, significantly reducing computing time for subsequent image calculations, especially registration. Thus, it is advantageous to remove any excessive blank space from the edges of your image before performing other operations.

To crop an image to your specifications, first determine the boundaries of the region you want to retain, using the cursors in the viewer (See ``Navigation controls in the viewer control panel'' in Working with the Orthogonal Viewer Display). Then, input the max and min values for X,Y, and Z (you may choose to use all of these, two, or just one) using the ``Range'' fields. The t parameter for cropping is used to crop images with a time dimension. Thus, if you choose a t range, only the images from within this set of the time series will be included in the result. Now, click the Crop button, and the image dimensions will change to fit the ranges you have specified. The lower bound in each dimension will become zero, and all data below it will be cut out, while the upper bound will become the maximum in that dimension of the image (the new maximum will be equal to the old maximum minus the summed amount cropped from both sides).

The ``Rate'' field specifies a sampling rate that can be applied to the image simultaneously to the crop operation. Thus, if you put two into the Rate field for X, the resulting image will be resampled using only every other voxel in the X direction, lowering its resolution in this dimension, but leaving the others unchanged.

Blank operates almost the same as Crop , except it does not change the image dimensions. Instead of deleting the voxels outside the range altogether, it simply sets their value to zero.

The AutoCrop button can be used to automatically perform a crop of the image where all the region to be cropped has intensity equal to zero.

7.4.4 Reslice

Reslicing reevaluates the image as individual voxels and redefines the slices, effectively applying an affine transformation to the image. Thus, with this tool you can flip and rotate your image in all three dimensions.

7.4.4.0.1 Transformation:

Applying transformations to the image is very easy. Simply choose the flip or rotation operation you would like to apply from the ``Transform:'' menu:

Identity - the identity transformation does not change the image . Flip Slice Order(Z) - the slice order is reversed, flips in the Z-axis. FlipX - flip image in the X-axis. FlipY - flip image in the Y-axis .Transpose - performs matrix transposition on the image; that is, permutes X and Y: {X = Y and Y = X}. Essentially switches the X and Y dimensions. Flip Transpose - - Transposes as above, but inverts as well: {X = -Y and Y = -X}. Essentially switches X and Y as well as performing a flip. Custom Rotation - Enables the RotX, RotY, and RotZ slider bars, allowing you to rotate the image any specified amount in any dimension After selecting a transformation (and setting the rotation sliders if ``Custom Rotation'' is selected, hit the Reslice! button to perform the operation. The output will be shown in the viewer (stored in the results display).

7.4.4.0.2 Sampling:

The sampling factor determines the voxel size on output. Thus, a factor of x2.0 results in an image that has voxels twice the size of the original. The overall image dimensions are unchanged, the resolution is simply changed by the sampling factor. A sampling factor

1.0 will result in an image that has higher data density than the original, leading to increased computing times, while sampling factors

1.0 will lower density, lowering subsequent computing time requirements.

Use the ``Sample factor:'' menu to choose a rate at which to sample the image. The entries preceded by ``x'' (e.g. ``x0.5'') use a scaled voxel size based on the current voxel size, while those without the ``x'' (e.g. ``1.0'') are absolute sampling rates. These are isotropic sampling parameters; if you want to sample at different rates in different directions, select the menu entry ``Custom'', and set the sampling rate for each dimension individually using the X,Y, and Z sliders.

**Figure 7.4:** The Threshold Image Control. The thresholding range can be set using the two sliders (A1) or the entry box (A2). The row marked B has controls for specifying whether the output is binary or not (and whether to use 1 or 100 as the output of a binary thresholding - this is controlled by the ``x100'' checkbox). The ``Inverse Output'' checkbox determines whether the allowed range is the once specified or its compliment. Thresholding via range is done by pressing the ``Threshold'' (C) button, whereas thresholding via a list (specified in A2) is performed by pressing the ``Threshold By List'' (D) button.
**Figure 7.5:** BioImage Suite can be used to blank out regions of the image as shown above. These are specified using the current position of the oblique slice. This is accomplished by (i) positioning the oblique slice and (ii) pressing one of the two buttons ``Oblique mask Above'' or ``Oblique Mask Below'' in the threshold control - these are marked as E in Figure 7.4..

7.4.5 Thresholding Images

Thresholding images is essentially a process where the value of the intensity outside a certain allowed range is mapped to zero. The value in the allowed range is either left untouched or set to a fixed value (binary output). BioImage Suite allows two types of thresholding: (i) by range (setting the upper and lower value of the allowed range) and (ii) by list - which specifies a list of allowed values. For a description of all the functionality see Figures 7.4 and 7.5.

7.4.6 Piecewise Map

The piecewise map utility allows you to specify a piecewise linear function that maps input voxel intensities to new output intensity values. Thus, you can create a custom mapping function to generate many different types of output images. This essentially takes a set of input/output pairs and performs linear interpolation between them. In order to view the equation as it is being generated, click the Show Equation button, which will bring up a console showing the piecewise mapping function as defined by the input and output fields.

An example is shown in figure 7.6.

**Figure 7.6:** An example of using the Piecewise linear mapping tool to adjust image intensities. In this case, an image is remapped such that voxels with intensity below 0 are kept at 0, voxels between 0 and 3000 are unchanged and any voxel with intensity greater than 3000 is mapped to 3000. The popup (PXTkConsole) shows the result of pressing the ``Show Equation'' button, which describes the function used to remap the image intensities.

7.4.6.0.1 Output Type:

The output type is the data type to be used to store the output values. The default is ``Same as Input'', which is self-explanatory. The other choices are arranged in order of increasing precision.

Click the Multi Linear Operation button to perform the piecewise mapping operation on the image. As usual, the results will be placed in the ``Results'' image of the viewer, and must be copied to the ``Image'' image before any further processing can be done.

7.5 EXAMPLE: Reorientation of Images

Many 3D whole brain acquisitions are taken in the sagittal orientation. If your reference brain has a different orientation (i.e. the MNI Brain is in the axial orientation) than your single subject brain, you must Reorient the single subject brain to match the orientation of your reference brain. The non-linear registration described later cannot be run between two different orientation types.

**Figure 7.7:** Methods used to reorient an image using the Image Processing Utility.

The step-by step procedure, with reference to Figure 7.7, is as follows:

Check the header of your window, it will list the following information about your image: Viewer being used, the name of the file that was loaded, the orientation and the image size. In this case the orientation begins as Sagittal.
Choose (Image Processing | Reorient/Crop Image)
A new Image Processing Utility window will appear. Choose proper orientation (axial if using the Colin Brain). Click Reorient
Click Copy Results to Image
Save File (you cannot continue onto the Brain Extraction unless you save the current image). Click (File | Save)
A new window will appear where you can type in your new filename and click Save.
Once saved, the header of your window will display the new filename as well as the new orientation.

Next: 8. The Interactive Segmentation Up: 2 B. Anatomical Image Previous: 2 B. Anatomical Image Contents