The Viewers and Colormap Editor

 


Introduction: The Viewers


Figure 1. The Viewers There are four core viewers in BioImage Suite. (A) The Mosaic Viewer which displays multiple images of the same orientation (e.g. multiple axial sections),  (B) The Orthogonal Viewer which displays a number of different combinations of either individual slices, linked orthogonal views (as shown in B), volume rendering, oblique slices etc. (C) The Objectmap Orthogonal Viewer -- used primarily in the SurfaceEditor and can overlay an image objectmap on top of the main image using transparency (controlled using the Mask slider) and (D) the 4D Orthogonal Viewer which is a true 4D viewer and can be used to play cine-loops of 4D images. The other viewers can display 4D images (a Frame Slider appears if the image is 4D) but they do not have movie playing capabilities.

The viewers are the heart of most BioImage Suite applications. This is where you are able to interact directly with your images. There are multiple viewing options, and a variety of coloring schemes. The viewers also seamlessly integrate with the registration and calculation tools. There are two main flavors of the viewer: The Simple or "Mosaic Viewer" (shown in Figure 1 A) and the "Orthogonal Viewer" shown in Figure 1B -- this last viewer has two extensions the "Objectmap Orthogonal Viewer" (Figure 1C) and the "4D Orthogonal Viewer" (shown in Figure 1D).

We described the viewers in more detail next, starting with the Orthogonal Viewer family.

Orthogonal Viewer Control


Figure 2. The Orthogonal Viewer. This is a complex control which lies at the heart of most BioImage Suite applications. 


The Orthogonal Viewer consists of essentially two parts, the viewer window (shown as A in Figure 2) and the viewer controls (B--H in Figure 2). The Viewer is capable of displaying different combinations of views as detailed in the display modes table below. The mouse can be used to pan/zoom the display as described in the Navigating the Display section below.

The controls (B--H in Figure 2) have the following functionality:
  1. The mode selectors. The drop menu "3-Slice Mode" is the Main Display mode control. The viewer can be switched from either a number of "2D" modes e.g. the 3-Slice Mode shown, to a 3D mode to combinations of 2D and 3D Modes. See the display modes table below for all the options. The drop menu "3-slice" mode is the 3D Mode Selector. This determines what is displayed in the 3D view (if it appears). These options include "none", orthogonal-"card" views, volume renderings, oblique slices and combinations of the above. The final drop menu "box" determines whether a box, or the coordinate axis are displayed in the 3D view.
  2. The two buttons marked "Volume" and "Oblique" bring up the Volume Rendering and Oblique Controls which control how volume renderings and oblique slices are displayed. See below for more detailed explanations.
  3. This is the coordinates frame, which can be used to display Talairach coordinates in neuroimaging applications. In the specific example the viewer cross-hairs are on the AC which has image coordinates 87,90,64 but Talairach coordinates 0,0,0 as shown in the black textbox -- see the Coordinates page for more details. The three checkboxes
    1. "x-hairs" -- disables the viewer changing positions when the left mouse button is clicked in the viewer. This viewer essentially locks/unlocks the viewer.
    2. "Lb" -- this turns on/off the label/axis display in the viewer
    3. "Interp" -- this turns on/off Open GL blending interpolation of the image. If this is off and the image is zoomed sufficiently then individual voxels can be seen!
    4. "T-Grid" -- This option to overlay a Talairach grid on top of an anatomical is only available in the 3D view.  This option is described in the Coordinate Systems manual page.
  4. Frame E is the image navigation frame where the user can set the X, Y and Z coordinates. These coordinates are in image space, i.e. Z=slice number, and X and Y are the in-plane voxel coordinates. The main panel (E1) is the set of controls for the regular orthogonal viewer. The objectmap orthogonal viewer has one additional control (See E2), titled mask which controls the transparency of the objectmap (0=completely transparent, 100=completely opaque). The 4D Viewer has two additional controls (See D3). These are the Frame scale (which also appears in the other viewers if the image is four-dimensional, i.e. 3D + time), and the Movie Control button which brings up the movie control for displaying cine-loops of the 4D image.
  5. This controls the image colormap. 
    1. St -- this uses a standard colormap where the minimum image intensity is mapped to black and the maximum to white with liner interpolation between them. This often gives bad contrast for MRIs.
    2. Nr -- this colormap -- which is the default -- is set to automatically "window" the image to saturate the image at the 99% of its cumulative histogram. This has been empirically shown to improve the contrast of MRI images
    3. F1, F2, F4 are overlay colormaps used by the Overlay Control to display merged functional/anatomical images. See the Registration page for more details.
    4. Cmap -- this brings up the Colormap Editor, described in more detail below, which can be used to create/edit custom colormaps.
  6. The zoom buttons can be used to zoom in/out simultaneously in all views of the orthogonal viewer. (Individual slices can be zoomed in/out using the middle mouse button).  The Reset button can be used to reset the display to a default orientation -- this is useful in 3D views when the image has been placed in a "bad angle". Finally the Save button can be used to grab a screen-shot of the viewer frame (A) and save it to either a JPEG (if the extension .jpeg or .jpg is specified) or a TIFF file otherwise. If the image is 4D, a screen-shot for each frame can be saved using the SaveAll button -- see Frame G2.
  7. The little "x" button can be used to hide the controls to maximize display area for the viewer. Clicking it back restores the controls.

Auxiliary Controls

There are five auxiliary controls which may be available, and which which have been referenced above, namely:
  1. The Volume Control
  2. The Oblique Slice Control
  3. The 3D Renderer Controls
  4. The Movie Controls
  5. The Colormap Editor

Navigating in the Display

The images are displayed in the black area on the left side of the viewer window. There are a variety of ways to manipulate the views and move through them. For the most efficient control, the use of a three-button mouse is recommended.

Mouse Control in the viewing area


Button 1 (left)
Click the left mouse button (button 1) anywhere on any of the 2-D slice views to set the cross-hairs to this point. Click and hold button 1 as you drag through the view space to move the cross-hairs cursor when it is active, and to rotate the 3-D display, when it is shown. (Unchecking the "x-hairs" box in the viewer control panel disables mouse-in-viewer movement of the cross-hairs. In this case, use the slider bars as described below.)

Button 2 (middle/wheel)
Click and drag with button 2 to scale the display. Drag down to zoom in, and drag up to zoom out.

Button 3 (right)
Clicking and dragging with mouse button 3 translates the display. Use it to slide the display up, down, and side to side. Pressing the shift key and the right mouse button brings up the Renderer Control.

The axes

The axis display is based on the image's native directions (usually X and Y are in the plane of the slices, and Z is across the slices). In the viewer, the X-axis is red, the Y-axis is green, and the Z-axis is blue.

Navigation controls in the viewer control panel

Value display box

The white text on black background in the viewer control panel continuously updates to display the location and value of the voxel under the cross-hairs. The first two letters in the display tell you the current coordinate system (ST = Standard: defined by the image voxels, TC = Talairach). The three coordinates in parentheses are the coordinates in that system of the crosshairs' current position (This matches the location of the "crosshair coordinate slider bars", below). The final number in the display is the value of the voxel at the current location.

Crosshair coordinate slider bars

As you move the crosshairs through the display, the "X-coord," "Y-coord," and "Z-coord" sliders continuously update their values to reflect the current location. Drag the sliders to move the crosshairs in the display to exact coordinate values.

Zoom control

In the viewer control panel, the two arrow buttons on either side of the "Zoom" label allow you to scale all viewer displays simultaneously. The left hand button zooms out, while the right hand button zooms in.

Reset

The reset button returns the display to its default view. All images are centered in their respective view space, sized to fit into the frame, and any 3-D views are rotated back to display their ventral side (XY plane, viewed from negative Z direction, in most orientations)

Display modes

You can switch between display modes using the large bar menu button at the top of the viewer control panel.

Available Display Modes


The Main Display Mode Selector       The 3D Mode Selector
3-Slice mode
The default display mode is "3-Slice mode," which shows three two-dimensional views of the image; an axial slice, a coronal slice, and a sagital slice, arranged to maximize available display space.
3-Slice
Show the three 2-D slices in their relative orthogonal orientation
3-Slice+3D Mode
This mode gives the three 2-D displays as in the default, but adds a large three-dimensional viewing space in the bottom right corner. This three dimensional view has multiple options, which can be accessed by the leftmost second-row menubutton in the viewer control panel
None
Do not display any image in the 3-D area
Simple mode
This display mode shows all three two-dimensional sections as in 3-Slice mode, but arranges them side-by-side.
Volume
Display a full volume rendering of the image
3-Slice+3D Mode(2)
This view mode is the same as the "3-Slice+3D Mode," except the four view spaces are allocated equal display space.
Obl+Vol
Show the volume with the oblique slice through it
Obl+3-s
Show the 3-slice rendering, along with a fourth oblique slice
3-s+vol
Show the full volume rendering, as well as the positions of the three slices, corresponding to the other three views.
3-s+vol+o
Show all three views as describe above, overlaid simultaneously
Sagittal-YZ
This shows only the 2-D view in the YZ plane
Coronal-XZ
Shows only the 2-D view in the XZ plane
Axial-XY
Shows only the 2-D view in the XY plane
3D Only
Fills viewer area with the 3-dimensional view, as described in "3-Slice+3D Mode"
Oblique
Show a slice through the image in any orientation

Volume Control

Clicking the Volume button in the third row of the viewer control panel (See Figure 2C) brings up the Volume Control Dialog box. This lets you set the display properties for the volume rendering of the image if it is shown.

There are two volume rendering methods used, a hardware accelerated (2D) Texture mapping method or a software ray-casting method. If the Texture Map checkbutton is enabled the Texture Mapping method is used. The ray casting method can yield higher quality displays, but it is much slower if a decent graphics card is available. The Resolution of the volume is controlled using the drop-menu labeled "Half-Original", just below the texture map button. Setting this value too high will place a large computational burden on the system. The image needs to be resampled prior to display, the interpolation mode for this is controlled using the drop menu marked "linear". Shading may be enabled using the Shading button.

The "Cropping" tools let you crop the image in all three dimensions, from either the max or min direction (or both). The sliders on the right side will, by default, be at the maximum value in their respective dimension. This is a quick check of the dimensions of your image.

The St Nr F1 F2 and F3 buttons at the bottom of the Volume Control Dialog box are some pre-set colormap options. If these are insufficient, clicking the Colormap button brings up the Colormap Editor, allowing you to create your own color scheme, or select from a wider variety of presets. See below. This is a separate colormap editor from that used for slice display (See Figure 2F).

The "Show Volume" checkbox sets whether the 3-D volume rendering should be displayed at all.



Oblique Control

Clicking the Oblique button in the third row of the viewer control panel (See Figure 2C) brings up the Oblique Control Dialog box. This lets the user manipulate the oblique slice, used to display non-orthogonal views of the current image.
This can either be positioned manually or automatically. For manual positioning, the user needs to adjust (i) the normal of the plane using the x-norm, y-norm and z-norm sliders and (ii) the offset of the plane centroid using the Distance slider. The Reset button is used to place the oblique slice in its default

"Automatic" positioning works by placing the slice perpendicular to the current viewer angle, this is enabled/disabled using the Follow Camera checkbox, which adjusts the plane in real time to follow the viewer camera. Alternatively the Look to Camera button may be used to perform this once. The Edge Camera button places the slice at the front of the camera clipping range (see the Renderer Controls for an explanation.) axial position.

The image display is controlled by (i) the main colormap – this is the same as for the main viewer (see Figure 2F), (ii) the resolution at which the slice samples the underlying image (this is set by the Res: drop menu – 256x256 is the default), and (iii) the opacity which can make the oblique slice more or less transparent – this is controlled by the Opacity slider.

There area also a number of miscellaneous controls.

The Show button enables/disables the display of the oblique slice. The Normal button shows/hides the normal direction arrow. The Load Plane and Save Plane buttons can be used to save the position and orientation of the current oblique slice in a text file for later retrieval. Finally the Save Image button can be used to save a 2D image of the current texture mapped on the oblique slice.


The 3D Renderer Control

3D Rendering techniques employ the formalism of a 3D scene being viewed by a camera positioned somewhere in the scene – the actual screen display is what the "camera" sees in the 3D scene. Manipulating the position and orientation of the camera can produce very different views.

While all viewers in BioImage Suite are true 3D viewers and employ 3D rendering techniques, when displaying a two image the user can only move the camera in very restricted ways – parallel to the image plane being displayed and can not change its orientation.

When part (or the whole of) the Orthogonal Viewer is set to 3D Mode, the user can explicitly rotate the volume etc. using the left mouse button. This moves the camera that generates the 3D Rendering. Pressing Shift+Right Mouse button in the 3D viewer brings up the 3D Renderer Controls which can be used to perform additional camera operations. In particular they can be used to move the camera to preset positions using the View drop-menu (currently displaying "Axial-XY"). A current position can be stored as a preset using the Add button and the current preset updated from the current camera position using the Upd button. All Camera position presets maybe loaded and saved using the Load and Save buttons respectively.

The two sliders Near Plane and Thickness control the camera clipping, and restrict the 3D rendering to display only objects that are in a range of distances perpendicular to the camera. The Rt button on the far right resets the clipping range to show all objects in the scene (e.g. image volumes, slices, surfaces etc.).

The top row contains controls to Zoom the camera (Zm), the reset to the original orientations (XY,Cl) and to save the scene to a jpeg/tiff file as described earlier (see Figure 2 and it's description).


The Movie Controls

The movie control is accessed using the "Movie Control" button in the 4D Orthogonal Viewer -- See Figure 2 E3). It has functionality for playing cine-loops of the contents of the viewer. It has two distinct play modes "Complete" which is equivalent to simply incrementing the frame and "Fast" which uses cached versions all frames prior to playing to achieve higher performance at the loss of interaction during movie mode. The caching is performed by clicking the "Prepare" button.



Mosaic Viewer Controls

The simple viewer, or "Mosaic Viewer" displays multiple 2-dimensional slices, simulating common radiology and MR figures. You can pull an image from the orthogonal viewer into the mosaic viewer in order to quickly compare multiple slices at once, or visualize it in standard ascending/descending slice order.

Manipulating the Slice Views

Figure 3. The Mosaic Viewer
This is an example of the type of 2-D display that can be generated from a 3-D image using the mosaic viewer. Here we see Axial slices in increments of 3, starting with slice 32. A label appears beneath each slice identifying its number. The whole image can be saved using the Save button.

Once an image has been loaded into the mosaic viewer, you can change a number of options regarding how it is displayed.

Slice Orientation

From the menu at the top of the viewer control panel you can choose the direction in which the slices will move through the image. Choose "Axial-XZ" to have the viewers display slices from top to bottom, "Sagittal-YZ" to have them display slices in profile, and "Coronal-XY" to display slices from front to back.

Number of Viewers

Sliding the button on the "Num Viewers" slider bar, will adjust the number of individual viewer panels in the display space. The size of the viewers is adjusted automatically so that they all fit in the display. Adding viewers allows you to see more slices at once but allocates less display space for each slice.

First Slice

Use the "First Slice" slider bar to select which slice will be displayed in the top left viewer. This will set the lowest slice displayed. Use it in combination with the Increment slider to maximize the usage of the display, and show the most interesting parts of your image in the highest slice density.

Increment

The "Increment" slider sets the number of slices to skip between successive viewers in the 2-D display. Thus, the display starts with the slice specified by the "First Slice" slider, and adds every nth slice to each following viewer in order, where n is the increment selected. If a large increment is selected, there may not be enough slices to fill all viewers, leaving those near the end blank. If a small increment is selected, consecutive slices will be very close together, and the viewers may fill up with slices within a short distance from the first slice.

Labels and Interpolation

The checkboxes for labels and interpolation turn the label display in each viewer on and off, and the display interpolation between voxels on and off, respectively.

Colormaps

The standard and custom colormaps can be applied to the 2-D displays just as for the orthogonal displays as described below.

Zoom

Clicking the arrows on either side of the Zoom label will zoom all viewers to the same level. To zoom an individual viewer's display, click and drag on it with the middle mouse button. (Clicking and dragging with the right mouse button will pan the display). These changes can be reversed by hitting the Reset button.

Save

Click the Save button to save the whole mosaic display as either a JPEG or TIFF image.





Using Colormaps

A colormap specifies how an image gets displayed. Specifically, it maps image voxel values to color values. A common display choice is a linear colormap, in which a linear increase in image voxel value yields a linear increase in the display brightness. The use of colormaps allows you to tinker with this relationship in an extremely flexible manner. You can make certain values transparent, or set up a non-linear colormap, or a map in which red, green, and blue values do not all follow the same dependence function.

Colormaps are saved along with image data. Thus, if you make changes to an image's colormap, and then save that image, the colormap will reflect your changes when you reopen that image later.

Accessing the Colormap for an Image

(Note: an image must be loaded to access the colormap editor). Once an image has been loaded into the viewer, you can simply click on the Cmap button on the lower right side of the viewer control panel. This will bring up the colormap editor window. (Fig. 4)

Colormap Settings

Figure 4. The Colormap Editor
This is the default colormap that is applied to any image when it is first loaded. A linear relationship is defined between the image intensity value and all three colors; red, green, and blue. Only the blue line shows up because it is drawn last, but the others are under there, trust us. (For proof, use the Complex Controls to move either the red or green line). The black line sets the transparency/opacity threshold for the image.

The default linear relationship is appropriate for many applications, but you may want to customize the colormap for you image. The simple controls to do this are in the main colormap window (Fig. 3). Use the menu right under the colormap function display to select which type of function you want to use to define your color map. You can also use the Complex Controls.

Step

Minimum Intensity

Sliding the top slider, "Min Intensity," sets the lowest image value that will be displayed as non-transparent. Notice that as you slide it above the lowest intensity value in the image, the black transparency line in the colormap function display will change so that the step from transparent to opaque coincides with the minimum intensity setting.

Figure 5. The Complex Colormap Control Window
This window allows you to set the mapping parameters for Red, Green, Blue, Alpha (Transparency), or RGB (all three colors simultaneously). Each of the sliders on the left side allows you to select an intensity in the image, and the associated slider on the right side allows you to set the corresponding parameter (for the color selected in the menu at the top). The top and bottom right side sliders correspond to the minimum and maximum image values, respectively.

Maximum Intensity

The maximum intensity is the intensity above which all values appear as saturated (full intensity). Use the "Max Intensity" slider to set this level.

Gamma

The slider in the colormap editor window when "Gamma" is selected sets the eccentricity of the gamma function that defines the colormap.

Complex Colormap Controls

The complex controls are accessed via the (Controls | Complex) menu selection at the top of the Colormap Editor. This brings up yet another window that allows you to control the mapping of individual levels to specific display values. Select Red, Green, Blue, RGB, or Alpha. This determines what color intensity the slider values will set. The RGB selection sets all three at once. The Alpha property determines the transparency of the image.

Now use one of the "Level" sliders to navigate to a given image intensity, and the "Value" slider just to its right to set what the corresponding intensity of the selected color (or Alpha - "transparency") will be. You will notice that there are two extra "Value" slider bars. The top one sets the display value for the minimum value present in the image, while the bottom slider sets the display value for the maximum value in the image.




About this manual.