Quantitative Big Imaging

• Jean Claude, Morphometry with R
  • Online through ETHZ
  • Buy it
• John C. Russ, “The Image Processing Handbook”,(Boca Raton, CRC Press)
  • Available online within domain ethz.ch (or proxy.ethz.ch / public VPN)

Advanced Segmentation

• Markov Random Fields for Image Processing Lecture
• Markov Random Fields Chapter
• Fuzzy set theory
• Superpixels

Contouring

• Active Contours / Snakes

• Motivation
  • Many Samples
  • Difficult Samples
  • Training / Learning
• Thresholding
  • Automated Methods
  • Hysteresis Method

• Feature Vectors
  • K-Means Clustering
  • Superpixels
  • Probabalistic Models
• Working with Segmented Images
  • Contouring
• Beyond
  • Fuzzy Models
  • Component Labeling

• Image acquisition and representations
• Enhancement and noise reduction
• Understanding models and interpreting histograms
• Ground Truth and ROC Curves

• Choosing a threshold
  • Examining more complicated, multivariate data sets
• Improving segementation with morphological operations
  • Filling holes
  • Connecting objects
  • Removing Noise
• Partial Volume Effect

With inconsistent or every changing illumination it may not be possible to apply the same threshold to every image.

Here is a bone slice

1. Find the larger cellular structures (osteocyte lacunae)
2. Find the small channels which connect them together

The first task

is easy using a threshold and size criteria (we know how big the cells should be)

The second

is much more difficult because the small channels having radii on the same order of the pixel size are obscured by partial volume effects and noise.

• The cortex is barely visible to the human eye
• Tiny structures hint at where cortex is located

• A simple threshold is insufficient to finding the cortical structures
• Other filtering techniques are unlikely to magicially fix this problem

Given that applying a threshold is such a common and signficant step, there have been many tools developed to automatically (unsupervised) perform it. A particularly important step in setups where images are rarely consistent such as outdoor imaging which has varying lighting (sun, clouds). The methods are based on several basic principles.

Taking a typical image of a bone slice, we can examine the variations in calcification density in the image

We can see in the histogram that there are two peaks, one at 0 (no absorption / air) and one at 0.5 (stronger absorption / bone)

Intermodes

• Take the point between the two modes (peaks) in the histogram

Otsu

Search and minimize intra-class (within) variance \[ \sigma^2_w(t)=\omega_{bg}(t)\sigma^2_{bg}(t)+\omega_{fg}(t)\sigma^2_{fg}(t) \]

Isodata

• \( \textrm{thresh}= \max(img)+\min(img)/2 \)
• while the thresh is changing
  • \( bg = img<\textrm{thresh}, obj = img>\textrm{thresh} \)
  • \( \textrm{thresh} = (\textrm{avg}(bg) + \textrm{avg}(obj))/2 \)

For some images a single threshold does not work

• large structures are very clearly defined
• smaller structures are difficult to differentiated (see partial volume effect)

ImageJ Source

Comparing the original image with the three phases

Now we apply two important steps. The first is to remove the objects which are not cells (too small) using an opening operation.

The second step to keep the between pixels which are connected (by looking again at a neighborhood \( \mathcal{N} \)) to the air voxels and ignore the other ones. This goes back to our original supposition that the smaller structures are connected to the larger structures

As we briefly covered last time, many measurement techniques produce quite rich data.

• Digital cameras produce 3 channels of color for each pixel (rather than just one intensity)
• MRI produces dozens of pieces of information for every voxel which are used when examining different contrasts in the system.
• Raman-shift imaging produces an entire spectrum for each pixel
• Coherent diffraction techniques produce 2- (sometimes 3) diffraction patterns for each point. \[ I(x,y) = \hat{f}(x,y) \]

A pairing between spatial information (position) and some other kind of information (value). \[ \vec{x} \rightarrow \vec{f} \]

We are used to seeing images in a grid format where the position indicates the row and column in the grid and the intensity (absorption, reflection, tip deflection, etc) is shown as a different color

The alternative form for this image is as a list of positions and a corresponding value

\[ \hat{I} = (\vec{x},\vec{f}) \]

This representation can be called the feature vector and in this case it only has Intensity

If we use feature vectors to describe our image, we are no longer to worrying about how the images will be displayed, and can focus on the segmentation/thresholding problem from a classification rather than a image-processing stand point.

Example

So we have an image of a cell and we want to identify the membrane (the ring) from the nucleus (the point in the middle).

A simple threshold doesn't work because we identify the point in the middle as well. We could try to use morphological tricks to get rid of the point in the middle, or we could better tune our segmentation to the ring structure.

In this case we add a very simple feature to the image, the distance from the center of the image (distance).

We now have a more complicated image, which we can't as easily visualize, but we can incorporate these two pieces of information together.

Now instead of trying to find the intensity for the ring, we can combine density and distance to identify it

\[ iff (5<\textrm{Distance}<10 \\ \& 0.3<\textrm{Intensity}>1.0) \]

The distance while illustrative is not a commonly used features, more common various filters applied to the image

• Gaussian Filter (information on the values of the surrounding pixels)
• Sobel / Canny Edge Detection (information on edges in the vicinity)
• Entroy (information on variability in vicinity)

The distributions of the features appear very different and can thus likely be used for identifying different parts of the images.

Combine this with our a priori information (called supervised analysis)

Input

• Feature vectors (\( \vec{v} \))

• Distance metric \[ D_{ij}=||\vec{v}_i-\vec{v}_j|| \]

• Group Count (\( N=2 \))

\[ \downarrow \]

Output

• Group 1

• Group 2

We give as an initial parameter the number of groups we want to find and possible a criteria for removing groups that are too similar

1. Randomly create center points (groups) in vector space
2. Assigns group to data point by the “closest” center
3. Recalculate centers from mean point in each group
4. Go back to step 2 until the groups stop changing

What vector space to we have?

• Sometimes represent physical locations (classify swiss people into cities)
• Can include intensity or color (K-means can be used as a thresholding technique when you give it image intensity as the vector and tell it to find two or more groups)
• Can also include orientation, shape, or in extreme cases full spectra (chemically sensitive imaging)

Note: If you look for N groups you will almost always find N groups with K-Means, whether or not they make any sense

Continuing with our previous image and applying K-means to the Intensity, Sobel and Gaussian channels looking for 2 groups we find

Looking for 5 groups

Including the position in the features as well

Since the distance is currently calculated by \( ||\vec{v}_i-\vec{v}_j|| \) and the values for the position is much larger than the values for the Intensity, Sobel or Gaussian they need to be rescaled so they all fit on the same axis \[ \vec{v} = \left\{\frac{x}{10}, \frac{y}{10}, \textrm{Intensity},\textrm{Sobel},\textrm{Gaussian}\right\} \]

An approach for simplifying images by performing a clustering and forming super-pixels from groups of similar pixels.

Drastically reduced data size, serves as an initial segmentation showing spatially meaningful groups

Segment the superpixels and apply them to the whole image (only a fraction of the data and much smaller datasets)

Superpixels (0.06% of the original)

Original

takes all of the points in a given slice or volume and finds the smallest convex 2D area or 3D volume (respectively) which encloses all of those points.

Depending on the type of sample the convex hull can make sense for filling in the gaps and defining the boundaries for a sample.

The critical short coming is it is very sensitive to single outlier points.

The convex hull very closely matches the area we would define as 'bone' without requiring any parameter adjustment, resolution specific adjustments, or extensive image-processing, for such a sample a convex hull is usually sufficient.

Here is an example of the convex hull applied to a region of a cortical bone sample. The green shows the bone and the red shows the convex hull. Compared to a visual inspection, the convex hull overestimates the bone area as we probably would not associate the region where the bone curves to the right with 'bone area'

Useful for a variety of samples (needn't be radially symmetric) and offers more flexibility in step size, smoothing function etc than convex hull.

1. Calculates the center of mass.
2. Transforms sample into Polar Coordinates
3. Calculates a piecewise linear fit \( r=f(\theta) \)

Fuzzy classification based on Fuzzy logic and Fuzzy set theory and is a general catagory for multi-value logic instead of simply true and false and can be used to build IF and THEN statements from our probabilistic models.

Instead of

\[ P(\{\vec{x} , I(\vec{x})\} | \alpha) \propto P(\alpha) + P(I(\vec{x}) | \alpha)+ \] \[ P(\sum_{x^{\prime} \in \mathcal{N}} I(\vec{x^{\prime}}) | \alpha) \]

Clear simple rules

which encompass aspects of filtering, thresholding, and morphological operations

• IF the intensity if dark (\( <100 \))
  • AND a majority of the neighborhood (\( \mathcal{N} \)) values are dark (\( <100 \))
• THEN it is a cell