Books

• Jean Claude, Morphometry with R
  • Online through ETHZ
  • Buy it
• John C. Russ, “The Image Processing Handbook”,(Boca Raton, CRC Press)
  • Available online within domain ethz.ch (or proxy.ethz.ch / public VPN)

Papers / Sites

• Comparsion of Tracking Methods in Biology

  • Chenouard, N., Smal, I., de Chaumont, F., Maška, M., Sbalzarini, I. F., Gong, Y., … Meijering, E. (2014). Objective comparison of particle tracking methods. Nature Methods, 11(3), 281–289. doi:10.1038/nmeth.2808
  • Maska, M., Ulman, V., Svoboda, D., Matula, P., Matula, P., Ederra, C., … Ortiz-de-Solorzano, C. (2014). A benchmark for comparison of cell tracking algorithms. Bioinformatics (Oxford, England), btu080–. doi:10.1093/bioinformatics/btu080

• Multiple Hypothesis Testing

  • Coraluppi, S. & Carthel, C. Multi-stage multiple-hypothesis tracking. J. Adv. Inf. Fusion 6, 57–67 (2011).
  • Chenouard, N., Bloch, I. & Olivo-Marin, J.-C. Multiple hypothesis tracking in microscopy images. in Proc. IEEE Int. Symp. Biomed. Imaging 1346–1349 (IEEE, 2009).

The course has covered imaging enough and there have been a few quantitative metrics, but “big” has not really entered.

What does big mean?

• Not just / even large
• it means being ready for big data
• volume, velocity, variety (3 V's)
• scalable, fast, easy to customize

So what is “big” imaging

• doing analyses in a disciplined manner
  • fixed steps
  • easy to regenerate results
  • no magic
• having everything automated
  • 100 samples is as easy as 1 sample
• being able to adapt and reuse analyses
  • one really well working script and modify parameters
  • different types of cells
  • different regions

• Motivation (Why and How?)
• Scientific Goals
• Experiments
  • Simulations
  • Experiment Design
• Object Tracking
• Distribution
• Topology

• Voxel-based Methods
  • Cross Correlation
  • DIC
  • DIC + Physics
• General Problems
  • Thickness - Lung Tissue
  • Curvature - Metal Systems
  • Two Point Correlation - Volcanic Rock

• 3D images are already difficult to interpret on their own
• 3D movies (4D) are almost impossible

• 2D movies (3D) can also be challenging

We can say that it looks like, but many pieces of quantitative information are difficult to extract

• how fast is it going?
• how many particles are present?
• are their sizes constant?
• are some moving faster?
• are they rearranging?

Rheology

Understanding the flow of liquids and mixtures is important for many processes

• blood movement in arteries, veins, and capillaries
• oil movement through porous rock
• air through dough when cooking bread
• magma and gas in a volcano

Deformation

Deformation is similarly important since it plays a significant role in the following scenarios

• red blood cell lysis in artificial heart valves
• microfractures growing into stress fractures in bone
• toughening in certain wood types

The first step of any of these analyses is proper experimental design. Since there is always

• a limited field of view
• a voxel size
• a maximum rate of measurements
• a non-zero cost for each measurement

There are always trade-offs to be made between getting the best possible high-resolution nanoscale dynamics and capturing the system level behavior.

• If we measure too fast
  • sample damage
  • miss out on long term changes
  • have noisy data
• Too slow
  • miss small, rapid changes
  • blurring and other motion artifacts
• Too high resolution
  • not enough unique structures in field of view to track
• Too low resolution
  • not sensitive to small changes

We start with a starting image

• A number of sphere objects with the same radius scattered evenly across the field of view

Analysis

• Threshold
• Component Label
• Shape Analysis
• Distribution Analysis

$$\vec{v}(\vec{x})=\langle 0,0.1 \rangle$$

$$\vec{v}(\vec{x})=0.3\frac{\vec{x}}{||\vec{x}||}\times \langle 0,0,1 \rangle$$

$$\vec{v}(\vec{x})=\langle 0,0.1 \rangle$$

$$\vec{v}(\vec{x})=0.3\frac{\vec{x}}{||\vec{x}||}\times \langle 0,0,1 \rangle$$

Under perfect imaging and experimental conditions objects should not appear and reappear but due to

• noise
• limited fields of view / depth of field
• discrete segmentation approachs
• motion artifacts
  • blurred objects often have lower intensity values than still objects

It is common for objects to appear and vanish regularly in an experiment.

Over many frames this can change the path significantly

The simulation can be represented in a more clear fashion by using single lines to represent each spheroid

We see that visually tracking samples can be difficult and there are a number of parameters which affect the ability for us to clearly see the tracking.

• flow rate
• flow type
• density
• appearance and disappearance rate
• jitter
• particle uniqueness

We thus try to quantify the limits of these parameters for different tracking methods in order to design experiments better.

Acquisition-based Parameters

• acquisition rate $\rightarrow$ flow rate, jitter (per frame)
• resolution $\rightarrow$ density, appearance rate

Experimental Parameters

• experimental setup (pressure, etc) $\rightarrow$ flow rate/type
• polydispersity $\rightarrow$ particle uniqueness
• vibration/temperature $\rightarrow$ jitter
• mixture $\rightarrow$ density

Input flow from simulation

$$\vec{v}(\vec{x})=\langle 0,0,0.05 \rangle+||0.01||\measuredangle b$$

Nearest Neighbor Tracking

Input flow from simulation

$$\vec{v}(\vec{x})=\langle 0,0,0.01 \rangle+||0.05||\measuredangle b$$

Nearest Neighbor Tracking

We can then quantify the success rate of each algorithm on the data set using the very simple match and mismatch metrics

Counting Misses

Time	NN	ONN	ANN
1.9	20%	27.5%	27.5%
3.7	22.5%	25%	27.5%
5.5	27.5%	20%	17.5%
7.3	17.5%	25%	15%
9.1	20%	32.5%	22.5%
10.9	15%	15%	10%
12.7	30%	27.5%	27.5%
14.5	22.5%	25%	22.5%
16.3	27.5%	35%	22.5%
18.1	22.5%	22.5%	20%

How does this help us to design experiments?

• density can be changed by adjusting the concentration of the substances being examined or the field of view
• flow per frame (image velocity) can usually be adjusted by changing pressure or acquisition time
• jitter can be estimated from images

How much is enough?

• difficult to create one number for every experiment
• 5% error in bubble position
  • $\rightarrow$ <5% in flow field
  • $\rightarrow$ >20% error in topology
• 5% error in shape or volume
  • $\rightarrow$ 5% in distribution or changes
  • $\rightarrow$ >5% in individual bubble changes
  • $\rightarrow$ >15% for single bubble strain tensor calculations

Bijective Requirement

• We define $\vec{P}_f$ as the result of performing the nearest neigbhor tracking on $\vec{P}_0$
• $$\vec{P}_f=\textrm{argmin}(||\vec{P}_0-\vec{y} || \forall \vec{y}\in I_1)$$
• We define $\vec{P}_i$ as the result of performing the nearest neigbhor tracking on $\vec{P}_f$
• $$\vec{P}_i=\textrm{argmin}(||\vec{P}_f-\vec{y} || \forall \vec{y}\in I_0)$$
• We say the tracking is bijective if these two points are the same
• $$\vec{P}_i \stackrel{?}{=} \vec{P}_0$$

Maximum Displacement

$$\vec{P}_1=\begin{cases} ||\vec{P}_0-\vec{y} ||<\textrm{MAXD}, & \textrm{argmin}(||\vec{P}_0-\vec{y} || \forall \vec{y}\in I_1) \\ \textrm{Otherwise}, & \emptyset \end{cases}$$

Prior / Expected Movement

$$\vec{P}_1=\textrm{argmin}(||\vec{P}_0+\vec{v}_{offset}-\vec{y} || \forall \vec{y}\in I_1)$$

Adaptive Movement

Can then be calculated in an iterative fashion where the offset is the average from all of the $\vec{P}_1-\vec{P}_0$ vectors. It can also be performed $$\vec{P}_1=\textrm{argmin}(||\vec{P}_0+\vec{v}_{offset}-\vec{y} || \forall \vec{y}\in I_1)$$

While nearest neighbor provides a useful starting tool it is not sufficient for truly complicated flows and datasets.

Better Approaches

1. Multiple Hypothesis Testing Nearest neighbor just compares the points between two frames and there is much more information available in most time-resolved datasets. This approach allows for multiple possible paths to be explored at the same time and the best chosen only after all frames have been examined

Shortcomings

1. Merging and Splitting Particles The simplicity of the nearest neighbor model does really allow for particles to merge and split (relaxing the bijective requirement allows such behavior, but the method is still not suited for such tracking). For such systems a more specific, physically-based is required to encapsulate this behavior.

For voxel-based approachs the most common analyses are digital image correlation (or for 3D images digital volume correlation), where the correlation is calculated between two images or volumes.

Standard Image Correlation

Given images $I_0(\vec{x})$ and $I_1(\vec{x})$ at time $t_0$ and $t_1$ respectively. The correlation between these two images can be calculated

$$C_{I_0,I_1}(\vec{r})=\langle I_0(\vec{x}) I_1(\vec{x}+\vec{r}) \rangle$$

With highly structured / periodic samples identfying a best correlation is difficult since there are multiple maxima.

The correlation function can be extended by adding rotation and scaling terms to the offset making the tool more flexible but also more computationally expensive for large search spaces.

$$C_{I_0,I_1}(\vec{r},s,\theta)=$$ $$\langle I_0(\vec{x}) I_1( \begin{bmatrix} s\cos\theta & -s\sin\theta\\ s\sin\theta & s\cos\theta \end{bmatrix} \vec{x}+\vec{r}) \rangle$$

DIC or DVC by themselves include no sanity check for realistic offsets in the correlation itself. The method can, however be integrated with physical models to find a more optimal solutions.

• information from surrounding points
• smoothness criteria
• maximum deformation / force
• material properties

$$C_{\textrm{cost}} = \underbrace{C_{I_0,I_1}(\vec{r})}_{\textrm{Correlation Term}} + \underbrace{\lambda ||\vec{r}||}_{\textrm{deformation term}}$$

There are a number of different ways to calculate strain and the strain tensor, but the most applicable for general image based applications is called the infinitesimal strain tensor, because the element matches well to square pixels and cubic voxels.

A given strain can then be applied and we can quantify the effects by examining the change in the small element.

We catagorize the types of strain into two main catagories:

$$\underbrace{\mathbf{E}}_{\textrm{Total Strain}} = \underbrace{\varepsilon_M \mathbf{I_3}}_{\textrm{Volumetric}} + \underbrace{\mathbf{E}^\prime}_{\textrm{Deviatoric}}$$

Volumetric / Dilational

The isotropic change in size or scale of the object.

Deviatoric

The change in the proportions of the object (similar to anisotropy) independent of the final scale

Data provided by Mattia Pistone and Julie Fife The air phase changes from small very anisotropic bubbles to one large connected pore network. The same tools cannot be used to quantify those systems. Furthermore there are motion artifacts which are difficult to correct.

We can utilize the two point correlation function of the material to characterize the shape generically for each time step and then compare.