EF3 – Model-Based Imaging

Project

EF3-5

Direct Reconstruction of Biophysical Parameters Using Dictionary Learning and Robust Regularization

Project Heads

Michael Hintermüller, Tobias Schäffter

Project Members

Staff: Guozhi Dong (HU & WIAS)

Associate: Kostas Papafitsoros (WIAS)

Project Duration

01.10.2019 – 31.03.2022

Located at

HU Berlin

Description

Model-based imaging exploit knowledge of physical models in imaging to enhance the reconstruction quality. This is often challenge in real applications, as the physical models might be not directly available, but hidden in different kinds of data, either experimental data or numerical one. This project aims to develop the concept of model-based imaging methods. The idea is integrating physical models, either dictionary based or learning-informed, into the image reconstruction process. Particularly, the physical model may involve some physical quantities, which are also of interests to estimate their precise values, as the case in quantitative imaging. These problems motivate the research of the this project.

As some methodology guideline, we are interested in the following optimization framework in this project but not limited to:

$\begin{eqnarray*} \text{minimize }& F(u,D)&\\ \text{subject to }& D\in \mathcal{D}_{ad},\quad & u\in \operatorname{argmin} \{G_D(v),\; v\in \mathcal{U}_{ad} \}. \end{eqnarray*}$

Here $F$ is an upper level objective which might be based on statistical measures, and u denotes some physical quantities. $D$ is an element either from an entry of some dictionary, or produced through some machine learning schemes. The lower level objective is usually a matching of fidelity with some regularization of $u$ involved. In particular, spatial distributed regularization parameters are of interest and will be investigated under this bilevel optimization framework.

These new mathematical objects then need to be analytically and numerically investigated, including also robust numerical solvers.

Exemplary case study focuses on magnetic resonance imaging. An integrated physics-based models are proposed for quantitatively estimating the tissue parameters, for instance, the T1/T2 relaxation time, the proton spin density. Approaches of using learning-informed physics or dictionary-based physics are under investigation. In particular, we are investigating the following problem in a (partial) differential equation constrained optimization setting:

$\begin{eqnarray*} \min_{y,u} \quad & \|Ay-g\|_{H}^2+ \alpha \mathcal{R}(u) \\ \text{ subject to } \quad & e(y,u)=0 \quad \text{ and } u\in C_{ad}. \end{eqnarray*}$

where $e(y,u)=0$ denotes some differential equations. The variable $y$ typically represents some imaging function one would like to reconstruct out of the detected data $g$ , and $u$ is some physical quantity which contains quantitative information. The operator $A$ is some linear or nonlinear operator, and $\mathcal{R}$ is a regularization functional.

For instance in the quantitative MRI problem, $y$ is the MRI image, $e(y,u)=0$ could be the Bloch equation, and $u$ represents the quantities like relaxation time $T_1$ and $T_2$ , proton spin density $\rho$ , as in the following

$\begin{eqnarray*} &\frac{\partial y}{\partial t}(t) &= y(t) \times \gamma B(t) - \left ( \frac{y_{1}(t)}{T_{2}}, \frac{y_{2}(t)}{T_{2}}, \frac{y_{3}(t)-\rho m_{e}}{T_{1}} \right ), \\ &y(0)&= \rho m_{0}.\\ \end{eqnarray*}$

Particularly, we have worked on an integrated physics-based approach, where the equation $e(y,u)=0$ is represented using an explicit map $y=\Pi(u)$ . With this, we study directly the reduced problem:

$\begin{eqnarray*} \min_{u} \quad & \|A\circ \Pi(u)-g\|_{H}^2+ \alpha \mathcal{R}(u) \\ \text{ subject to } & u\in C_{ad}. \end{eqnarray*}$

In practice, this explicit operator may not be available directly, but hidden in numerical or experimental data. Therefore, a dictionary or learning-informed scheme can play the role to replace the underline exact physics. Using this, we come up with the following problem:

$\begin{eqnarray*} \min_{u} & \|A\circ \Pi_N (u) - g\|_{H}^2+ \alpha \mathcal{R}(u) \\ \text{ subject to } & u\in C_{ad}. \end{eqnarray*}$

Questions like the feasibility of the replacement of $\Pi$ using $\Pi_N$ , existence of solutions, regularity, and differentiability of the solution map are important and of interest to study in this project, as well as the error bounds of the solutions, and their numerical analysis, implementations. Related to MRI problem, the dictionary or machine learning techniques are applied to simulate the Bloch map $\Pi: u \to y$ from the numerical solutions for Bloch equations.

In the bilevel optimization framework, a particular line of research which has been continued in this project is a generalization of the idea for automatically selecting the spatially dependent regularization parameters in total generalized variation (TGV) method for imaging and inverse problems.

Techniques using variational methods and also partial differential equation tools have been studied and will continue to be studied in different context.

Project Webpages

Selected Publications

G. Dong, M. Hintermüller, K. Papafitsoros, Quantitative magnetic resonance imaging: From fingerprinting to integrated physics-based models, SIAM Journal on Imaging Sciences, 2 (2019), pp. 927–971, DOI 10.1137/18M1222211 .
M. Hintermüller, K. Papafitsoros, Chapter 11: Generating structured nonsmooth priors and associated primal-dual methods, in: Processing, Analyzing and Learning of Images, Shapes, and Forms: Part 2, R. Kimmel, X.-Ch. Tai, eds., 20 of Handbook of Numerical Analysis, Elsevier, 2019, pp. 437–502, (Chapter Published), DOI 10.1016/bs.hna.2019.08.001.
G. Dong, M. Hintermüller, Y. Zhang, A class of geometric second order quasi-linear hyperbolic PDEs and their application in imaging science. WIAS Preprint No. 2591, (2019).
M. Hintermüller, K. Papafitsoros, C. N. Rautenberg, H. Sun, Dualization and automatic distributed parameter selection of total generalized variation via bilevel optimization, WIAS Preprint No. 2689, (2020).
G. Dong, M. Hintermüller, K. Papafitsoros, Optimization with learning-informed differential equation constraints and its applications, WIAS Preprint No. 2754, (2020).

Selected Pictures

T2 relaxation time of a brain phantom

T2 relaxation time estimated using BLIP method

T2 relaxation time estimated using integrated learning-informed method

The above is the T2 relaxation parameter map of a brain phantom. We compare our result to some result from state of the art method in the literature. We use an integrated model based imaging method where the physics characterized by Bloch equations are replaced by learning-informed method like neural networks. Our result shows that it is more stable in comparison to the so-called BLIP method applying to noise contaminated K-space MRI data. Note that here, only 10-frame 25% subsampled Fourier space data are used.

Spatially distributed TGV regularization parameters

Parrot image with noise

Denoising using weighted TGV via bilevel optimization

Using the bilevel optimization framework for the automatic selection of spatially dependent regularization parameters for total generalized variation (TGV) regularization. The computed regularization parameters not only result in preservation of fine scale image details, but also lead to elimination of the staircasing effect, a well-known artifact of total variation regularization.

Please insert any kind of pictures (photos, diagramms, simulations, graphics) related to the project in the above right field (Image with Text), by choosing the green plus image on top of the text editor. (You will be directed to the media library where you can add new files.)
(We need pictures for a lot of purposes in different contexts, like posters, scientific reports, flyers, website,…
Please upload pictures that might be just nice to look at, illustrate, explain or summarize your work.)

As Title in the above form please add a copyright.

And please give a short description of the picture and the context in the above textbox.

Don’t forget to press the “Save changes” button at the bottom of the box.

If you want to add more pictures, please use the “clone”-button at the right top of the above grey box.