Energy, Entropy, and Dissipative Dynamics

Mathematics is a key discipline for today’s research landscape. It provides a language that makes it possible to describe and quantify complex processes in the natural and engineering sciences. Many achievements and amenities of modern life would not have been possible without mathematics. The mathematical analysis of applied models not only drives progress within the discipline itself, but also has the potential to impact applications in ways that cannot be predicted a priori.

The Research Training Group Energy, Entropy, and Dissipative Dynamics combines analysis, modeling, and numerics of nonlinear partial differential equations coming from physics, materials science, and geometry. A common theme among the research projects is the use of energy and entropy functionals and their dissipation mechanisms as a tool for the investigation of the qualitative and quantitative behavior of solutions.

Important topics include energy minimization, properties of energy landscapes, maximization of energy dissipation or entropy production as a selection criterion in time-dependent situations. The research projects focus on different applied models such as moment models in gas dynamics, hyperbolic conservation laws, kinetic partial differential equations, flows on networks, Landau-Lifshitz equations, and the models of geometric knot theory. A list of representative research projects can be found below.

Click on headings to open/close sections.

Hyperbolic PDEs

Efficient numerical methods for stochastic hyperbolic conservation laws
Advisors: Michael Herty, Siegfried Müller

In hyperbolic conservation laws, uncertainties may be introduced, for instance, by initial or boundary data as well as fluxes. In order to capture the effect of such uncertainties on the solution we are interested in quantities like expectation and variance of solutions. The resulting equations pose serious challenges in the analysis as well as in the numerical resolution. While there has been tremendous progress in the field of stochastic elliptic equations similar results for hyperbolic are still at large. Within the research project we want to examine efficient and reliable numerical schemes for stochastic hyperbolic conservation laws. Here we plan to focus on intrusive methods based on recent results of the applicants on entropy-entropy flux pairs for one-dimensional stochastic hyperbolic equations. In contrast to non-intrusive methods like Monte-Carlo methods, where many deterministic problems are solved and averaged over the stochastic domain, intrusive methods modify the underlying equations, e.g., by stochastic Galerkin projection. These methods are based on a generalized polynomial chaos expansion of the solution $u$ in the space of all square integrable random variables with respect to an orthogonal polynomial basis.

Flow solutions for transport equations
Advisors: Michael Herty, Siegfried Müller, Michael Westdickenberg

In recent years there has been tremendous progress in the mathematical theory of transport and continuity equations, starting with the seminal work by DiPerna-Lions (1989) and Ambrosio (2004) who generalized the classical theory to the case of velocity fields with Sobolev and $\mathrm{BV}$-regularity, respectively. The main difficulty stems from the fact if the velocity field is not Lipschitz continuous, then solutions of the corresponding ordinary differential equations may no longer be unique. The Lagrangian point of view plays an important role in these developments, as formalized in Ambrosio’s regular Lagrangian flow and the interpretation of solutions as superpositions of integral curves of the underlying ordinary differential equation. Bianchini-Bonicatto (2017) proved the stability of the regular Lagrangian flows, known as Bressan’s conjecture, for convergent sequences of $\mathrm{BV}$-regular velocity fields. The existence of a suitable flow enables us to split transport equations into a geometry part, captured by the flow, and the transport part, which significantly simplifies since the evolution only needs to be studied along the characteristics. This research project aims at developing novel numerical schemes for transport equations based on this insight, by connecting these theoretical results with recent breakthroughs in discontinuous Petrov-Galerkin methods for transport equations by Broersen-Dahmen-Stevenson (2015) and others.

Minimal acceleration for dissipative solutions
Advisor: Michael Westdickenberg

Well-posedness of the compressible Euler equations of gas dynamics is still on open problem in more than one space dimension. Recent work by De Lellis-Székelyhidi (2009) and others showed that for suitable initial data infinitely many weak solutions can be constructed using techniques inspired by convex integration. It is still unclear whether a suitable entropy condition can restore uniqueness and result in the well-posedness of the problem. In this research project we will investigate an approach based on a suitable selection procedure: Within the class of dissipative solutions, a solution concept akin to measure-valued solutions, we seek to find the one that minimizes the acceleration (or the total energy) at a.e. instance of time. The existence of such a minimizing dissipative solution has been established recently. In a sense, this solution is as close to a weak solution as possible. We will further investigate the properties of such extremal solutions and explore possible connections with the approach by De Lellis-Skékelyhidi.

Selection principles for transport equations
Advisor: Michael Westdickenberg

It is well-known for ordinary differential equations of the form $\dot{x} = \mathbf{b}(t,x)$, that if the velocity field $\mathbf{b}$ is not Lipschitz continuous, then solutions of the corresponding initial value problem may not be unique. As a consequence, it is not clear how to define a flow: which integral curves should one pick? This difficulty is mirrored in the question whether the transport or continuity equations involving the same velocity field $\mathbf{b}$ may be ill-posed. A lot of effort has been devoted in the last decades to identify assumptions on $\mathbf{b}$ that would still allow to define a suitable version of flow, such as Ambrosio’s regular Lagrangian flow. These assumptions typically come in the form of Sobolev- or $\mathrm{BV}$-regularity of $\mathbf{b}$. In this research project we will investigate a different approach, which we introduced recently: Instead of trying to force essential uniqueness of solutions of the underlying ordinary differential equation, we try to select integral curves for a.e. starting point in such a way that the resulting family of curves form a flow. This approach is motivated by Krylov’s Markov selection in the theory of stochastic processes and is possible under fairly weak regularity assumptions on the velocity. Once a flow is selected, the corresponding transport problem is well-posed.

Wasserstein-closure for moment equations
Advisors: Michael Herty, Manuel Torrilhon

The closure problem of moment equations of the Boltzmann equation requires the inference of a distribution of the particle velocities from a finite set of moments. This reconstruction task is underdetermined and requires additional conditions on the distribution, like the minimization of a certain functional. Classic closures use a weighted $L^2$-norm (Grad-Hermite closure) or the entropy (minimum-entropy closure) as functional. In this project we want to explore a closure that minimizes the Wasserstein distance of the distribution to the local equilibrium (Maxwell) distribution under the constraint of given moments. Through the interpretation as optimal transport this closure would also have a physical motivation of choosing a distribution that requires minimal transport costs of the particle from the equilibrium state into the current state. In some sense this also resonates with the entropy approach where the most likely distribution is chosen. The optimal transport map would have implications on the resulting structure of the moment equations, like hyperbolicity and H-theorem. Numerically, this closure would be implemented in a discretized distribution setting and tested on scenarios like a Riemann problem or stationary shock profile.

Machine Learning

Kinetic Theory For Swarm-Based Optimization Methods
Advisors: Michael Herty, Manuel Torrilhon

In the context of large–scale machine learning problems modern optimization methods are based on the evolution of a particle system. Here, the particles evolve in a possibly highly non-trivial landscape towards a (local) minimum of a suitable energy functional. Currently most of the proposed methods are heuristics and suitable convergence results are still incomplete. In this project we propose to analyze and exploit possible energy and entropy dissipation properties of corresponding mean-field equations to gain insight on convergence rates and stability of the discovered local minima. A first preliminary example has been studied by Carrillo et al. (2018) where a set of particles evolves within an energy landscape according to a Kuramoto-Vicsek type model. The particles are instantaneously driven towards a weighted average of the particle positions which represents an approximation of the function minimum. An additional stochastic component is here required to ensure robustness of the method and to obtain convergence estimates for the corresponding mean-field equation. In this project, we plan to extend the concept of particle methods in several directions. This will be done by examining a variety of dynamic models also posed on suitable, not necessarily metric, spaces that will be studied both analytically and numerically. Moreover, we plan to extend the current results on mean-field level to Boltzmann-type equations that allow for the existence of an analogue to the classical H-Theorem. Large time behavior of the system will also be studied by suitable mean-field descriptions as well as constrained dynamics. Ideally, decay rates for suitable energy and entropy functionals in Wasserstein distance will be obtained. Furthermore, the stability of the local minima will be investigated. Finally, the theoretical results should be accompanied by fast and efficient, possibly higher-order discretizations to obtain feasible methods. Those will also be used to solve numerically general constrained optimization problems.

Gradient flows on matrix spaces
Advisor: Michael Westdickenberg

Gradient flows have been extensively studied in the context of Hilbert spaces or Riemannian manifold for a very long time. In recent decades, gradient flows have also been considered on spaces of probability measures since various parabolic partial differential equations can be (formally) interpreted as a gradient flows with respect to a suitable metric structure, induced for example by the Wasserstein distance. This has led—among other things—to the development of the theory of curves of maximal slopes, which generalizes gradient flows to rather general metric spaces. In this research project, we will investigate gradient flows on space of matrices where completely new features appear due to the non-commutativity of the matrix multiplication. There are two important applications for this research: First, gradient flows on matrix spaces occur naturally in the context of dissipative quantum systems, as observed by Carlen-Mas (2014). Second, deep learning of neural networks often amounts to a minimization over a space of weights, which are matrices. These minimization problems can be solved using gradient descent methods, which can be understood as time discretized versions of gradient flows. Since the objective functions are typically non-convex, there are many open problems related to the convergence of such descents/flows and the characterization of their long-time limits.

Infinite depth/width limits of neural networks
Advisors: Maria G. Westdickenberg, Michael Westdickenberg

Mathematically, deep learning of artificial neural networks can be understood as the attempt to find a function that maps given data to expected outcomes. A prototypical example is to learn, from an image of a hand-written letter, which letter the image actually represents. The ansatz space for this approximation problem consists of functions that are compositions of affine maps and pointwise activation functions (such as the rectified linear unit ReLU), over several layers. Deep learning then amounts to optimizing the matrix weights and other parameters that specify the affine functions in each layer. Typically they are defined on high-dimensional vector spaces, where the dimension can be the number of pixels of an image, for example. In this project, we will be interested in the limits of either (1) infinite width: the dimension of the vector spaces is sent to infinity, in which case matrices are replaced by functions/measures; or (2) inifinite depth, i.e., the number of layer becomes unbounded. In either of these cases, the discrete nature of artificial neural networks gives way to continuously defined structures whose properties can be studied using ordinary or partial differential equations. In the case of infinite depth, for example, deep learning can sometimes be rephrased as an optimal control problem; for infinitely wide networks, one may obtain a description in terms of stochastic processes and Fokker-Planck-equations.

Materials Science

Hamiltonian vortex dynamics
Advisors: Christof Melcher, Umberto L. Hryniewicz

This project aims to explore the connection between Landau-Lifshitz equations and point vortex systems as an example for finite dimensional reductions of Hamiltonian systems. From a general perspective this is a very difficult task. Compared to dissipative gradient systems the applicability of asymptotic variational methods and regularity theory is rather limited. Rather than aiming at a general theory, we shall focus on special classes of solutions, in particular periodic solutions. Popular approaches are based on variational principles taking into account conserved quantities arising from symmetries, and appear to be a good starting point for our investigations. A further step is to connect more complex many body type dynamics of point vortices to non-trivial periodic solutions of the Landau-Lifshitz equation.

Entropic effects in stochastically perturbed PDEs
Advisors: Michael Herty, Maria G. Westdickenberg

It is well-known that large deviation theory provides sharp exponential bounds on rare events in the small noise limit. Entropic effects are less well understood. In an Allen-Cahn type equation with an asymmetric two-well potential, the broad well is preferred by the stochastic process over the narrow one even though the local minimizers of the potential have the same energy. As a consequence, the equilibrium distribution of transition layers is no longer uniform and pairs of transition layers tend to stay close together. The entropic effect of the large system size, on the other hand, is that transition layers tend to spread. The dynamics of stochastically perturbed PDEs in the presence of entropic effects can be formally approximated using a generalization to infinite dimensions of a theorem of Katzenberger. Some of the effects observed there are ones that we would like to look at more closely numerically and analytically. This project combines the development of efficient numerical methods for stochastic dynamics with the analysis of coarsening and nucleation events. Part of the work extends the notion of asymptotic preserving schemes to this context in order to correctly resolve dynamics independent of the numerical discretization. The topic of asymptotic preserving schemes has been studied extensively, e.g., in the context of kinetic equations and their hyperbolic limit. Also, in the context of continuous optimization, studies of limiting schemes and limiting optimization problems have been investigated recently in simpler systems.

Global existence of Mullins-Sekerka in two dimensions
Advisors: Umberto L. Hryniewicz, Maria G. Westdickenberg

For an initial curve asymptotically close to a circle, Xinfu Chen (1993) established global existence of the Mullins-Sekerka evolution on a bounded two dimensional domain. His method relies on linearization and a Schauder contraction argument. Joint work of ours with Chugreeva and Otto (2019) applied a more nonlinear, energy-based argument to establish sharp convergence rates to the half-plane for initial data order one away from that state. Extending such a framework to functions order one away from a circle requires new estimates and/but could yield a much stronger global existence result as well as convergence rates. The underlying theme/tool is the use of our Distance-Energy-Dissipation method developed in Otto-Westdickenberg (2014), which extends an observation of Brézis to the nonconvex setting.

Optimal Transport

Monge-Ampère-type equations for reflector/refractor problems
Advisors: Siegfried Müller, Michael Westdickenberg

Optimal transport is concerned with the problem of moving one probability measure into another one while minimizing the total cost of the transport. If the cost is given by the squared distance, then the optimal transport plan is contained in a cyclically monotone set, i.e., the subdifferential of a convex function. With sufficient regularity, the condition expressing compatibility of the initial and final measures then amounts to a Monge-Ampère equation for the transport map, which is a nonlinear elliptic equation. It is indeed possible to solve the optimal transport problem by solving the Monge-Ampère equation. In the available literature the main focus is on solving a suitable discretization of this nonlinear equation using some sophisticated variant of Newton’s method, for example. What has not been fully addressed, however, is the question of monotonicity of the transport map. This will be investigated in this research project. Similar issues arise in the design of mirrors/lenses that have the property that light from a point source is reflected/refracted so that the resulting intensity matches a prescribed light distribution. The goal of this project is to develop efficient numerical methods for these and related problems.

Symplectic Geometry

Polynomial and exponential growth of periodic motions in low-dimensional Hamiltonian systems
Advisors: Umberto L. Hryniewicz, Michael Westdickenberg

The notion of entropy in dynamical systems (topological or measure theoretic) goes back to the work of Kolmogorov, Bowen, and Adler, Konheim, and McAndrew. Topological entropy is a conjugacy invariant that quantifies chaotic behaviour inside the system. During the second half of the twentieth century entropy has become a central concept in dynamical systems theory and remarkable results have been proved. For instance, Katok showed that (in low dimensions) a smooth system with positive topological entropy admits compact invariant sets where the dynamics is conjugated to that of a topological Markov chain with exponential growth of periodic orbits. This research project is about detecting entropy in low dimensional Hamiltonian systems. We wish to explore to what extent a given collection of periodic orbits of a smooth area-preserving diffeomorphism on a surface, or of a Hamiltonian flow on a three-dimensional energy level, forces positivity of the entropy. Notice that there are situations where the ambient phase space has a complicated structure that forces positive entropy for all Hamiltonian systems defined on it. This will, however, not be the focus of this research project. Instead our intention is to study simple phase spaces that do admit zero entropy systems, and then to investigate patterns that will force the entropy to be positive.

Variational convergence and symplectic homogenization
Advisors: Umberto L. Hryniewicz, Heiko von der Mosel

The aim of this research project is to investigate systematically the relations between the $\Gamma$-convergence of energy functionals when homogenized in configuration variables, and the corresponding limiting processes on the dual Hamiltonian side. Possible applications include regularity problems for obstacle problems for elastic rods where a varying parameter in the energy functional could penalize extensibility of the minimizing rod in favour of an inextensible limit regime. Sets of vanishing velocity of the extensible rod’s centerline, which cause regularity problems, will then be reduced during the limiting process. More generally, the dependence of holonomic constraints in dynamical systems on varying parameters in the energy density is reflected on the Hamiltonian side, and this needs to be examined rigorously in the various multiplier formulations suggested, e.g., by Gonzalez, Maddocks, and Pego. On the symplectic side, the Lagrangian point of view might be a way to better understand the topology of the group of Hamiltonian diffeomorphisms.