Thèses

Jeudi 31 Mars 2022 à 13h30.

Model-based and model-free macroscopic control of the shape of low-dimensional systems with stochastic and deterministic dynamics

''This thesis explores the control of the morphology of extended physical systems involving stochastic or nonlinear dynamics. More precisely, we are interested in the problem of achieving in finite time a target morphology that is as arbitrary as possible through "model-based" or "model-free" control methods. In the model-based approach, complete knowledge of the physical laws governing the system is exploited to compute an optimal control strategy. If this knowledge is not available, a control strategy can be learned by the model-free approach by interacting with the dynamical system itself and "reinforcing" the actions that maximize a certain reward signal.

We have applied model-based control to the case of small two-dimensional islands of a few particles under the effect of an external macroscopic field, such as an electric field or a temperature gradient, which acts as a control parameter to reach a target shape. This model describes single-layer clusters of atoms, nanoparticles, or colloids. We considered the case of a dynamics governed by stochastic diffusion of particles along the periphery of the island, which thus conserves the number of particles during the dynamics. Reaching a target shape for the island can be seen as a first-passage problem in the space of configurations, and the choice of the external field can be studied in the framework of Markov decision processes. The finite number of configurations allowed us to apply tabular algorithms (which list all the states of the system in an array). We have also derived some analytical results using a high temperature expansion.

In the absence of an external field, we showed that sufficiently large compact shapes exhibit an optimal temperature at which they are reached in minimum time. In the presence of field, we used the so-called dynamic programming method to solve the Markov decision process and find an optimal control strategy to reach the target shapes in minimum time. This strategy results in a time gain that increases when increasing the size of the islands or decreasing the temperature. Moreover, the optimal strategy is not unique, and its degeneracy is mainly related to the symmetries of the system. Furthermore, the optimal strategy presents a discrete set of transitions as the temperature varies. As the cluster size increases, a continuous density of transitions emerges.

With model-free control, we modeled a situation that mimics an experimental setup, where an automated controller learns to manipulate a small cluster. We have shown that the reinforcement learning methods of Monte Carlo and Q-learning type allow one to reach control performances close to the optimal performances computed by dynamic programming, except at high temperature where the fluctuations are important and the influence of the external field is weak.

Finally, we have studied the model-free control of the morphology of extended deterministic continuous systems. The control is obtained by adjusting a global parameter that governs the stability of the system, such as the temperature difference from the critical point in a system undergoing a phase transition. In the framework of a one-dimensional model based on a time-dependent generalization of the Allen-Cahn (or Ginzburg-Landau) equation, we have been able to control the number of phase domains appearing in the system by using neural network-based approximation techniques (deep Q-learning). Preliminary results have also allowed us to classify the shape of domains undergoing a fingering instability obtained by a two-dimensional phase-field model, a result that marks a first step towards the morphological control of these systems.

Membres du jury :

Médéric Argentina, Professeur des Universités, Université Côte d'Azur (Rapporteur)
Nicolas Combe, Professeur des Universités, Université Toulouse (Rapporteur)
Catherine Barentin, Professeur des Universités, Université Lyon 1 (Examinatrice)
Laetitia Matignon, Maître de Conférences, Université Lyon 1 (Examinateur)
Francesco Montalenti, Professeur, Université de Milan Bicocca (Examinateur)
Olivier Pierre-Louis, Directeur de Recherche, CNRS Lyon (Directeur de thèse)
Freddy M. Bouchet, Directeur de Recherche, CNRS Lyon (Invité)

''