Title: The Model Behind ChatGPT

Abstract: This will be a general talk to introduce a deep learning model called Generative Pre-training Transformer (GPT). First, I will discuss in concept the machine learning approach for the task of question-answering. Then I will describe language modeling and some models such as recurrent neural network (RNN) models for that task. Finally, I will present the Transformer as well as the GPT models that have led to ChatGPT.

Title: The nonlinearity of regulation in biological networks

Abstract: The extent to which the components of a biological system are (non)linearly regulated determines how amenable they are to therapy and control. To better understand this property termed `regulatory nonlinearity', we analyzed a suite of 137 published Boolean network models, containing a variety of complex nonlinear interactions, using a probabilistic generalization of Boolean logic that George Boole himself had proposed. Leveraging the continuous-nature of this formulation, we used Taylor decomposition to approximate the models with various levels of nonlinearity. A comparison of the resulting series of approximations of the biological models with appropriate random ensembles revealed that biological regulation tends to be less nonlinear than expected. A further categorical analysis of the biological models revealed that the nonlinearity of cancer and disease networks could not only be sometimes higher than expected but are also relatively more variable. We show that this variation is caused by differences in the apportioning of information among the various orders of nonlinearity. Taken together, our results suggest, but do not imply, that biological regulation may have evolved to be more linear on average, and certain systems such as cancer may have, on the other hand, evolved to be more nonlinear.

Title: Robust Estimation of Smooth Graph Signals from Randomized Space-time Samples

Abstract: Heat diffusion processes have found wide applications in modeling dynamical system over graphs. In this talk, I will talk about the recovery of a k-bandlimited graph signal that is an initial signal of a heat diffusion process from its space-time samples. In this work, we have proposed three random space-time sampling regimes, termed dynamical sampling techniques, that consist in selecting a small subset of space-time nodes at random according to some probability distribution. We show that the number of space-time samples required to ensure stable recovery for each regime depends on a parameter called the spectral graph weighted coherence, that depends on the interplay between the dynamics over the graphs and sampling probability distributions. Then, we propose a computationally efficient method to reconstruct k-bandlimited signals from their space-time samples. We prove that it yields accurate reconstructions and that it is also stable to noise. Finally, we test dynamical sampling techniques on a wide variety of graphs. The numerical results on support our theoretical findings and demonstrate the efficiency.

Title: A generic framework to coarse-grain stochastic reaction networks by Abstract Interpretation

To make decisions we are guided by the evidence we collect and the opinions of friends and neighbors. How do we combine our private beliefs with information we obtain from our social network? To understand the strategies humans use to do so, it is useful to compare them to observers that optimally integrate all evidence. Here we derive network models of rational (Bayes optimal) agents who accumulate private measurements and observe the decisions of their neighbors to make an irreversible choice between two options. The resulting information exchange dynamics has interesting properties: When decision thresholds are asymmetric, the absence of a decision can be increasingly informative over time. In a recurrent network of two agents, the absence of a decision can lead to a sequence of belief updates akin to those in the literature on common knowledge. We then consider large networks under the same framework. Using a combination of asymptotic methods and first passage time calculations, we find that when the network is sufficiently large, most agents decide correctly irrespective of whether the first agent’s decision is right or wrong. Interestingly, individuals in networks with both hasty and deliberate agents can make the right choice more quickly and more often than in networks of identical agents: Observing the choices of a small group of hasty agents can allow the more deliberate agents to make accurate decisions. Our model is tractable and readily generalizable, paving the way for the future study of different social network topologies. We conclude that diverse groups make quicker, more accurate decisions than homogenous groups.

Title: Extrapolation for eigenvalue problems: Upcycling data for faster convergence

Abstract: We will discuss accelerating convergence to numerical solutions of eigenvalue problems using a simple post-processing step applied to standard eigensolver techniques. First we will consider accelerating the standard power iteration, one of the most basic and powerful but sometimes very slow iterative methods. We will review some recent results on how we can make the power iteration faster by recombining previous iterates to form our next approximation to a solution; and, we will discuss why this works. We can also apply a similar technique to a restarted Arnoldi method to boost its performance with little additional computational cost. Numerical examples will illustrate the theory.

Title: Theory and Algorithms for Nonlinear Eigenvector Problems with Affine-Linear Structures

Abstract: Eigenvector-dependent Nonlinear Eigenvalue Problems (NEPv) have long played critical roles in computational physics and chemistry and are becoming increasingly important in numerous data science applications. In this talk, we consider a class of NEPv where the coefficient matrices have a special affine-linear structure. One important origin of affine-linear NEPv is the Rayleigh-quotient-related optimization, including the trace-ratio optimization for dimension reduction and robust Rayleigh-quotient optimization for handling data uncertainties. We will establish variational characterizations for particular affine-linear NEPv, and then provide a geometric interpretation of a Self-Consistent Fields (SCF) iteration for solving the NEPv. The geometric interpretation reveals the global convergence of SCF in many cases and explains its potential non-convergence issues in others. New improvements to SCF, including the local acceleration schemes and the global verification techniques, are also discussed. Numerical experiments demonstrate the effectiveness of our approach.