Paper proposes a method to reduce hallucinations in diffusion models using Laplacian score sharpening.
problem Hallucinations in diffusion models create incoherent or unrealistic samples.
method Post-hoc adjustment to the score function during inference using Laplacian approximation.
result Significantly reduces the rate of hallucinated samples across various data types.
The paper improves GP regression for sparse sensor data in structural mode shape reconstruction.
problem Reconstructing full-field structural mode shapes from sparse sensor data.
method Physics-Constrained Single-Output Gaussian Process (CONS-SOGP) framework.
result The proposed method provides more accurate and reliable mode shapes.
This work ensures stability in POD basis interpolation for pMOR in hyperelasticity.
problem Stability of POD basis interpolation on Grassmann manifolds for pMOR in hyperelasticity.
method Stability conditions derived from Grassmannian Exponential map and principal angles.
result Explicit stability conditions for practical pMOR applications and non-monotonic error behavior.
A new GAN model α-GAN with tunable loss function addresses gradient vanishing and mode collapse issues.
problem Addressing vanishing gradients and mode collapse in GANs.
method Introduced a tunable GAN α-GAN using a supervised α-loss function. result Holistic understanding of α-GAN related to Arimoto divergence and convergence properties. New MCMC method improves sampling from multimodal distributions.
problem Sampling from multimodal distributions is challenging for classical MCMC methods.
method Interpolating along the diffusion path, preserving mode weights and mixing properties.
result MAD-Path sampler improves global exploration and mode-weight estimation.
The paper optimizes interpolation schedules in generative models to improve sampling accuracy.
problem Improving sampling accuracy in generative models with fewer resources.
method Minimizing the averaged squared Lipschitzness of the drift field, using transfer formulas.
result Designed schedules yield more accurate fine-scale statistics at fixed integrator budget.
Proposes AEGAN for stable GAN training.
problem Training instability in GANs.
method Four-network model with adversarial and reconstruction losses.
result Stabilizes GAN training and prevents mode-collapse.
Survey of GANs and autoencoders, addressing mode collapse and likelihood issues.
problem Addressing mode collapse and likelihood issues in GANs and autoencoders.
method Explains various GAN and autoencoder variants, their applications, and methods to resolve issues.
result Various methods to resolve mode collapse and improve likelihood in GANs and autoencoders.
We address the problem of imitation learning with multi-modal demonstrations. Instead of attempting to learn all modes, we argue that in many tasks it is sufficient to imitate any one of them. We show that the state-of-the-art methods such as GAIL and behavior cloning, due to their choice of loss function, often incorr…
In portable, three dimensional, and ultra-fast ultrasound imaging systems, there is an increasing demand for the reconstruction of high quality images from a limited number of radio-frequency (RF) measurements due to receiver (Rx) or transmit (Xmit) event sub-sampling. However, due to the presence of side lobe artifact…
This work tackles GAN training instability through parallel tempering.
problem Training instability and mode collapse in GANs.
method Introduces a parallel tempering framework to stabilize GAN training.
result Significantly reduces gradient variance and improves training efficiency.
Optimal self-distillation improves generative models' velocity risk and mode recovery.
problem Improving generative models' velocity risk and mode recovery.
method Proved optimal self-distillation for rectified flow via linear probing, derived mixing coefficient, and provided validation tuning.
result Optimal self-distillation improves velocity risk and mode recovery.
LMC loss barrier decreases to zero with large network width.
problem Understanding the LMC phenomenon in neural networks.
method Fine-grained analysis of LMC for two-layer ReLU networks.
result LMC loss barrier decreases to zero at a rate of O(m^-1/2) for large network width.
The convergence rate and final performance of common deep learning models have significantly benefited from heuristics such as learning rate schedules, knowledge distillation, skip connections, and normalization layers. In the absence of theoretical underpinnings, controlled experiments aimed at explaining these strate…
LoRA-Curve connects independent LoRA optima through continuous low-loss valleys, improving Bayesian model averaging.
problem Challenges in estimating epistemic uncertainty in LoRA-based Bayesian inference.
method Introduces LoRA-Curve, a segmented Bézier curve parameterization in the LoRA space, with free and anchored configurations.
result Empirically shows that connecting independent LoRA optima through continuous low-loss valleys improves mutual information of the predictive distribution.
Unified framework connects different neural network models.
problem Understanding the geometry of neural network loss landscapes.
method Unified framework capturing four symmetry classes.
result First discovery of low- and zero-barrier linear interpolation paths.
Periodic surfaces have a limited number of bending modes, equal to their membrane modes.
problem Understanding the limitations of bending modes in periodic surfaces.
method Analyzing deformation modes of periodic, piecewise smooth, simply connected surfaces.
result Effective membrane modes and bending modes are orthogonal, limiting the total number of modes to 3.
A new method uses string method to explore diffusion models.
problem Understanding the geometry of learned distributions in diffusion models.
method String method to compute continuous paths between samples.
result The string method identifies realistic morphing sequences and transition pathways.
Sparse-mode DMD disambiguates local and global modes in spatiotemporal data.
problem Disambiguating local and global modes in spatiotemporal data.
method Sparse-mode DMD with sparsity-promoting regularization.
result Explicitly constructs discrete and continuous spectra.
Motivated by advantages of current-mode design, this brief contribution explores the implementation of weight matrices in neuromemristive systems via current-mode memristor crossbar circuits. After deriving theoretical results for the range and distribution of weights in the current-mode design, it is shown that any we…
This paper shows how forward rate interpolations are equivalent to discount factor interpolations in yield curve construction.
problem The challenge of choosing between different interpolation methods for yield curve construction.
method Demonstrates the equivalence between forward rate interpolations and discount factor interpolations.
result Some popular interpolation methods on forward rates are equivalent to classical interpolation methods on discount factors.
For dynamical systems that can be modelled as asymptotically stable linear systems forced by Gaussian noise, this paper develops methods to infer or estimate their modes from observations in real time. The modes can be real or complex. For a real mode, we wish to infer its damping rate and mode shape. For a complex mod…
Characterizes neutral deformation modes of minimal surfaces.
problem Understanding the energy content of deformation modes of minimal surfaces.
method Analyzes the energy content of stretching, drilling, and bending modes of minimal surfaces.
result All isometries of a minimal surface are globally neutral and give rise to soft elasticity.
Mathematical analysis shows annealing prevents mode collapse in Gaussian mixtures.
problem Mode collapse in variational inference for multimodal distributions.
method Analyzed annealing strategies for Gaussian mixtures, derived formulas, and tested on neural networks.
result Appropriately chosen annealing schemes can robustly prevent mode collapse.
New method estimates log-determinant using trace powers, avoiding classical limitations.
problem Estimating log-determinant of large matrices efficiently and accurately.
method Interpolating moment-generating function and its derivative at zero using trace powers.
result No continuous estimator using finite moments can be uniformly accurate over unbounded conditioning.
Parsimonious Dynamic Mode Decomposition selects sparse modes robustly.
problem Manual tuning of sparsity parameters in traditional DMD.
method Time-delay embedding and Orthogonal Matching Pursuit.
result Autonomously determines optimally sparse subset of modes.
We consider the problem of recovering a low-rank tensor from its noisy observation. Previous work has shown a recovery guarantee with signal to noise ratio O(n⌈K/2⌉/2) for recovering a Kth order rank one tensor of size n×⋯×n by recursive unfolding. In this paper, we first improve…
The paper proposes a method for generating uniform interpolations on data manifolds.
problem Generating high-quality interpolations between data samples on complex manifolds.
method Autoencoder network with interpolation network, regularized by a Riemannian metric.
result The method generates interpolations that remain within the manifold's distribution.
Geodesics connect model modes in neural network loss landscapes.
problem Connecting modes in neural network loss landscapes.
method Reframed mode connectivity in Information Geometry, hypothesized geodesics as mode-connecting paths, proposed algorithm to approximate geodesics.
result Geodesics achieve mode connectivity in neural networks.
A new, efficient k-modes algorithm improves clustering of categorical data.
problem Clustering categorical data using existing methods like k-means is inefficient. method Developed a novel k-modes algorithm called OTQT, which improves on existing methods. result OTQT finds more accurate clusters per iteration and is faster overall.
Multimodal clustering is an unsupervised technique for mining interesting patterns in n-adic binary relations or n-mode networks. Among different types of such generalized patterns one can find biclusters and formal concepts (maximal bicliques) for 2-mode case, triclusters and triconcepts for 3-mode case, closed $n…
New FX option interpolations impact implied volatilities.
problem Different interpolations of FX option quotes lead to varying implied volatilities.
method Analysis of various exact interpolations of broker quotes.
result Different interpolations result in different implied volatilities.
EDLP samples flat modes in discrete spaces using entropy.
problem Sampling flat modes in discrete spaces is challenging.
method EDLP uses a continuous auxiliary variable and local entropy to guide sampling.
result EDLP consistently outperforms traditional methods in various tasks.
Proposes a Gaussian process for Koopman mode decomposition.
problem Estimating Koopman mode decomposition quantities and latent variables.
method Unsupervised Gaussian process for simultaneous estimation.
result Efficient parameter estimation through low-rank approximations.
Deep learning helps remove secondary B-mode polarization to detect primordial gravitational waves.
problem Removing secondary B-mode polarization from CMB data to detect primordial gravitational waves. method Applied deep learning (ResUNet-CMB) to estimate and remove multiple sources of secondary B-mode polarization. result Deep learning can produce nearly optimal, unbiased estimates of the amplitude of primordial gravitational waves.
Kernel interpolation is inconsistent for norms with smoothness above a constant.
problem Inconsistency of kernel interpolation in reproducing kernel Hilbert spaces.
method Lower bounds for generalization error in Sobolev norms.
result Kernel interpolation is always inconsistent for norms with smoothness above a constant.
Bayesian interpolants explain neural network inferences concisely.
problem Understanding neural network inferences.
method Adapting Craig interpolants for neural networks.
result Produces precise, understandable explanations.
A new method for continual learning in GANs learns new modes with limited data.
problem Learning new target modes with limited samples while preserving previously learned ones.
method Mode-affinity score for generative modeling, generator replay, and weighted label generation.
result Gains over state-of-the-art methods, even with fewer training samples.
Near-interpolating models grow norms quickly, affecting generalization.
problem Understanding the trade-off between interpolation and generalization in near-interpolating models.
method Random matrix theory and eigendecay analysis of data covariance matrix.
result Near-interpolating models exhibit rapid norm growth and worse generalization trade-offs.
The paper improves interpolation in generative models by using specific base distributions.
problem Unexpected side effects in linear interpolations of normalizing flows.
method Enforces a specific manifold using Dirichlet and von Mises-Fisher base distributions.
result Superior performance in terms of bits per dimension, FID, and KID scores for interpolation.
Empirical study shows GANs overfit and drop modes when training is deterministic.
problem Understanding overfitting and mode drop in GAN training.
method Empirical analysis of GAN training with and without stochasticity.
result GANs overfit and drop modes when training is deterministic.
The paper finds shape modes for vortices in a specific sigma model.
problem Existence of internal modes in CP1 vortices. method Developed a geometric formalism based on the Bogomol'nyi decomposition of the energy functional.
result Proved the existence of at least one shape mode for a general CP1 vortex solution. Deep neural networks can interpolate any dataset in the overparametrized regime.
problem Interpolating any dataset with deep neural networks in the overparametrized regime.
method Proving universal approximations and interpolating any dataset with deep neural networks, considering specific conditions on activation functions.
result Interpolation of any dataset is possible in the overparametrized regime with deep neural networks.
Interpolation hurts robust generalization even without noise.
problem The challenge of robust generalization in the absence of noise.
method Avoiding interpolation through ridge regularization.
result Ridge regularization improves robust generalization.
Dynamic Mode Decomposition (DMD) yields a linear, approximate model of a system's dynamics that is built from data. We seek to reduce the order of this model by identifying a reduced set of modes that best fit the output. We adopt a model selection algorithm from statistics and machine learning known as Least Angle Reg…
This paper is inspired from the nice result of Andrew Hassell on the eigenfunctions in the stadium billiard. From a classical paper of V. Arnol'd, we know that quasi-modes are not always close to exact modes. We show that, for almost all Riemannian metrics on closed surfaces with an elliptic generic closed geodesic C, …
Uniform convergence of interpolators proven for Gaussian data.
problem Interpolation learning in high-dimensional linear regression with Gaussian data.
method Generic uniform convergence guarantee in terms of Gaussian width.
result Consistency of interpolators for minimum-norm and near-minimal-norm cases.
The paper characterizes vector fields as interpolating sesqui-harmonic maps on Riemannian manifolds.
problem Characterizing vector fields as interpolating sesqui-harmonic maps on Riemannian manifolds.
method Characterization theorem and critical point condition for interpolating sesqui-harmonic vector fields.
result Conditions for vector fields to be interpolating sesqui-harmonic maps on compact manifolds.