Study geometric flows with varying parameters and prove continuous dependence.
problem Continuous dependence of flows on parameters in geometric settings.
method Derived suitable topologies for vector fields and flows, proved new continuous dependence.
result Proved continuous dependence of flows on parameters in a general topological space.
CAM-GAN improves GANs for continual learning with efficient feature map transformations.
problem Efficient continual learning for GANs with reduced parameter growth.
method Designing and leveraging parameter-efficient feature map transformations, including global and task-specific parameters, residual bias, and Fisher information matrix.
result Significantly improved model performance and high-quality samples with fewer parameters.
Continuized Nesterov acceleration accelerates stochastic gradient descent and gossip algorithms.
problem Improving the convergence rate of stochastic gradient descent and gossip algorithms.
method Introducing a continuized variant of Nesterov acceleration, which mixes variables continuously and takes gradient steps at random times.
result The continuized Nesterov acceleration achieves convergence rates similar to Nesterov's original acceleration but with random parameters.
Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data. SGDCT performs…
A new framework for federated continual learning reduces interference and improves performance.
problem Learning from a sequence of tasks with limited data from each client.
method Federated Weighted Inter-client Transfer (FedWeIT) framework.
result FedWeIT significantly outperforms existing methods with reduced communication cost.
While recent continual learning methods largely alleviate the catastrophic problem on toy-sized datasets, some issues remain to be tackled to apply them to real-world problem domains. First, a continual learning model should effectively handle catastrophic forgetting and be efficient to train even with a large number o…
PARMESAN learns from memory without parameters for fast, efficient continual learning.
problem Inflexibility in deep learning methods for continual learning.
method Transductive reasoning and memory search for parameter-free learning.
result 3-4 orders of magnitude faster than baselines, comparable performance.
New bounds on continuous random variables' right-tail probabilities.
problem Finding precise upper and lower limits for right-tail probabilities of continuous random variables.
method Developed new bounds based on PDF, first derivative, and two parameters.
result The new bounds are tight for various continuous random variables.
Paper proves EM algorithm convergence for mixtures of discrete and continuous parameters.
problem Nontrivial convergence analysis for EM algorithms with mixed-integer parameters.
method Introduces conditions for EM convergence in mixed-integer optimization.
result Proves convergence of EM-based sparse Bayesian learning algorithm.
CLOPS improves deep learning for continuous physiological data.
problem Deep learning struggles with streaming, multi-sensor clinical data.
method CLOPS is a replay-based continual learning strategy.
result CLOPS outperforms state-of-the-art methods in continual learning scenarios.
New method prevents forgetting in LLMs by dynamically identifying task-specific subspaces.
problem Catastrophic forgetting in continual learning of LLMs.
method Adaptive Singular Value Decomposition (SVD) for constrained full fine-tuning.
result Achieves state-of-the-art results in continual learning benchmarks.
Proposes a new method for continual learning in neural networks.
problem Challenges in applying sequential Bayesian inference to neural networks.
method Sequential function-space variational inference.
result Neural networks trained with the proposed method achieve better predictive accuracy.
Continuous-time analysis shows SGD with noise prefers flat minima.
problem Optimizing neural networks using SGD with noise.
method Continuous-time model for SGD with noise analysis.
result Optimization prefers flat minima in certain noise regimes.
Continuous time Bayesian networks (CTBNs) describe structured stochastic processes with finitely many states that evolve over continuous time. A CTBN is a directed (possibly cyclic) dependency graph over a set of variables, each of which represents a finite state continuous time Markov process whose transition model is…
Optimized neural network approximates high-dimensional functions with minimal parameters.
problem Achieving optimal approximation of high-dimensional continuous functions with minimal parameters.
method Developed a neural network with a specific activation function and architecture to achieve super approximation property.
result A composed network with at most 10889d + 10887 nonzero parameters achieves super approximation property, suggesting optimality in parameter growth.
Method estimates mixture components without discretizing parameters.
problem Learning from mixtures of continuous features with noise.
method Off-the-grid optimization method for continuous parameter space.
result Prediction error bound similar to Lasso predictor.
EVCL combines VCL and EWC to prevent forgetting new tasks.
problem Preventing catastrophic forgetting in continual learning.
method Hybrid model integrating VCL and EWC.
result Consistently outperforms baselines in learning new tasks.
We define a family of kernels for mixed continuous/discrete hierarchical parameter spaces and show that they are positive definite.
We model how Lipschitz continuity changes during neural network training.
problem Understanding how Lipschitz continuity evolves during training.
method We use a system of stochastic differential equations to capture the dynamics of Lipschitz continuity under SGD.
result We identify three factors driving the evolution of Lipschitz continuity: gradient flow projection, gradient noise, and Hessian projection.
Solves complex equation for specific geometric solitons.
problem Solving complex Monge-Ampère equation for specific geometric solitons.
method Aubin continuity path and continuity method.
result Initial value of the path parameter has a solution and is open to all.
This paper proposes the first-ever algorithmic framework for tuning hyper-parameters of stochastic optimization algorithm based on reinforcement learning. Hyper-parameters impose significant influences on the performance of stochastic optimization algorithms, such as evolutionary algorithms (EAs) and meta-heuristics. Y…
Efficiently learns deep factor graphs using Gaussian belief propagation.
problem Learning in deep factor graphs with efficient inference.
method Treats all relevant quantities as random variables, uses belief propagation for inference.
result Efficiently solves training and prediction problems in deep factor graphs with belief propagation.
Evolutionary Strategies optimize hyper-parameters for off-policy learning.
problem Hyper-parameter sensitivity in off-policy learning.
method Application of Evolutionary Strategies for online hyper-parameter tuning.
result Our method outperforms state-of-the-art baselines.
Paper uses diffusion models to design electric aircraft quickly.
problem Designing electric aircraft efficiently and accurately.
method Simulation-based inference with hierarchical diffusion models.
result Rediscovers known aircraft design trends and laws.
New method optimizes SDE models using continuous-time gradient descent.
problem Optimizing over the stationary distribution of SDE models.
method Continuous-time stochastic gradient descent for SDE models.
result Asymptotic convergence to the direction of steepest descent.
Estimates signals from a continuous dictionary with sparse mixtures using optimization.
problem Estimating signals from a continuous dictionary with unknown mixtures and noise.
method Formulates a regularized optimization problem with data fidelity and (ℓ1,Lp)-penalty. result High probability bounds on prediction error for the Group-Nonlinear-Lasso solution.
Paper tackles continual learning with single-index models, proving regret bounds.
problem Continual learning with single-index models across multiple tasks.
method Proposes a randomized strategy to learn a common single-index and task-specific link functions.
result Proves regret bounds for the proposed strategy under various loss function assumptions.
New framework discovers non-affine continuous symmetries in neural networks.
problem Lack of efficient methods for detecting non-affine continuous symmetries in neural networks.
method Computational framework for discovering infinitesimal generators of multi-parameter group actions.
result Framework can discover non-affine continuous symmetries in neural networks.
This paper tackles continuous domain generalization, improving model performance across unseen domains.
problem Existing domain generalization approaches fail to capture the complex, multidimensional nature of real-world variation.
method Introduces Continuous Domain Generalization (CDG), a principled framework grounded in geometric and algebraic theories. Proposes a Neural Lie Transport Operator (NeuralLio) for structure-preserving parameter transitions and a gating mechanism for robust generalization.
result Demonstrates significant improvement in generalization accuracy and robustness across various datasets.
Proposes efficient model for continual learning that grows model over task-specific parameters.
problem Limited transfer learning ability and forgetting of earlier knowledge in existing methods.
method Filter and channel expansion method that grows model over previous task parameters.
result Better knowledge transfer and improved performance in task incremental learning.
Proposes a new estimator for causal mediation with continuous treatments.
problem Estimation of direct and indirect effects with continuous treatments.
method Kernel smoothing approach with cross-fitting for non-parametric estimation.
result Multiply robust and asymptotically normal estimator for continuous treatments.
We introduce two approaches for combining neural evolution strategy (NES) and proximal policy optimization (PPO): parameter transfer and parameter space noise. Parameter transfer is a PPO agent with parameters transferred from a NES agent. Parameter space noise is to directly add noise to the PPO agent`s parameters. We…
Training-free model learns SDE dynamics without training, accelerating parameter studies.
problem High computational cost of simulating parameter-dependent SDEs.
method Training-free conditional diffusion model with joint kernel-weighted Monte Carlo estimator.
result Accurate approximation of conditional distributions across varying parameter values.
Consider the one-parameter generalizations of the logarithmic and exponential functions which are obtained from the integration of non-symmetrical hyperboles. These generalizations coincide to the one obtained in the context of non-extensive thermostatistics. We show that these functions are suitable to describe and un…
La-MAML improves fast online continual learning with a look-ahead approach.
problem Fast online continual learning with limited model capacity.
method Optimisation-based meta-learning with look-ahead and episodic memory.
result Superior performance on visual classification benchmarks.
Study improves policy search in continuous control by using heavy-tailed distributions.
problem Challenges in continuous space policy search due to non-convexity and myopic-farsighted incentives.
method Introduced heavy-tailed policy parameterizations and analyzed convergence rates and stability.
result Convergence rate to stationarity depends on policy's tail index and exploration tolerance.
Paper improves parameter estimation of continuous distributions using preference feedback.
problem Improving parameter estimation of continuous distributions.
method Preference-based M-estimators and deterministic preferences.
result Preference-based estimators achieve an estimation error scaling of O(1/n), significantly faster than sample-only methods.
New framework generalizes neural network parameters to C∗-algebra for more efficient feature learning.
problem Efficient feature learning and adaptability of neural network models.
method Generalizes neural network parameters to C∗-algebra-valued parameters and combines models continuously. result Shows improved feature learning with limited data using the new framework.
Invertible DenseNets improve model efficiency and performance.
problem Improving model efficiency and performance in neural networks.
method Enforcing invertibility in DenseNets by satisfying the Lipschitz constraint and proposing a learnable concatenation.
result i-DenseNets outperform Residual Flows in negative log-likelihood on various datasets.
New neural network architecture with height adds expressive power.
problem Expressiveness of neural networks limited by width and depth.
method Introduces height as a new hyper-parameter in neural network architecture.
result Neural networks with height achieve significantly better approximation of functions.
We revisit the problem of estimating the parameters of a partially observed diffusion process, consisting of a hidden state process and an observed process, with a continuous time parameter. The estimation is to be done online, i.e. the parameter estimate should be updated recursively based on the observation filtratio…
Efficiently scales continuous kernels with sparse Fourier domain learning.
problem High computational and memory demands, spectral bias in continuous kernels.
method Sparse learning in the Fourier domain.
result Efficient scaling of continuous kernels, reduced computational and memory requirements, mitigated spectral bias.
A new method learns from multi-modal sequences with external memory.
problem Learning new modes in a dynamic environment without prior knowledge.
method Maintains a neural episodic memory with a Dirichlet Process prior to store mode descriptors and transfers knowledge through retrieval.
result Performs continual learning favorably compared to mainstream approaches.
Solves complex Monge-Ampère equation for Kähler-Ricci solitons.
problem Complex Monge-Ampère equation for shrinking gradient Kähler-Ricci solitons.
method Aubin continuity path, implementing another continuity method.
result Shows existence of solution for the equation.
New CTBNs with clocks allow for non-exponential survival times.
problem Modeling phenomena with non-exponential survival times in continuous time.
method Introduced node-wise clocks to construct graph-coupled semi-Markov chains, enabling non-exponential survival times without auxiliary states.
result Parameter and structure inference algorithms provided, demonstrating advantages over current CTBN extensions.
New insights into continual learning for deep models, showing convergence issues but local linear solutions.
problem Challenges in continual learning for homogeneous deep models.
method Sequential projections onto task margin sets, leveraging nonconvex projection theory.
result Local linear convergence under certain conditions for homogeneous deep networks.
Indecomposable symmetric Lorentzian manifolds of non-constant curvature are called Cahen-Wallach spaces. Their isometry classes are described by continuous families of real parameters. We derive necessary and sufficient conditions for the existence of compact quotients of Cahen-Wallach spaces in terms of these paramete…
George Cybenko's landmark 1989 paper showed that there exists a feedforward neural network, with exactly one hidden layer (and a finite number of neurons), that can arbitrarily approximate a given continuous function f on the unit hypercube. The paper did not address how to find the weight/parameters of such a networ…