Solutions of an implicit ODE form a web. Already for cubic ODEs the 3-web of solutions has a nontrivial local invariant, namely the curvature form. Thus any local classification of implicit ODEs necessarily has functional moduli if no restriction on the class of ODEs is imposed. Here the most symmetric case of hexagona…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “implicit assumptions”
Stochastic recurrent neural networks with latent random variables of complex dependency structures have shown to be more successful in modeling sequential data than deterministic deep models. However, the majority of existing methods have limited expressive power due to the Gaussian assumption of latent variables. In t…
REALFIN benchmarks financial reasoning by removing implicit assumptions, revealing model weaknesses.
problem Models struggle when implicit assumptions are missing, leading to incorrect answers.
method Developed a bilingual benchmark that systematically removes essential premises from financial questions.
result General-purpose models over-commit, while finance-specialized models fail to identify missing premises.
Deep equilibrium models converge globally without explicit computation.
problem Global convergence of deep learning models with implicit layers.
method Analysis of gradient dynamics and proof of convergence rate.
result Deep equilibrium models converge to global optimum at a linear rate.
Most successful machine intelligence systems rely on gradient-based learning, which is made possible by backpropagation. Some systems are designed to aid us in interpreting data when explicit goals cannot be provided. These unsupervised systems are commonly trained by backpropagating through a likelihood function. We i…
This work extends implicit bias analysis to multiclass classification using a new loss framework.
problem The implicit bias of gradient descent on multiclass data without explicit regularization.
method Employing the PERM framework to introduce a multiclass extension of the exponential tail property.
result Extended implicit bias result to multiclass classification using a new loss framework.
PVI improves SIVI by directly optimizing ELBO without parametric assumptions.
problem Intractable variational densities in SIVI methods.
method Particle Variational Inference (PVI) using empirical measures to approximate optimal mixing distributions.
result PVI directly optimizes the ELBO and performs favorably compared to other SIVI methods.
Large models collapse epistemic uncertainty, challenging traditional wisdom.
problem Epistemic uncertainty collapse in large models.
method Implicit ensembling and decomposition techniques.
result Larger models can collapse epistemic uncertainty, contrary to expectations.
In implicit models, one often interpolates between sampled points in latent space. As we show in this paper, care needs to be taken to match-up the distributional assumptions on code vectors with the geometry of the interpolating paths. Otherwise, typical assumptions about the quality and semantics of in-between points…
The paper improves convergence for linear systems using entropic mirror descent with Polyak stepsizes.
problem Convergence analysis for linear systems with unbounded domain.
method Entropic mirror descent with Polyak stepsizes, sublinear and linear convergence results.
result Generalized convergence result for arbitrary convex functions.
We propose graph-dependent implicit regularisation strategies for distributed stochastic subgradient descent (Distributed SGD) for convex problems in multi-agent learning. Under the standard assumptions of convexity, Lipschitz continuity, and smoothness, we establish statistical learning rates that retain, up to logari…
We consider whether algorithmic choices in over-parameterized linear matrix factorization introduce implicit regularization. We focus on noiseless matrix sensing over rank- positive semi-definite (PSD) matrices in , with a sensing mechanism that satisfies restricted isometry properties (RIP)…
SGD with large learning rates can achieve better test accuracy than expected.
problem SGD with large learning rates often outperforms expected convergence bounds.
method Proved that SGD with small learning rates stays close to gradient flow path on modified loss.
result Explicitly adding an implicit regularizer to the loss improves test accuracy.
SetRank tackles collaborative ranking from implicit feedback using setwise Bayesian approach.
problem Challenges in pairwise and listwise approaches for implicit feedback.
method SetRank is a novel setwise Bayesian approach that accommodates implicit feedback characteristics.
result SetRank outperforms state-of-the-art baselines on real-world datasets.
Enhances graph modeling with hyperbolic geometry and variational inference.
problem Challenges in modeling relational data with complex dependencies.
method Semi-implicit hierarchical variational Bayes with Poincaré embedding and mutual information regularization.
result Improves graph representation quality and flexibility in edge prediction and node classification.
The paper challenges the belief that more inner iterations at test time improve performance in implicit deep learning.
problem The performance improvement of implicit deep learning models with increased inner iterations at test time.
method Theoretical analysis of a simple setting, validation on implicit deep learning problems.
result Overparametrization plays a key role; increasing the number of iterations at test time does not improve performance for overparametrized networks.
Simple greedy algorithms can excel in multi-objective bandits with multiple good arms.
problem Optimizing multiple objectives in bandits is traditionally harder.
method Introduced greedy algorithms that exploit multiple good arms for multiple objectives.
result Simple greedy algorithms achieve strong performance in multi-objective bandits.
We introduce the implicitly constrained least squares (ICLS) classifier, a novel semi-supervised version of the least squares classifier. This classifier minimizes the squared loss on the labeled data among the set of parameters implied by all possible labelings of the unlabeled data. Unlike other discriminative semi-s…
Noise in RNNs promotes flatter minima and more stable dynamics.
problem Understanding and optimizing the training of RNNs with noise.
method Formalizing RNNs as stochastic differential equations and analyzing the effect of noise in the hidden states.
result Noise injection in RNNs leads to flatter minima, more stable dynamics, and improved robustness.
Gradient flow on ReLU networks converges to a simple model with few regions.
problem Understanding the dynamics of gradient flow in shallow ReLU networks.
method Analysis of gradient flow dynamics on univariate ReLU neural networks.
result Gradient flow converges to a network with at most O(r) linear regions.
Paper proposes bypassing implicit assumption in GM-based AD methods.
problem Lack of anomalous data and implicit assumption in GM-based AD methods.
method Integrating Discriminative idea to GMM for AD tasks (DiGMM).
result Establishes a connection between generative and discriminative models for AD.
Study efficient derivative computation for nondifferentiable maps in machine learning.
problem Efficiently compute derivatives of fixed-point of nondifferentiable contractions.
method Iterative Differentiation (ITD), Approximate Implicit Differentiation (AID), and New Stochastic Implicit Differentiation (NSID).
result Established convergence rates for ITD, AID, and NSID, matching or improving smooth setting rates.
Gradient flow with infinitesimal initialization converges to Greedy Low-Rank Learning for matrix factorization.
problem Understanding implicit regularization in gradient descent for matrix factorization.
method Theoretical and empirical analysis of gradient flow with infinitesimal initialization and Greedy Low-Rank Learning.
result Gradient flow with infinitesimal initialization is mathematically equivalent to Greedy Low-Rank Learning for depth-2 matrix factorization under reasonable assumptions.
Implicit Generative Copulasstat.ML
Flexible copula model using implicit generative neural networks.
problem Limited flexibility of parametric copulas and curse of dimensionality in non-parametric methods.
method Implicit generative neural networks to model high-dimensional copula distributions with unspecified marginals.
result Demonstrated flexibility and performance on various datasets.
CDF Transform-and-Shift: An effective way to deal with datasets of inhomogeneous cluster densitiescs.LG
The problem of inhomogeneous cluster densities has been a long-standing issue for distance-based and density-based algorithms in clustering and anomaly detection. These algorithms implicitly assume that all clusters have approximately the same density. As a result, they often exhibit a bias towards dense clusters in th…
Generative adversarial nets (GANs) have generated a lot of excitement. Despite their popularity, they exhibit a number of well-documented issues in practice, which apparently contradict theoretical guarantees. A number of enlightening papers have pointed out that these issues arise from unjustified assumptions that are…
The paper analyzes two ISGD modes for statistical inference, deriving error bounds and confidence intervals.
problem Statistical inference with implicit SGD for smooth convex functions.
method Proximal Robbins-Monro (proxRM) and proximal Polyak-Ruppert (proxPR) procedures for ISGD.
result Derives non-asymptotic error bounds and confidence interval estimators for model parameters.
New Hamiltonian Monte Carlo method for non-canonical dynamics.
problem Incompatibility of canonical symplectic structure with non-canonical dynamics.
method Developed a framework for Hamiltonian Monte Carlo using non-canonical symplectic structures with implicit integration.
result Non-canonical Hamiltonian Monte Carlo provides sampling advantages.
This work addresses the instability in asynchronous data parallel optimization. It does so by introducing a novel distributed optimizer which is able to efficiently optimize a centralized model under communication constraints. The optimizer achieves this by pushing a normalized sequence of first-order gradients to a pa…
New tensor formulation reveals gradient flow's bias in linear neural networks.
problem Understanding implicit bias in linear neural network training.
method Tensor formulation of neural networks, including fully-connected, diagonal, and convolutional networks.
result Gradient flow on linear tensor networks converges to solutions of specific optimization problems.
Gradient flow on softmax attention minimizes nuclear norm of weight matrices.
problem Classification with separate key and query weight matrices.
method Gradient flow on exponential loss, separability assumption, reparameterization, approximate KKT conditions.
result Gradient flow implicitly minimizes nuclear norm of weight matrices, contrasting with Frobenius norm minimization.
Implicit feedback is the simplest form of user feedback that can be used for item recommendation. It is easy to collect and domain independent. However, there is a lack of negative examples. Existing works circumvent this problem by making various assumptions regarding the unconsumed items, which fail to hold when the …
We have discovered a "little" gap in our proof of the sharp conjecture that in with volume and perimeter densities and , balls about the origin are uniquely isoperimetric if , that is, if they are stable (and ). The implicit unjustified assumption is that the g…
CATE Lasso: Conditional Average Treatment Effect Estimation with High-Dimensional Linear Regressionecon.EM
Estimates CATEs using high-dimensional linear regression models.
problem Estimating individualized causal effects (CATEs) in two treatments.
method Proposes a Lasso regression method for consistently estimating CATEs under high-dimensional and non-sparse parameters, leveraging the assumption of implicit sparsity.
result The proposed method is consistent for estimating CATEs.
Canonical Correlation Analysis (CCA) is a classic technique for multi-view data analysis. To overcome the deficiency of linear correlation in practical multi-view learning tasks, various CCA variants were proposed to capture nonlinear dependency. However, it is non-trivial to have an in-principle understanding of these…
New approach uses 'forward-looking' counterfactuals for treatment choice.
problem Using traditional 'retrospective' counterfactuals in treatment choice leads to counterintuitive results.
method Introduces 'counterfactual treatment choice' for forward-looking counterfactuals.
result Mismatches between interventional and forward-looking counterfactuals can lead to counterintuitive results.
This paper shows how to train only the implicit layer of overparameterized implicit neural networks.
problem Understanding how the implicit layer contributes to the training of overparameterized implicit neural networks.
method Restricting training to only the implicit layer and analyzing the generalization error for ReLU-activated networks.
result Global convergence is guaranteed even if only the implicit layer is trained, and gradient flow with proper random initialization can achieve small generalization errors.
New SGD variant makes neural networks compressible without assumptions.
problem Improving neural network compressibility without strong assumptions.
method Introducing heavy-tailed noise to SGD iterates.
result Compressible outputs with high probability for any compression rate.
Paper studies the theoretical equivalence between implicit and explicit neural networks in high dimensions.
problem Lack of theoretical analysis of implicit and explicit neural networks.
method Examined high-dimensional implicit neural networks and established their equivalence to explicit networks.
result Equivalence between implicit and explicit neural networks in high dimensions.
Gradient matching method estimates implicit regularization in complex deep learning systems.
problem Estimating implicit regularization in modern deep learning systems with complex modifications.
method Gradient matching methods to empirically estimate implicit regularization.
result Empirical estimation of implicit regularization in arbitrary networks, including dropout.
Deep neural networks perform well due to optimization effects.
problem Understanding why deep neural networks perform well empirically.
method Proved convergence theorem for nonconvex optimization, applied to learning problems.
result Wide deep neural networks converge to global optima with gradient descent.
Continuous semi-implicit models enable faster training and better performance in generative modeling.
problem Slow convergence in hierarchical semi-implicit models during training.
method CoSIM, a continuous semi-implicit model that incorporates a continuous transition kernel for efficient training.
result CoSIM achieves superior performance on image generation tasks compared to existing methods.
RINS-T solves time series inverse problems robustly without pretraining.
problem Recovering original signals from corrupted time series data.
method Implicit neural solvers with robust optimization techniques.
result RINS-T achieves high recovery performance without pretraining.
Presentation bias is one of the key challenges when learning from implicit feedback in search engines, as it confounds the relevance signal with uninformative signals due to position in the ranking, saliency, and other presentation factors. While it was recently shown how counterfactual learning-to-rank (LTR) approache…
Gradient descent converges to a global minimum in nonlinear ReLU implicit networks with linear width.
problem Understanding convergence of gradient methods in nonlinear, infinitely deep ReLU networks.
method Introduced a scaling constant to ensure well-posedness of the equilibrium equation, proving convergence to a global minimum for linear width networks.
result Gradient descent converges to a global minimum at a linear rate for nonlinear ReLU implicit networks with linear width.
Study finds implicit government guarantee improves municipal investment bond ratings.
problem Questioning the objectivity of municipal investment bond ratings due to implicit government guarantee.
method Text mining of policy documents and PMC index model for implicit guarantee strength calculation.
result Implicit government guarantee boosts municipal investment bond ratings, especially in less developed regions.
Study shows SGD's generalization is not explained by implicit bias.
problem Explaining the generalization ability of overparameterized learning algorithms.
method Revisited Stochastic Convex Optimization with SGD, demonstrating limitations of implicit bias.
result No distribution-independent or distribution-dependent implicit regularizer can explain SGD's generalization.
This paper measures the intensity of implicit government guarantees using PMC index model.
problem Excessive local government debt due to implicit government guarantees.
method Text mining of policy documents related to municipal investment bonds, PMC index model.
result Recent policies have reduced the intensity of implicit government guarantees.