Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

3837661,1481,531 · Jun 202019922001200920172026
48 results for Unconstrained Feature Model

This paper extends neural collapse to class-imbalanced datasets using an unconstrained ReLU feature model.

problem Understanding neural collapse in class-imbalanced datasets with cross-entropy loss.
method Generalized neural collapse to class-imbalanced settings using an unconstrained ReLU feature model.
result Class-means converge to orthogonal vectors with different lengths, and classifier weights align to these vectors.

This paper extends neural collapse to imbalanced data under cross-entropy loss.

problem Analyzing neural collapse in deep networks with imbalanced data.
method Using the unconstrained feature model and cross-entropy loss, the paper studies neural collapse in imbalanced datasets.
result Feature vectors within the same class collapse to a single mean vector, but angles between them depend on sample size.

Quantum computing improves feature selection in machine learning.

problem Optimizing feature selection in machine learning problems.
method Formulated feature selection as a QUBO problem and compared quantum and classical methods.
result Quantum computing can outperform classical methods in feature selection, depending on data set.

Study shows 'Ordinal Neural Collapse' in deep OR tasks, revealing simple geometric relationships.

problem Understanding neural collapse in deep Ordinal Regression tasks.
method Combining cumulative link models and Unconstrained Feature Model to investigate neural collapse.
result Demonstrates 'Ordinal Neural Collapse' (ONC) with three key properties.

Training shapes the geometry of neural network feature maps, revealing local area magnification.

problem Understanding how training affects the geometric structure of neural network feature maps.
method Analyzing the Riemannian geometry induced by neural network feature maps at infinite width and after training.
result Training breaks the symmetry of the geometry induced by random neural network feature maps, magnifying local areas along decision boundaries.

Study optimal portfolio management with periodic evaluations in stochastic models, considering convex constraints.

problem Optimal portfolio management under ratio-type periodic evaluations in stochastic factor models with convex trading constraints.
method Transformed infinite horizon optimal control problem into an auxiliary terminal wealth optimization problem. Introduced an auxiliary unconstrained optimization problem in a modified market model. Used martingale duality approach to establish dual minimizer and optimal unconstrained wealth process.
result Derived and verified the optimal constrained portfolio process for the original problem over an infinite horizon.

Proposes an evolutionary approach to fitting acyclic VAR models.

problem Cycles in multivariate time series systems obscure hierarchical analysis.
method Evolutionary approach to fitting acyclic VAR processes with hierarchical representation.
result Outperforms unconstrained models and captures key structural properties.

DP-GD achieves dimension-independent convergence for unconstrained private GLMs.

problem Differentially private empirical risk minimization for unconstrained GLMs.
method Differentially private gradient descent (DP-GD).
result DP-GD achieves an excess empirical risk of $ ilde O\left(\sqrt{ exttt{rank}}/εn ight)$ for unconstrained GLMs.

Wide neural networks with weight decay exhibit neural collapse.

problem Proving neural collapse in wide neural networks trained with weight decay.
method Generic guarantees on neural collapse for wide networks with weight decay, proving low training error and balancedness, and bounded conditioning.
result First proof of neural collapse in end-to-end training of wide neural networks with weight decay.

Unconstrained MLIPs outperform constrained ones in accuracy and speed.

problem Improving the efficiency and accuracy of machine-learned interatomic potentials.
method Investigated unconstrained models trained on large datasets compared to physically constrained models.
result Unconstrained MLIPs can be superior in accuracy and speed compared to physically constrained models.

We consider a variant of online convex optimization in which both the instances (input vectors) and the comparator (weight vector) are unconstrained. We exploit a natural scale invariance symmetry in our unconstrained setting: the predictions of the optimal comparator are invariant under any linear transformation of th…

2017-08-23abs ↗pdf ↗

We analyze neural collapse in neural networks, showing that features collapse to vertices of a Simplex ETF.

problem Understanding and optimizing the features learned in the last layer of neural networks during training.
method Simplified unconstrained feature model, studying the global optimization landscape of cross-entropy loss with weight decay.
result The global minimizers of the loss are Simplex ETFs, and other critical points are strict saddles with negative curvature.

This work justifies neural collapse under MSE loss and analyzes the optimization landscape.

problem Understanding neural collapse in deep neural networks under MSE loss.
method Global landscape analysis of vanilla nonconvex MSE loss.
result The only global minimizers are neural collapse solutions.

This paper analyzes the landscape of supervised contrastive loss in over-parameterized networks.

problem Understanding the structure of solutions in over-parameterized networks under supervised contrastive loss.
method Analytical approach using unconstrained features model (UFM) to study the solutions of SC loss minimization.
result All local minima of SC loss are global minima in over-parameterized networks, and the minimizer is unique (up to rotation).

Paper explains neural collapse in neural networks using a new model.

problem Understanding neural collapse in neural networks during training.
method Introducing the unconstrained layer-peeled model (ULPM) to prove gradient flow convergence to critical points of a minimum-norm separation problem.
result Proves that all critical points are strict saddle points except the global minimizers exhibiting neural collapse.

Study shows neural collapse is invariant to class imbalances under certain conditions.

problem Neural collapse properties are only valid for balanced data.
method Adopted UFM and introduced SELI for invariant characterization.
result Embeddings and classifiers always interpolate a simplex-encoded label matrix regardless of class imbalances.

Unconstrained models learn physical symmetries effectively with simple data augmentation.

problem Ensuring physical symmetries in machine learning models.
method Rigorous metrics to measure symmetry content, data augmentation strategy, architectural analysis.
result Unconstrained models can learn approximate equivariant behavior with simple data augmentation.

New approach reduces unconstrained linear bandits to simpler optimization problems.

problem Unconstrained linear bandits problem.
method Perturbation-based approach combined with comparator-adaptive OLO algorithms.
result First high-probability guarantees for both static and dynamic regret in unconstrained linear bandits.

Obtaining compact and discriminative features is one of the major challenges in many of the real-world image classification tasks such as face verification and object recognition. One possible approach is to represent input image on the basis of high-level features that carry semantic meaning which humans can understan…

2012-11-13abs ↗pdf ↗

The paper explores solving inverse problems for ODEs with and without constraints.

problem Understanding when second order ODEs can represent Lagrangian models with or without constraints.
method Geometric techniques to address the inverse problem for both constrained and unconstrained systems of second order ODEs.
result The constrained case presents more ambiguities and complexities than the unconstrained one.

Proposes ConstraintMatch for semi-supervised clustering with unconstrained data.

problem Leveraging unconstrained data alongside constraints for clustering models.
method Semi-supervised context with pseudo-constraining and pseudo-labeling mechanisms.
result Demonstrates effectiveness of ConstraintMatch over baselines.

This paper evaluates conformal prediction for aerial image classification in challenging environments.

problem Challenging aerial image classification in data-scarce, unconstrained environments.
method Conformal prediction applied to pretrained models (MobileNet, DenseNet, ResNet) with limited labeled data.
result Conformal prediction can provide valuable uncertainty estimates even with small labeled samples.

Despite significant progress made over the past twenty five years, unconstrained face verification remains a challenging problem. This paper proposes an approach that couples a deep CNN-based approach with a low-dimensional discriminative embedding learned using triplet probability constraints to solve the unconstraine…

2016-04-19abs ↗pdf ↗

Tree ensemble kernels improve Bayesian optimization for mixed features and constraints.

problem Optimizing over mixed-feature spaces with known constraints.
method Kernel interpretation of tree ensembles as Gaussian Process prior, compatible optimization formulation for acquisition function, integration of known constraints.
result Framework outperforms state-of-the-art methods for mixed-feature spaces and constraints.

Low rank matrix factorisation is often used in recommender systems as a way of extracting latent features. When dealing with large and sparse datasets, traditional recommendation algorithms face the problem of acquiring large, unrestrained, fluctuating values over predictions especially for users/items with very few co…

2018-07-15abs ↗pdf ↗

We clarify what fairness guarantees we can and cannot expect to follow from unconstrained machine learning. Specifically, we characterize when unconstrained learning on its own implies group calibration, that is, the outcome variable is conditionally independent of group membership given the score. We show that under r…

2018-08-29abs ↗pdf ↗

Neural collapse occurs in normalized features over a Riemannian manifold.

problem Understanding neural collapse in normalized feature models.
method Simplified multi-class classification task to a nonconvex optimization problem over the Riemannian manifold, analyzing the landscape of critical points.
result The only global minimizers are neural collapse solutions, with all other critical points being strict saddles.

New algorithms reduce online learning regret by tracking gradient variation.

problem Online learning with unconstrained losses and gradient variation.
method Parameter-free algorithms with adaptive updates for LL-smooth convex losses.
result Regret bounds of order O~(uVT(u)+Lu2+G4)\widetilde{O}(\|u\|\sqrt{V_T(u)} + L\|u\|^2+G^4) achieved without prior knowledge of comparator norm or Lipschitz constant.

Recurrent neural networks can be large and compute-intensive, yet many applications that benefit from RNNs run on small devices with very limited compute and storage capabilities while still having run-time constraints. As a result, there is a need for compression techniques that can achieve significant compression wit…

2019-06-12abs ↗pdf ↗

Solves VaR-constrained portfolio optimization in markets with stochastic volatility.

problem Optimizing portfolio in markets with stochastic volatility under VaR constraints.
method Dynamic programming approach to Heston's stochastic volatility model.
result Optimal investment strategy linked to unconstrained problem via a vega-neutral derivative.

VAV method optimizes learning rate for faster, stable SGD convergence.

problem Optimizing learning rate for efficient and stable machine learning models.
method Energy-based self-adaptive learning rate with auxiliary variable rr.
result VAV method achieves faster convergence and superior stability with larger learning rates.

This work explains neural collapse in shallow neural networks and its impact on generalization.

problem Understanding neural collapse in shallow neural networks and its effect on generalization.
method Analysis of two and three-layer ReLU neural networks, focusing on data dimension, sample size, and signal-to-noise ratio.
result Neural collapse occurs in shallow ReLU networks under certain conditions related to data properties and network architecture.

ParamBoost uses gradient boosting to create interpretable non-linear models with constraints.

problem Creating interpretable non-linear models with expert knowledge constraints.
method Gradient Boosting of cubic polynomials with specified constraints.
result ParamBoost outperforms state-of-the-art GAMs in real-world datasets.

Deep linear networks exhibit collapsing features and classifiers across datasets.

problem Understanding the collapse of features and classifiers in deep linear networks.
method Theoretical and empirical analysis of deep linear networks with MSE and CE losses.
result Deep linear networks exhibit NC properties, collapsing features and classifiers to orthogonal vectors.

A new L-BFGS method tackles large-scale optimization with fewer evaluations.

problem Efficiently solving large-scale unconstrained optimization problems.
method Proposes a regularized L-BFGS method with line search techniques.
result Shows global convergence and robust performance in numerical tests.

Develops methods for estimating constrained function-valued parameters in infinite-dimensional models.

problem Estimating function-valued parameters with structural constraints in complex models.
method Characterizes constrained solutions as minimizers of penalized population risk, using a Lagrange-type formulation and path through unconstrained space.
result Proposes estimators that achieve optimal risk and constraint satisfaction, applicable across various statistical learning approaches.

This paper uses quantum computing to solve sparse linear regression problems efficiently.

problem Sparse linear regression to identify important features from a large set of variables.
method Formulates the 0\ell_0 optimization problem as a QUBO problem and solves it using the D-Wave adiabatic quantum computer.
result The QUBO solution matches the optimal solution for a wide range of sparsity penalty values across datasets.

New algorithm reduces online regression error in RKHS.

problem Online regression with time-varying functions in RKHS.
method Hierarchical Vovk-Azoury-Warmuth with discounting.
result Achieves optimal dynamic regret with O(T2/3PT1/3+TlnT)O(T^{2/3}P_T^{1/3} + \sqrt{T}\ln T) regret bound.