New methods show sparse portfolios offer no advantage over mean-variance in diversification.
problem Investment diversification and risk management with sparse portfolios.
method Developed and implemented a new estimation procedure for sparse second-order stochastic spanning using a greedy algorithm and Linear Programming.
result No benefit from expanding a sparse opportunity set beyond 45 assets; optimal sparse portfolio reduces tail risk.
This paper explores efficient neural networks by identifying sparse structures.
problem Reducing computational complexity in neural networks without sacrificing accuracy.
method Identifying and utilizing sparse structures in neural network weights and activations.
result Large and sparse models are more beneficial for practical problems.
This paper considers the sparse eigenvalue problem, which is to extract dominant (largest) sparse eigenvectors with at most k non-zero components. We propose a simple yet effective solution called truncated power method that can approximately solve the underlying nonconvex optimization problem. A strong sparse recove…
We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We es…
Paper proposes embedding models to capture semantic similarities of categorical attributes in financial bonds.
problem Challenges in finding similar bonds due to overshadowing of categorical non-financial attributes.
method Embedding models to capture semantic similarities of categorical attributes.
result Improves risk modeling and curve construction via sparse-issuer augmentation.
Paper develops methods for analyzing forms with synchronized singularities.
problem Analyzing forms with synchronized singularities.
method Exact reduction, analytic transfer, and geometric recomposition.
result Transfer of sparse domination principle to synchronized singular forms.
ACOWA improves distributed sparse classification with extra communication round.
problem Efficiently optimizing sparse classification with limited communication.
method Introducing ACOWA, a new technique with an extra communication round.
result ACOWA achieves better approximation quality and higher accuracy.
Novel approximation hierarchy for sparse quadratic programs.
problem Sparse Quadratic Programs with Cardinality Constraints.
method Exploits rank-dominating eigenvectors for min-max optimization over binary variables.
result Efficient screening of nonzero elements with scalable optimization algorithms.
In this paper, we will investigate the efficacy of IMAT (Iterative Method of Adaptive Thresholding) in recovering the sparse signal (parameters) for linear models with missing data. Sparse recovery rises in compressed sensing and machine learning problems and has various applications necessitating viable reconstruction…
Sparse-mode DMD disambiguates local and global modes in spatiotemporal data.
problem Disambiguating local and global modes in spatiotemporal data.
method Sparse-mode DMD with sparsity-promoting regularization.
result Explicitly constructs discrete and continuous spectra.
Sparse PCA is a widely used technique for high-dimensional data analysis. In this paper, we propose a new method called low-rank principal eigenmatrix analysis. Different from sparse PCA, the dominant eigenvectors are allowed to be dense but are assumed to have a low-rank structure when matricized appropriately. Such a…
Various problems in data analysis and statistical genetics call for recovery of a column-sparse, low-rank matrix from noisy observations. We propose ReFACTor, a simple variation of the classical Truncated Singular Value Decomposition (TSVD) algorithm. In contrast to previous sparse principal component analysis (PCA) al…
Deep neural networks (DNNs) have emerged as key enablers of machine learning. Applying larger DNNs to more diverse applications is an important challenge. The computations performed during DNN training and inference are dominated by operations on the weight matrices describing the DNN. As DNNs incorporate more layers a…
REALITrees uses a Rashomon ensemble approach for active learning in sparse decision trees.
problem Active learning reduces labeling costs by selecting informative samples, but current methods often sacrifice model diversity and direct characterization of the hypothesis space.
method REALITrees constructs a committee of all near-optimal sparse decision tree models using a Rashomon Set and a Gibbs posterior to weight them by empirical risk.
result REALITrees outperforms randomized ensembles, especially in noisy environments, by leveraging expanded model multiplicity.
Various ℓ1-penalised estimation methods such as graphical lasso and CLIME are widely used for sparse precision matrix estimation. Many of these methods have been shown to be consistent under various quantitative assumptions about the underlying true covariance matrix. Intuitively, these conditions are related to …
The recently proposed Lottery Ticket Hypothesis of Frankle and Carbin (2019) suggests that the performance of over-parameterized deep networks is due to the random initialization seeding the network with a small fraction of favorable weights. These weights retain their dominant status throughout training -- in a very r…
Deviance-style normalization for sparse, jointly overdispersed count matrices
problem Jointly overdispersed count matrices
method Dirichlet-multinomial deviance residualization
result Preserves exact sparsity, evaluates in constant time, recovers multinomial residual
Model shows loss curve with two distinct exponents due to sparse activations.
problem Sparse activations impact neural network scaling laws.
method Introduced a model for neural scaling laws under sparse activations, derived asymptotic population loss, and analyzed gradient-descent dynamics.
result Loss curve exhibits double-descent peak near interpolation threshold with two distinct scaling exponents.
We introduce Block Sparse Canonical Correlation Analysis which estimates multiple pairs of canonical directions (together a "block") at once, resulting in significantly improved orthogonality of the sparse directions which, we demonstrate, translates to more interpretable solutions. Our approach builds on the sparse CC…
We propose to execute deep neural networks (DNNs) with dynamic and sparse graph (DSG) structure for compressive memory and accelerative execution during both training and inference. The great success of DNNs motivates the pursuing of lightweight models for the deployment onto embedded devices. However, most of the prev…
New method improves tensor completion by selectively preserving important elements.
problem Recovering corrupted high-dimensional tensor data with missing entries and noise.
method Tensor weighted correlated total variation (TWCTV) regularizer with ADMM algorithm.
result Superior performance in image completion, denoising, and background subtraction tasks.
The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty te…
New method improves deep learning models robustness to label noise.
problem Improving deep learning models' robustness to corrupted labels.
method Sparse over-parameterization and implicit regularization.
result State-of-the-art test accuracy against label noise on various datasets.
New algorithm finds sparse matrices on Stiefel manifold for optimisation.
problem Finding sparse matrices on Stiefel manifold for optimisation.
method Modified Orthogonal Iteration algorithm for sparse global optimality.
result Proposed method finds globally optimal sparse Stiefel matrices.
FF layers in transformers are nearly as interpretable as sparse autoencoders.
problem Comparing interpretability of feature vectors in FF layers vs. sparse autoencoders.
method Revisited interpretability of FF layers as key-value memories using modern benchmarks.
result FF and SAE feature vectors are similarly interpretable, but FFs can be better in some aspects.
Transformers learn to integrate information from past positions incrementally, specializing heads in distinct patterns.
problem How transformers learn to integrate information from multiple past positions with varying statistical significance.
method High-order Markov chain task, incremental learning, sparse attention patterns, simplified differential equations, stage-wise convergence, early stopping as regularizer.
result Transformers learn to specialize heads in distinct patterns, shifting from competitive to cooperative learning dynamics.
Sparse Gaussian process quantile regression tackles computational challenges in Bayesian quantile regression.
problem Nonconjugacy and computational cost in Gaussian process quantile regression.
method Sparse Gaussian process framework with Laplace approximation, adaptive inducing-input placement, and sequential data acquisition.
result Accuracy of Laplace approximation and effectiveness of adaptive mechanisms in reducing predictive uncertainty.
The search for efficient, sparse deep neural network models is most prominently performed by pruning: training a dense, overparameterized network and removing parameters, usually via following a manually-crafted heuristic. Additionally, the recent Lottery Ticket Hypothesis conjectures that, for a typically-sized neural…
Sparse portfolio strategy from mutual funds' favorite stocks in China A share market.
problem Building a sparse portfolio from mutual funds' favorite stocks in a market with limited fund information.
method Analyzed mutual fund favorite stocks, used portfolio optimizer with constraints, and compared different methods.
result Sparse portfolios consistently outperform the benchmark index 930950.CSI.
SGM combines deep learning and planning for robust long-horizon tasks.
problem Combining deep learning and planning for robust long-horizon tasks.
method Sparse Graphical Memory (SGM) that stores states and feasible transitions in a sparse memory, aggregating states according to a two-way consistency objective.
result SGM significantly outperforms current state of the art methods on long horizon, sparse-reward visual navigation tasks.
Proposes a nonparametric tensor factorization for sparse data.
problem Handling sparse tensor data with structural and interpretability benefits.
method Hierarchical Gamma processes and Poisson random measures for tensor-valued process, Dirichlet processes for sampling entry indices, Gaussian processes for values.
result Demonstrates superior performance on benchmark datasets.
Efficient ANN search for sparse embeddings in ads targeting.
problem Efficiently searching near neighbors in sparse data for applications like ads targeting.
method Graph-based ANN algorithms (HNSW, chi-square two-tower model, Sign Cauchy Projections).
result Sparse embeddings and ANN algorithms improve efficiency in EBR applications.
Entropy regularization improves interpretability of probabilistic clustering models.
problem Bayesian nonparametric mixture models often produce unbalanced cluster frequencies.
method Interpreting the posterior as penalized likelihood, entropy regularization reduces sparsely-populated clusters.
result The proposed entropy-regularized estimator enhances interpretability without sacrificing computational convenience.
New framework shows much of equity market risk may come from asset returns themselves.
problem Understanding the sources of risk in equity markets.
method Decomposes asset returns into endogenous and exogenous components, using statistical methods.
result Most of the risk in equity markets may be explained by a sparse network of interacting assets.
Connected domination numbers found for plane triangulations up to 13 vertices.
problem Finding connected domination numbers for plane triangulations.
method Analyzing triangulations of up to 13 vertices and proving the difference between connected and regular domination numbers can be arbitrarily large.
result Connected domination numbers for triangulations up to 13 vertices and upper bound for larger triangulations.
Manifolds can be dominated by hypersurfaces in a sphere.
problem Dominating manifolds with hypersurfaces.
method Proving any smooth, closed, oriented manifold can be dominated by a codimension 1 submanifold of the sphere.
result Any smooth, closed, oriented manifold can be dominated by a codimension 1 submanifold of the sphere.
New GCNs solve graph embedding problems efficiently and interpretably.
problem Graph embedding for scalable and interpretable machine learning.
method Proposed two GCNs: CAFE-GCN and sphere-GCN, based on constrained optimization.
result Both GCNs yield good approximations of dominant eigenvectors and perform dimensionality reduction.
This note considers softmax parameter estimation when little/no labeled training data is available, but a priori information about the relative geometry of class label log-odds boundaries is available. It is shown that `data-free' softmax model synthesis corresponds to solving a linear system of parameter equations, wh…
New method ranks multivariate distributions in SMOOP using q-dominance.
problem Lack of reliable methods to rank multivariate distributions in SMOOP.
method Introduces center-outward q-dominance and develops empirical test procedures.
result Proves q-dominance implies FSD and establishes a sample size threshold.
Complexity helps identify sparse risk factors in asset pricing.
problem Tension between feature richness and economic parsimony in high-dimensional asset pricing.
method Expanding feature space and using basis pursuit to discover sparse risk factors.
result Nonlinear feature expansions combined with basis pursuit yield superior out-of-sample performance.
Proposes a new framework for predicting stock market movements using sparse neural architectures.
problem Challenging problem of predicting stock market movements using technical indicators.
method Multi-criteria optimization approach to evolve sparse neural architectures.
result Evolved parsimonious networks with better generalization capabilities.
We show that non-domination results for targets that are not dominated by products are stable under Cartesian products.
New framework for ranking distributions using variable fractional parameters.
problem Ordering distributions with varying steepness and local non-concavities.
method Introducing a function γ:Ro[0,1] to replace the fixed parameter in fractional SD. result Enables ranking of a broader range of distributions and incorporates dynamic greediness.
The paper analyzes how adversarial attacks affect sparse regression models.
problem Effects of adversarial attacks on sparse regression models.
method Primal-dual witness paradigm to analyze support of estimated regression parameter vector.
result Adversaries can influence sample complexity by corrupting irrelevant features.
The Bethe free energy approximation is reliable when convex on a submanifold, the 'Bethe box'.
problem Accuracy of the Bethe free energy approximation in probabilistic inference.
method Analysis of convexity and verification conditions based on the Bethe Hessian matrix.
result The Bethe approximation is mostly accurate if it is convex on a submanifold, the 'Bethe box'.
Research examines when 4-manifolds are dominated by geometric ones.
problem When is an orientable closed 4-manifold dominated by another?
method Focuses on geometric or fibred cases.
result Characterizes conditions for domination.
3-manifolds can virtually dominate others with positive simplicial volume.
problem Domination of 3-manifolds with positive simplicial volume.
method Proving existence of finite covers with degree-1 maps.
result Virtual domination of 3-manifolds with positive simplicial volume.
Study domination between non-Fuchsian surface group representations and anti-de Sitter geometry.
problem Domination problem between non-Fuchsian representations of closed surface groups.
method Analysis of branched harmonic immersions and construction of anti-de Sitter 3-manifolds.
result Found that representations admitting branched harmonic immersions dominate other representations, and constructed large families of branched anti-de Sitter 3-manifolds.