We propose an algorithm, semismooth Newton coordinate descent (SNCD), for the elastic-net penalized Huber loss regression and quantile regression in high dimensional settings. Unlike existing coordinate descent type algorithms, the SNCD updates each regression coefficient and its corresponding subgradient simultaneousl…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “coordinate loss”
BCD algorithm finds global minima in neural networks.
problem Training deep neural networks to find global minima.
method Block coordinate descent with skip connections and non-negative projection.
result Proves convergence to global minima for strictly monotonic and ReLU activations.
We introduce a proximal version of dual coordinate ascent method. We demonstrate how the derived algorithmic framework can be used for numerous regularized loss minimization problems, including regularization and structured output SVM. The convergence rates we obtain match, and sometimes improve, state-of-the-…
Develops a fast algorithm for high-dimensional LASSO penalized quantile regression.
problem Computational challenges in high-dimensional penalized quantile regression.
method Pathwise coordinate descent algorithm to solve exact coordinatewise minimum of the nonsmooth loss function.
result Algorithm runs faster than existing alternatives and maintains estimation accuracy.
Note: Variational Encoding of Protein Dynamics Benefits from Maximizing Latent Autocorrelationphysics.chem-ph
As deep Variational Auto-Encoder (VAE) frameworks become more widely used for modeling biomolecular simulation data, we emphasize the capability of the VAE architecture to concurrently maximize the timescale of the latent space while inferring a reduced coordinate, which assists in finding slow processes as according t…
In this paper we develop and analyze Hydra: HYbriD cooRdinAte descent method for solving loss minimization problems with big data. We initially partition the coordinates (features) and assign each partition to a different node of a cluster. At every iteration, each node picks a random subset of the coordinates from tho…
We design a randomised parallel version of Adaboost based on previous studies on parallel coordinate descent. The algorithm uses the fact that the logarithm of the exponential loss is a function with coordinate-wise Lipschitz continuous gradient, in order to define the step lengths. We provide the proof of convergence …
Stochastic Gradient Descent (SGD) has become popular for solving large scale supervised machine learning optimization problems such as SVM, due to their strong theoretical guarantees. While the closely related Dual Coordinate Ascent (DCA) method has been implemented in various software packages, it has so far lacked go…
Stochastic dual coordinate ascent (SDCA) is an effective technique for solving regularized loss minimization problems in machine learning. This paper considers an extension of SDCA under the mini-batch setting that is often used in practice. Our main contribution is to introduce an accelerated mini-batch version of SDC…
We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure. We analyze the runtime of the framework and obtain rates that improve state-of-the-art results for various key machine learning optimization problems including …
Paper proves robust M-estimators' coordinates' normality in high dimensions.
problem High-dimensional robust M-estimators' asymptotic normality.
method Develops Stein formulae for high-dimensional random vectors on the sphere.
result Asymptotic normality holds for most coordinates of robust M-estimators with convex penalty.
In modern large-scale machine learning applications, the training data are often partitioned and stored on multiple machines. It is customary to employ the "data parallelism" approach, where the aggregated training loss is minimized without moving data across machines. In this paper, we introduce a novel distributed du…
TripleSurv improves survival analysis by ranking samples with time-adaptive adjustments.
problem Modeling censored time-to-event data with high accuracy and robustness.
method Introduces a time-adaptive coordinate loss function to rank samples and calibrate robustness.
result TripleSurv outperforms state-of-the-art methods on various survival datasets.
There are two important statements regarding the Trautman-Bondi mass at null infinity: one is the positivity, and the other is the Bondi mass loss formula, which are both global in nature. In this note, we compute the limit of the Wang-Yau quasi-local mass on unit spheres at null infinity of an asymptotically flat spac…
The paper examines mass aspects at future null infinity and limits of quasilocal mass.
problem Understanding mass aspects and limits of quasilocal mass at future null infinity.
method Review and extension of Bondi mass and mass loss formula in Bondi-Sachs coordinate system.
result New results about the limit of quasilocal mass of unit spheres at null infinity.
In this paper, we study two general classes of optimization algorithms for kernel methods with convex loss function and quadratic norm regularization, and analyze their convergence. The first approach, based on fixed-point iterations, is simple to implement and analyze, and can be easily parallelized. The second, based…
In representation learning and non-linear dimension reduction, there is a huge interest to learn the 'disentangled' latent variables, where each sub-coordinate almost uniquely controls a facet of the observed data. While many regularization approaches have been proposed on variational autoencoders, heuristic tuning is …
01 loss neural network improves adversarial robustness on CIFAR10.
problem Improving machine learning models' resistance to adversarial attacks.
method Proposes a single hidden layer 01 loss neural network trained with stochastic coordinate descent.
result 01 loss network has the largest adversarial distortion compared to other models.
New algorithm optimizes Bayesian network learning from Gaussian data.
problem Learning Bayesian networks from Gaussian observational data.
method Proposes a coordinate descent algorithm for -penalized maximum likelihood estimation.
result The algorithm converges to a coordinate-wise minimum and achieves optimal objective value as sample size increases.
A new method uses Coordinate Descent to optimize ResNet networks for private inference.
problem Reducing ReLU count in ResNet networks for private inference.
method Directly optimizing in the discrete domain using Coordinate Descent.
result Our method yields a sparse solution and is state-of-the-art.
We propose a new stochastic coordinate descent method for minimizing the sum of convex functions each of which depends on a small number of coordinates only. Our method (APPROX) is simultaneously Accelerated, Parallel and PROXimal; this is the first time such a method is proposed. In the special case when the number of…
The 01 loss is robust to outliers and tolerant to noisy data compared to convex loss functions. We conjecture that the 01 loss may also be more robust to adversarial attacks. To study this empirically we have developed a stochastic coordinate descent algorithm for a linear 01 loss classifier and a single hidden layer 0…
This study considers that the collective route choices of travelers en route represent a resolution of their competition on network routes. Well understanding this competition and coordinating their route choices help mitigate urban traffic congestion. Even though existing studies have developed such mechanisms (e.g., …
A method models nonlinear dynamics from data using barycentric coordinates and memory.
problem Modeling complex dynamical systems from data.
method SPA for data projection, barycentric coordinates, delay-embedding theorem for memory.
result Stable models of chaotic dynamics and attractors are reproduced.
The one-bit quantization is implemented by one single comparator that operates at low power and a high rate. Hence one-bit compressive sensing (1bit-CS) becomes attractive in signal processing. When measurements are corrupted by noise during signal acquisition and transmission, 1bit-CS is usually modeled as minimizing …
Large-scale L1-regularized loss minimization problems arise in high-dimensional applications such as compressed sensing and high-dimensional supervised learning, including classification and regression problems. High-performance algorithms and implementations are critical to efficiently solving these problems. Building…
Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems. In this work, we introduce new adaptive rules for the random selection of their updates. By adaptive, we mean that our selection rules are based on the dual residual or the primal-du…
Researchers improved Minecraft game performance using imitation learning.
problem Achieving state-of-the-art performance in immersive environments like Minecraft.
method Applied imitation learning to Minecraft, optimizing network architecture, loss function, and data augmentation.
result Reported stronger results than previous experiments, reaching second place in a competition.
A new subdivision scheme for Heisenberg group values with central smoothness loss.
problem Regularity of limit curves in Heisenberg group-valued subdivision schemes.
method Interpolatory subdivision scheme with central correction based on group law.
result Central part of limit curve converges to a continuous limit with logarithmic modulus of continuity.
A meta-learning method learns adaptive robust loss functions for noisy labels.
problem Handling robust learning with noisy labels and optimizing hyperparameters.
method Adaptive learning of robust loss hyperparameters through mutual improvement with network parameters.
result Generalized and effective robust loss functions with good generalization capability.
Unified analysis of stochastic gradient methods for convex and smooth optimization.
problem Minimizing composite convex and smooth functions.
method Unified convergence analysis of various stochastic gradient methods.
result Unified convergence rates for a variety of methods including proximal SGD, variance reduced methods, quantization, and coordinate descent.
We study the performance of a family of randomized parallel coordinate descent methods for minimizing the sum of a nonsmooth and separable convex functions. The problem class includes as a special case L1-regularized L1 regression and the minimization of the exponential loss ("AdaBoost problem"). We assume the input da…
DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimizationmath.OC
Machine learning with big data often involves large optimization models. For distributed optimization over a cluster of machines, frequent communication and synchronization of all model parameters (optimization variables) can be very costly. A promising solution is to use parameter servers to store different subsets of…
SLERP interpolation optimizes dynamic weight rebalancing in AMMs.
problem Optimizing dynamic weight rebalancing in automated market makers (AMMs).
method Riemannian geometry and SLERP interpolation.
result SLERP interpolation minimizes the KL divergence loss in dynamic weight rebalancing.
Robust support vector model based on bounded asymmetric elastic net loss for binary classificationstat.ML
BAEN-SVM improves SVM robustness to noisy data.
problem Noise and geometric irrationalities in SVM.
method Bounded asymmetric elastic net loss combined with SVM.
result BAEN-SVM is robust to noise and geometrically well-defined.
Model shows loss curve with two distinct exponents due to sparse activations.
problem Sparse activations impact neural network scaling laws.
method Introduced a model for neural scaling laws under sparse activations, derived asymptotic population loss, and analyzed gradient-descent dynamics.
result Loss curve exhibits double-descent peak near interpolation threshold with two distinct scaling exponents.
In supervised binary hashing, one wants to learn a function that maps a high-dimensional feature vector to a vector of binary codes, for application to fast image retrieval. This typically results in a difficult optimization problem, nonconvex and nonsmooth, because of the discrete variables involved. Much work has sim…
Machine learning solves Einstein equations without symmetry assumptions.
problem Solving the Euclidean vacuum Einstein equations with a cosmological constant.
method Semi-supervised machine learning with patch-based architecture and coordinate consistency loss.
result Hints against the existence of Ricci-flat metrics on spheres in dimensions 4 and 5.
Formulae track evolution of angular momentum and center of mass at null infinity.
problem Tracking the evolution of conserved quantities at null infinity.
method Evolution formulae in Bondi-Sachs coordinates, expressed in terms of shear and news tensors.
result Supertranslation invariance of fluxes, conservation law of angular momentum, duality paradigm.
Developing a differentially private deep learning algorithm is challenging, due to the difficulty in analyzing the sensitivity of objective functions that are typically used to train deep neural networks. Many existing methods resort to the stochastic gradient descent algorithm and apply a pre-defined sensitivity to th…
Cautious Weight Decay modifies weight decay for better optimization.
problem Improving optimization in deep learning models.
method Applies weight decay selectively based on parameter sign alignment.
result Consistently improves model performance across various tasks and scales.
In this paper, a geometric framework for neural networks is proposed. This framework uses the inner product space structure underlying the parameter set to perform gradient descent not in a component-based form, but in a coordinate-free manner. Convolutional neural networks are described in this framework in a compact …
Deep learning has aroused extensive attention due to its great empirical success. The efficiency of the block coordinate descent (BCD) methods has been recently demonstrated in deep neural network (DNN) training. However, theoretical studies on their convergence properties are limited due to the highly nonconvex nature…
Efficient packages solve SLOPE problem in multiple languages.
problem Efficiently solving SLOPE problem for various data types.
method Hybrid coordinate descent algorithm for GLMs with multiple loss functions.
result Packages outperform existing SLOPE implementations in speed.
Principal component regression (PCR) is a widely used two-stage procedure: principal component analysis (PCA), followed by regression in which the selected principal components are regarded as new explanatory variables in the model. Note that PCA is based only on the explanatory variables, so the principal components a…
Stochastic gradient descent converges to universal limits in high dimensions.
problem Statistical tasks in high dimensions with specific data projections.
method Stochastic gradient descent applied to mixture distributions, proving universality of limits.
result The ODE limits are universal for mixtures of arbitrary product distributions.
Privileged Coordinates and Nilpotent Approximation for Carnot Manifolds, II. Carnot Coordinatesmath.DG
This paper is a sequel of arxiv:1709.09045 and deals with privileged coordinates and nilpotent approximation of Carnot manifolds. By a Carnot manifold it is meant a manifold equipped with a filtration by subbundles of the tangent bundle which is compatible with the Lie bracket of vector fields. In this paper, we single…
We solve a high-dimensional model where nonlinear autoencoders detect hidden structure missed by PCA.
problem Hidden structure in high-dimensional data not detected by PCA.
method Tractable spiked model with two latent factors, one visible and one uncorrelated.
result Nonlinear autoencoders can extract hidden structure missed by PCA, even if reconstruction loss is higher.