This paper studies convergence properties of multivariate distributions constructed by endowing empirical margins with a copula. This setting includes Latin Hypercube Sampling with dependence, also known as the Iman--Conover method. The primary question addressed here is the convergence of the component sum, which is r…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,291 papers · 148 categories
Trend · papers per month
48 results for “Empirical Sums”
Deep sum-product networks learn faster than shallow models.
problem The speed of parameter optimization in sum-product networks.
method Theoretical analysis and empirical experiments on overparameterized sum-product networks.
result Gradient-based optimization in deep sum-product networks is equivalent to gradient ascent with adaptive and time-varying learning rates and additional momentum terms.
Paper develops a simple estimator for high-dimensional superposition models with various component structures.
problem Estimating high-dimensional superposition models with different component structures.
method Presented a simple estimator for general superposition models with any number of component parameters and any norm structure.
result Geometric condition and high probability non-asymptotic bounds for accurate component estimation.
Large deviations for interacting particle dynamics for finding mixed equilibria in zero-sum gamesstat.ML
This work finds mixed equilibria in zero-sum games using interacting particle dynamics.
problem Finding mixed equilibrium points in continuous minmax games.
method A method based on entropic regularisation of two-layer zero-sum games with interacting particle dynamics.
result The sequence of empirical measures of the particle system satisfies a large deviation principle as the number of particles grows to infinity, implying convergence of the empirical measure and the Nikaidô-Isoda error.
We describe a novel optimization method for finite sums (such as empirical risk minimization problems) building on the recently introduced SAGA method. Our method achieves an accelerated convergence rate on strongly convex smooth problems. Our method has only one parameter (a step size), and is radically simpler than o…
SVRN accelerates Newton methods by reducing variance and improving performance.
problem Improving the efficiency of Newton methods for large-scale optimization problems.
method Stochastic Variance-Reduced Newton (SVRN) algorithm that accelerates Subsampled Newton and Iterative Hessian Sketch algorithms.
result SVRN accelerates Newton methods by reducing the number of passes over the data, achieving a significant improvement in performance.
This paper tackles minimizing clipped convex functions with heuristics and mixed-integer convex programming.
problem Minimizing a sum of clipped convex functions.
method Heuristics and mixed-integer convex programming.
result Heuristics can find good solutions, and the perspective transformation yields tractable lower bounds.
RSPFs combine multiple SPNs for better density estimation.
problem Creating large SPNs for complex data.
method Random sum-product forests with residual links.
result RSPFs outperform individual SPNs and improve with residual links.
Sum-Product Graphical Modelsstat.ML
SPGM combines SPNs and GMs for efficient probabilistic inference.
problem Efficient probabilistic inference with tractable models.
method Combines SPNs and GMs, proposes a learning algorithm.
result Competitive density estimation performance.
Averaged Gradient Descent improves performance in rough landscapes.
problem Minimizing strongly non-convex functions in high-dimensional estimation problems.
method Empirical average of gradients over random positions in parameter space.
result Averaged Gradient Descent outperforms existing methods in tensor PCA.
Develops efficient nonparametric testing with random projections.
problem High computational complexity in nonparametric inference with large data.
method Random projection strategy for kernel ridge regression.
result Achieves testing optimality with minimum number of projections.
ZeroS improves Transformers by adding negative weights, matching or beating softmax attention.
problem Limited performance of linear attention methods, especially in long context sequences.
method Proposes Zero-Sum Linear Attention (ZeroS) that removes the zero-order term and reweights zero-sum softmax residuals.
result ZeroS matches or exceeds standard softmax attention across various benchmarks, theoretically expanding representable functions.
Belief Propagation solves a relaxed network flow problem.
problem Generalized Min-Cost Network Flow with relaxed flow conservation constraints.
method Extends Belief Propagation to solve a new class of network flow problems.
result Belief Propagation converges to the exact solution of the relaxed network flow problem.
Paper develops efficient algorithms for robust optimization across multiple groups.
problem Minimizing maximal empirical risk across distinct groups in robust optimization.
method Develops ALEG and ALEM algorithms for two-level finite-sum convex-concave minimax optimization.
result Achieves ε-accuracy with complexity O(m√(nlnm/ε)) and outperforms state-of-the-art methods.
Recent advances in optimization theory have shown that smooth strongly convex finite sums can be minimized faster than by treating them as a black box "batch" problem. In this work we introduce a new method in this class with a theoretical convergence rate four times faster than existing methods, for sums with sufficie…
Improved Optimization of Finite Sums with Minibatch Stochastic Variance Reduced Proximal Iterationsmath.OC
New optimization method speeds up learning from data.
problem Efficiently optimizing large datasets for machine learning.
method Minibatch stochastic variance reduced proximal iterations.
result Improved convergence speed for quadratic objectives.
TSAW improves MCMC integral estimation with faster convergence.
problem Estimating integrals using MCMC with standard random walks is slow.
method Introduces TSAW to penalize overuse in finite-state adaptive sampling.
result TSAW-based estimators converge faster, achieving error.
Accelerates optimization in asynchronous systems with sparse updates.
problem Optimizing finite-sum objectives in asynchronous lock-free environments.
method New accelerated SVRG variant with sparse updates.
result Achieves optimal incremental gradient complexity.
Novel Newton method for large-scale kernel methods using random features.
problem Efficiently solving large-scale finite-sum minimization problems in RKHS.
method Randomized feature-based Newton method for empirical risk minimization.
result Local superlinear and global linear convergence of the method.
XSPNs combine SPNs and MEVMs for efficient inference in data with repeated parts.
problem Efficient inference in data with repeated interchangeable parts.
method Introducing Exchangeability-Aware Sum-Product Networks (XSPNs) that combine SPNs and MEVMs.
result XSPNs can be more accurate than conventional SPNs when data contains repeated parts.
We present results about financial market observables, specifically returns and traded volumes. They are obtained within the current nonextensive statistical mechanical framework based on the entropy ($S_{1} \equiv S_{BG}=-k\sum\limits_{i=1}^{W}p_{i} \l…
A new method for forming learning objectives using the sum of ranked range.
problem Forming learning objectives from aggregated values.
method Sum of ranked range (SoRR) minimization with DCA.
result The proposed method effectively forms learning objectives and is applicable to binary and multi-label/multi-class classification.
Sum-Product Networks simplify building models for mixed data types.
problem Building probabilistic models for mixed data types is difficult and time-consuming.
method Proposes Sum-Product Networks (SPNs) with piecewise polynomial leave distributions and novel decomposition and conditioning steps.
result Sum-Product Networks can effectively approximate any continuous distribution and are efficient for learning and inference.
Learning Nash Equilibria in Zero-Sum Stochastic Games via Entropy-Regularized Policy Approximationcs.LG
Algorithm learns Nash equilibria in stochastic games using entropy-regularized policies.
problem Learning Nash equilibria in zero-sum stochastic games is computationally expensive.
method Entropy-regularized soft policies for Q-function updates.
result Algorithm converges to Nash equilibrium under certain conditions.
Research tackles alliance formation in many-player zero-sum games, showing reinforcement learning fails but a contract mechanism can help.
problem Tackles the challenge of alliance formation in many-player zero-sum games.
method Demonstrates the social dilemma aspect of alliance formation, introduces a contract mechanism to augment reinforcement learning.
result Naïve reinforcement learning fails to form alliances, but a contract mechanism can help.
Two new methods improve block-sparse signal recovery from noisy data.
problem Recovering block-sparse signals with unknown partitions.
method LogLOP-l2/l1 and AdaLOP-l2/l1 methods using log-sum penalty and MCP.
result Our methods outperform existing techniques in estimation accuracy.
AIS optimally adapts sampling policies based on past and current information.
problem Optimizing sampling policies in adaptive importance sampling.
method Adaptive Importance Sampling (AIS) updates sampling policy based on past and current information.
result AIS is asymptotically optimal, matching the performance of an oracle strategy.
Paper introduces a new kernel model for PSD-valued functions with theoretical guarantees and applications.
problem Enforcing positive semi-definiteness (PSD) in function models with good performance and theoretical guarantees.
method Kernel sum-of-squares model for PSD-valued functions, extending previous models for non-negative scalar functions.
result The model constitutes a universal approximator of PSD functions and can represent any smooth and strongly convex function.
Efficient algorithms estimate moments robustly to outliers.
problem Estimating moments of unknown distributions with adversarial outliers.
method Sum-of-squares relaxation of optimization problem.
result Improved guarantees and algorithms for independent component analysis and mixture learning.
Theoretical analysis of cross-entropy loss functions and their robustness.
problem Guarantees for using cross-entropy as a surrogate loss function.
method Theoretical analysis of a broad family of loss functions, including cross-entropy.
result First -consistency bounds for comp-sum losses and smooth adversarial comp-sum losses.
Algorithm learns from changing zero-sum games with no regret.
problem Learning in time-varying zero-sum games.
method Developed a single parameter-free algorithm with three performance measures.
result Algorithm recovers best known results for fixed games and adapts to non-stationarity.
Method learns reward functions that are independently obtainable and sum to original reward.
problem Learning reward functions that are independent and meaningful.
method Defining independent obtainability and optimizing a novel objective function.
result Learned reward functions generalize well to modified environments and have optimal policies.
Expanding FCCO to non-smooth weakly-convex problems, improving deep learning performance.
problem Addressing the limitations of current FCCO methods by tackling non-smooth weakly-convex problems.
method Developed a single-loop algorithm for non-smooth weakly-convex FCCO and extended it to tri-level problems.
result Established the complexity for finding ε-stationary points in the Moreau envelop of the objective function.
A new algorithm reduces the computational cost of RTRL while maintaining performance.
problem Large computational costs in RTRL for learning long-term dependencies.
method Optimal Kronecker-Sum Approximation (OK) of RTRL.
result OK is optimal for a class of RTRL approximations and matches TBPTT in performance.
The paper parallelizes HMM inference for efficient long-term computations.
problem Efficiently computing inference in long-term hidden Markov models.
method Parallelization using associative elements and operators for sum-product and max-product algorithms.
result The proposed parallel algorithms are computationally efficient for long time horizons.
New gossip algorithms improve robustness of rank-based statistics in decentralized systems.
problem Ensuring robustness in decentralized AI and edge intelligence systems, especially in the presence of corrupted or adversarial data.
method Developed asynchronous gossip algorithms for computing rank-based statistics.
result First convergence rate bound for asynchronous gossip-based rank estimation.
Improved guarantees for sparse random embeddings with explicit bounds and empirical superiority.
problem Improving the explicitness and sharpness of guarantees for sparse random embeddings.
method Explicit bounds, tighter estimates for quadratic chaos, extreme properties of sparse linear forms, and improved bounds for sums of independent random variables.
result Significantly outperforms prior works on various real-world datasets.
In this on-going work, I explore certain theoretical and empirical implications of data transformations under the PCA. In particular, I state and prove three theorems about PCA, which I paraphrase as follows: 1). PCA without discarding eigenvector rows is injective, but looses this injectivity when eigenvector rows are…
Empirical evidence suggests link polynomials can detect causality in spacetimes.
problem Detect causality in -dimensional globally hyperbolic spacetimes.
method Introduced a new invariant of certain tangles related to the Conway polynomial.
result The Conway polynomial does not detect causality in certain spacetime scenarios.
The SPS method constructs confidence regions for true parameters with optimal sample complexity.
problem Constructing exact, non-asymptotic confidence regions for true system parameters.
method Sign-Perturbed Sums (SPS) method, generalized to various types of problems.
result High probability upper bounds for SPS confidence regions show optimal shrinkage rate.
Optimizes bilevel empirical risk minimization with improved oracle calls.
problem Optimizing bilevel empirical risk minimization problems.
method Proposes a bilevel extension of the SARAH algorithm.
result Demonstrates improved oracle calls to achieve stationarity.
Paper develops probabilistic bounds for a stochastic gradient algorithm in non-convex problems.
problem Stochastic optimization in non-convex finite sum problems.
method Develops a new dimension-free Azuma-Hoeffding type bound for a martingale difference sequence.
result Empirical results show superior probabilistic performance of Prob-SARAH compared to other algorithms.
Analyzes gradient-based learning in games, avoiding local Nash equilibria.
problem Analyzes gradient-based learning in competitive games.
method Formulates a general framework and uses dynamical systems theory.
result Characterizes local Nash equilibria avoided by gradient-based learning.
New measures quantify mutual dependence between multiple random vectors.
problem Measuring mutual dependence between multiple random vectors.
method Proposes three measures based on generalized distance covariance.
result Empirical and simplified empirical measures effectively test mutual independence.
We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimization (ERM), and in particular linear least-squares regression, across a wide range of problem settings. To achieve this, we establish a framewo…
GANs may not have Nash equilibria, but proximal training can find solutions.
problem Existence of Nash equilibria in GANs optimization.
method Proximal training approach to find solutions.
result Proximal training finds solutions to GAN problems.
Freya PAGE optimizes nonconvex optimization with heterogeneous, asynchronous workers.
problem Optimizing nonconvex finite-sum problems with varying worker processing times.
method Freya PAGE, a parallel method robust to stragglers and adaptive to slow computations.
result Freya PAGE offers improved time complexity guarantees compared to previous methods.
Market lawsq-fin.ST
Power laws detected in financial data, modeled with random multipliers.
problem Detecting power laws in financial data.
method Investigated data from financial instruments, proposed a model based on sums of Maxwell-Boltzmann distributions with random multipliers.
result Detected power laws with various exponents in financial data, proposed a universal model.