Structural pruning of neural network parameters reduces computation, energy, and memory transfer costs during inference. We propose a novel method that estimates the contribution of a neuron (filter) to the final loss and iteratively removes those with smaller scores. We describe two variations of our method using the …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,786 papers · 148 categories
Trend · papers per month
48 results for “parameter contributions”
GENIE balances domain-invariant feature learning and gradient alignment for improved DG performance.
problem Domain Generalization (DG) overfitting to domain-specific features
method GENIE (Generalization-ENhancing Iterative Equalizer) optimizer
result Prevents a small subset of parameters from dominating optimization, promoting domain-invariant feature learning
Improves privacy amplification by shuffling for differential privacy.
problem Enhancing privacy guarantees in systems with anonymous data contributions.
method Theoretical and numerical analysis of Rényi differential privacy parameters and privacy amplification by shuffling.
result First asymptotically optimal analysis of Rényi differential privacy parameters for shuffled outputs.
Investigates chaotic financial time series with monthly contributions and devaluation.
problem Analyzing chaotic behavior in financial processes with piecewise contributions and negative interest rates.
method Examines a financial process with monthly contributions and devaluation, showing dichotomy in behavior.
result Financial time series exhibit either periodic sequences or Cantor set of ω-limit points, with chaotic behavior at points of a Cantor attractor.
Topological data analysis and its main method, persistent homology, provide a toolkit for computing topological information of high-dimensional and noisy data sets. Kernels for one-parameter persistent homology have been established to connect persistent homology with machine learning techniques. We contribute a kernel…
Deep learning estimates time-varying Markov model parameters.
problem Estimating time-dependent parameters in Markov models.
method Reframes parameter estimation as an optimization problem using maximum likelihood.
result Real solution close to SDE with neural network-derived parameters under specific conditions.
Global optimization in Bayesian inference yields little additional benefit.
problem Improving psychometric parameter estimation using global optimization strategies.
method Experimental simulations comparing myopic and global strategies in multiple models.
result Global optimization strategies provide negligible additional utility improvement beyond the immediate next steps.
LCA measures neural network training changes, revealing parameter contributions.
problem Limited understanding of neural network training process.
method LCA decomposes training trajectory using Runge-Kutta integrator.
result 50% of parameters help during training, some layers hurt overall.
Determinantal point processes (DPPs) offer a powerful approach to modeling diversity in many applications where the goal is to select a diverse subset. We study the problem of learning the parameters (the kernel matrix) of a DPP from labeled training data. We make two contributions. First, we show how to reparameterize…
The present contribution suggests the use of a multidimensional scaling (MDS) algorithm as a visualization tool for manifold-valued elements. A visualization tool of this kind is useful in signal processing and machine learning whenever learning/adaptation algorithms insist on high-dimensional parameter manifolds.
Data science relies on pipelines that are organized in the form of interdependent computational steps. Each step consists of various candidate algorithms that maybe used for performing a particular function. Each algorithm consists of several hyperparameters. Algorithms and hyperparameters must be optimized as a whole …
Improves scalability and efficiency of mixture models in black-box variational inference.
problem Scaling mixture models in black-box variational inference leads to high parameter and time costs.
method Introduces MISVAE for amortized mixture parameter space and new ELBO estimators.
result Achieves superior estimation performance with fewer parameters and shorter inference time.
We briefly review the approach to optimization of portfolios according to the theory of Markowitz and propose a further modification that can improve the outcome of the optimization process. The modification takes account of the entropic contribution from the time series used to compute the parameters in the Markowitz …
One of the main obstacles to broad application of reinforcement learning methods is the parameter sensitivity of our core learning algorithms. In many large-scale applications, online computation and function approximation represent key strategies in scaling up reinforcement learning algorithms. In this setting, we hav…
Federated learning is a recent advance in privacy protection. In this context, a trusted curator aggregates parameters optimized in decentralized fashion by multiple clients. The resulting model is then distributed back to all clients, ultimately converging to a joint representative model without explicitly having to s…
The paper studies geometric constants under modified Ricci flows with variable parameters.
problem Understanding geometric constants under variable coupling parameters in Ricci flows.
method Introduced modified Ricci flows with variable coefficients, derived evolution formulas, and proved monotonicity conditions.
result Conditions for maintaining monotonicity of geometric constants under modified Ricci flows.
In the presence of a layer of metaprobabilities (from uncertainty concerning the parameters), the asymptotic tail exponent corresponds to the lowest possible tail exponent regardless of its probability. The problem explains "Black Swan" effects, i.e., why measurements tend to chronically underestimate tail contribution…
Neural network models and deep models are one of the leading and state of the art models in machine learning. Most successful deep neural models are the ones with many layers which highly increases their number of parameters. Training such models requires a large number of training samples which is not always available…
A generalized public goods game with coupling of individual ability and project benefitphysics.soc-ph
Facing a heavy task, any single person can only make a limited contribution and team cooperation is needed. As one enjoys the benefit of the public goods, the potential benefits of the project are not always maximized and may be partly wasted. By incorporating individual ability and project benefit into the original pu…
Sound synthesis is a complex field that requires domain expertise. Manual tuning of synthesizer parameters to match a specific sound can be an exhaustive task, even for experienced sound engineers. In this paper, we introduce InverSynth - an automatic method for synthesizer parameters tuning to match a given input soun…
Machine learning in high-energy physics faces challenges from nuisance parameters, which are reviewed and techniques to mitigate their impact are discussed.
problem Impact of nuisance parameters on machine learning performance in high-energy physics.
method Review and discussion of techniques including nuisance-parameterized models, modified or adversary losses, semi-supervised learning, and inference-aware techniques.
result Various methods to reduce the impact of nuisance parameters and improve model performance in high-energy physics.
Capsule network (CapsNet) was introduced as an enhancement over convolutional neural networks, supplementing the latter's invariance properties with equivariance through pose estimation. CapsNet achieved a very decent performance with a shallow architecture and a significant reduction in parameters count. However, the …
Study identifies key parameters and input dimensions making LLMs and VLMs brittle.
problem Vulnerability of large language and vision-language models to perturbations.
method Proposed FI measure based on information geometry to quantify sensitivity.
result Small subset of high FI parameters significantly contribute to brittleness.
Artificial neural networks (ANNs) suffer from catastrophic forgetting when trained on a sequence of tasks. While this phenomenon was studied in the past, there is only very limited recent research on this phenomenon. We propose a method for determining the contribution of individual parameters in an ANN to catastrophic…
Semi-analytical approach for optimal wealth management contributions.
problem Optimizing contributions to achieve a financial goal with uncertain returns.
method Controlled backward Kolmogorov equation and Schrodinger equation solution.
result Semi-analytical solutions for efficient frontiers in control space.
Paper develops a new method for optimal stopping in American options.
problem Optimal stopping in American options with singular generators.
method Entropy-regularized penalization scheme for reflected BSDEs with singular generators.
result Limit of the penalization scheme solves a reflected BSDE with a logarithmically singular generator.
We provide a general theory of the expectation-maximization (EM) algorithm for inferring high dimensional latent variable models. In particular, we make two contributions: (i) For parameter estimation, we propose a novel high dimensional EM algorithm which naturally incorporates sparsity structure into parameter estima…
ShapleyBO explains BO's decisions, enhancing human-AI collaboration in robotics.
problem Lack of explainability in BO's black box nature, especially in human-in-the-loop applications.
method Uses game-theoretic Shapley values to quantify and interpret BO's parameter contributions.
result ShapleyBO identifies exploration and exploitation contributions, disentangling aleatoric and epistemic uncertainty.
We present a system that enables rapid model experimentation for tera-scale machine learning with trillions of non-zero features, billions of training examples, and millions of parameters. Our contribution to the literature is a new method (SA L-BFGS) for changing batch L-BFGS to perform in near real-time by using stat…
Parameter inference from a non-stationary unknown processphysics.data-an
Unified review of methods for inferring non-stationary process parameters.
problem Inferring parameters of non-stationary processes without a known model.
method Unified review and categorization of algorithms for Parameter Inference from a Non-stationary Unknown Process (PINUP).
result Simple statistical features can perform well on non-stationary systems, highlighting gaps in existing methods.
Piecewise constant denoising can be solved either by deterministic optimization approaches, based on the Potts model, or by stochastic Bayesian procedures. The former lead to low computational time but require the selection of a regularization parameter, whose value significantly impacts the achieved solution, and whos…
The present paper proposes generalized Gaussian kernel adaptive filtering, where the kernel parameters are adaptive and data-driven. The Gaussian kernel is parametrized by a center vector and a symmetric positive definite (SPD) precision matrix, which is regarded as a generalization of the scalar width parameter. These…
Using Quantum Mechanics to Cluster Time Seriesphysics.data-an
In this article we present a method by which we can reduce a time series into a single point in . We have chosen 13 dimensions so as to prevent too many points from being labeled as "noise." When using a Euclidean (or Mahalanobis) metric, a simple clustering algorithm will with near certainty label the…
Model projection transfers convolutional network properties to feedforward networks.
problem Transferring properties between feedforward and convolutional networks.
method Unified node-level framework with tensor-valued activations, model projection.
result Projected CNN nodes inherit GFFN-style trainable structure.
Bayesian units improve speech recognition with minimal parameters.
problem Improving speech recognition models with fewer parameters.
method Derived Bayesian recurrent units integrated into deep learning frameworks.
result Adding Bayesian units improves speech recognition performance.
Learning from the crowd has become increasingly popular in the Web and social media. There is a wide variety of crowdlearning sites in which, on the one hand, users learn from the knowledge that other users contribute to the site, and, on the other hand, knowledge is reviewed and curated by the same users using assessm…
This paper studies systematic exploration for reinforcement learning with rich observations and function approximation. We introduce a new model called contextual decision processes, that unifies and generalizes most prior settings. Our first contribution is a complexity measure, the Bellman rank, that we show enables …
Gaussian Mixture Model with unknown diagonal covariances via continuous sparse regularizationmath.ST
Paper estimates GMMs with unknown covariances using sparse regularization.
problem Estimating GMMs with unknown diagonal covariances from samples.
method Employed Beurling-LASSO (BLASSO) for sparse estimation of component means, covariances, and weights.
result Established non-asymptotic recovery guarantees with nearly parametric convergence rates.
We consider the task of low-multilinear-rank functional regression, i.e., learning a low-rank parametric representation of functions from scattered real-valued data. Our first contribution is the development and analysis of an efficient gradient computation that enables gradient-based optimization procedures, including…
This paper improves federated learning efficiency by auto-tuning secure aggregation parameters.
problem Efficient communication in federated learning with quantized updates.
method Auto-tuning secure aggregation parameters based on random rotation properties.
result Improved communication efficiency in federated learning with secure aggregation.
New CDC scheme avoids intergenerational subsidies, offering better outcomes.
problem Intergenerational cross-subsidies in UK CDC schemes.
method Collective-Drawdown CDC approach using explicit insurance contracts.
result Better pension outcomes with no intergenerational cross-subsidies.
New method calculates degrees of freedom for sparse estimation in continuous models.
problem Quantifying effective parameters in over-parameterized models with large continuous parameter spaces.
method Develops a continuous Lasso method for sparsity-inducing optimization over measure spaces.
result Proof of a continuous degrees of freedom formula for Beurling Lasso.
This note studies a method for the efficient estimation of a finite number of unknown parameters from linear equations, which are perturbed by Gaussian noise. In case the unknown parameters have only few nonzero entries, the proposed estimator performs more efficiently than a traditional approach. The method consists o…
Generative model identifies temporal count data components with regime-dependent contributions.
problem Modeling temporal count data with regime-dependent dynamics.
method Generative framework combining regime-adaptive dynamics with Poisson log-normal emissions.
result Established identifiability of the model and revealed co-variation patterns and regime shifts.
The paper explores how symmetries and noise in SGD influence parameter dynamics.
problem Understanding the dynamics of parameter updates in SGD with symmetries.
method Proved the existence of noise equilibria and showed their role in balancing gradient noise.
result Gradient noise creates a systematic motion of parameters to a unique fixed point, called noise equilibria.
Proposes LFGP for likelihood-free Gaussian process regression.
problem Inability to set likelihood functions in unknown probability models.
method Clusters and approximates likelihood using asymptotic normality.
result Reduces assumptions and computational costs for scalable problems.
Don't Fall for Tuning Parameters: Tuning-Free Variable Selection in High Dimensions With the TREXstat.ME
Lasso is a seminal contribution to high-dimensional statistics, but it hinges on a tuning parameter that is difficult to calibrate in practice. A partial remedy for this problem is Square-Root Lasso, because it inherently calibrates to the noise variance. However, Square-Root Lasso still requires the calibration of a t…
Study decomposes uncertainty in HK-distribution parameter estimation for QUS.
problem Uncertainty in HK-distribution parameter estimation for quantitative ultrasound.
method Bayesian Neural Networks (BNNs) for parameter estimation and uncertainty decomposition.
result Decomposes total predictive uncertainty into epistemic and aleatoric components.