DPERC efficiently estimates covariance matrices for mixed data with missing values.
problem Estimating covariance matrices for datasets with missing values and mixed features.
method Direct Parameter Estimation for Randomly Missing Data with Categorical Features (DPERC).
result DPERC outperforms other methods in estimating covariance matrices for mixed data with missing values.
This paper compares imputation and direct parameter estimation methods for missing data in correlation matrix visualization.
problem Missing data challenges in estimating correlation coefficients for accurate visualization.
method Comparison of imputation and direct parameter estimation methods for handling missing data.
result Direct parameter estimation (DPER) outperforms imputation for accurate correlation matrix visualization.
Develops a direct debiased machine learning framework using Bregman divergence.
problem Reduces bias in machine learning estimates of causal effects or structural models.
method Neyman targeted estimation and generalized Riesz regression using Bregman divergence.
result Improves estimation of parameters of interest in causal models.
We present a distributed (non-Bayesian) learning algorithm for the problem of parameter estimation with Gaussian noise. The algorithm is expressed as explicit updates on the parameters of the Gaussian beliefs (i.e. means and precision). We show a convergence rate of O(1/k) with the constant term depending on the numb…
Estimates causal effects using machine learning for binary treatment and mediator.
problem Estimating direct and indirect quantile treatment effects under selection-on-observables.
method Double/debiased machine learning estimators based on efficient score functions.
result Uniform consistency and asymptotic normality of effect estimators.
LayerNorm transformers have dead directions that can be read from their parameters alone.
problem Locating dead directions in LayerNorm transformers
method Using the inverse-scale direction of LayerNorm affine parameters
result Predicted dead direction matches measured bottom singular direction
Paper tackles hyper-gradient estimation in decentralized FL over time-varying networks.
problem Excessive communication costs and inability to use robust networks.
method Introduces an optimality condition and uses Push-Sum for averaging model parameters and gradients over time-varying directed networks.
result Derives a hyper-gradient estimator that operates over time-varying directed networks and converges to the true hyper-gradient.
Regularization is a popular technique in machine learning for model estimation and avoiding overfitting. Prior studies have found that modern ordered regularization can be more effective in handling highly correlated, high-dimensional data than traditional regularization. The reason stems from the fact that the ordered…
The modelling of data on a spherical surface requires the consideration of directional probability distributions. To model asymmetrically distributed data on a three-dimensional sphere, Kent distributions are often used. The moment estimates of the parameters are typically used in modelling tasks involving Kent distrib…
We propose directed time series regression, a new approach to estimating parameters of time-series models for use in certainty equivalent model predictive control. The approach combines merits of least squares regression and empirical optimization. Through a computational study involving a stochastic version of a well …
New algorithms estimate Jacobian matrices for large-scale machine learning.
problem Efficiently computing search directions for large nonlinear least squares.
method Exploit low-rank structure in Hessian to estimate Jacobian matrices.
result Two algorithms perform well compared to state-of-the-art methods.
Proposes a new estimator for causal mediation with continuous treatments.
problem Estimation of direct and indirect effects with continuous treatments.
method Kernel smoothing approach with cross-fitting for non-parametric estimation.
result Multiply robust and asymptotically normal estimator for continuous treatments.
Paper proposes DAG-DB for learning discrete DAGs via backpropagation.
problem Learning Directed Acyclic Graphs (DAGs) from data.
method DAG-DB uses Discrete Backpropagation with I-MLE and Straight-Through Estimation.
result DAG-DB learns DAGs effectively using probabilistic sampling and backpropagation.
New algorithm for estimating MLR parameters with non-Gaussian noise.
problem Estimating MLR parameters with non-Gaussian noise.
method Combining ADMM with EM algorithm idea.
result Our method outperforms EM algorithm in non-Gaussian noise case.
We introduce the "NoBackTrack" algorithm to train the parameters of dynamical systems such as recurrent neural networks. This algorithm works in an online, memoryless setting, thus requiring no backpropagation through time, and is scalable, avoiding the large computational and memory cost of maintaining the full gradie…
A new method optimizes neural sequence models for better task performance.
problem Training neural sequence models with maximum likelihood estimation ignores task losses.
method Maximum likelihood guided parameter search (MGS) in the parameter space.
result MGS optimizes sequence-level losses, reducing repetition and non-termination.
Paper efficiently infers differential parameters in time-varying models using time score matching.
problem Efficiently inferring differential parameters in time-varying probabilistic models.
method Directly estimates the differential parameter using time score matching and proves consistency of the method.
result Consistent estimation of parameter derivatives in high-dimensional settings.
Parametric images provide insight into the spatial distribution of physiological parameters, but they are often extremely noisy, due to low SNR of tomographic data. Direct estimation from projections allows accurate noise modeling, improving the results of post-reconstruction fitting. We propose a method, which we name…
Direct approach for handling contextual bandits with latent state dynamics.
problem Handling contextual bandits with latent state dynamics, especially when rewards depend on posterior probabilities of hidden states.
method Direct reduction to standard linear contextual bandits, extended analysis of HMM parameters, periodic update of reward-model parameters.
result Periodic update of reward-model parameters allows handling complex dependencies in hidden states.
Novel approach for SEM in small samples with p>n.
problem Small sample size and p>n issues in factor-based SEM. method Reformulates covariance structure into self-covariance and cross-covariance, defines a feasible set with relative error constraint.
result Improved stability and directional information in small-sample settings.
A new ES method improves reinforcement learning speed and accuracy.
problem Slow convergence and local maxima in reinforcement learning.
method Directional Gaussian Smoothing Evolution Strategy (DGS-ES)
result DGS-ES accelerates RL training with high accuracy and nonlocal search direction.
Proposes a new measure to evaluate stability of statistical parameters under distributional shifts.
problem Difficulty in transferring knowledge across data sets due to distributional changes.
method Introduces a measure of instability quantifying sensitivity of statistical parameters to Kullback-Leibler divergence and directional shifts.
result The proposed measure can elucidate the type of shifts a parameter is sensitive to and improve estimation accuracy under shifted distributions.
Estimation of density derivatives is a versatile tool in statistical data analysis. A naive approach is to first estimate the density and then compute its derivative. However, such a two-step approach does not work well because a good density estimator does not necessarily mean a good density-derivative estimator. In t…
Estimates parameters in max-linear Bayesian networks with noise.
problem Causal inference in extreme-value settings with noise parameters.
method Max-plus algebra and logarithm transformation, normal distribution estimation, EM algorithm and quadratic optimization.
result An estimator of a parameter for each edge in a DAG is normally distributed.
IGNIS uses neural networks to estimate copula parameters robustly.
problem Pathological properties of Archimedean copulas make traditional estimators brittle.
method Unified neural estimation framework with multi-input architecture and softplus output layer.
result Accurate and stable estimates for real-world datasets.
Direct neural ratio estimator for likelihood-free inference.
problem Efficient likelihood estimation for complex models.
method Amortized likelihood ratio estimation using neural networks.
result DNRE often outperforms previous ratio estimators.
A new method recovers latent potentials from graph flows, preserving ordering and stability.
problem Recovering latent potentials from graph flows is ill-posed and standard methods collapse the ordering.
method Gauge-invariant, parameter-insensitive regularization using Dirichlet energy.
result The method preserves ordering and stability across different regularization strengths.
We outline a representation for discrete multivariate distributions in terms of interventional potential functions that are globally normalized. This representation can be used to model the effects of interventions, and the independence properties encoded in this model can be represented as a directed graph that allows…
Gibbs sampling is a workhorse for Bayesian inference but has several limitations when used for parameter estimation, and is often much slower than non-sampling inference methods. SAME (State Augmentation for Marginal Estimation) \cite{Doucet99,Doucet02} is an approach to MAP parameter estimation which gives improved pa…
Efficiently estimates missing data parameters without iterations.
problem Randomly missing data in statistics and bioinformatics.
method Novel algorithms for maximum likelihood estimation.
result Less time-consuming and superior estimation performance.
Paper estimates Hurst parameter from implied volatilities.
problem Estimating Hurst parameter from implied volatilities.
method Uses covariance between asset return and realized volatility, and applies limit theorems for stochastic volatility models.
result Direct relation between covariance and slope of at-the-money implied volatility established.
New meta-learning method improves domain generalization by balancing parameters closer to domain centroids.
problem Improving domain generalization by reducing overfitting to specific domains.
method Arithmetic meta-learning with arithmetic-weighted gradients to balance parameters closer to domain centroids.
result Experimental validation of improved domain generalization performance.
Algorithm identifies Pareto front using multiple context directions and reuses exploration samples.
problem Identifying a set of arms with undominated mean reward vectors in linear bandits.
method Proposes a new estimator that updates estimates along multiple context directions and reuses exploration samples.
result Optimal sample complexity and logarithmic regret compared to optimal algorithms.
Paper tackles efficient policy gradient estimation from off-policy data.
problem Estimating policy gradients from off-policy data is challenging and inefficient.
method Derives asymptotic lower bounds, proposes a meta-algorithm with 3-way robustness, and establishes convergence guarantees.
result Meta-algorithm achieves the lower bound on mean-squared error without parametric assumptions.
Study identifies parameters in causal models with latent confounding.
problem Parameter identification in linear non-Gaussian causal models with latent confounding.
method Graphical criterion for necessary and sufficient identifiability of direct causal effects, with polynomial-time algorithm.
result Developed a graphical criterion for identifying direct causal effects in latent variable models with arbitrary non-linear confounding.
This work simplifies Bayesian inference for neural networks by identifying influential parameter directions.
problem High computational complexity in Bayesian inference for neural networks due to high-dimensional parameter space.
method Constructing an active subspace of influential parameter directions to reduce dimensionality.
result Effective and scalable Bayesian inference achieved via reduced active subspace.
Directed acyclic graphs (DAGs) are a popular framework to express multivariate probability distributions. Acyclic directed mixed graphs (ADMGs) are generalizations of DAGs that can succinctly capture much richer sets of conditional independencies, and are especially useful in modeling the effects of latent variables im…
Psychiatric neuroscience is increasingly aware of the need to define psychopathology in terms of abnormal neural computation. The central tool in this endeavour is the fitting of computational models to behavioural data. The most prominent example of this procedure is fitting reinforcement learning (RL) models to decis…
This paper analyzes quantiles of heavy-tailed distributions, separating projection direction and quantile threshold effects.
problem Analyzing quantiles of heavy-tailed distributions with estimated parameters.
method Introduces a Q-Q orthogonality formulation to separate projection-direction and quantile-threshold effects.
result Decomposes the difference between empirical and population quantiles into three terms.
Simulation of high-speed train aerodynamics using RANS and machine learning.
problem Aerodynamic analysis of high-speed trains under turbulent flow conditions.
method RANS equations with turbulence model, machine learning (GEP, GPR, RF) for predictions.
result Random Forest (RF) provides the most accurate predictions for aerodynamic coefficients.
Log-linear models are the popular workhorses of analyzing contingency tables. A log-linear parameterization of an interaction model can be more expressive than a direct parameterization based on probabilities, leading to a powerful way of defining restrictions derived from marginal, conditional and context-specific ind…
New IDS algorithm refines parameter norm bounds for better bandit performance.
problem Frequentist IDS requires tight norm bounds, which are often unavailable in practice.
method Iteratively refines a high-probability upper bound on true parameter norm using data.
result Regret bounds independent of assumed parameter norm, outperforming state-of-the-art algorithms.
In stochastic gradient descent, especially for neural network training, there are currently dominating first order methods: not modeling local distance to minimum. This information required for optimal step size is provided by second order methods, however, they have many difficulties, starting with full Hessian having…
New CRB derived for curved models using extrinsic geometry.
problem Estimate curved statistical families accurately.
method Vector generalization of CRB with curvature correction using SDP and SOS relaxations.
result Directional curvature correction provides more accurate estimation.
Deterministic algorithm estimates ridge regression with minimal space.
problem Estimating ridge regression solutions efficiently.
method Deterministic space-efficient algorithm using Frequent Directions.
result First o(d2) space deterministic streaming algorithm with guaranteed error. Estimates high-dimensional posterior densities by marginal distributions and neural networks.
problem High-dimensional probability density estimation for inference is difficult.
method Direct estimation of lower-dimensional marginal distributions, using Moment Networks for fast computation of moments.
result Demonstrates estimation of gravitational wave time series and applications in cosmology.
This article addresses the modeling of reverberant recording environments in the context of under-determined convolutive blind source separation. We model the contribution of each source to all mixture channels in the time-frequency domain as a zero-mean Gaussian random variable whose covariance encodes the spatial cha…
Optimal neural network approximation for Wasserstein gradient direction via convex optimization.
problem Approximating Wasserstein gradient direction with limited data.
method Two-layer networks with squared-ReLU activations, SDP relaxation.
result Optimal approximation of Wasserstein gradient direction in two-layer networks.