Latent feature models (LFM)s are widely employed for extracting latent structures of data. While offering high, parameter estimation is difficult with LFMs because of the combinational nature of latent features, and non-identifiability is a particularly difficult problem when parameter estimation is not unique and ther…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Parameter Non-identifiability”
Hypothesis testing in singular models is fundamentally about identifiable vs. non-identifiable parameters.
problem Testing in singular models is inherently problematic due to non-identifiability and degeneracy of Fisher information.
method Formalized the overlap obstruction and showed that hypotheses over non-identifiable parameters are untestable, while those over identifiable parameters reduce to classical testing.
result Hypotheses over non-identifiable parameters are untestable, while those over identifiable parameters reduce to classical testing.
Overparametrized neural networks retain significant epistemic uncertainty even with sufficient data.
problem Epistemic uncertainty in overparametrized neural networks persists despite model identifiability.
method Analysis of non-identifiability and characterization of residual uncertainty in one-hidden-layer ReLU networks.
result Substantial parameter uncertainty remains even when the underlying function is fully identified.
Study optimizes step size for Metropolis algorithm in non-identifiable cases.
problem Optimizing step size for Metropolis algorithm in non-identifiable models.
method Analytical derivation of average acceptance rate for non-identifiable cases.
result Developed optimization principle for step size based on average acceptance rate.
New method uses logical relations to derive bounds and inequality constraints from causal models.
problem Recovering bounds and inequality constraints from unobserved confounding.
method Using rules of probability and restrictions on counterfactuals implied by causal graphical models.
result Powerful method to recover known and novel bounds and constraints.
This paper tackles non-identifiability in financial market simulations using multivariate time series data.
problem Non-identifiability issue in social simulation models, leading to indistinguishable simulated time series data.
method Proposes a maximization-based aggregation function to form a new calibration objective function using multiple time series features.
result Significant improvements in alleviating non-identifiability and achieving higher simulation fidelity.
Analysis of DPPs and k-DPPs via spectral decomposition reveals identifiable parameters and non-identifiability gaps.
problem Identifying parameters of DPPs and k-DPPs through spectral decomposition.
method Spectral decomposition of the covariance matrix, analysis of invariances, and counting arguments.
result Identifiability of parameters changes fundamentally for k-DPPs, with specific invariances and non-identifiability gaps.
Solves parameter non-identifiability in Bayesian LTI system identification.
problem Parameter non-identifiability in standard Bayesian approaches for LTI system identification.
method Embedding canonical forms of LTI systems within the Bayesian framework.
result Unlocking the use of meaningful priors and robust uncertainty estimates.
Bayesian Neural Networks with Latent Variables (BNN+LVs) capture predictive uncertainty by explicitly modeling model uncertainty (via priors on network weights) and environmental stochasticity (via a latent input noise variable). In this work, we first show that BNN+LV suffers from a serious form of non-identifiability…
New research shows LLMs can't be explained by statistical generalization alone.
problem Understanding why large language models (LLMs) perform well despite statistical generalization limitations.
method Examined the non-identifiability of AR probabilistic models and their implications for LLMs.
result Non-identifiability of LLMs leads to different behaviors and requires a separate theoretical explanation.
Variational autoencoders often collapse, showing latent variables are non-identifiable.
problem Posterior collapse in variational autoencoders due to non-identifiable latent variables.
method Proves latent variable non-identifiability causes posterior collapse. Proposes latent-identifiable models using Brenier maps and input convex neural networks.
result Latent-identifiable models resolve posterior collapse and provide meaningful representations.
New method warns of counterfactual non-identifiability in DSCMs.
problem Counterfactual inference from observational data is non-identifiable even without unobserved confounding.
method Prove counterfactual identifiability for monotonic generation mechanisms, provide impossibility result for general mechanisms, propose method for estimating worst-case errors.
result Non-identifiability of counterfactual inference from observational data, even in absence of unobserved confounding.
A new method resolves non-identifiability in reward modeling using anchor labels.
problem Non-identifiability in reward modeling from pairwise preferences alone.
method Anchor-guided Variance-aware Reward Modeling (AVRM) framework.
result AVRM resolves non-identifiability and improves reward modeling performance.
IMA addresses non-identifiability in nonlinear ICA by assuming orthogonal Jacobian columns.
problem Non-identifiability in nonlinear ICA.
method IMA assumes orthogonal Jacobian columns and extends to manifold settings.
result IMA circumvents non-identifiability issues and can be beneficial for higher-dimensional observations.
We introduce thermodynamic response functions for singular Bayesian models.
problem Singular Bayesian models violate regular asymptotics due to non-identifiability and degenerate Fisher geometry.
method Posterior tempering induces thermodynamic response functions, linking WAIC, WBIC, and singular fluctuation.
result WAIC, WBIC, and singular fluctuation are unified within a thermodynamic response framework.
Unified framework for singular statistical models using observable charts.
problem Non-identifiability and breakdown of classical asymptotic theory in singular models.
method Invariant framework based on observable charts to define local coordinate systems in model space.
result Observable order provides a lower bound on KL divergence vanishing rate in singular models.
The paper proposes a new method to calibrate multiple computer models simultaneously.
problem Calibrating multiple computer models one at a time is inefficient.
method Developed a probabilistic framework using customized neural networks.
result Simultaneous calibration improves predictive accuracy but can be non-identifiable in high dimensions.
Unified framework for SGMoE resolves estimation and selection issues.
problem Non-identifiability, coupled differential relations, and tight coupling in softmax-Gated models.
method Unified statistical framework with Voronoi-type loss functions and dendrograms of mixing measures.
result Consistent selection of the number of experts without model sweeps, optimal parameter rates under overfitting.
Transformers without skip connections collapse token representations to a single direction.
problem Rapid convergence of token representations to a single direction in self-attention-only Transformers.
method Analysis of layer normalization, residual connections, and multi-head attention mechanisms.
result Residual connections prevent rank collapse in real Transformers, while MLPs generate new feature directions.
This work explores how overparametrization and priors affect Bayesian neural network posteriors.
problem Symmetries, non-identifiabilities, and weight-space priors fragment and inflate BNN posteriors.
method We study the interplay between overparametrization and priors in BNN posteriors, deriving key phenomena and validating through experiments.
result Overparametrization induces structured, prior-aligned weight posterior distributions.
Memory-efficient optimizers fail to track a subspace, leading to unpredictable model performance.
problem Memory-efficient optimizers fail to track a subspace, leading to unpredictable model performance.
method Analyzing the behavior of memory-efficient optimizers like GaLore, which project gradients onto a rank-r subspace recomputed every T steps.
result Memory-efficient optimizers fail to track a subspace, leading to unpredictable model performance.
Neural networks can learn relationships that traditional models cannot.
problem Identifying factors that differentiate neural networks from traditional models.
method Proving non-identifiability of neural networks compared to smooth parametric models.
result Neural networks can learn nontrivial relationships that traditional models cannot.
New framework estimates treatment effects based on preferences.
problem Estimating treatment effects with flexible outcomes.
method Preference-based Conditional Treatment Effect (CPTE) framework.
result CPTE provides interpretable targets and new identifiability conditions.
We consider a blind identification problem in which we aim to recover a statistical model of a network without knowledge of the network's edges, but based solely on nodal observations of a certain process. More concretely, we focus on observations that consist of single snapshots taken from multiple trajectories of a d…
Single sample estimation for hard-constrained models like SAT and coloring problems.
problem Estimating parameters of Markov Random Fields with hard constraints using a single sample.
method Pseudo-likelihood estimator with coupling techniques.
result Single-sample estimation is not always possible for hard constraints, and existence of an estimator is related to satisfiability.
E&E uses contrastive learning to speed up SBI for high-dimensional systems.
problem Challenges in training high-dimensional emulators for complex systems.
method Contrastive learning for low-dimensional latent embedding and fast emulator.
result Superior performance in non-identifiable parameter estimation tasks.
New framework calibrates computer models using deep learning and quantile regression.
problem Uncertainty in computer model input parameters due to high-dimensional time series data.
method Deep neural network with long-short term memory layers for inverse modeling, quantile regression for interval predictions.
result Accurate point and interval estimates for input parameters in WRF-hydro model.
Inversion-free natural gradient method for Riemannian manifolds.
problem Hindered by the need for Euclidean space, Fisher information matrix inversion, and computational cost.
method Intrinsic, inversion-free natural gradient method on Riemannian manifolds, using moving approximation of inverse FIM.
result Almost-sure convergence rates and sub-quadratic storage complexity for large-scale applications.
A new method for binary ICA using non-stationary sources.
problem Independent component analysis of binary data.
method Linear mixing model in latent space, followed by binary observation model with non-stationary sources.
result Proves non-identifiability with few observed variables but identifies with more variables.
Proposes efficient bounds for causal effect estimation under weak confounding.
problem Estimating causal effects with weakly confounded variables.
method Develops an efficient linear program to derive upper and lower bounds on causal effect under small entropy of unobserved confounders.
result Bounds are consistent and tighter for weakly confounded variables.
The Rashomon effect shows many models can perform similarly, explored in this paper.
problem Why do many models perform similarly in machine learning?
method Categorized causes into statistical, structural, and procedural sources.
result Structural multiplicity persists and cannot be resolved without additional assumptions.
Proposes clustering and pruning to simplify causal data fusion models.
problem Combining observational and experimental data to identify causal effects.
method Generalizes pruning and clustering operations for multiple data sources.
result Derives conditions for inferring causal effects from simplified models.
This paper improves parameter estimation in cardiac models using Gaussian process-based MH sampling.
problem Uncertainty in estimating patient-specific model parameters from sparse and noisy clinical data.
method Integrates surrogate modeling into Metropolis-Hastings sampling to improve computational efficiency and accuracy.
result Significant gain in computational efficiency without compromising accuracy, and insights into tissue heterogeneity.
Nonnegative matrix factorization (NMF) is a popular dimension reduction technique that produces interpretable decomposition of the data into parts. However, this decompostion is not generally identifiable (even up to permutation and scaling). While other studies have provide criteria under which NMF is identifiable, we…
POSCMs extend SCMs for causal modeling with latent contexts.
problem Causal modeling with latent contexts and endogenous mechanisms.
method Kolmogorov-Arnold-Sprecher edge-functional decomposition for explicit parametrization.
result Identifiability of structure and mechanisms under latent context.
We investigate the non-identifiability issues associated with bidirectional adversarial training for joint distribution matching. Within a framework of conditional entropy, we propose both adversarial and non-adversarial approaches to learn desirable matched joint distributions for unsupervised and supervised tasks. We…
New method identifies SDE drift and diffusion from temporal data.
problem Learning SDE parameters from temporal data, especially in noisy or incomplete data.
method Entropy-regularized optimal transport, APPEX algorithm.
result Can almost always recover drift and diffusion from temporal marginals.
Study clarifies Bayesian generalization error in CBM for 3-layered linear neural networks.
problem Understanding the generalization error in concept bottleneck models.
method Mathematical analysis of Bayesian generalization error and free energy in CBM for 3-layered linear neural networks.
result CBM significantly alters the parameter region and Bayesian generalization error compared to standard models.
Paper proposes a new sparse VAR model for high-dimensional time series.
problem Non-identifiability, computational intractability, and difficulty of interpretation for high-dimensional time series.
method Sparse infinite-order VAR model with -regularized estimation methods.
result Greater statistical efficiency and interpretability achieved with little loss of temporal information.
Study homotopy groups in GIT quotients using transversality methods.
problem Homotopy groups of stable loci in affine GIT.
method Infinite-dimensional transversality framework extended to GIT setting.
result Generic homotopies avoid unstable and strictly semistable strata under certain conditions.
The study uses pre-trained neural networks to adjust for confounding in non-tabular data.
problem Neglecting non-tabular data sources can lead to biased ATE estimates.
method Leverages latent features from pre-trained neural networks to adjust for confounding.
result Neural networks can achieve fast convergence rates for ATE estimation with latent features.
Efficient Calibration of Multi-Agent Simulation Models from Output Series with Bayesian Optimizationq-fin.ST
The paper proposes a framework to calibrate multi-agent simulation models from output series using Bayesian optimization.
problem Calibrating multi-agent simulation models from observable output series.
method Novel eligibility set concept, two-sample Kolmogorov-Smirnov test with Bonferroni correction, Bayesian optimization (BO), and trust-region BO (TuRBO).
result Demonstrated the efficiency of the proposed framework using numerical experiments.
LLMs can memorize economic data and recall exact values before their training cutoff.
problem Evaluating the trustworthiness of LLMs' economic forecasts during their training period.
method Demonstrated through counterfactual forecasting and analysis of LLMs' recall ability.
result LLMs have memorized economic and financial data, leading to recall-level accuracy before their knowledge cutoff.
Deep Discrete Encoders (DDEs) tackle interpretable generative models for rich data with discrete latent layers.
problem Overparametrized, non-identifiable, and uninterpretable deep generative models in high-stakes applications.
method Directed graphical model with multiple binary latent layers, transparent identifiability conditions, scalable estimation pipeline.
result Transparent identifiability conditions and scalable estimation pipeline for interpretable DDEs.
Bayesian models' singular fluctuation is shown to be akin to specific heat, influencing model complexity and generalization.
problem Understanding the thermodynamic interpretation of singular fluctuation in Bayesian models.
method Showed singular fluctuation as the curvature of Bayesian free energy and variance of log-likelihood observable under a Gibbs posterior.
result Singular fluctuation is the statistical analogue of specific heat, controlling model complexity and generalization.
Training-free source selection for LLM families with shared vocabularies
problem Source selection for LLM families with shared vocabularies
method Fisher alignment at vocabulary scale
result Fisher alignment is a cosine between kernel mean embeddings in the joint activation-error space
The paper discovers a hidden component in data using an autoencoder with a discriminator.
problem Discovering a single independent latent variable in data.
method An autoencoder with a discriminator is used to recover the hidden component.
result The approach can recover the hidden component up to entropy-preserving transformations.
Physics-Informed Neural Networks (PINNs) benchmarked against clinical estimator and reveal parameter identifiability
problem Chemotherapy pharmacokinetics (PK) with tissue concentration not measured
method Physics-Informed Neural Networks (PINNs) benchmarked against clinical estimator and reveal parameter identifiability
result PINN recovers tissue concentration and identifies non-identifiable parameters