New estimator stabilizes higher-order influence functions for bilinear forms.
problem Stability issues in estimating bilinear forms using higher-order influence functions.
method Proposes a new stabilized higher-order estimator for a class of bilinear forms without sample splitting.
result New estimator exhibits more stable finite-sample performance compared to the empirical higher-order estimator.
New estimator stabilizes higher-order influence functions for stable statistical inference.
problem Numerical instability in estimating inverse population Gram matrix.
method Proposes a new stabilized higher-order estimator without sample splitting.
result Stabilized estimator exhibits more stable performance and similar statistical guarantees.
New stable HOIF estimators for statistical functionals.
problem Constructing numerically stable HOIF estimators for statistical functionals.
method Developed new sHOIF estimators with provable guarantees.
result 2nd order sHOIF estimators were validated in synthetic experiments.
A model predicts influential nodes in complex networks by considering indirect interactions.
problem Identifying influential nodes in complex networks using indirect interactions.
method Proposes MOGen, a multi-order generative model that considers all indirect influences up to a maximum distance.
result MOGen consistently outperforms network models and path-based approaches in predicting influential nodes.
JAWS audits predictive uncertainty under covariate shift using jackknife+ weighted methods.
problem Auditing predictive uncertainty under data distribution shifts.
method JAW and JAWA methods for distribution-free uncertainty quantification.
result JAW relaxes the jackknife+'s assumption of data exchangeability for covariate shift.
Discriminative jackknife estimates deep learning uncertainty.
problem Quantifying uncertainty in deep learning models.
method Discriminative jackknife using influence functions of loss.
result DJ satisfies frequentist coverage and discriminative accuracy.
The paper identifies causal effects in latent variable models using higher-order cumulants.
problem Challenges in identifying causal effects in latent variable models with latent confounders.
method Using higher-order cumulants, the paper addresses two challenging setups: a single proxy variable and underspecified instrumental variables.
result Causal effects are identifiable with a single proxy or instrument.
Unified framework for feature-based explanations using ANOVA and game theory.
problem Differences between feature-based explanations methods limit their applicability.
method Introduces a unified framework combining fANOVA and cooperative game theory.
result Uncovered similarities and differences between various explanation techniques.
Study embeddings between Barron spaces with various activation functions, focusing on RePU.
problem Understanding the influence of activation functions on infinitely wide neural networks.
method Prove embeddings by constructing push-forward maps on measures representing functions.
result Barron spaces with RePU activation have a hierarchical structure similar to Sobolev spaces.
Unified framework for output analysis using Monte Carlo sampling.
problem Accurately assess the quality of estimated values in predictive models.
method Unified output analysis framework through Monte Carlo sampling, leveraging fast iterative bootstrap sampling and higher-order influence functions.
result Clear advantage in building more robust confidence intervals with higher coverage probability.
We use the Frölicher-Nijenhuis formalism to reformulate the inverse problem of the calculus of variations for a system of differential equations of order 2k in terms of a semi-basic 1-form of order k. Within this general context, we use the homogeneity proposed by Crampin and Saunders in [14] to formulate and discuss t…
New model estimates higher-order interactions in stochastic processes using lower-dimensional projections.
problem Estimating higher-order interaction effects in stochastic processes with limited data.
method Additive Poisson Process (APP) combines information geometry and generalized additive models to model intensity functions in lower dimensions.
result The model can estimate higher-order intensity functions with sparse data.
We introduce nonlinear higher-order label spreading for semi-supervised learning.
problem Efficient semi-supervised learning on graphs with complex label spreading.
method We add nonlinearity to label spreading through higher-order graph structures, proving convergence and demonstrating efficiency on various datasets.
result Our nonlinear higher-order label spreading algorithm converges to the global solution and performs favorably compared to classical methods.
The study examines higher-order modern portfolio theory with complex critical points and feasible portfolio variety.
problem Understanding the complex critical points and feasible portfolio variety in higher-order modern portfolio theory.
method Established genericity conditions for utility functions with higher-order cumulants, analyzed discriminant loci, and determined the dimension and degree of the feasible portfolio variety.
result The utility function has a constant number of complex critical points under genericity conditions, and the feasible portfolio variety has a determined dimension and degree.
The paper defines and analyzes higher-order Yang-Mills-Higgs functionals and their gradient flows.
problem Analyzing the behavior of higher-order Yang-Mills-Higgs functionals and their gradient flows.
method Gauge fixing technique, L2-bound of the Higgs field, local L2-derivative estimates, energy estimates, blow-up analysis. result Solutions to the gradient flow do not hit finite time singularities under certain conditions.
Influence functions are inaccurate in deep learning models, especially for deeper networks.
problem Inaccuracies in influence functions in deep learning models.
method Empirical study of influence functions in neural network models trained on various datasets.
result Influence estimates are often erroneous for deeper networks and require regularization.
Paper introduces a framework for diagnosing Alzheimer's disease using higher-order topological features from fMRI.
problem Diagnosing Alzheimer's disease using brain network topology.
method Persistent homology to extract higher-order features (cycles, cavities) from fMRI data.
result Framework significantly outperforms existing methods in AD classification.
Social recommendation has emerged to leverage social connections among users for predicting users' unknown preferences, which could alleviate the data sparsity issue in collaborative filtering based recommendation. Early approaches relied on utilizing each user's first-order social neighbors' interests for better user …
Bayesian-TPNN improves ANOVA-TPNN for detecting higher-order components.
problem Difficulty in incorporating higher-order components in ANOVA-TPNN due to computational and memory constraints.
method Bayesian inference procedure for functional ANOVA model with TPNN basis functions.
result Bayesian-TPNN detects higher-order components with reduced computational cost.
Sharp spectral gap estimates for higher-order operators on hyperbolic spaces.
problem Estimating spectral gaps for higher-order operators on Cartan-Hadamard manifolds.
method Symmetrization-free proofs based on general functional inequalities.
result Solves a sharp asymptotic problem from Cheng and Yang and answers a question from Kristály.
The paper constructs denoisers that recover the Brenier map from higher-order score functions.
problem Estimating the Brenier map from noisy data.
method Constructs a hierarchy of denoisers using higher-order score functions.
result The T∞ denoiser recovers the Brenier map from the additive Gaussian model. RelatIF selects more intuitive training examples for explaining model predictions.
problem Influence functions identify outliers as explanatory examples, leading to poor explanations.
method RelatIF separates global and local influence, optimizing for local relative to global effects.
result Examples selected by RelatIF are more intuitive than those from influence functions.
Establishes statistical and computational bounds for influence diagnostics.
problem Identifying influential datapoints or subsets in machine learning models.
method Finite-sample statistical bounds and computational complexity for influence functions and approximate maximum influence perturbations.
result Established statistical and computational guarantees for influence diagnostics.
A key feature of inductive logic programming (ILP) is its ability to learn first-order programs, which are intrinsically more expressive than propositional programs. In this paper, we introduce techniques to learn higher-order programs. Specifically, we extend meta-interpretive learning (MIL) to support learning higher…
Influence functions help study large language model generalization, revealing surprising decay patterns.
problem Understanding and mitigating risks in large language models (LLMs).
method Eigenvalue-corrected Kronecker-Factored Approximation (EK-FAC) to scale influence functions to LLMs.
result Influences decay to near-zero when key phrases order is flipped, revealing a surprising limitation.
Paper tackles stochastic control with mean and higher-order moments, finding Nash equilibria.
problem Time-inconsistent stochastic control problems with mean and higher-order moments.
method Developed closed-loop and open-loop Nash equilibrium controls using PDEs and maximum principles.
result Identical closed-loop and open-loop Nash equilibria controls, independent of state value and random path.
New research investigates why influence functions are fragile and proposes new validation procedures.
problem Understanding and mitigating the fragility of influence functions in deep learning model explanations.
method Verification of influence functions using various conditions and procedures, including convexity and non-convexity.
result Validation procedures may cause the observed fragility of influence functions.
We generalize the Lagrangian-Hamiltonian formalism of Skinner and Rusk to higher order field theories on fiber bundles. As a byproduct we solve the long standing problem of defining, in a coordinate free manner, a Hamiltonian formalism for higher order Lagrangian field theories. Namely, our formalism does only depend o…
Study shows influence functions are poor for neural networks but useful for identifying influential examples.
problem Influence functions misalign with leave-one-out retraining in neural networks.
method Decomposed the discrepancy into five terms and studied their contributions across different architectures and datasets.
result Influence functions are a good approximation to the proximal Bregman response function (PBRF), useful for identifying influential examples.
In this paper we determine a number of meaningful compositions of higher order of a set of functions, which is considered in Malesevic (1998), in implicit and explicit form. Results which are obtained are applied to the vector analysis in order to determine the number of meaningful differential operations of higher ord…
Study optimizes zero-order strongly convex function minimization with higher order smoothness.
problem Optimizing a strongly convex function with noisy evaluations.
method Randomized approximation of projected gradient descent with smoothing kernel.
result Upper bounds and minimax lower bounds for the algorithm, showing near-optimality.
Paper introduces a multi-stage influence function to track model predictions.
problem Improving natural language processing and computer vision performance.
method Develops a multi-stage influence function to track predictions from finetuned models back to pretraining data.
result Identifies pretraining examples contributing most to finetuning task predictions.
The paper extends influence functions to sequence tagging tasks for better model interpretability.
problem Lack of interpretability methods for sequence tagging models.
method Define and compute influence of training instance segments on test segment predictions.
result The segment influence method tracks with true influence and identifies annotation errors.
Better Hessian approximations improve influence function attributions in deep learning.
problem Influence functions are difficult to compute due to ill-conditioned Hessians, leading to poor data attribution performance.
method Investigated the impact of Hessian approximation quality on influence-function attributions in a controlled setting.
result Better Hessian approximations consistently yield better influence score quality.
The main goal of this paper is to develop a concept of approximate differentiability of higher order for subsets of the Euclidean space that allows to characterize higher order rectifiable sets, extending somehow well known facts for functions. We emphasize that for every subset A of the Euclidean space and for eve…
Separation of the sources and analysis of their connectivity have been an important topic in EEG/MEG analysis. To solve this problem in an automatic manner, we propose a two-layer model, in which the sources are conditionally uncorrelated from each other, but not independent; the dependence is caused by the causality i…
Enhances influence functions for deep models without costly Hessian inversion.
problem Challenges in computing Hessian matrix inverse for deep models.
method Substitutes Hessian inverse with identity matrix and extends applications.
result TracIn method performs well in measuring model utility and fairness.
The present paper develops two concepts of pointwise differentiability of higher order for arbitrary subsets of Euclidean space defined by comparing their distance functions to those of smooth submanifolds. Results include that differentials are Borel functions, higher order rectifiability of the set of differentiabili…
The paper simplifies influence computations for large-scale machine learning models.
problem Improving training efficiency and accuracy in large-scale models.
method Study influence functions, define memorization, simplify computations.
result Influence functions can be practical for large-scale models, indicating memorization.
Sharp inequalities for radial functions on hyperbolic spaces without boundary conditions.
problem Establishing inequalities for radial functions on hyperbolic spaces without zero boundary conditions.
method Novel approach considering both bounded and unbounded domains, focusing on weighted Sobolev and Adams-Trudinger-Moser embeddings.
result Theorems 1.2, 1.3, and 1.4 for weighted Sobolev embedding theorems, and Theorems 1.5 and 1.6 for Adams-Trudinger-Moser type embedding theorems.
Unified approach to discrete and smooth isoperimetric inequalities of arbitrary order.
problem Finding higher order isoperimetric inequalities for both discrete and smooth curves.
method Unified approach via Fourier analysis of linear operators.
result Unified upper and lower bounds for isoperimetric deficit in smooth curves.
We deduce from the work of Chen, that the restriction morphism from closed free iterated integrals to closed iterated integrals on loops is onto. We use this to show that the module of higher order invariants of smooth functions is generated by free closed iterated integrals.
New IF method improves accuracy in deep neural networks with noisy data.
problem Inaccurate influence estimates in deep neural networks, especially with noisy data.
method Established a connection between influence estimation error, validation set risk, and sharpness, introducing a novel estimation form for flat validation minima.
result Our novel Influence Function approach provides more accurate influence estimates, validated across various tasks.
We define a generalized mass for asymptotically flat manifolds using some higher order symmetric function of the curvature tensor. This mass is non-negative when the manifold is locally conformally flat and the σk curvature vanishes at infinity. In addition, with the above assumptions, if the mass is zero, then, nea…
A new method, VIF, calculates influence for non-decomposable losses efficiently.
problem Efficiently calculating influence for complex machine learning models with non-decomposable losses.
method Revisiting influence function from robust statistics, proposing Versatile Influence Function (VIF) for any non-decomposable loss.
result VIF method is up to 10^3 times faster than brute-force methods and closely matches influence results.
The paper examines conditions for the equator map to be minimizing or unstable for higher order energy functionals.
problem Conditions for the equator map to be minimizing or unstable for extrinsic k-energy functionals.
method Analyzes the extrinsic k-energy functional and the equator map to establish conditions for minimization or instability.
result Establishes necessary and sufficient conditions for the equator map to be minimizing or unstable for extrinsic k-energy functionals.
We present a geometric approach to the field theory with higher order anisotropic interactions. The concepts of higher order space, or locally anisotropic, space (in brief, h-space, or la-space) are introduced as general ones for various types of higher order extensions of Lagrange and Finsler geometry and higher dimen…
Develops a faster model selection method using influence functions.
problem Time-consuming and computationally inefficient model selection.
method Model-specific data subsampling guided by influence functions.
result Empirically shows improved selection of high-quality models.