This paper introduces a new feature learning technique based on error representation.
problem Learning high-level features for classification from diverse and imbalanced data.
method Inverse feature learning using error representation approach.
result Significantly better performance compared to state-of-the-art techniques.
Paper proposes a new method to learn features from error representations.
problem Learning from error representations in machine learning.
method Inverse feature learning (IFL) based on deep clustering.
result IFL leads to improved performance in classification and clustering.
Proves error bounds for state representation in RL using graph spectral features.
problem Addressing the curse of dimensionality in RL with unknown transition graphs.
method Proves upper bounds on approximation error of linear value function approximation using learned spectral features of the state-graph.
result Error bounds scale with algebraic connectivity and eigenvector estimation error.
iTimER learns from reconstruction errors to represent irregularly sampled time series.
problem Learning from irregularly sampled time series with missing data.
method iTimER models reconstruction errors as a proxy for unobserved values, using a mixup strategy and a Wasserstein metric.
result iTimER outperforms state-of-the-art methods in classification, interpolation, and forecasting tasks.
The study examines how equivariance in networks affects generalization error using PAC-Bayesian bounds.
problem Understanding how equivariance in networks impacts generalization error.
method Utilized PAC-Bayesian analysis for equivariant networks, deriving norm-based bounds for generalization error.
result The bound indicates that using larger group size in the model improves generalization error.
Spotlight method finds hidden errors in deep learning models.
problem Systematic errors in deep learning models on rare data subsets.
method Shining a spotlight on hidden layer representations to find poor performance areas.
result Identifies semantically meaningful areas of weakness in various models.
This paper tackles fairness in PCA by balancing it with reconstruction error.
problem Fairness concerns in PCA due to different group representation errors.
method A multi-objective optimization approach to balance fairness and reconstruction error.
result Achieving fairness with minimal loss in reconstruction error.
Estimates model performance under distribution shift using domain-invariant predictors.
problem Poor performance of models on test distributions different from training distributions.
method Uses domain-invariant predictors as a proxy for unknown target labels.
result Shows that the complexity of latent representations influences target risk.
Framework for continuous-time network data representation learning.
problem Learning reliable representations of dynamic network interactions.
method Three-stage process: intensity estimation, projection learning, evolving node representation construction.
result Trajectories satisfy structural and temporal coherence, providing robust inference.
Paper analyzes error in stochastic approximation for discontinuous functions.
problem Estimating expected error in discontinuous stochastic approximation.
method Uses finite differences and O(n−1/5) error estimate for discontinuous functions. result Achieves error estimate of O(n−1/5) for discontinuous stochastic representation. The paper tackles stock prediction models by improving their generalizability to out-of-sample domains using causal representation learning.
problem Low signal-to-noise ratio and nonstationary nature of financial markets lead to poor performance of stock prediction models.
method The paper investigates Domain Generalization techniques, focusing on causal representation learning to improve model generalizability. It introduces a novel error bound and a causal discovery technique to mitigate spurious correlations.
result The proposed approach enhances the generalizability of stock prediction models, as demonstrated by numerical results.
There is intense interest in applying machine learning to problems of causal inference in fields such as healthcare, economics and education. In particular, individual-level causal inference has important applications such as precision medicine. We give a new theoretical analysis and family of algorithms for predicting…
Develops UKP for comparing feature representations in multitask learning.
problem Comparing feature representations learned by different models without access to test data.
method Uniform Kernel Prober (UKP) for comparing representations in kernel ridge regression tasks.
result UKP provides a uniform measure of prediction error on test data without access to test data.
This work improves understanding of reinforcement learning state representations.
problem Lack of precise characterization of how and when state representations generalize.
method Developed a bound on the generalization error based on effective dimension.
result Bound quantifies the tension between generalization and approximation.
New metric explains neural network performance, simplifying generalization error calculation.
problem Precise characterization of neural network generalization error.
method Introducing Representation Gap, linking to intrinsic dimension and equivariant diffusion models.
result Asymptotic equivalent of Representation Gap is governed by intrinsic dimension, easy to estimate.
Leveraging reference-only samples for two-sample testing under size asymmetry
problem Two-sample testing under size imbalance
method Adaptive aggregation of reference-dependent representations
result Strong performance with type I error control
Due to the ability of deep neural nets to learn rich representations, recent advances in unsupervised domain adaptation have focused on learning domain-invariant features that achieve a small error on the source domain. The hope is that the learnt representation, together with the hypothesis learnt from the source doma…
Neural network accuracy improves with denser training samples.
problem Improving neural network accuracy on unseen test samples.
method Bounding empirical training error smoothed across activation regions and using it to discard high-risk test samples.
result Discarding high-risk test samples based on error bounds improves prediction accuracy by up to 20%.
Metalearning optimizes autoencoder dimensions for efficient data representation.
problem Selecting optimal dimension for autoencoder output to balance accuracy and complexity.
method Metalearning approach using actor-critic algorithm to dynamically adjust dimension.
result Automatic selection of minimum number of bases for optimal reconstruction.
This paper explores how representation learning can improve design-based causal inference.
problem Estimating causal effects in design-based studies is challenging due to the need for optimal weights.
method The authors propose an end-to-end estimation procedure that learns a flexible representation to minimize the error in choosing a representation.
result The proposed method is competitive in various causal inference tasks and shows promise for improving design-based weights.
A typical approach in estimating the learning rate of a regularized learning scheme is to bound the approximation error by the sum of the sampling error, the hypothesis error and the regularization error. Using a reproducing kernel space that satisfies the linear representer theorem brings the advantage of discarding t…
In the era of deep learning several unsupervised models have been developed to capture the key features in unlabeled handwritten data. Popular among them is the Restricted Boltzmann Machines RBM. However, due to the novelty in handwritten multidialect data, the RBM may fail to generate an efficient representation. In t…
We consider the reconstruction problem in compressed sensing in which the observations are recorded in a finite number of bits. They may thus contain quantization errors (from being rounded to the nearest representable value) and saturation errors (from being outside the range of representable values). Our formulation …
This paper aims to develop a new and robust approach to feature representation. Motivated by the success of Auto-Encoders, we first theoretical summarize the general properties of all algorithms that are based on traditional Auto-Encoders: 1) The reconstruction error of the input can not be lower than a lower bound, wh…
Researchers propose a new SSL risk decomposition method to evaluate and improve self-supervised learning models.
problem Self-supervised learning evaluation is limited to a single metric, providing little insight into model performance and improvement.
method Proposes an SSL risk decomposition that considers four error components: approximation, representation usability, probe generalization, and encoder generalization.
result Analysis of 169 SSL vision models reveals the main sources of error and provides insights for improving SSL models in specific settings.
This paper describes a hierarchical learning strategy for generating sparse representations of multivariate datasets. The hierarchy arises from approximation spaces considered at successively finer scales. A detailed analysis of stability, convergence and behavior of error functionals associated with the approximations…
In this paper, we investigate the robust dictionary learning (DL) to discover the hybrid salient low-rank and sparse representation in a factorized compressed space. A Joint Robust Factorization and Projective Dictionary Learning (J-RFDL) model is presented. The setting of J-RFDL aims at improving the data representati…
The paper proposes a method to balance fairness and prediction accuracy by adjusting data representations.
problem Machine learning models can inherit and amplify historical biases, leading to unfair outcomes.
method The paper uses subspace decomposition and influence analysis to control the fairness-utility trade-off.
result The method effectively improves fairness while preserving predictive performance.
Offline RL with pre-trained features amplifies errors even under mild shifts.
problem Sample-efficient offline RL with pre-trained features under mild distribution shift.
method Empirical study of offline RL with pre-trained neural representations.
result Substantial error amplification occurs even with pre-trained features, requiring stronger conditions for successful offline RL.
Generative models, such as GANs, learn an explicit low-dimensional representation of a particular class of images, and so they may be used as natural image priors for solving inverse problems such as image restoration and compressive sensing. GAN priors have demonstrated impressive performance on these tasks, but they …
The exclusive or (xor) function is one of the simplest examples that illustrate why nonlinear feedforward networks are superior to linear regression for machine learning applications. We review the xor representation and approximation problems and discuss their solutions in terms of probabilistic logic and associative …
The paper analyzes the stability of an observer error in a vibrating string system.
problem Stability analysis of observer error in a vibrating string system.
method Abstract Cauchy problem reformulation and application of LaSalle's invariance principle for infinite-dimensional systems.
result The observer error is asymptotically stable.
Training-free source selection for LLM families with shared vocabularies
problem Source selection for LLM families with shared vocabularies
method Fisher alignment at vocabulary scale
result Fisher alignment is a cosine between kernel mean embeddings in the joint activation-error space
Study error bounds in evaluating distributional computational graphs.
problem Error analysis in evaluating graphs with inputs as probability distributions.
method Establish non-asymptotic error bounds using Wasserstein-1 distance.
result Non-asymptotic error bounds for discretization errors in distributional computational graphs.
The Immersed Boundary (IB) method is a widely-used numerical methodology for the simulation of fluid-structure interaction problems. The IB method utilizes an Eulerian discretization for the fluid equations of motion while maintaining a Lagrangian representation of structural objects. Operators are defined for transmit…
In this study we examined the question of how error correction occurs in an ensemble of deep convolutional networks, trained for an important applied problem: segmentation of Electrocardiograms(ECG). We also explore the possibility of using the information about ensemble errors to evaluate a quality of data representat…
Machine Learning (ML) is increasingly being used for computer aided diagnosis of brain related disorders based on structural magnetic resonance imaging (MRI) data. Most of such work employs biologically and medically meaningful hand-crafted features calculated from different regions of the brain. The construction of su…
Optimal estimator derived for partially observable LTI systems.
problem Optimal estimator for partially observable LTI systems.
method State-space representation for derivation of optimal estimator.
result Derivation of minimum error variance estimator for partially observable LTI systems.
We analyze generalization in deep learning models using random matrix theory.
problem Understanding the generalization error in deep learning models with random feature representations.
method Applying Random Matrix Theory to derive asymptotic generalization error formulas for various architectures.
result Linear ESNs are equivalent to ridge regression with exponentially time-weighted input covariance, revealing an inductive bias towards recent inputs.
Concept Factorization (CF) and its variants may produce inaccurate representation and clustering results due to the sensitivity to noise, hard constraint on the reconstruction error and pre-obtained approximate similarities. To improve the representation ability, a novel unsupervised Robust Flexible Auto-weighted Local…
The paper tackles transfer learning for growing matrix representations, improving estimation accuracy.
problem Structured matrix estimation under growing ambient dimensions and latent representations.
method Proposes a general transfer framework decomposing target parameters into embedded source components, low-rank innovations, and sparse edits. Develops an anchored alternating projection estimator.
result Establishes deterministic error bounds that separate target noise, representation growth, and source estimation error, yielding improved rates.
MSRL learns a representation maximizing mutual info with response variables.
problem Learning sufficient representations for complex, multi-dimensional data.
method Variational mutual information, deep neural networks, generalized Dudley's inequality.
result MSRL achieves consistent and accurate representation learning.
Manifold learning seeks a low dimensional representation that faithfully captures the essence of data. Current methods can successfully learn such representations, but do not provide a meaningful set of operations that are associated with the representation. Working towards operational representation learning, we endow…
Nowadays Knowledge Graphs constitute a mainstream approach for the representation of relational information on big heterogeneous data, however, they may contain a big amount of imputed noise when constructed automatically. To address this problem, different error detection methodologies have been proposed, mainly focus…
Study shows label noise impacts neural representations' information content, revealing double descent behavior.
problem Impact of label noise on neural network hidden representations.
method Information Imbalance proxy of conditional mutual information to compare hidden representations.
result Representations learned with noisy labels are more informative than those with clean labels in the underparameterized regime, and equally informative in the overparameterized regime.
MEDIDA discovers model errors in chaotic systems using sparse regression and data assimilation.
problem Model errors in chaotic systems lead to significant discrepancies between model predictions and real-world states.
method MEDIDA combines Bayesian sparse regression and data assimilation to estimate and interpret model errors from noisy observations.
result MEDIDA successfully identifies different types of model errors in the chaotic Kuramoto-Sivashinsky system.
Paper analyzes how contrastive learning structures learned representations.
problem Understanding the structure of learned representations in contrastive learning.
method Kernel-based contrastive learning framework (KCL) and statistical dependency viewpoint.
result Theoretical upper bound and generalization error bound for KCL.
Study on CVA in volatility models, including rough volatility.
problem Calculating CVA in fractional and rough volatility models.
method General representation formula, specialized for volatility models, numerical and theoretical error analysis.
result Roughness influences the claim's price, and provides accurate approximations.