New model optimizes feature alignment, improving statistical and computational efficiency.
problem Statistical and computational challenges in feature alignment.
method Covariance alignment model, Gromov-Wasserstein algorithm.
result Gromov-Wasserstein algorithm is minimax optimal and practical for covariance alignment.
NTKs explain GNNs' alignment for graph prediction.
problem Understanding GNNs' alignment for graph prediction.
method Analyzing NTKs and alignment in GNNs, focusing on cross-covariance.
result Optimizing alignment in GNNs optimizes graph representation.
ITSPACE improves covariance alignment faster than other methods.
problem Optimizing covariance matrices for machine learning tasks.
method Proximal majorization-minimization method that directly optimizes the Bures-Wasserstein objective.
result ITSPACE achieves lower BW gap solutions faster than other methods.
Paper introduces MSA for weakly supervised covariance alignment in MEG signals.
problem Limited labeled signals in target datasets for MEG applications.
method Mixing model Stiefel Adaptation (MSA) leveraging unlabeled data.
result MSA outperforms recent methods in brain-age regression with MEG signals.
New method groups similar functional covariates for better modeling.
problem Analyzing functional covariates with similar shapes.
method Coefficient shape alignment regularization approach.
result True grouping structure can be accurately identified under certain conditions.
Deep model predicts shapes of curves with multiple covariates.
problem Predicting shapes of planar curves with various covariates.
method Deep learning model using complex-valued functions, conditional covariance smoother with modality-specific encoders.
result Model accurately predicts shapes of curves with multimodal covariates.
FIRE method improves model performance in federated learning by penalizing fragmentation-induced covariate shifts.
problem Performance degradation in federated learning due to data fragmentation and covariate shift.
method FIRE method accumulates fragmentation-induced covariate shift divergences via approximate Fisher information and uses it as a per-fragment loss penalty.
result FIRE outperforms importance weighting and federated learning benchmarks by up to 5.3% on shifted validation sets.
Sparse models for high-dimensional linear regression and machine learning have received substantial attention over the past two decades. Model selection, or determining which features or covariates are the best explanatory variables, is critical to the interpretability of a learned model. Much of the current literature…
Attention learns PCA on Gaussian data, proving its connection to principal component analysis.
problem Principal component analysis on Gaussian data.
method Analysis of attention mechanisms through PCA, covering finite and infinite prompt regimes.
result Attention aligns with principal eigenvectors of covariance matrices, converging to optimal solutions in the infinite-prompt limit.
Proposes a new method for handling domain shift in samples with biases in both covariates and labels.
problem Domain shift in samples with biases in both covariates and labels.
method Factorizable Joint Shift (FJS) and Joint Importance Aligning (JIA).
result Our method can handle co-existence of sampling bias in covariates and labels.
The paper examines how spike strengths and alignments affect overfitting in linear regression models.
problem The impact of spike strengths and alignments on overfitting in linear regression models.
method Characterization of generalization error through exact expressions and analysis of spike strengths, aspect ratio, and target alignment.
result Increasing spike strength can lead to catastrophic overfitting before benign overfitting, especially in well-specified aligned problems.
The paper develops a theory for random forests, separating variance components and providing methods for estimating prediction intervals.
problem Understanding the variance and uncertainty in random forest predictions.
method Design-based theory, Monte Carlo averaging, PASR resampling.
result The floor of prediction uncertainty is positive and persists even without observation overlap, providing conservative prediction intervals.
New covariance estimator for financial portfolios.
problem Estimating large financial covariances in non-stationary environments.
method Exponentially weighted averages and cross-validation for nonlinearly shrinking sample eigenvalues.
result Our estimator performs well in large dimensions compared to existing estimators.
Improved graph matching using covariates for network data integration.
problem Matching networks without unique identifiers.
method Two novel covariate-assisted seeded graph matching methods.
result Improved alignment accuracy through covariate information.
New meta-reinforcement learning method improves performance in finite-horizon MDPs.
problem Improving meta-reinforcement learning in finite-horizon MDPs with shared optimal action-value functions.
method Proposes MTSRL and MTSRL+ algorithms with learned priors and covariance, coupled with prior-alignment technique for meta-regret guarantees.
result Achieves meta-regret guarantees with learned priors and covariance, outperforming prior-independent RL and bandit-only meta-baselines.
The paper develops methods to reduce deployment risk under dynamic covariate shifts.
problem Reduction of deployment risk under dynamic covariate shifts.
method Time-domain Poincare inequality and Jacobian-velocity theorem to identify and control directional tangent energy.
result Drift-aligned tangent regularization (DTR) reduces risk volatility and directional gain in low-rank drift regimes.
New study reveals task alignment is key to ICL performance.
problem Understanding how task alignment affects generalization in in-context learning.
method Derived an exact expression for ICL generalization error in high dimensions under task covariance mismatch.
result Identified train-test task alignment as a key determinant of generalization in ICL.
This paper extends neural collapse to regression problems, revealing key features and structures.
problem Understanding the structure learned by deep neural networks in regression tasks.
method Established Neural Regression Collapse (NRC) across different models, analyzing feature and weight alignments.
result Deep neural regression models exhibit a collapsed feature space, aligning with target dimensions and covariances.
Domain adaptation addresses the common problem when the target distribution generating our test data drifts from the source (training) distribution. While absent assumptions, domain adaptation is impossible, strict conditions, e.g. covariate or label shift, enable principled algorithms. Recently-proposed domain-adversa…
New kernel interprets 3D anisotropic data with rotations and improved predictions.
problem Capturing rotated anisotropy in 3D spatial fields.
method Introduces a Lie-algebraic kernel with three principal length-scales and an explicit rotation.
result Posterior recovers rotated anisotropy and improves prediction over axis-aligned kernels.
Proves the Kundt conjecture in arbitrary dimensions, confirming its validity.
problem Determining spacetimes not characterized by scalar polynomial curvature invariants.
method New bilinear map and analysis of covariant derivatives of the Riemann tensor.
result Confirms the Kundt conjecture in arbitrary dimensions, removing regularity assumptions.
This work investigates how gradient-based learning performs with structured data, revealing issues and improvements.
problem Gradient-based learning under structured data, particularly with a spiked covariance structure.
method Investigates the effect of a spiked covariance structure on gradient-based feature learning and proposes weight normalization.
result Gradient-based dynamics may fail to recover the true direction in anisotropic settings, but weight normalization can improve performance.
Study on clustering in high dimensions with anisotropic Gaussian mixtures, showing interpolation can be optimal and robust.
problem Clustering in high-dimensional anisotropic Gaussian mixtures.
method Derive minimax bounds, analyze ℓ2-regularized classifiers, and investigate interpolation's robustness. result Interpolating solutions can be optimal and robust under certain conditions.
This paper rethinks confidence calibration under covariate shifts.
problem Calibration methods struggle with covariate shifts and unstable importance weighting.
method Derives Expectation consistency condition and proposes Expectation consistency loss (ECL).
result ECL loss is compatible with various types of calibration and has the same sample complexity as ECE.
This study optimizes covariate density and propensity score for efficient ATE estimation.
problem Efficiently estimating average treatment effects (ATEs) with minimal variance.
method Adaptive experiment optimizing both covariate density and propensity score.
result Proposed method minimizes the semiparametric efficiency bound for ATE estimation.
CASP improves portfolio optimization by considering asset covariance.
problem Infeasibility in cardinality-constrained portfolio optimization.
method CASP uses volatility-normalized selection and covariance-aware projection.
result CASP-Basic delivers lower portfolio variance than standard Euclidean repair.
Study improves treatment effect estimation using unlabeled covariates.
problem Estimating treatment effects with limited labeled data.
method Developed efficiency bounds and estimators for semi-supervised setting.
result Estimators using unlabeled covariates have lower asymptotic variance.
SpecGD mitigates misalignment in phase retrieval models with anisotropic inputs.
problem Misalignment during gradient descent in phase retrieval models with anisotropic inputs.
method Spectral gradient descent modifies gradient updates to preserve directional information and remove spike amplification.
result SpecGD removes spike amplification, leading to stable alignment and accelerated noise contraction.
This article addresses the modeling of reverberant recording environments in the context of under-determined convolutive blind source separation. We model the contribution of each source to all mixture channels in the time-frequency domain as a zero-mean Gaussian random variable whose covariance encodes the spatial cha…
PAMA learns covariate importance for better matching in observational studies.
problem Poor performance of conventional matching methods when covariates differ in relevance.
method PAMA is a semi-supervised framework that learns covariate importance from paired data and optimizes a weighted quadratic score.
result PAMA outperforms standard methods, particularly in high-dimensional settings and under model misspecification.
Sharp-SSL uses random projections to identify important variables for semi-supervised learning.
problem High-dimensional semi-supervised learning problems.
method Careful aggregation of low-dimensional results from many axis-aligned random projections.
result Sharp-SSL algorithm can recover signal coordinates with high probability.
MPVAE learns latent embeddings and label correlations for multi-label classification.
problem Challenging task of predicting multiple targets with label correlations.
method Proposes MPVAE, a novel framework that learns latent embedding spaces and label correlations using a Multivariate Probit model.
result MPVAE outperforms state-of-the-art methods on various application domains and is robust under noisy settings.
Investigates portfolio optimization with and without gearing constraints.
problem Improving portfolio weights for better alignment with expected returns.
method Extends the alpha-weight angle bound to include gearing constraints and uses theoretical arguments and simulations.
result Equally weighted portfolios are not preferable to mean-variance portfolios even with poor forecast ability and a badly conditioned covariance matrix.
New method estimates covariance in multi-view data with better accuracy and uncertainty.
problem Estimating covariance in multi-view data with shared and view-specific latent factors.
method Spectral decompositions and conditional conjugate priors for factor loadings and residual variances.
result Proves favorable asymptotic properties and excellent performance in simulations and real data.
Study nonparametric contextual bandits with batched updates, achieving optimal regret.
problem Optimal regret in nonparametric contextual bandits with batch constraints.
method Dynamic binning of covariate space, optimal regret achieved.
result Achieves optimal regret (up to logarithmic factors) for nonparametric contextual bandits.
Paper proposes a probabilistic alignment method for domain adaptation.
problem Latent distribution mismatch and miscalibrated uncertainty in adapting large-scale models.
method Bayesian latent transport framework with PAC-Bayesian regularization.
result Reduction in latent manifold discrepancy and improved uncertainty calibration.
We introduce a new method for sparse principal component analysis, based on the aggregation of eigenvector information from carefully-selected axis-aligned random projections of the sample covariance matrix. Unlike most alternative approaches, our algorithm is non-iterative, so is not vulnerable to a bad choice of init…
Study optimal ridge regularization for out-of-distribution prediction.
problem Optimal ridge regularization for predicting out-of-distribution data.
method Established conditions for optimal regularization under covariate and regression shifts, proving monotonic risk in data aspect ratio.
result Negative regularization can be optimal under shifts, even with isotropic or underparameterized training features.
Bayesian method uses data spectra to estimate non-sparse high-dimensional models.
problem Handling many parameters in high-dimensional Bayesian statistics.
method Data-adaptive Gaussian prior aligned with leading eigenvectors of sample covariance.
result Posterior contraction rates reveal the effect of spectral mass on prediction error.
A new method routes EEG covariance matrices across domains using adaptive subspace selection.
problem Challenges in cross-domain EEG decoding due to distinct SPD manifold regions.
method Dynamic Stiefel routing with expert filters and cross-attention for adaptive subspace projection.
result Consistent gains across three datasets: balanced accuracy improves from 0.773 to 0.823, 0.757 to 0.809, and 0.801 to 0.839.
This paper improves STL inference reliability under covariate shift.
problem Ensuring correct STL formulas in real-world settings with distribution shift.
method Proposes a conformalized STL inference framework that addresses covariate shift.
result Significantly improves symbolic learning reliability at deployment time.
A new method uses diffusion models to simulate financial markets accurately.
problem Generating realistic synthetic financial market data.
method Diffusion model approach with efficient numerical integration.
result Synthetic data closely matches observed market data in key aspects.
A geometric theory explains loss functions for robust representation learning.
problem Treats robustness, domain adaptation, and sensor drift as separate literatures.
method Estimates covariance Sigma_task and uses it to pin Jacobian penalties.
result Proves optimality and necessity of range coverage for penalty matrices.
Proposes a method to compare noisy high-dimensional datasets with low-dimensional manifolds.
problem Comparing distributions on manifolds in noisy high-dimensional datasets.
method Linking low-rank structure to manifold geometry, developing a scale-invariant distance measure.
result Superior robustness and statistical power compared to existing methods.
Framework enhances AI explainability by aligning with human cognitive models.
problem Lack of explainability in AI models hinders trust and accountability.
method Integrates explainability techniques with Malle's five category model of behavior explanation.
result Demonstrates practical relevance in credit risk assessment and regulatory analysis.
Inflating the minimum norm interpolator improves linear regression generalization error.
problem Highly anisotropic covariances and diverging d/n in linear regression. method Inflating the minimum ℓ2 norm interpolator by a constant greater than one. result Inflating the minimum norm interpolator improves generalization error.
We consider the class of locally boost isotropic spacetimes in arbitrary dimension. For any spacetime with boost isotropy, the corresponding curvature tensor and all of its covariant derivatives must be simultaneously of alignment type D relative to some common null frame. Such spacetimes are known as type ${\b…
In the scenario of real-time monitoring of hospital patients, high-quality inference of patients' health status using all information available from clinical covariates and lab tests is essential to enable successful medical interventions and improve patient outcomes. Developing a computational framework that can learn…