Inference-Time Scaling can be extended to domains prone to systematic failure using intrinsic statistics.
problem Scaling inference time in domains prone to systematic failure
method Intrinsic Selection (iS), Intrinsic Particle Filtering (iPF), and Particle Distillation (dPF)
result Intrinsic Selection improves engineering design selection by 20% and pass@1 by 6.1 points on average.
Study provides bounds for estimating intrinsic dimension using Gaussian kernels.
problem Estimating intrinsic dimension from data.
method Finite-sample concentration and anti-concentration bounds for Gaussian kernel sums.
result Explicit dependence on sample size, bandwidth, and geometric parameters.
This work introduces a protocol to automatically select the correct range of scales for meaningful Intrinsic Dimension estimation.
problem The Intrinsic Dimension (ID) varies with scale in real-world datasets, leading to erroneous results.
method The protocol selects the correct range of scales by ensuring constant density of data points.
result The method provides a robust and scale-adaptive approach to estimating meaningful Intrinsic Dimension.
GeLoRA optimizes LoRA fine-tuning by dynamically adjusting ranks based on intrinsic dimensionality.
problem Efficient fine-tuning of large language models with limited computational resources.
method GeLoRA computes intrinsic dimensionality to adaptively select LoRA ranks, balancing expressivity and efficiency.
result GeLoRA consistently outperforms recent baselines within the same parameter budget on multiple tasks.
A novel method for feature selection using a reparameterized logitNormal distribution.
problem Feature selection for reconstruction in high-dimensional data.
method Introducing a reparameterization of the logitNormal distribution to address differentiability and covariance issues.
result The method provides an effective exploration scheme and efficient feature selection for reconstruction.
Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning. In the single-agent setting, this challenge is addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces; however, applying these techniques naively to the multi-age…
Paper reviews intrinsic motivations and their role in open-ended learning.
problem Understanding intrinsic motivations and their role in open-ended learning.
method Defining intrinsic motivations, presenting psychological/neuroscientific and computational models.
result Links between psychological/neuroscientific and computational models of intrinsic motivations.
This paper deals with a new filter algorithm for selecting the smallest subset of features carrying all the information content of a data set (i.e. for removing redundant features). It is an advanced version of the fractal dimension reduction technique, and it relies on the recently introduced Morisita estimator of Int…
Data acquisition, storage and management have been improved, while the key factors of many phenomena are not well known. Consequently, irrelevant and redundant features artificially increase the size of datasets, which complicates learning tasks, such as regression. To address this problem, feature selection methods ha…
This research examines how data transformations affect adversarial robustness in recurrent neural networks.
problem Adversarial examples reduce machine learning accuracy, especially in high-dimensional datasets.
method Analysis of feature selection, dimensionality reduction, and trend extraction techniques on recurrent neural networks.
result Data transformations may increase vulnerability to adversarial samples, but only if they approximate intrinsic dimensionality and maintain manifold coverage.
A kinematic method selects the deformation Laplacian for fluid dynamics on Riemannian manifolds.
problem Ambiguity in viscous operator choice for Navier-Stokes equations on Riemannian manifolds.
method Kinematic construction of strain rate from Lie-dragged vectors, excluding Hodge Laplacian due to antisymmetric part.
result Kinematic selection uniquely identifies the deformation Laplacian, resolving analytical obstructions.
The article presents a new entropy model for assessing stock market interest.
problem Assessing investor interest and market sentiment in exchange-traded securities.
method Intrinsic entropy model using actual trading data, without exogenous factors.
result Empirical evidence supports the model's ability to predict trading activity.
Measuring domain relevance of data and identifying or selecting well-fit domain data for machine translation (MT) is a well-studied topic, but denoising is not yet. Denoising is concerned with a different type of data quality and tries to reduce the negative impact of data noise on MT training, in particular, neural MT…
We present a Bayesian model selection approach to estimate the intrinsic dimensionality of a high-dimensional dataset. To this end, we introduce a novel formulation of the probabilisitic principal component analysis model based on a normal-gamma prior distribution. In this context, we exhibit a closed-form expression o…
DeepICMGP surrogate models multiple outputs efficiently.
problem Challenges in modeling dependencies between multiple outputs using traditional multi-output GPs.
method Introduces hierarchical coregionalization structures across layers in DGPs.
result Demonstrates competitive performance and active learning strategies.
Feature selection methods are usually evaluated by wrapping specific classifiers and datasets in the evaluation process, resulting very often in unfair comparisons between methods. In this work, we develop a theoretical framework that allows obtaining the true feature ordering of two-dimensional sequential forward feat…
In this paper, we propose a test, called Flagged-1-Bit (F1B) test, to study the intrinsic capability of recurrent neural networks in sequence learning. Four different recurrent network models are studied both analytically and experimentally using this test. Our results suggest that in general there exists a conflict be…
Active learning selects high-quality examples for text-to-SQL systems.
problem Efficiently annotate large language models for text-to-SQL systems.
method Formalizes example selection as a constrained experimental design problem over semantic query embeddings, proposing a stratified greedy algorithm that maximizes heteroscedastic mutual information.
result Proposed method significantly reduces labeling effort while maintaining high text-to-SQL retrieval accuracy.
A new method for efficient inference and model selection in SBMs using OT.
problem Efficient inference and model selection in stochastic block models.
method Interpreting MLVI as srGW with entropic regularization, then unregularizing for sparse solutions, and adding a sparsity-promoting regularizer.
result The method consistently recovers SBM parameters and selects the number of clusters in finite samples.
A new method monitors unstructured 3D shapes without registration.
problem Error-prone registration and mesh reconstruction steps in PCD monitoring.
method Intrinsic geometric properties of shapes, using Laplacian and geodesic distances.
result Effective monitoring of defects without registration and mesh reconstruction.
A stability metric compares feature selection algorithms in machine learning.
problem Stability of feature selection algorithms in machine learning.
method Rank-based instability index to compare MDA, LIME, and SHAP algorithms.
result LIME and SHAP are more stable than MDA, with LIME being best for human interpretability.
Kernel methods on discrete domains have shown great promise for many challenging data types, for instance, biological sequence data and molecular structure data. Scalable kernel methods like Support Vector Machines may offer good predictive performances but do not intrinsically provide uncertainty estimates. In contras…
Proposes GRAB-MDM for robust multiview data fusion.
problem Limited theoretical guarantees for multiview fusion methods in noisy high-dimensional data.
method Generalized Robust Adaptive-Bandwidth Multiview Diffusion Maps (GRAB-MDM) with adaptive bandwidth selection.
result Adaptive bandwidths lead to robust recovery of shared intrinsic structure in noisy multiview data.
New method selects neural network architectures without needing data.
problem Choosing efficient deep neural network architectures.
method Developed the deep frame potential to quantify network capacity.
result Deep frame potential correlates with generalization error.
Regularized training of an autoencoder typically results in hidden unit biases that take on large negative values. We show that negative biases are a natural result of using a hidden layer whose responsibility is to both represent the input data and act as a selection mechanism that ensures sparsity of the representati…
New method finds significant high-order interactions efficiently.
problem Finding statistically significant high-order interactions in high-dimensional data.
method Extends selective inference to high-order interaction models with pruning strategy.
result Demonstrated efficient and powerful method for high-order interactions.
A new classifier uses weighted orthogonal regression for robust classification with limited data.
problem Challenges in classification with insufficient training data.
method Exploits intrinsic structure of data through Eigen components with specific weights determined by eigenvalues.
result Robust learning in classification problems with limited data.
New optimizer improves privacy-protected hyperparameter tuning.
problem No practical methods for differentially private hyperparameter selection.
method Study honest hyperparameter selection under DP, show adaptive optimizers like DPAdam have an advantage.
result DPAdam optimizes hyperparameters more efficiently under DP constraints.
A new method ranks and selects features without model fitting.
problem Feature importance measures algorithm-specific and need improvement.
method Integrates global sensitivity analysis with forward selection and backward elimination.
result Demonstrates clear advantage over state-of-the-art methods.
Deep learning has become increasingly popular in both supervised and unsupervised machine learning thanks to its outstanding empirical performance. However, because of their intrinsic complexity, most deep learning methods are largely treated as black box tools with little interpretability. Even though recent attempts …
Study portfolio selection with exogenous and endogenous transaction costs using deep learning.
problem Portfolio selection with both exogenous and endogenous transaction costs.
method Deep learning-driven policy iteration scheme for high-dimensional HJB equations.
result Proposes a scheme to address the curse of dimensionality and adapt to high-dimensional control spaces.
A new method for feature selection in high-dimensional data.
problem Dealing with noise and high-dimensional data in unsupervised feature selection.
method Sparse PCA via l2,p-norm regularization, combined with an efficient optimization algorithm. result The proposed method effectively selects features from real-world data sets.
Introduces intrinsic Hopf-Lax semigroup linking to intrinsic slope.
problem Understanding intrinsic Hopf-Lax semigroup and its relation to intrinsic slope.
method Introduces and proves the link between intrinsic Hopf-Lax semigroup and intrinsic slope.
result Intrinsic Hopf-Lax semigroup is a subsolution of Hamilton-Jacobi type equality.
Neuroimage analysis usually involves learning thousands or even millions of variables using only a limited number of samples. In this regard, sparse models, e.g. the lasso, are applied to select the optimal features and achieve high diagnosis accuracy. The lasso, however, usually results in independent unstable feature…
This works extends the Random Embedding Bayesian Optimization approach by integrating a warping of the high dimensional subspace within the covariance kernel. The proposed warping, that relies on elementary geometric considerations, allows mitigating the drawbacks of the high extrinsic dimensionality while avoiding the…
ERM uses energy-based selection to improve recursive reasoning.
problem Lack of principled inference mechanism in recursive models.
method Energy-guided Recursive Model (ERM) introduces Hopfield energies for trajectory selection.
result ERM achieves optimal solutions on various puzzles.
The web link selection problem is to select a small subset of web links from a large web link pool, and to place the selected links on a web page that can only accommodate a limited number of links, e.g., advertisements, recommendations, or news feeds. Despite the long concerned click-through rate which reflects the at…
In this paper we present a novel model of the primary visual cortex (V1) based on orientation, frequency and phase selective behavior of the V1 simple cells. We start from the first level mechanisms of visual perception: receptive profiles. The model interprets V1 as a fiber bundle over the 2-dimensional retinal plane …
Paper shows noisy labels can improve PLR variable selection.
problem Variable selection in PLR is challenging due to noisy labels.
method Proposes a novel ADMM-based algorithm to fuse noisy labels.
result Fused noisy labels improve PLR performance in estimation and classification.
New model selects uncorrelated and discriminative features for unsupervised feature selection.
problem Selecting uncorrelated and discriminative features in high-dimensional data.
method Adaptive graph-based generalized regression model with uncorrelated constraint and ℓ2,1-norm regularization. result The model effectively selects uncorrelated and discriminative features, improving clustering performance.
Study local exploration on dynamic graphs with time-varying edges.
problem Learning optimal actions in a network with changing connections.
method Local explore-then-commit algorithms under a structural condition ensuring intrinsic walk stability.
result Sublinear expected regret for reward-aware strategies.
Survey of intrinsically linked or knotted graphs.
problem Understanding graphs that are inherently knotted or linked.
method Survey of existing results.
result Discussion of known properties of intrinsically linked or knotted graphs.
The paper studies properties of intrinsically Lipschitz constants in metric spaces.
problem Investigating properties of intrinsically Lipschitz constants.
method Introduced Leibniz and product formulas for intrinsic slope.
result Formulated Leibniz and product formulas for intrinsic slope.
Study uses CSIE to estimate portfolio volatility relative to market.
problem Estimating relative volatility risk of stock portfolios.
method Cross-sectional intrinsic entropy (CSIE) model to estimate cross-sectional volatility.
result Discover sets of symbols that outperform market indices in terms of return with similar or lower risk.
Markowitz (1952, 1959) laid down the ground-breaking work on the mean-variance analysis. Under his framework, the theoretical optimal allocation vector can be very different from the estimated one for large portfolios due to the intrinsic difficulty of estimating a vast covariance matrix and return vector. This can res…
Acquisition of labeled training samples for affective computing is usually costly and time-consuming, as affects are intrinsically subjective, subtle and uncertain, and hence multiple human assessors are needed to evaluate each affective sample. Particularly, for affect estimation in the 3D space of valence, arousal an…
A new criterion for deep active learning selects minimal labeled data points.
problem Efficiently select minimal labeled data points for deep neural networks.
method Diffuses label information over a graph of data representations to switch between exploration and refinement.
result The diffusion-based criterion outperforms existing methods in deep active learning.
Long-term investors, different from short-term traders, focus on examining the underlying forces that affect the well-being of a company. They rely on fundamental analysis which attempts to measure the intrinsic value an equity. Quantitative investment researchers have identified some value factors to determine the cos…