WeakNAS uses a set of weaker predictors to find top architectures with fewer samples.
problem Finding the best neural architecture with heavy computation costs.
method Proposes a paradigm shift from fitting the whole architecture space to progressively fitting a search path through a set of weaker predictors.
result WeakNAS produces coarse-to-fine iteration to gradually refine the ranking of sampling space, requiring fewer samples to find top-performance architectures.
The paper develops predictors for functional data on manifolds.
problem Functional data prediction on time-varying manifolds.
method Least-squares local linear Fréchet curve predictor and weighted Fréchet mean approach.
result Asymptotical optimality of the proposed predictors.
LESS combines local predictors for subsets to learn from heterogeneous input-output pairs.
problem Learning from heterogeneous input-output pairs in populations with varied behavior.
method LESS algorithm: generates subsets, trains local predictors, combines them.
result LESS is highly competitive compared to state-of-the-art methods.
GATES improves neural architecture search by modeling operations as information transformation.
problem Improving predictor-based neural architecture search efficiency.
method GATES models operations as information transformation, covering both node and edge cell search spaces.
result GATES boosts sample efficiency and improves predictor performance.
This work improves structured prediction by learning the balance between signal and random noise.
problem Structured prediction with random perturbations.
method Learning the variance of randomized structured predictors to balance signal and noise.
result Learning the balance improves structured prediction effectiveness.
Predict accuracy of neural architectures using non-neural models.
problem Improving efficiency and effectiveness of neural architecture search.
method Used gradient boosting decision tree (GBDT) for accuracy prediction and pruned the search space based on features learned from GBDT.
result GBDT-based predictor achieves comparable or better accuracy than neural network-based predictors and is 22x more sample efficient on NASBench-101.
This paper presents Sparse Partitioning, a Bayesian method for identifying predictors that either individually or in combination with others affect a response variable. The method is designed for regression problems involving binary or tertiary predictors and allows the number of predictors to exceed the size of the sa…
New method uses quotient predictor space for better PAC-Bayes bounds, reducing KL divergence and improving model performance.
problem Overparameterized models with continuous symmetries can lead to biased predictions.
method Perform PAC-Bayesian analysis on quotient predictor space, constructing a canonical prior that reflects model's implicit bias.
result The new prior reduces KL divergence and improves model performance in experiments.
A graph VAE framework optimizes neural architectures in a continuous space.
problem Discovering efficient neural architectures in a discrete space.
method Graph VAE framework with VAE and GNN components, joint learning of predictors and decoders.
result The framework discovers powerful neural architectures with both excellent performance and high computational efficiency.
Optimal trading strategy with predictor and costs, derived equations and shape.
problem Optimal trading strategy in presence of price predictor, costs, and risk control.
method Path-integral method to derive equations for band edges, solved explicitly for Ornstein-Uhlenbeck predictor.
result Explicit equations and shape of the optimal band strategy derived and analyzed.
NPENAS improves neural architecture search efficiency and accuracy.
problem Efficient and accurate neural architecture search (NAS) for minimizing search costs.
method Proposes NPENAS, a neural predictor guided evolutionary algorithm that enhances exploration ability of evolutionary algorithms.
result NPENAS-BO and NPENAS-NP outperform existing NAS algorithms on NASBench-201, NASBench-101, and DARTS.
DFNNs predict non-Euclidean responses from Euclidean predictors.
problem Regression with non-Euclidean responses.
method Deep Fréchet neural networks (DFNNs) approximating conditional Fréchet means.
result DFNNs consistently outperform existing methods in empirical studies.
GIDS reduces high-dimensional response and predictor spaces, improving interpretability and computational efficiency.
problem Challenges in modeling interactions among high-dimensional multimodal data.
method Graph Independence Dual Screening (GIDS) framework that reduces both response and predictor dimensions.
result GIDS reduces feature space to 9,000 CpGs and 2,000 transcripts, revealing coordinated regulatory mechanisms.
Paper discusses binary classification with metric space predictors, privacy constraints, and convergence rates.
problem Binary classification with metric space predictors under privacy constraints.
method Derives convergence rates for Proto-NN classifier with and without privacy constraints.
result Proto-NN classifier is universally consistent under privacy constraints.
Efficiency criteria improve conformal predictors' performance.
problem Improving the performance of conformal predictors.
method Learning classifiers by minimizing observed fuzziness as a training objective function.
result Conformal predictors trained by minimizing observed fuzziness perform better than traditional ones.
Post-processing predictors reduces calibration errors for decision-making.
problem Predictors with low calibration error for machine learning may have high error for decision-making.
method Post-processing with ε distance to calibration adds noise to make predictions differentially private.
result Post-processing achieves O(√ε) ECE and CDL, asymptotically optimal.
UWM-JEPA predicts future scenarios in belief space, improving accuracy in partially observed environments.
problem Predicting future scenarios in partially observed environments with uncertainty.
method Introduces UWM-JEPA, a JEPA world model with a density-matrix latent and learned unitary predictor.
result UWM-JEPA achieves 0.77 accuracy on a hidden-velocity indicator task, outperforming LSTM-JEPA.
Feature selection, as a critical pre-processing step for machine learning, aims at determining representative predictors from a high-dimensional feature space dataset to improve the prediction accuracy. However, the increase in feature space dimensionality, comparing to the number of observations, poses a severe challe…
A new Bayesian approach to linear system identification has been proposed in a series of recent papers. The main idea is to frame linear system identification as predictor estimation in an infinite dimensional space, with the aid of regularization/Bayesian techniques. This approach guarantees the identification of stab…
We consider rules for discarding predictors in lasso regression and related problems, for computational efficiency. El Ghaoui et al (2010) propose "SAFE" rules that guarantee that a coefficient will be zero in the solution, based on the inner products of each predictor with the outcome. In this paper we propose strong …
Autoencoder performance is predicted by eigenvalues of weight matrices.
problem Predicting an autoencoder's generalization ability without dataset knowledge.
method Analyze Jacobian matrices' eigenvalues to bound mean squared errors.
result Eigenvalues are good predictors of MSE on test points.
LIT-LVM improves linear predictors by estimating interaction terms with latent vectors.
problem Accurately estimating coefficients for interaction terms in linear predictors.
method Structured regularization using latent vectors to represent features.
result LIT-LVM achieves superior prediction accuracy compared to other methods.
Lasso is a widely used regression technique to find sparse representations. When the dimension of the feature space and the number of samples are extremely large, solving the Lasso problem remains challenging. To improve the efficiency of solving large-scale Lasso problems, El Ghaoui and his colleagues have proposed th…
In this paper we explore the richness of information captured by the latent space of a vision-based generative model. The model combines unsupervised generative learning with a task-based performance predictor to learn and to exploit task-relevant object affordances given visual observations from a reaching task, invol…
The problem of forecasting conditional probabilities of the next event given the past is considered in a general probabilistic setting. Given an arbitrary (large, uncountable) set C of predictors, we would like to construct a single predictor that performs asymptotically as well as the best predictor in C, on any data.…
Bayesian model averaging under predictor redundancy
problem Reporting Bayesian model averaging posterior without changing the Bayesian target
method Using hard or soft regions of support space
result Region reports often give shorter and clearer summaries while preserving the main posterior information
In spite of several notable efforts, explaining the generalization of deterministic non-smooth deep nets, e.g., ReLU-nets, has remained challenging. Existing approaches for deterministic non-smooth deep nets typically need to bound the Lipschitz constant of such deep nets but such bounds are quite large, may even incre…
New method detects out-of-distribution samples in regression tasks.
problem Detecting instances far from training data in regression models.
method Estimating predictor entropy based on nearest neighbors and generative models.
result A new method for robust OOD detection in regression tasks.
Deep neural networks estimate regression functions on manifolds.
problem Estimating regression functions on manifolds from data.
method Fully connected deep neural networks with ReLU activation, analyzing convergence rates.
result Estimates achieve a rate of convergence dependent on manifold dimension, not predictor dimension.
This paper proposes a method to reduce complexity in GLMs with categorical predictors.
problem Wasteful, hard-to-interpret, and prone to overfitting of traditional one-hot encoding for high-cardinality categorical predictors.
method Clustering categories of categorical predictors through a numerical method that preserves or improves accuracy while reducing the number of coefficients.
result Clustering categories of categorical predictors reduces complexity substantially without harming accuracy.
The book explores universal time-series forecasting using mixture predictors.
problem Sequential probability forecasting in a general setting.
method Mixture predictors combining multiple predictors.
result Universality of mixture predictors in a general probabilistic setting.
APQ jointly optimizes neural architecture, pruning, and quantization for efficient inference.
problem Efficient deep learning inference on resource-constrained hardware.
method Joint optimization of neural architecture, pruning, and quantization policy using a quantization-aware accuracy predictor.
result Joint optimization leads to 2.3% higher ImageNet accuracy with reduced latency and energy consumption.
The article compares predictor importance in classification problems with categorical outcomes.
problem Comparing predictor importance in classification problems with categorical response variables.
method The approach is based on the categorical Gini correlation (CGC) and tests differences in CGCs across predictor groups.
result The proposed methodology accommodates predictors of arbitrary and unequal dimensions and allows for dependence between predictor groups.
Paper proposes a sparse synthetic control method to select important predictors.
problem Choosing and weighting predictors affects synthetic control estimator performance.
method Sparse synthetic control procedure that penalizes predictors, derived in a linear factor model.
result Sparse synthetic control achieves lower bias and better post-treatment performance.
New theory explains how noisy, high-dimensional data can still lead to robust predictions.
problem Modern machine learning models achieve high performance with noisy, high-dimensional data.
method Synthesizes principles from Information Theory, Latent Factor Models, and Psychometrics to clarify predictive robustness.
result Predictive robustness arises from data architecture and model capacity, not just data cleanliness.
Interactive visualization helps understand complex machine learning models.
problem Low interpretability of machine learning models.
method Interactive slice visualization of predictor space, using interaction or touring algorithms.
result Enhances understanding and validation of machine learning model fits.
A new model for learning and unlearning predictors efficiently.
problem Learning and unlearning predictors from subsets of data.
method Proposes a ticketed model for learning-unlearning, allowing examples to present tickets for unlearning.
result Space-efficient schemes for various concept classes, including thresholds and parities.
ML helps select variables for minimum-variance portfolios, reducing risk and improving performance.
problem Optimizing minimum-variance portfolios with relevant predictors.
method Parameterized minimum-variance portfolio weights using a large pool of firm-level characteristics and their transformations.
result ML-selected predictors lead to lower risk and better performance in minimum-variance portfolios.
Proposes a method to create fair, robust predictors that remain consistent across different scenarios.
problem Creating fair and robust machine learning models that behave consistently across different scenarios.
method Graphical criteria and a model-agnostic framework called CIP based on HSCIC.
result Demonstrates the effectiveness of CIP in enforcing counterfactual invariance across various datasets.
This paper continues study, both theoretical and empirical, of the method of Venn prediction, concentrating on binary prediction problems. Venn predictors produce probability-type predictions for the labels of test objects which are guaranteed to be well calibrated under the standard assumption that the observations ar…
Derives bounds for deterministic predictors using smooth loss functions.
problem Generalizing probabilistic predictors to deterministic ones.
method Exploits smoothness properties of loss and predictor classes, controlling the Jensen gap class through Rademacher complexity.
result Derives bounds for deterministic predictors involving flatness quantities from Jacobians and Hessians.
DONNA rapidly finds optimal neural networks across diverse spaces.
problem Efficient scaling and handling of diverse architectural search-spaces in NAS.
method Three-phase pipeline: accuracy predictor, rapid evolutionary search, and optimal model finetuning.
result 100x faster than MNasNet in finding state-of-the-art architectures on-device.
Study shows competition feedback can make ML predictors biased towards specific user groups.
problem How competition affects machine learning predictors and user prediction quality.
method Flexible model of competing ML predictors, empirical and mathematical analysis.
result Competition causes predictors to specialize for specific sub-populations at the cost of general performance.
Study on merging predictors in causal and anticausal directions using CMAXENT.
problem Comparing merging predictors in causal and anticausal directions.
method Using CMAXENT as inductive bias, study differences in merging predictors.
result CMAXENT solution reduces to logistic regression in causal direction and LDA in anticausal direction.
Neural network predicts functional responses from scalar inputs.
problem Regression of functional responses with large scalar predictors and nonlinear relationships.
method Transform functional response to finite dimensions, design feed-forward neural network, modify output via objective functions, apply roughness penalty.
result Proposed neural network outperforms conventional methods in multiple scenarios.
The Partial Information Decomposition (PID) [arXiv:1004.2515] provides a theoretical framework to characterize and quantify the structure of multivariate information sharing. A new method (Idep) has recently been proposed for computing a two-predictor PID over discrete spaces. [arXiv:1709.06653] A lattice of maximum en…
Approximate computing is being considered as a promising design paradigm to overcome the energy and performance challenges in computationally demanding applications. If the case where the accuracy can be configured, the quality level versus energy efficiency or delay also may be traded-off. For this technique to be use…
This work tackles long-term visual planning by goal-conditioned hierarchical predictors.
problem Current learning approaches fail on long-horizon tasks due to lack of goal information and coarse-to-fine planning.
method Formulate goal-conditioned predictors (GCPs) and hierarchical models to predict trajectories between observations.
result GCPs enable effective long-term planning with much longer horizons than before.