Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance fu…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “over-fitting avoidance”
AutoCompete automates machine learning competition tasks.
problem Tackling machine learning competitions with minimal human intervention.
method A highly automated framework that learns from past competitions.
result Produces better results with less runtime compared to other approaches.
We present techniques for effective Gaussian process (GP) modelling of multiple short time series. These problems are common when applying GP models independently to each gene in a gene expression time series data set. Such sets typically contain very few time points. Naive application of common GP modelling techniques…
Researchers predict NBA player salaries using machine learning, avoiding overfitting.
problem Predicting NBA player salaries based on performance statistics.
method Selected important determinants, used Random Forest machine learning, avoided overfitting.
result Very satisfactory salary predictions identified for important factors.
Interpolation improves nearest neighbor algorithms' performance.
problem Improving nearest neighbor algorithms' performance.
method Considered a class of interpolated weighting schemes and characterized their asymptotic performances.
result Mild degree of data interpolation strictly improves prediction accuracy and statistical stability.
The accurate prediction of time-changing covariances is an important problem in the modeling of multivariate financial data. However, some of the most popular models suffer from a) overfitting problems and multiple local optima, b) failure to capture shifts in market conditions and c) large computational costs. To addr…
Learning multiple tasks across heterogeneous domains is a challenging problem since the feature space may not be the same for different tasks. We assume the data in multiple tasks are generated from a latent common domain via sparse domain transforms and propose a latent probit model (LPM) to jointly learn the domain t…
The study compares feature learning techniques for predicting Alzheimer's disease from MRI.
problem Predicting cognitive impairment from MRI data.
method Review and comparison of feature learning and selection techniques.
result Stacked auto-encoders outperformed other methods in MRI-based Alzheimer's disease prediction.
Bayesian co-clustering model for mixed data tables.
problem Co-clustering of mixed-type data tables.
method MAP Bayesian approach inferring optimal segmentation and minimizing model selection cost function.
result Optimal co-clustering model with exact model quality measure.
Improves GATs by adding margin-based constraints to prevent over-fitting and over-smoothing.
problem Over-fitting and over-smoothing in GATs.
method Margin-based constraints on attention weights and graph structure.
result Significant improvements over previous GATs on various datasets.
The paper improves dropout's utility by reducing interactions in deep neural networks.
problem Over-fitting problem in deep learning.
method Game-theoretic interactions analysis and interaction loss.
result Interaction loss improves dropout's utility and boosts DNN performance.
The paper addresses over-fitting in deep learning models trained on imbalanced data.
problem Over-fitting to minor classes in deep learning models trained on imbalanced data.
method Investigated feature deviation and proposed class-dependent temperatures (CDT) to compensate for it.
result CDT helps in overcoming feature deviation, improving model performance on test data of minor classes.
Modern machine learning practices contradict traditional bias-variance theory.
problem Modern machine learning models often fit data perfectly, yet perform well.
method Introducing a 'double descent' curve to reconcile classical and modern practices.
result Increasing model capacity beyond interpolation improves performance.
DropEdge improves deep GCNs for node classification by reducing over-fitting and over-smoothing.
problem Over-fitting and over-smoothing in deep GCNs for node classification.
method Randomly removes edges from the input graph at each training epoch to reduce over-fitting and over-smoothing.
result DropEdge improves performance on various GCN models and prevents over-smoothing.
Despite having various attractive qualities such as high prediction accuracy and the ability to quantify uncertainty and avoid over-fitting, Bayesian Matrix Factorization has not been widely adopted because of the prohibitive cost of inference. In this paper, we propose a scalable distributed Bayesian matrix factorizat…
Early stopping improves nonparametric testing optimality.
problem Improving minimax optimal testing in nonparametric settings.
method Applying early stopping to functional gradient descent in RKHS to obtain a Wald-type test.
result Sharp stopping rule for optimal testing in nonparametric settings.
Package {mlr3spatiotempcv} simplifies spatiotemporal resampling methods in R.
problem Assessing and tuning spatial and spatiotemporal machine learning models.
method Integrates various spatiotemporal resampling methods into the {mlr3} framework.
result Provides a consistent interface for spatiotemporal resampling methods.
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
Unified framework for adaptive connection sampling in GNNs improves performance and robustness.
problem Over-smoothing and over-fitting in deep GNNs.
method Adaptive connection sampling trained jointly with GNN model parameters.
result Adaptive connection sampling mathematically equivalent to Bayesian GNNs approximation.
Enhances graph classification models on small datasets.
problem Over-fitting and undergeneralization on small-scale benchmark datasets.
method Data augmentation via graph structure transformation and model evolution framework.
result Average improvement of 3 - 13% accuracy on graph classification tasks.
A new credit scoring method using Gaussian Mixture Models.
problem Differentiating good from bad borrowers in credit scoring.
method Gaussian Mixture Models for consumer classification.
result The model performs comparably to others and avoids overfitting.
Combines deep learning with instrumental variables for causal effect prediction.
problem Causal effect estimation in the presence of unobserved confounders.
method Deep instrumental variables networks (D-IV) combining first and second stage neural nets.
result Flexible D-IV framework resolves causal estimation into manageable prediction tasks.
A new matrix factorization method for high-dimensional data.
problem Exploiting sparse structures in complex data for better interpretability.
method Bayesian shrinkage priors and flexible sparse patterns modeled through row and column dependencies.
result Demonstrated practical advantages through simulation and soccer heatmap analysis.
Method selects variables for malaria risk prediction using Lasso and GLM.
problem Overcoming expert pretreatment in epidemiological data.
method Automatic variable selection using Lasso and GLM for high-dimensional data.
result Only few climatic and environmental variables are main malaria risk factors.
LayerNorm improves training and generalization by re-centering and re-scaling backward gradients.
problem Understanding the effectiveness of LayerNorm and addressing over-fitting.
method Re-centering and re-scaling backward gradients; proposing AdaNorm.
result LayerNorm without bias and gain performs better than LayerNorm on most datasets.
New method finds sparse groups of input variables for neural networks.
problem Finding optimal groups of input variables for neural networks.
method Developed a new loss function and optimization algorithm for multi-layer non-linear neural networks to achieve group sparsity.
result Achieved group sparsity in three real-world datasets, improving model performance and excluding a significant number of variables.
Develops a Bayesian method for an infinite mixture of inverted Dirichlet distributions.
problem Overcoming the need to pre-determine the number of mixture components in Dirichlet process models.
method Adopted the extended variational inference framework to derive an analytically tractable solution.
result Demonstrates good performance and effectiveness compared to other DP-related methods.
Interpolated nearest neighbor algorithms minimize bias in machine learning models.
problem Understanding and reducing overfitting in machine learning models.
method Proves the interpolated nearest neighbor algorithm achieves minimax optimal rates in regression and classification.
result Interpolated nearest neighbor algorithms are statistically optimal and perform better than traditional methods in some cases.
Proposes InfoSSM for interpretable unsupervised learning of complex dynamics.
problem Learning complex nonparametric dynamics from multi-modal data.
method InfoSSM framework using multiple Gaussian process transition models and mutual information regularizer.
result Demonstrates improved interpretability and performance in multi-modal dynamics.
Paper combines scalable BMF algorithms for web-scale datasets.
problem High computational cost of Bayesian Matrix Factorization.
method Combines Posterior Propagation and asynchronous distributed implementation.
result Substantial improvements in scalability on web-scale datasets.
Optimal feature learning strength improves generalization in deep networks.
problem Understanding how feature learning strength affects generalization in practical settings.
method Empirical studies and theoretical analysis of gradient flow dynamics in two-layer ReLU nets.
result Optimal feature learning strength yields substantial generalization gains, contrary to the prevailing intuition.
The optimization of large portfolios displays an inherent instability to estimation error. This poses a fundamental problem, because solutions that are not stable under sample fluctuations may look optimal for a given sample, but are, in effect, very far from optimal with respect to the average risk. In this paper, we …
Improved prediction accuracy in linear models with missing data.
problem Improving prediction accuracy in linear models with missing data.
method Introduced Soft Weighted Prediction (SWP) algorithm and optimized it for missing data scenarios.
result Improved Mean Squared Error (MSE) on test set compared to state-of-the-art methods.
Paper presents a self-adaptive learning model for robust classification and regression.
problem Dealing with various datasets of different complexity.
method Combines DNDN and DSP, an end-to-end training approach with multiple randomly initialized softmax layers and adaptive soft pruning.
result The model demonstrates no performance loss compared with unpruned models and higher robustness over different data and feature distributions.
CNAPs adapts image classifiers to new tasks efficiently.
problem Adapting image classifiers to new tasks after initial training.
method Conditional Neural Adaptive Processes (CNAPs) using a modulated classifier and adaptation network.
result CNAPs achieves state-of-the-art results on Meta-Dataset, demonstrating robust transfer-learning.
This paper describes a novel method to approximate the polynomial coefficients of regression functions, with particular interest on multi-dimensional classification. The derivation is simple, and offers a fast, robust classification technique that is resistant to over-fitting.
Symmetry improves machine learning models by reducing overfitting and complexity.
problem Ignoring symmetry in machine learning models can lead to overfitting and increased complexity.
method Incorporating symmetry into machine learning models, specifically neural networks for classifying handwritten digits.
result Incorporating symmetry into machine learning models reduces overfitting and complexity, requiring less training data and less time to train.
The paper defines and analyzes feature complexity in DNNs, proposing metrics for feature disentanglement and evaluation.
problem Understanding and quantifying the complexity of features learned by deep neural networks.
method Proposes a definition and disentanglement of feature complexity orders, introduces metrics for reliability and over-fitting evaluation.
result Establishes a relationship between feature complexity and DNN performance, and proposes a generic mathematical tool for network compression and knowledge distillation.
Paper explains word embedding dimensionality and proposes a new metric.
problem Understanding the optimal dimensionality of word embeddings.
method Proposed Pairwise Inner Product (PIP) loss to measure dissimilarity.
result Revealed a bias-variance trade-off in dimensionality selection.
New deep learning model for matrix completion combining linear and nonlinear relationships.
problem Matrix completion considering only linear or nonlinear relations, ignoring latent relationships.
method Combines linear and nonlinear models in a latent variables framework, using a deep neural network with two branches for columns and rows, and manifold learning as an auxiliary task.
result Experimental results show the proposed method outperforms state-of-the-art matrix completion methods.
Paper uses virtual big data to improve autoencoder training and address imbalanced data classification.
problem Imbalanced data classification and autoencoder over-fitting.
method Cross-concatenation using Virtual Big Data.
result Cross-concatenation method effectively balances imbalanced class distributions.
This study uses TDA to map corporate failure, revealing distinct regions of risk.
problem Understanding and predicting corporate default risk.
method Topological Data Analysis (TDA) applied to Altman's Z-score model.
result Firms do not cluster neatly along default predictors, suggesting complex risk landscapes.
Study develops a method to select penalty parameters for sparse neural networks without cross-validation.
problem Selecting optimal penalty parameters for sparse neural networks without cross-validation.
method Established theoretical foundation to bound the infinite norm of the gradient of the loss function at zero.
result Proposed method effectively selects penalty parameters for sparse neural networks.
New activation function SERLU improves neural network performance.
problem Improving neural network performance and avoiding overfitting.
method Introducing a new activation function (SERLU) that breaks monotonicity while preserving self-normalizing property and developing shift-dropout for regularization.
result SERLU-based neural networks provide consistently promising results compared to other activation functions.
Adaptive weight-decay improves deep neural network performance.
problem Overfitting in deep neural networks.
method AdaDecay adjusts weight-decay adaptively based on gradient norms within each layer.
result AdaDecay improves generalization and accuracy across various datasets and models.
It has always been a great challenge for clustering algorithms to automatically determine the cluster numbers according to the distribution of datasets. Several approaches have been proposed to address this issue, including the recent promising work which incorporate Bayesian Nonparametrics into the -means clusterin…
ELMs benefit from using equal number of hidden nodes to training samples.
problem Overfitting and underfitting in ELMs with hidden nodes.
method ELMs with hidden nodes equal to training samples achieve perfect training.
result ELMs with larger hidden nodes outperform traditional ELMs.
Improves GCNNs with node transition probabilities and DropNode regularization.
problem Over-fitting and over-smoothing issues in GCNNs.
method Message passing based on node transition probabilities and DropNode regularization.
result Improved GCNNs with better node representations and reduced over-fitting and over-smoothing.