While variable selection is essential to optimize the learning complexity by prioritizing features, automating the selection process is preferred since it requires laborious efforts with intensive analysis otherwise. However, it is not an easy task to enable the automation due to several reasons. First, selection techn…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “automated threshold selection”
Paper improves risk estimation for rare events in sequential decisions.
problem Estimating tail risk in high confidence levels with limited data.
method Combines extreme value theory and automated threshold selection.
result Improves performance in estimating tail risk for some distributions.
Optimal fees protect passive LPs in AMMs under varying market conditions.
problem Adverse selection losses in AMMs are not offset by static trading fees.
method Dynamic reduced-form model with parallel AMM and CEX, large-scale simulations, real market data analysis.
result Optimal AMM fees are stable under normal conditions but high in volatile periods to protect LPs.
Automates model selection for GLMs using optimization.
problem Automating model selection for generalized linear models.
method Mixed-integer conic optimization for feature subset selection and model constraints.
result Optimization of AIC and BIC criteria with multicollinearity constraints.
Auto-Surprise automates recommender system selection and optimization.
problem Finding the best algorithm and hyperparameters for recommender systems.
method Extends Surprise library with TPE optimization for algorithm selection and hyperparameter tuning.
result Significantly faster in finding optimal hyperparameters compared to grid search.
To be effective, state of the art machine learning technology needs large amounts of annotated data. There are numerous compelling applications in healthcare that can benefit from high performance automated decision support systems provided by deep learning technology, but they lack the comprehensive data resources req…
When Stability Fails: Hidden Failure Modes Of LLMS in Data-Constrained Scientific Decision-Makingcs.LG
LLMs fail to match statistical ground truth despite stable run-to-run performance.
problem LLMs lack validation against statistical ground truth in automated scientific workflows.
method Introduced a behavioral evaluation framework for LLMs, separating four decision-making dimensions.
result LLMs can exhibit near-perfect stability but diverge from statistical ground truth.
Feature Selection (FS) plays an important role in learning and classification tasks. The object of FS is to select the relevant and non-redundant features. Considering the huge amount number of features in real-world applications, FS methods using batch learning technique can't resolve big data problem especially when …
One of the longstanding problems in spectral graph clustering (SGC) is the so-called model order selection problem: automated selection of the correct number of clusters. This is equivalent to the problem of finding the number of connected components or communities in an undirected graph. In this paper, we propose AMOS…
This paper presents an approach for automation of interpretable feature selection for Internet Of Things Analytics (IoTA) using machine learning (ML) techniques. Authors have conducted a survey over different people involved in different IoTA based application development tasks. The survey reveals that feature selectio…
Automated method selects best model from many for production systems.
problem Selecting the best model for production systems from a large pool.
method Automated online experimentation mechanism using Bayesian surrogate models.
result Efficiently identifies the best model with small online experiments.
Automates selection and visualization of model responses in various directions.
problem Manual selection and limited visualization of PDPs.
method Formalizes method for automating PDP selection and extends to arbitrary directions.
result Demonstrates usefulness across model selection, bias detection, and latent space exploration.
New formula identifies and quantifies costs for automated market makers.
problem Adverse selection costs faced by liquidity providers in automated market makers.
method Derives a Black-Scholes-like formula for AMMs and identifies loss-versus-rebalancing cost.
result Closed-form expressions for LVR applicable to all automated market makers.
A simple thresholding technique improves graph selection in neural connectivity studies.
problem Graphical model selection for functional neural connectivity in the presence of latent variables.
method Apply a hard thresholding operator to graphical Lasso, neighborhood selection, or CLIME estimators.
result Thresholded estimators outperform existing methods in graph selection consistency and empirical results.
Fast, fully-automated histograms for large data sets.
problem Efficiently constructing histograms for large-scale data.
method G-Enum histograms using Minimum Description Length (MDL) principle for model selection and a greedy search heuristic.
result Construct histograms in linearithmic time, significantly faster than previous methods.
DriveML automates machine learning tasks in R.
problem Manual machine learning tasks are time-consuming and error-prone.
method Automated data preparation, feature engineering, model building, and model explanation.
result DriveML performs best across different parameters compared to other packages.
The paper develops a test for independence of selected Gaussian variables after thresholding correlations.
problem Testing independence of selected Gaussian variables after thresholding correlations.
method The approach involves conditioning on the selection event and using a new characterization of the conditioning event in terms of canonical correlation.
result The proposed test has higher power than a naive approach that ignores selection effects.
To estimate a sparse linear model from data with Gaussian noise, consilience from lasso and compressed sensing literatures is that thresholding estimators like lasso and the Dantzig selector have the ability in some situations to identify with high probability part of the significant covariates asymptotically, and are …
MCU-Net combines U-Net and Monte Carlo Dropout for uncertainty in medical image segmentation.
problem Lack of uncertainty representation in deep learning methods for patient-centered healthcare decisions.
method MCU-Net framework using U-Net and Monte Carlo Dropout with four uncertainty metrics.
result MCU-Net maximizes automated performance and refers truly uncertain cases.
AutoFS combines trainers to improve feature selection efficiency and effectiveness.
problem Balancing feature selection efficiency and effectiveness.
method Interactive Reinforced Feature Selection (IRFS) framework with diverse trainers.
result Improved feature selection efficiency and effectiveness compared to existing methods.
FILTER model uses fusion penalized logistic threshold regression for high-dimensional data with unknown cut points.
problem Modeling high-dimensional data with unknown cut points and binary responses.
method Fusion penalized logistic threshold regression (FILTER) model with fused lasso penalty for variable selection.
result Established non-asymptotic error bounds for coefficient estimation and model selection consistency.
With the maturation of metabolomics science and proliferation of biobanks, clinical metabolic profiling is an increasingly opportunistic frontier for advancing translational clinical research. Automated Machine Learning (AutoML) approaches provide exciting opportunity to guide feature selection in agnostic metabolic pr…
This paper uses machine learning to select kernels for machine learning models on various devices.
problem Traditional kernel auto-tuning is limited for machine learning research with changing network topologies and hyperparameters.
method Combines auto-tuning and machine learning to select kernels for SYCL on various devices.
result Initial results show high performance kernel selection with little developer effort.
One of the longstanding open problems in spectral graph clustering (SGC) is the so-called model order selection problem: automated selection of the correct number of clusters. This is equivalent to the problem of finding the number of connected components or communities in an undirected graph. We propose automated mode…
Automation of machine learning model development is increasingly becoming an established research area. While automated model selection and automated data pre-processing have been studied in depth, there is, however, a gap concerning automated model adaptation strategies when multiple strategies are available. Manually…
The article examines different thresholding methods for improving PAM algorithm in cancer classification.
problem High-dimensional classification with too many features selected by PAM.
method Extends PAM with hard and order thresholding methods and a deep search algorithm.
result Improved cancer status prediction accuracy and smaller number of features.
BOOST automates kernel and acquisition function selection in Bayesian optimization.
problem Inappropriate kernel and acquisition function combinations lead to poor performance in Bayesian optimization.
method BOOST uses offline evaluation to predict and select the best kernel-acquisition function pair.
result BOOST consistently improves over fixed-hyperparameter BO and is competitive with state-of-the-art adaptive methods.
Review of automation's role in chemical discovery, emphasizing future challenges.
problem Improving automation's contribution to chemical discovery.
method Analysis of exemplary studies and open research directions.
result Future autonomous systems need improvement in data handling, model building, and experiment automation.
The paper provides high-probability bounds on false discovery proportions in conformal inference.
problem Existing methods fail to provide high-probability bounds on the realized false discovery proportion.
method Constructing a high-probability envelope for the empirical distribution function of null conformal p-values by sampling from their joint distribution.
result Establishes finite-sample, distribution-free upper bounds on the FDP that hold simultaneously over all possible rejection thresholds.
Classical stochastic gradient methods for optimization rely on noisy gradient approximations that become progressively less accurate as iterates approach a solution. The large noise and small signal in the resulting gradients makes it difficult to use them for adaptive stepsize selection and automatic stopping. We prop…
Optimal algorithm selects biological models without prior info.
problem Determining the correct biological model without prior knowledge.
method Systems biology models and likelihood-free inference.
result Improved model selection performance over conventional methods.
This paper describes the autofeat Python library, which provides scikit-learn style linear regression and classification models with automated feature engineering and selection capabilities. Complex non-linear machine learning models, such as neural networks, are in practice often difficult to train and even harder to …
Proposes a Siamese NN for algorithm selection focusing on alike performing instances.
problem Lack of effective meta-features for algorithm selection via meta-learning.
method Siamese Neural Network architecture with 'Algorithm-Performance Personas' concept.
result Proposed metric outperforms standard performance metrics in training sample selection.
Evaluation of the marginal likelihood plays an important role in model selection problems. The widely applicable Bayesian information criterion (WBIC) and singular Bayesian information criterion (sBIC) give approximations to the log marginal likelihood, which can be applied to both regular and singular models. When the…
New method optimizes tail dependence coefficient estimation.
problem Estimating tail dependence in nonparametric data.
method Optimal threshold selection combining mean squared error and copula estimation.
result Improved accuracy in tail dependence coefficient estimation.
LASSO-PCA combines LASSO and PCA for automated forecast averaging.
problem Automating the selection of forecast averaging methods and tuning parameters.
method LASSO estimation combined with PCA, using information criteria for parameter selection.
result LASSO-PCA outperforms other methods in forecast error reduction.
Proposes a method to choose thresholds for LLM evaluation metrics.
problem Ensuring reliable large language models (LLMs) with correct threshold selection.
method Identify risks, stakeholders' risk tolerance, and use ground-truth data to determine thresholds.
result Demonstrates a concrete example with the Faithfulness metric and HaluBench dataset.
Lasso proves consistent model selection for high-dimensional Ising models.
problem Model selection consistency of Lasso for high-dimensional Ising models.
method Theoretical analysis of Lasso with and without post-thresholding for Ising models.
result Lasso without post-thresholding is model selection consistent in the whole paramagnetic phase with .
Approximate Bayesian computation (ABC) can be used for model fitting when the likelihood function is intractable but simulating from the model is feasible. However, even a single evaluation of a complex model may take several hours, limiting the number of model evaluations available. Modelling the discrepancy between t…
The paper proposes a method to assess when automated predictions are reliable.
problem Ensuring reliability and safety of automated decision-making in machine learning.
method Clustering to measure distances between outputs and class centroids, defining a safety threshold based on these distances.
result The proposed metric can efficiently determine when automated predictions are acceptable and when they should be deferred.
Method selects the best deep learner for time-series prediction using Bayesian networks.
problem Selecting the most effective deep learning model for time-series prediction.
method Bayesian network selects deep learners based on input variables and cluster training data.
result Threshold value determines which deep learners predict time-series data robustly.
Multi-Likelihood Methods for Developing Stock Relationship Networks Using Financial Big Dataq-fin.ST
Development of stock networks is an important approach to explore the relationship between different stocks in the era of big-data. Although a number of methods have been designed to construct the stock correlation networks, it is still a challenge to balance the selection of prominent correlations and connectivity of …
Interactive RL and DT feedback improve feature selection efficiency.
problem Balancing feature selection effectiveness and efficiency.
method Interactive Reinforcement Learning (IRL) and Decision Tree Feedback (DTF) architecture.
result Improved feature selection performance on real-world datasets.
Study automates feature selection and clustering for HFT stock price forecasting.
problem Manual feature selection and clustering for high-frequency trading (HFT) stock price forecasting.
method Dual competitive feature importance mechanism and clustering via shallow neural network topology.
result Enhanced forecasting ability of the RBFNN regressor through automated feature selection and clustering.
Graphical lasso may fail to fit models when data points are insufficient.
problem When does graphical lasso fail to select and fit a graphical model?
method Computational experiments with graphical lasso.
result Graphical lasso may fail when the number of data points is less than the maximum likelihood threshold.
Ridge regression is revisited with debiasing and thresholding, offering advantages over Lasso.
problem High-dimensional data challenges classical ridge regression's sparsity detection and bias issues.
method Debiasing and thresholding ridge regression, introducing a wild bootstrap for confidence regions and hypothesis testing, and a hybrid bootstrap for prediction intervals.
result Debiased and thresholded ridge regression can offer similar performance to thresholded Lasso and may be preferable in some settings.
Hard Thresholding Pursuit (HTP) is an iterative greedy selection procedure for finding sparse solutions of underdetermined linear systems. This method has been shown to have strong theoretical guarantee and impressive numerical performance. In this paper, we generalize HTP from compressive sensing to a generic problem …
Improved estimation of hedge fund tail risks using a novel model.
problem Estimation inefficiencies and need for manual threshold selection in extreme value regression models.
method Extended tail regression model with automatic threshold selection and artificial censoring.
result Significant link between tail risks and factors like equity momentum and financial stability index.