Model selection for time series forecasting can be biased by the distribution of scores.
problem Model selection for probabilistic forecasting on time series data.
method Using proper scoring rules to aggregate scores across multiple time series.
result The mean score is immune to the skewness of the score distribution.
This work improves Gaussian process model selection for large datasets.
problem Prohibitively high computational cost in Gaussian process model selection.
method Linear-time scaling and computational uncertainty tradeoff.
result Computation-aware Gaussian processes can be trained on large datasets efficiently.
Inference-Time Scaling can be extended to domains prone to systematic failure using intrinsic statistics.
problem Scaling inference time in domains prone to systematic failure
method Intrinsic Selection (iS), Intrinsic Particle Filtering (iPF), and Particle Distillation (dPF)
result Intrinsic Selection improves engineering design selection by 20% and pass@1 by 6.1 points on average.
The study extracts market direction from transaction data.
problem Extracting market direction from transaction data.
method Dynamic equation with time scale selection from past transactions.
result Automatic determination of time scale for price calculation.
The analysis of temporal networks has a wide area of applications in a world of technological advances. An important aspect of temporal network analysis is the discovery of community structures. Real data networks are often very large and the communities are observed to have a hierarchical structure referred to as mult…
Efficient knockoffs for large-scale feature selection.
problem Large-scale feature selection problems.
method Gaussian model-X knockoffs with efficient methods for solving semidefinite programs.
result Efficient knockoffs can be generated with linear complexity in the dimension.
Large-scale Hierarchical Classification (HC) involves datasets consisting of thousands of classes and millions of training instances with high-dimensional features posing several big data challenges. Feature selection that aims to select the subset of discriminant features is an effective strategy to deal with large-sc…
UniShape improves time series classification by selecting relevant subsequences.
problem Classifying time series data requires capturing interpretable shapelets.
method UniShape uses a shape-aware adapter to aggregate multiscale subsequences into class tokens.
result UniShape achieves state-of-the-art classification performance.
Aims to describe neural network training dynamics using two-time-scale models.
problem Lack of a general mathematical description of neural network training.
method Introduces a theoretical framework based on two-time-scale population dynamics.
result Derives selection-mutation equations and effective fitness for hyperparameters.
Investment strategies for rank-dependent utility agents are derived in a continuous-time market.
problem Time inconsistency in rank-dependent utility models.
method Study of consistent planners seeking intra-personal equilibrium strategies.
result Explicit final wealth profile replicating equilibrium strategies, with scaling function derived.
Sampling more can make models more confident in wrong answers, not better.
problem The modal ceiling and correlation ceiling limit the benefit of increased sampling.
method Analyzes the trade-offs between sampling more and selecting the best answer.
result Extra sampling beyond a certain point does not improve model performance and can even degrade it.
This study examines memory effects in S&P500 market correlations using Langevin models.
problem The neglect of memory effects in market correlations for optimal portfolio selection.
method Fit a generalised Langevin equation (GLE) to S&P500 market correlation data.
result Memory effects in market correlations significantly improve forecasting accuracy and suggest a hidden slow time scale.
We investigate finite-time decoupled convergence in nonlinear two-time-scale stochastic approximation.
problem Achieving decoupled convergence in nonlinear two-time-scale stochastic approximation.
method Nested local linearity assumption, suitable step size selection, convergence analysis of matrix cross term, fourth-order moment convergence rates.
result Finite-time decoupled convergence rates can be achieved in nonlinear two-time-scale stochastic approximation with proper step size selection.
Enhanced framework selects features for unbiased causal inference.
problem Unbiased estimation of causal quantities in causal inference.
method Three-stage computational framework balancing treatment and non-treatment variables.
result Significantly reduces bias and variance in estimating causal quantities.
New method reduces memory usage for high-dimensional variable selection.
problem Scalability issues in high-dimensional variable selection, especially in genomics.
method Adaptive sampling of null features to eliminate dummy matrix materialization.
result Reduces memory and runtime by several orders of magnitude while preserving FDR control.
STAS selects optimal spatio-temporal scales for bias correction in precipitation forecasts.
problem Limited prior data and fixed ST scale in existing BCoPs lead to biases in numerical weather predictions.
method End-to-end deep-learning BCoP model STAS with SFM/TFM to automatically adjust spatial and temporal scales.
result STAS outperforms 8 published BCoP methods on threat scores (TS).
New classical algorithm outperforms quantum in neural network subnetwork selection.
problem Selecting sparse subnetworks from large neural networks efficiently.
method Quantum-inspired classical algorithm using ridgelet transform sampling.
result Runs in polynomial time, outperforming naive classical methods.
BanditLP optimizes personalized recommendations for large-scale systems.
problem Optimizing personalized recommendations for large-scale systems with constraints.
method Unified neural Thompson Sampling for learning and large-scale linear programming for action selection.
result Consistent gains over strong baselines in experiments and business win in LinkedIn's email marketing system.
A sparse modeling is a major topic in machine learning and statistics. LASSO (Least Absolute Shrinkage and Selection Operator) is a popular sparse modeling method while it has been known to yield unexpected large bias especially at a sparse representation. There have been several studies for improving this problem such…
Fast, fully-automated histograms for large data sets.
problem Efficiently constructing histograms for large-scale data.
method G-Enum histograms using Minimum Description Length (MDL) principle for model selection and a greedy search heuristic.
result Construct histograms in linearithmic time, significantly faster than previous methods.
Feature selection is an important challenge in machine learning. It plays a crucial role in the explainability of machine-driven decisions that are rapidly permeating throughout modern society. Unfortunately, the explosion in the size and dimensionality of real-world datasets poses a severe challenge to standard featur…
We study two time-scale linear stochastic approximation algorithms, which can be used to model well-known reinforcement learning algorithms such as GTD, GTD2, and TDC. We present finite-time performance bounds for the case where the learning rate is fixed. The key idea in obtaining these bounds is to use a Lyapunov fun…
Automating statistical modelling is a challenging problem in artificial intelligence. The Automatic Statistician takes a first step in this direction, by employing a kernel search algorithm with Gaussian Processes (GP) to provide interpretable statistical models for regression problems. However this does not scale due …
WALNUTS improves sampling efficiency and robustness for multi-scale distributions.
problem Adapting leapfrog step size for multi-scale posterior distributions.
method Adapts leapfrog step size at fixed intervals of simulated time, selecting the largest step size to keep energy error below a threshold.
result Substantial improvements in sampling efficiency and robustness compared to standard NUTS.
CorePPR combines PageRank and CoreRank for scalable GNNs.
problem Expensive message passing in GNNs for large graphs.
method Learnable convex combination of approximate PPR and CoreRank, dynamic node selection.
result CorePPR outperforms PPRGo on large graphs.
Agents' heterogeneity is recognized as a driver mechanism for the persistence of financial volatility. We focus on the multiplicity of investment strategies' horizons, we embed this concept in a continuous time stochastic volatility framework and prove that a parsimonious, two-scale version effectively captures the lon…
Paper compares ML methods for credit scoring, highlighting feature selection and scaling impacts.
problem Determining default risk in credit scoring models.
method Eight ML methods (SVM, Naive Bayes, DT, RF, XGBoost, KNN, MLP, LR) with feature selection and scaling.
result Feature selection and scaling improve model performance in credit scoring.
Preformer improves Transformer for long-term time series forecasting.
problem Transformer's quadratic complexity and lack of context-awareness for long-term forecasting.
method Introduces Multi-Scale Segment-Correlation mechanism for efficient time series segmentation and context-aware attention.
result Preformer outperforms other Transformer-based methods in long-term time series forecasting.
In [1], we have explored the theoretical aspects of feature selection and evolutionary algorithms. In this chapter, we focus on optimization algorithms for enhancing data analytic process, i.e., we propose to explore applications of nature-inspired algorithms in data science. Feature selection optimization is a hybrid …
This study evaluates feature scaling across 14 datasets and 12 techniques in ML.
problem Impact of feature scaling on machine learning performance and computational costs.
method Systematic evaluation of 12 scaling techniques across 14 datasets and 16 ML algorithms.
result Wide variation in model performance due to feature scaling, especially for non-ensemble models.
New method selects diffusion scales for graph wavelets.
problem Choosing optimal diffusion scales for graph wavelets.
method Proposes an unsupervised method using information theory.
result Method selects diffusion scales for graph wavelets.
A method to select important experts for Gaussian processes to balance computational efficiency and uncertainty quantification.
problem Balancing computational efficiency and uncertainty quantification in Gaussian processes for big data.
method Using graphical models to select important experts and aggregate their predictions while ensuring uncertainty quantification.
result Substantially reduces computational cost of aggregating dependent experts while ensuring calibrated uncertainty quantification.
Benchmarking deep learning models for financial time series, focusing on risk-adjusted performance.
problem Optimizing risk-adjusted performance in financial time series prediction.
method Evaluation of various deep learning architectures including linear models, RNNs, transformers, state space models, and sequence representation approaches.
result Hybrid models like VSN with LSTM and xLSTM achieve the highest overall Sharpe ratio and superior downside adjusted characteristics.
Biodiversity monitoring using audio recordings is achievable at a truly global scale via large-scale deployment of inexpensive, unattended recording stations or by large-scale crowdsourcing using recording and species recognition on mobile devices. The ability, however, to reliably identify vocalising animal species is…
SHAKE-GNN scales GNNs for large graphs with multi-scale representations.
problem Scaling Graph Neural Networks (GNNs) to large graphs.
method SHAKE-GNN uses a hierarchy of Kirchhoff Forests for stochastic multi-resolution graph decompositions.
result SHAKE-GNN achieves competitive performance on large-scale graph classification benchmarks.
ES improves training efficiency by dynamically selecting data samples.
problem Efficiently selecting informative data samples for faster learning.
method Evolved Sampling (ES) dynamically selects data samples based on loss dynamics and differences.
result ES achieves significant training acceleration without compromising model performance.
Hard optimisation problems such as Boolean Satisfiability typically have long solving times and can usually be solved by many algorithms, although the performance can vary widely in practice. Research has shown that no single algorithm outperforms all the others; thus, it is crucial to select the best algorithm for a g…
The paper introduces a framework to select efficient datasets for preserving model rankings.
problem Efficient evaluation of machine learning models on small, representative datasets.
method Bootstrap aggregation, clustering, design criteria, random baselines, and greedy farthest-first (FAFI).
result Several selection strategies improve rank preservation compared to random subsets, especially in time series classification.
In this paper, several modifications are introduced to the functional approximation method iterLap to reduce the approximation error, including stopping rule adjustment, proposal of new residual function, starting point selection for numerical optimisation, scaling of Hessian matrix. Illustrative examples are also prov…
New methods detect continuous variation in single-cell data.
problem Continuous variation within and between cell types not detected by discrete analyses.
method Three topologically motivated mathematical methods for unsupervised feature selection.
result Detect additional biologically meaningful genes with coherent expression patterns.
Algorithm identifies fractal system's scaling exponents in high dimensions.
problem Statistical identification of Hurst distribution in high-dimensional fractal systems.
method Wavelet random matrices, modified spectral clustering, model selection.
result Algorithm consistently estimates Hurst distribution in moderately high dimensions.
PRISM infers model structures and parameters from simulations, controlling complexity at test time.
problem Choosing among large model families for scientific discovery.
method Simulation-based encoder-decoder that infers model structures and parameters, with test-time complexity control.
result PRISM scales to large model families and performs model selection in biophysical diffusion MRI.
A new approach selects tuning parameters for embedding methods.
problem Difficulty in selecting tuning parameters for embedding methods.
method Minimize a stress notion to supervise tuning parameter selection.
result Uncover a new bias--variance tradeoff phenomenon.
We present a novel event embedding algorithm for crime data that can jointly capture time, location, and the complex free-text component of each event. The embedding is achieved by regularized Restricted Boltzmann Machines (RBMs), and we introduce a new way to regularize by imposing a ℓ1 penalty on the conditiona…
The analysis of scientific data of increasing size and complexity requires statistical machine learning methods that are both interpretable and predictive. Union of Intersections (UoI), a recently developed framework, is a two-step approach that separates model selection and model estimation. A linear regression algori…
New algorithm reduces bandit regret to log^3(T).
problem Noise model for linear stochastic bandits with vanishing noise.
method Weighted least-squares estimation, leveraging eigenvalue relation.
result Minimax regret scaling as log^3(T) for time horizon T.
MDS selects assets by combining daily returns and intraday risk curves, improving portfolio performance.
problem High estimation error in large-scale asset selection.
method Metric Dependence Screening (MDS) incorporating high frequency information as object valued data.
result MDS improves portfolio performance over benchmarks by preserving intraday risk dynamics.
ALEXP improves model selection in linear bandits with exponential regret improvement.
problem Model selection in linear bandits is challenging due to balancing exploration and exploitation.
method ALEXP uses online learning with favorable bias-variance trade-off to emulate full-information feedback.
result ALEXP achieves an exponentially improved (logM) regret dependence on the number of models M.