SPlit optimizes dataset splitting for better model performance.
problem Improving model performance through optimal dataset splitting.
method Adapting Support Points (SP) algorithm for subsampling and categorical variables in a sequential nearest neighbor approach.
result SPlit significantly improves worst-case testing performance compared to random splitting.
New method escapes local optima in neural architecture optimization.
problem Escaping local optima in neural architecture optimization.
method Signed neural splitting in steepest descent framework.
result Escapes local optima, leading to better performance.
Optimizes decision-making with uncertain variables using auxiliary observations.
problem Contextual stochastic optimization problems with uncertain variables and rich auxiliary observations.
method Trains forest decision policies by growing trees that optimize downstream decision quality, using optimization perturbation analysis for efficient approximations.
result Proves asymptotic optimality and empirical validation of the method's performance and efficiency.
Optimal data split ratio is sqrt(p):1 for linear regression.
problem Lack of clear guidance on optimal training/testing data split ratio.
method Showed that optimal ratio is sqrt(p):1 for linear regression.
result Optimal ratio for training/testing split is sqrt(p):1.
Optimizes data splitting for shorter conformal prediction intervals.
problem Minimizing prediction interval length while maintaining coverage.
method Theoretical framework for optimal data splitting in split conformal prediction.
result Analytical characterizations of length-optimal split ratios in various settings.
This paper optimizes high-dimensional oblique splits for decision trees, enhancing performance and computational efficiency.
problem Enhancing decision tree performance and computational efficiency in high-dimensional data.
method Established Sufficient Impurity Decrease (SID) convergence for s0-sparse oblique splits, proposing progressive trees for iterative refinement. result Demonstrated that SID function class expands with s0-sparsity, enabling capture of complex data-generating processes. The study examines how altering impurity functions influences optimal splits in binary classification trees.
problem Understanding how altering impurity functions affects optimal splits in binary classification trees.
method Investigates how skewing impurity functions biases optimal splits towards isolating points of a particular class.
result A necessary and sufficient condition for skewing an impurity function to bias optimal splits towards isolating points of a particular class is provided.
New optimization algorithms for neural networks using operator splitting.
problem Training efficiency and convergence in neural networks.
method Sequential operator splitting technique applied to neural network training.
result Empirical rate of convergence towards local minimum of loss function validated.
MCP extends conformal prediction to vector-valued score functions without data splitting.
problem Fixed prediction set shapes in scalar score functions limit coverage guarantees.
method MCP uses a single optimization problem for prediction set design and calibration, eliminating data splitting.
result RemMCP and RelMCP achieve target coverage with smaller or comparable prediction set sizes, reducing variance.
A new method for growing neural networks by splitting neurons, improving efficiency.
problem Optimizing neural network structures, especially for lightweight architectures.
method A progressive training approach using steepest descent to adaptively grow and split neurons.
result The method provides a computationally efficient way to optimize neural network structures.
Reshuffling splits improves hyperparameter optimization's generalization performance.
problem Improving peak performance of machine learning models through better hyperparameter optimization.
method Reshuffling splits for every hyperparameter configuration improves generalization performance.
result Reshuffling leads to better generalization performance compared to fixed splits.
Optimally estimates a functional using nuisance function tuning and sample splitting.
problem Estimating optimal rates for a doubly robust functional.
method Combines nuisance function tuning and sample splitting strategies.
result Shows optimal rates of convergence for various estimators.
The paper explores how splitting data samples influences optimal neural network hyperparameters.
problem Understanding the effectiveness of neural networks and their hyperparameters.
method Investigates the role of sample splitting in neural network hyperparameter selection.
result Optimal hyperparameters derived from sample splitting lead to a neural network model that minimizes prediction risk asymptotically.
This work improves data reconstruction methods by ensuring unique solutions and refining optimization.
problem Ensuring unique solutions and optimizing reconstruction from KKT conditions.
method Discussion of sufficient conditions for unique solutions and introduction of sample splitting for optimization.
result Sample splitting improves reconstruction performance across various methods.
Improved neural architecture optimization for energy efficiency.
problem Designing energy-efficient deep learning networks for mobile and edge devices.
method Incorporates energy cost in splitting process and uses a scalable stochastic gradient algorithm to speed up the splitting.
result Trains highly accurate and energy-efficient networks on challenging datasets like ImageNet.
The paper introduces a new concept of frame vorticity and uses it to find optimal sections in specific geometric settings.
problem Finding optimal sections in geometric settings using frame vorticity.
method Defining frame vorticity, relating it to split pseudo-Riemannian metrics, and using split special Lagrangian calibrations.
result Explicit homologically volume maximizing sections and optimal sections for specific manifolds.
Twinning splits data into fast, statistically similar sets.
problem Creating statistically similar data splits for Big Data.
method Twinning is a method based on SPlit for fast, model-independent dataset splitting.
result Twinning is orders of magnitude faster than SPlit.
Improves tree model performance by considering future node splits.
problem Improving tree model performance.
method Next-Depth Lookahead Tree (NDLT) model that evaluates future node splits.
result Enhanced tree model performance.
Meta-learning performance depends on train-validation split type.
problem Understanding the importance of train-validation split in meta-learning.
method Theoretical and experimental study comparing train-val and train-train methods.
result Train-train method can achieve strictly better excess loss in realizable cases.
This work introduces a transformation-based learner model for classification forests. The weak learner at each split node plays a crucial role in a classification tree. We propose to optimize the splitting objective by learning a linear transformation on subspaces using nuclear norm as the optimization criteria. The le…
TSSM splits neural networks for parallel training with minimal accuracy loss.
problem Accuracy degradation in parallel training of deep neural networks.
method TSSM reformulates alternating minimization to achieve parallelism with minimal accuracy loss.
result TSSM achieves significant speedup without accuracy loss on multiple datasets.
New approach connects stochastic gradient descent to ODE splitting schemes.
problem Improving convergence in stochastic optimization.
method Connection between stochastic gradient descent and ODE splitting schemes.
result Derive a new upper bound on global splitting error.
New random forest variants achieve optimal performance in high dimensions.
problem Handling dependencies between features in high-dimensional data.
method Using oblique splits in random forests with general split directions.
result Achieved minimax optimal convergence rates in arbitrary dimension.
A new method for kernel tests without data splitting increases power.
problem Lack of power in kernel-based tests due to data splitting.
method Selective inference framework to learn hyperparameters and test on full sample.
result Empirically larger test power without data splitting, regardless of split proportion.
Subagging improves regression tree performance, especially with many splits.
problem Improving regression tree performance with subsample aggregating.
method Formalized bias and variance dependencies, compared subagging to single trees, and analyzed optimal tree sizes.
result Subagging improves tree performance, especially with many splits.
Novel approach for creating interpretable classifiers using bilevel optimization of split-rules in NLDTs.
problem Creating highly accurate and easily interpretable classifiers for practical applications.
method Representing classifiers as assemblies of simple mathematical rules using NLDTs with evolutionary bilevel optimization.
result The approach ensures interpretability while achieving high accuracy on various classification problems.
The paper focuses on the sparse approximation of signals using overcomplete representations, such that it preserves the (prior) structure of multi-dimensional signals. The underlying optimization problem is tackled using a multi-dimensional split Bregman optimization approach. An extensive empirical evaluation shows ho…
Study evaluates when splitting classifiers can improve performance despite disparate treatment.
problem Impact of disparate treatment in classification models.
method Comparison of split classifiers and group-blind classifiers, quantifying performance improvement.
result Proves an equivalent expression for the benefit-of-splitting which can be efficiently computed.
New algorithm solves maximal monotone inclusion problems.
problem Solving maximal monotone inclusion problems.
method Bregman Douglas-Rachford splitting method and variants.
result Convergence of algorithms under certain assumptions.
In-network learning outperforms Federated and Split learning in wireless networks.
problem Efficiently using distributed features for inference in wireless networks.
method Proposes 'in-network learning' architecture, uses neural networks for optimization, compares with Federated and Split learning.
result In-network learning offers better accuracy and bandwidth savings.
We study the Dirac spectrum on compact Riemannian spin manifolds M equipped with a metric connection ∇ with skew torsion T∈Λ3M in the situation where the tangent bundle splits under the holonomy of ∇ and the torsion of ∇ is of `split' type. We prove an optimal lower bound for the first eige…
Paper analyzes nonconvex bandit problems with improved adaptive methods.
problem Continuous armed bandit problems for nonconvex cost functions.
method Simple and adaptive bin splitting methods.
result Adaptive method achieves locally minimax optimal expected cumulative regret.
The classical multi-set split feasibility problem seeks a point in the intersection of finitely many closed convex domain constraints, whose image under a linear mapping also lies in the intersection of finitely many closed convex range constraints. Split feasibility generalizes important inverse problems including con…
A new random forest algorithm improves tree construction for optimal performance.
problem Improving the performance of random forests, especially in complex and smooth scenarios.
method Adaptive split-balancing method using permutation-based splitting criterion.
result Achieves minimax optimality under various Lipschitz and Hölder classes.
We prove a splitting theorem for Riemannian n-manifolds with scalar curvature bounded below by a negative constant and containing certain area-minimising hypersurfaces (Theorem 3). Thus we generalise [25,Theorem 3] by Nunes. This splitting result follows from an area comparison theorem for hypersurfaces with non-positi…
A new approach for efficient data compression in split DNN computing.
problem Optimizing data compression for DNN models split between mobile devices and edge servers.
method Systematic design and training of bottleneck units that can be inserted at the split point.
result Achieves excellent rate-distortion performance with minimal compute and storage overhead.
Novel methods for splitting Gaussian mixtures improve uncertainty propagation in nonlinear systems.
problem Improving accuracy and efficiency in nonlinear uncertainty propagation.
method Preserving mean and covariance, novel heuristics for selecting splitting direction informed by initial uncertainty and nonlinear function properties.
result Improved accuracy and efficiency in uncertainty propagation compared to existing techniques.
We adapt the Douglas-Rachford (DR) splitting method to solve nonconvex feasibility problems by studying this method for a class of nonconvex optimization problem. While the convergence properties of the method for convex problems have been well studied, far less is known in the nonconvex setting. In this paper, for the…
A new method speeds up sampling in diffusion models.
problem Slow sample generation in diffusion models.
method Proposed Splitting Integrators for fast stochastic sampling.
result Achieved FID score of 2.36 in 100 NFE, significantly faster than baselines.
A new algorithm speeds up optimal transport for machine learning.
problem Optimal transport for machine learning with additional terms.
method Forward-backward splitting algorithm based on Bregman distances.
result Significant improvement in speed and performance for domain adaptation.
The most popular approach for analyzing survival data is the Cox regression model. The Cox model may, however, be misspecified, and its proportionality assumption may not always be fulfilled. An alternative approach for survival prediction is random forests for survival outcomes. The standard split criterion for random…
In this work, we propose a deep neural network architecture motivated by primal-dual splitting methods from convex optimization. We show theoretically that there exists a close relation between the derived architecture and residual networks, and further investigate this connection in numerical experiments. Moreover, we…
To reduce the label complexity in Agnostic Active Learning (A^2 algorithm), volume-splitting splits the hypothesis edges to reduce the Vapnik-Chervonenkis (VC) dimension in version space. However, the effectiveness of volume-splitting critically depends on the initial hypothesis and this problem is also known as target…
Two conformal methods create flexible conditional predictive bands without strong assumptions.
problem Achieving asymptotic conditional coverage without strong dependence assumptions.
method Flexible distribution-free conditional predictive bands using density estimators.
result Methods achieve better control of conditional coverage and smaller length than previous methods.
A new method for causal inference in high-dimensional data using machine learning.
problem Causal inference in high-dimensional observational data.
method Support Points Sample Splitting (SPSS) for efficient double machine learning (DML) in causal inference.
result Deep learning with SPSS and hybrid methods outperform SVM with SPSS in computational efficiency and estimation quality.
New algorithms solve nonconvex federated learning problems efficiently.
problem Nonconvex federated composite optimization in federated learning.
method FedDR and asyncFedDR algorithms combining Douglas-Rachford splitting, randomized block-coordinate strategies, and asynchronous implementation.
result Match communication complexity lower bound up to a constant factor.
FIESTA optimizes model selection by efficiently evaluating multiple splits and seeds.
problem Inefficient model selection leading to unreliable performance comparisons.
method Adaptive bandit algorithms to determine optimal number of data splits and random seeds.
result Significantly reduces model evaluations while ensuring correct optimal model selection.
The three operator splitting scheme was recently proposed by [Davis and Yin, 2015] as a method to optimize composite objective functions with one convex smooth term and two convex (possibly non-smooth) terms for which we have access to their proximity operator. In this short note we provide an alternative proof for the…