A new type of distributional regression tree uses soft split rules for better predictive performance.
problem Estimating complete conditional distributions in regression.
method Distributional adaptive soft regression trees using multivariate soft split rules.
result The method outperforms various benchmark methods, especially in complex non-linear interactions.
SBAMDT uses adaptive soft splits to model complex decision boundaries.
problem Limited ability of standard decision trees to capture complex decision boundaries.
method Probabilistic additive decision tree model with adaptive soft multivariate splits.
result Demonstrated improved predictive performance on synthetic and real datasets.
This paper introduces TNTK to study infinite soft tree ensembles.
problem Understanding the behavior of infinite soft tree ensembles.
method Introduced Tree Neural Tangent Kernel (TNTK) to analyze infinite soft tree ensembles.
result Identified several non-trivial properties of infinite soft tree ensembles.
Rectified decision trees improve machine learning interpretability and effectiveness.
problem Combining interpretability and effectiveness in machine learning models.
method Knowledge distillation and modified decision tree splitting criteria.
result Soft labels improve model performance and reduce model size.
Sequence-to-sequence models with soft attention have been successfully applied to a wide variety of problems, but their decoding process incurs a quadratic time and space cost and is inapplicable to real-time sequence transduction. To address these issues, we propose Monotonic Chunkwise Attention (MoChA), which adaptiv…
Paper analyzes soft tree ensembles using NTK, finding only leaf count matters.
problem Understanding impact of various tree architectures in ensemble learning.
method Formulated and analyzed Neural Tangent Kernel (NTK) for soft tree ensembles.
result Only the number of leaves at each depth is relevant for tree architecture in ensemble learning.
In Domain Adaptation (DA), the category-relevant losses usually occupy a dominant position, while they are usually built with hard or soft labels in existing models. We observed that hard labels are overconfident due to hard samples existed, and soft labels are ambiguous as too many small noisy probabilities involved, …
Study improves fair opinion aggregation by balancing voter attributes.
problem Aggregation of opinions can be biased by voter attributes.
method Combines majority voting and D&S model with fairness options.
result Effective combination of Soft D&S and fairness options for different data types.
A new adaptive splitting method improves accuracy for Cox-Ingersoll-Ross model.
problem Improving numerical solution accuracy for Cox-Ingersoll-Ross model.
method Adaptive splitting method over deterministic and random meshes, with uniform moment bound and strong error results.
result Uniform moment bound and strong error results of order 1/4 in L1 and L2 for κθ>σ^2, and order 1 for large noise.
Motivated by recently published methods using frequency decompositions of convolutions (e.g. Octave Convolutions), we propose a novel convolution scheme to stabilize the training and reduce the likelihood of a mode collapse. The basic idea of our approach is to split convolutional filters into additive high and low fre…
We discuss an autoencoder model in which the encoding and decoding functions are implemented by decision trees. We use the soft decision tree where internal nodes realize soft multivariate splits given by a gating function and the overall output is the average of all leaves weighted by the gating values on their path. …
The paper introduces a new model to correct bias in treatment effect estimates due to sample selection.
problem Bias in treatment effect estimates due to sample selection.
method Type 2 Tobit Bayesian Additive Regression Trees (TOBART-2) with Dirichlet Process Mixture distribution and soft trees.
result Corrects bias in treatment effect estimates by accounting for nonlinearities and model uncertainty.
Sparse coding is a core building block in many data analysis and machine learning pipelines. Typically it is solved by relying on generic optimization techniques, such as the Iterative Soft Thresholding Algorithm and its accelerated version (ISTA, FISTA). These methods are optimal in the class of first-order methods fo…
Sparse reconstruction approaches using the re-weighted l1-penalty have been shown, both empirically and theoretically, to provide a significant improvement in recovering sparse signals in comparison to the l1-relaxation. However, numerical optimization of such penalties involves solving problems with l1-norms in the ob…
A new MMD-based test combines kernels for two-sample testing without splitting data.
problem Efficiently testing if two datasets come from the same distribution without splitting data.
method Proposes a novel statistic based on Maximum Mean Discrepancy (MMD) that combines kernels, proving concentration bounds and showing data-dependent kernel selection.
result Exponential concentration bounds and improved test power compared to existing methods.
Employers actively look for talents having not only specific hard skills but also various soft skills. To analyze the soft skill demands on the job market, it is important to be able to detect soft skill phrases from job advertisements automatically. However, a naive matching of soft skill phrases can lead to false pos…
We present an algorithm for learning decision trees using stochastic gradient information as the source of supervision. In contrast to previous approaches to gradient-based tree learning, our method operates in the incremental learning setting rather than the batch learning setting, and does not make use of soft splits…
Soft labeling impacts OOD detection in neural networks.
problem Impact of soft labeling on OOD detection in deep neural networks.
method Empirical analysis of how soft labeling affects OOD detection performance.
result Soft labeling can either improve or deteriorate OOD detection performance.
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
Soft cells fill space without gaps, derived from minimal surfaces and deformed using edge bending.
problem Creating space-filling shapes without sharp corners.
method Edge bending algorithm to deform polyhedral tilings into soft tilings.
result Soft tilings derived from minimal surfaces can be continuously transformed into one another.
ASBART accelerates Soft BART for faster Bayesian regression.
problem Slow computation in Soft BART.
method Proposed ASBART, a variant of Soft BART.
result ASBART is about 10 times faster than Soft BART with similar accuracy.
Maximum entropy deep reinforcement learning (RL) methods have been demonstrated on a range of challenging continuous tasks. However, existing methods either suffer from severe instability when training on large off-policy data or cannot scale to tasks with very high state and action dimensionality such as 3D humanoid l…
New method improves stability of soft FQI for offline RL.
problem Stability issues in soft FQI under function approximation.
method Stationary reweighting to align operator norms.
result Local linear convergence proved under certain conditions.
Paper proposes a new method to stabilize noisy gradient algorithms.
problem Stochastic-gradient Langevin algorithms can introduce bias when taming denominators depend on stochastic-gradient realizations.
method Proposes a structure-preserving framework for designing tamed denominators that avoid unnecessary taming and maintain the stabilizing effect of taming.
result The method avoids stationary bias and explains the stationary error split into bias and remaining error.
Generalizes soft noncommutative schemes to flag varieties.
problem Applying soft noncommutative schemes to flag varieties.
method Generalization via toric geometry and distinguished affine charts.
result Soft noncommutative schemes can be applied to flag varieties.
Paper proposes a new loss function for conditional models using soft targets.
problem Improving generalization performance of deep neural networks on supervised classification tasks.
method Introduces a new loss function compatible with soft targets, based on noise contrastive estimation.
result Soft target InfoNCE loss performs on par with cross-entropy baselines and outperforms other losses.
Introduces Soft-SVM for binary classification bridging logistic and SVM.
problem Data separability issues in binary classification.
method Soft-SVM regression using convex relaxation of hinge loss with softness and class-separation parameters.
result Soft-SVM performs well in classification and prediction errors.
Deep model learns from labeled and unlabeled data for industrial soft sensing.
problem Lack of data limits soft sensor development in industrial processes.
method Hierarchical, generative deep latent variable model for semi-supervised multi-unit soft sensing.
result Model outperforms current methods for soft sensing in industrial processes.
Improves deep neural networks using soft labels through alternating minimization.
problem Improving deep neural networks training with soft labels.
method Co-Learns DNNs and soft labels via Alternating Minimization of two objectives.
result COLAM achieves improved performance on many tasks with better testing classification accuracy.
The paper develops a multi-unit soft sensing model for virtual flow meters that improves few-shot learning.
problem Improving soft sensor performance for new wells with limited data.
method Formulates a probabilistic, hierarchical model using a deep neural network for multi-unit soft sensing.
result Multi-unit models trained on many wells can perform well on new wells with just a few data points.
In this article supervised learning problems are solved using soft rule ensembles. We first review the importance sampling learning ensembles (ISLE) approach that is useful for generating hard rules. The soft rules are then obtained with logistic regression from the corresponding hard rules. In order to deal with the p…
Recently, deep learning becomes the main focus of machine learning research and has greatly impacted many important fields. However, deep learning is criticized for lack of interpretability. As a successful unsupervised model in deep learning, the autoencoder embraces a wide spectrum of applications, yet it suffers fro…
Proposes a new method for deep ensembles that improves accuracy and calibration.
problem Improving accuracy and calibration of deep ensembles.
method Estimates confusion matrices of ensemble members and weighs them according to their inferred performance.
result Empirically shows superiority of soft Dawid Skene over ensemble averaging.
CoCP optimizes prediction intervals by jointly learning center and radius, improving efficiency and coverage.
problem Inefficient conformal prediction intervals under heteroscedasticity and skewness.
method Co-optimization framework that learns center and radius through alternating optimization steps.
result CoCP yields consistently shorter intervals and state-of-the-art conditional coverage diagnostics.
Bayesian neural networks update beliefs with soft evidence, improving accuracy and calibration.
problem Updating neural network weights with uncertain or soft evidence.
method Developed two algorithms to approximate Jeffrey's rule for updating neural network weights.
result Jeffrey-based methods outperform traditional approaches in accuracy and calibration, especially in noisy data.
A method for selecting pseudo-labeled data in semi-supervised learning using generalized Bayes and soft revision.
problem Selecting pseudo-labeled data for semi-supervised learning with robustness to uncertainty.
method Using credal sets and the Gamma-Maximin method with soft revision to update priors and select pseudo-labeled data.
result The Gamma-Maximin method with soft revision can achieve promising results, especially in scenarios with low labeled data proportions.
Floer homology is a good example of homological invariants living in the infinite dimension. We suggest a way to construct this kind of invariants using only soft essentially finite-dimensional tools; no hard analysis or PDE is involved. This work is partially inspired by the M. Gromov's survey ``Soft and hard symplect…
MSLG generates soft labels to improve DNN performance on noisy datasets.
problem Significant performance degradation of DNNs due to noisy labels.
method Meta-learning techniques to estimate optimal label distribution and iteratively update soft labels.
result MSLG outperforms state-of-the-art methods by a large margin on various datasets.
Constructs noncommutative spaces for D-branes on complex algebraic spaces.
problem Mathematical model for D-branes on noncommutative spaces.
method Toric geometry, Azumaya schemes, invertible sheaves.
result Embeds algebraic Calabi-Yau spaces into soft noncommutative schemes.
In industrial systems, certain process variables that need to be monitored for detecting faults are often difficult or impossible to measure. Soft sensor techniques are widely used to estimate such difficult-to-measure process variables from easy-to-measure ones. Soft sensor modeling requires training datasets includin…
HyperFair integrates fairness in recommender systems using probabilistic soft logic.
problem Ensuring fairness in recommender systems across diverse domains.
method Soft fairness constraints integrated as regularization in a joint inference objective function.
result HyperFair improves fairness of predictions from black-box models and hybrid systems.
Sparse coding is a core building block in many data analysis and machine learning pipelines. Typically it is solved by relying on generic optimization techniques, that are optimal in the class of first-order methods for non-smooth, convex functions, such as the Iterative Soft Thresholding Algorithm and its accelerated …
Partial soft-matching distance improves neural representation comparison by allowing some neurons to remain unmatched.
problem Neural representations are noisy and contain outliers, making traditional matching methods unreliable.
method Extends soft-matching distance to a partial optimal transport setting, allowing some neurons to remain unmatched.
result Partial soft-matching provides robust correspondences that are more reliable under noise and outliers.
A framework learns dynamic soft labels to improve model generalization and accuracy.
problem Models trained on one-hot labels overfit and are sensitive to noisy annotations.
method Proposes a framework where labels are treated as learnable parameters, adapting dynamically during optimization.
result Consistent gains across different datasets and architectures, improving ResNet18 by 2.1% on CIFAR100.
Recently proposed budding tree is a decision tree algorithm in which every node is part internal node and part leaf. This allows representing every decision tree in a continuous parameter space, and therefore a budding tree can be jointly trained with backpropagation, like a neural network. Even though this continuity …
New algorithms optimize a soft-robust criterion in reinforcement learning, reducing conservatism.
problem Computing robust policies for high-stakes decisions with limited data.
method Soft-robust criterion using risk measures, two algorithms for optimization.
result Our algorithms produce less conservative solutions than existing methods.
We comment on the fact that gradient ascent for logistic regression has a connection with the perceptron learning algorithm. Logistic learning is the "soft" variant of perceptron learning.
Soft geometric bias improves physical dynamics predictions.
problem Learning physical dynamics with exact group equivariance can degrade performance.
method Object-centric world models using geometric algebra neural networks.
result Soft geometric inductive bias leads to better physical fidelity predictions.