End-to-end TTS framework uses hard alignment to improve accuracy.
problem End-to-end TTS systems struggle with accurate alignment between input text and output acoustic features.
method Proposes a constrained alignment scheme with hard monotonic alignments, marginalized during training.
result Improves alignment learning and prediction in end-to-end TTS systems.
The paper explores the difficulty of robust machine learning models.
problem Understanding the vulnerability of machine learning models to adversarial attacks.
method The study uses computational learning theory to analyze the feasibility of robust learning from both sample and computational complexity perspectives.
result No non-trivial concept class can be robustly learned in the distribution-free setting against a single-bit adversary, and the class of monotone conjunctions cannot be robustly learned under the uniform distribution against an adversary that can perturb ω(logn) bits. New work shows FP potential monotonicity equals low-degree polynomial estimators limits.
problem Establishing a precise mathematical relationship between statistical physics and polynomial estimators limits.
method Analyzing Gaussian additive models (GAMs) to show FP potential monotonicity equals low-degree polynomial estimators limits.
result For a broad family of Gaussian additive models, the power of low-degree polynomials is equivalent to the monotonicity of the annealed FP potential.
Article addresses practical challenges in conformal prediction.
problem Challenges in determining, computing, and controlling conformal prediction regions.
method Proposes a quadratic-polynomial non-conformity measure.
result Allows circumventing three challenges in full conformal prediction framework.
The monotonic linear interpolation in deep networks often leads to plateaus, revealing biases in optimization.
problem Plateaus in the optimization landscape of deep networks during monotonic linear interpolation.
method Investigated monotonic linear interpolation on deep neural networks, focusing on biases in weights and biases.
result Interpolating weights and biases differently can lead to significant differences in loss and accuracy, revealing biases in optimization.
Curriculum Learning - the idea of teaching by gradually exposing the learner to examples in a meaningful order, from easy to hard, has been investigated in the context of machine learning long ago. Although methods based on this concept have been empirically shown to improve performance of several learning algorithms, …
Paper tackles hard shape constraints in kernel machines.
problem Enforcing shape requirements in a hard fashion is challenging.
method Tightened second-order cone constrained reformulation for kernel machines.
result Performance guarantees and efficiency demonstrated in various applications.
Submodular functions have many applications. Matchings have many applications. The bitext word alignment problem can be modeled as the problem of maximizing a nonnegative, monotone, submodular function constrained to matchings in a complete bipartite graph where each vertex corresponds to a word in the two input senten…
Trivial links are unique up to number of link components, but they can be hard to recognize from arbitrary diagrams. We define a new measure of the complexity of a link embedding, the crumple, and show how this may be used to measure progress toward a trivial embedding. In conjunction with a modified form of arc presen…
Unified framework for hard affine SDP constraints in vRKHSs.
problem Incorporating shape constraints into predictive models for rich function classes.
method Unified convex optimization framework using second-order cone tightening.
result Unified and modular approach for handling multiple shape constraints.
Applications in machine learning, optimization, and control require the sequential selection of a few system elements, such as sensors, data, or actuators, to optimize the system performance across multiple time steps. However, in failure-prone and adversarial environments, sensors get attacked, data get deleted, and a…
Optimal intervention in economic networks modeled as influence maximization, with hard computational problems.
problem Optimal intervention in economic networks modeled as influence maximization.
method Transformed into influence maximization-like form, with theoretical and practical implications.
result Optimal intervention is NP-hard and cannot be approximated to a constant factor in polynomial time.
Optimizes hard-to-optimize metrics using adaptive surrogates.
problem Training models with black-box and hard-to-optimize metrics.
method Expresses metric as a function of surrogates, solves optimization problem over relaxed surrogate space.
result Approach performs on par with known methods and adds value when metric form is unknown.
Improves GANs training through game theory.
problem Hard training of GANs due to antagonistic networks.
method Rewrote GAN training as a variational inequality and introduced a stochastic relaxed forward-backward algorithm.
result Algorithm converges to an exact solution or a neighborhood of it under monotonicity.
Derives a new formula for optimal stopping problems with exploding derivatives.
problem Optimal stopping problems with complex boundary conditions.
method Develops a change of variable formula for functions with exploding derivatives near a surface.
result Derives a formula similar to Itô's but with less restrictive conditions.
Paper proposes DG-ETC for online submodular maximization with stochastic bandit feedback.
problem Online unconstrained submodular maximization with stochastic bandit feedback.
method Double-Greedy - Explore-then-Commit (DG-ETC) approach.
result DG-ETC achieves logarithmic regret O(dlog(dT)) for 1/2-approximate pseudo-regret. A key problem in reinforcement learning for control with general function approximators (such as deep neural networks and other nonlinear functions) is that, for many algorithms employed in practice, updates to the policy or Q-function may fail to improve performance---or worse, actually cause the policy performance …
There is a growing body of literature showing that deep neural networks are vulnerable to adversarial input modification. Recently this work has been extended from image classification to malware classification over boolean features. In this paper we present several new methods for training restricted networks in this …
Boosted GFlowNets improve exploration by sequentially training GFlowNets with residual rewards.
problem GFlowNets struggle to evenly explore reward landscapes, leading to poor coverage of high-reward areas.
method Sequential training of an ensemble of GFlowNets, each optimizing a residual reward.
result Boosted GFlowNets achieve better exploration and sample diversity on multimodal benchmarks and peptide design tasks.
New equivalences found between subsampling and ridge regularization methods.
problem Establishing precise structural and risk equivalences between subsampling and ridge regularization.
method Proved structural and risk equivalences between subsample ridge estimators and different ridge regularization levels and subsample aspect ratios.
result Optimally tuned ridge regression exhibits a monotonic prediction risk in the data aspect ratio.
Applications of safety, security, and rescue in robotics, such as multi-robot target tracking, involve the execution of information acquisition tasks by teams of mobile robots. However, in failure-prone or adversarial environments, robots get attacked, their communication channels get jammed, and their sensors may fail…
This paper considers stochastic optimization problems for a large class of objective functions, including convex and continuous submodular. Stochastic proximal gradient methods have been widely used to solve such problems; however, their applicability remains limited when the problem dimension is large and the projecti…
We describe k-MLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases the incomplete (expec…
New method finds rare dense clusters in asymmetric binary perceptrons, resolving algorithmic hardness.
problem Resolving algorithmic hardness in asymmetric binary perceptrons.
method Fully lifted random duality theory (fl RDT) and large deviation upgrade (sfl LD RDT).
result Local entropy breaks down for constraint densities in (0.77, 0.78) interval, matching current solver limits.
Study on sequential defaulting in financial networks, analyzing stability and optimal timing.
problem Understanding which banks default and how much they can fulfill in a sequential financial network.
method Sequential model of financial networks, analyzing stability and optimal timing of defaults.
result Stabilization time can heavily depend on the ordering of announcements, and finding the best time for default is NP-hard.
Optimizes classifiers for varying levels of automation.
problem Supervised learning models often perform worse than human experts on specific instances.
method Focuses on convex margin-based classifiers, showing the problem is NP-hard. For SVMs, the objective function is decomposed into monotone and modular components, allowing efficient algorithms to solve the problem.
result The approach demonstrates that classifiers optimized for varying levels of automation can outperform full automation and human-only models.
We solve ReLU regression with efficient approximations for various distributions.
problem Finding the best fitting ReLU function with square loss from unknown distributions.
method Introduced efficient constant-factor approximation algorithm and polynomial-time approximation scheme.
result First constant-factor approximation algorithm for ReLU regression with weak concentration conditions.
The control and sensing of large-scale systems results in combinatorial problems not only for sensor and actuator placement but also for scheduling or observability/controllability. Such combinatorial constraints in system design and implementation can be captured using a structure known as matroids. In particular, the…
The paper addresses monotonicity in machine learning models for fairness and accountability.
problem Ensuring fairness and accountability in transparent machine learning models.
method Study of three types of monotonicity (individual, weak pairwise, strong pairwise) and propose monotonic groves of neural additive models.
result Monotonic groves of neural additive models maintain transparency, accountability, and fairness.
The paper tackles non-monotonic learning performance and proposes algorithms to make models more monotone.
problem Non-monotonic learning performance where more data does not always improve model quality.
method Proposes three algorithms to make supervised learning models more monotone, proving consistency and monotonicity with high probability.
result The algorithm MT-HT reduces less than 1% non-monotonic decisions on MNIST while maintaining competitive error rates.
Probit Monotone BART estimates binary outcomes using monotonic functions.
problem Estimating conditional mean functions for binary outcomes with monotonicity constraints.
method Proposes a new BART variant that incorporates monotonicity constraints for binary outcomes.
result Allows for more precise estimation of monotonic functions in binary outcome models.
Monotone neural networks can approximate and interpolate functions efficiently.
problem Understanding the efficiency and expressiveness of monotone neural networks.
method Solving the monotone interpolation problem using depth-4 networks and comparing size bounds with arbitrary networks.
result Monotone neural networks can approximate and interpolate functions efficiently, but may require exponential size in high dimensions.
GD at EoS edge minimizes logistic loss without monotonic convergence.
problem Understanding GD's implicit bias at the edge of stability.
method Theoretical analysis of logistic regression with constant stepsize GD.
result GD with any constant stepsize minimizes logistic loss over long time scales.
Improves k-NN for monotonic data with robustness against noise.
problem Class noise in real-life data violates monotonic constraints in k-NN.
method Monotonic Fuzzy k-NN (MonFkNN) with new fuzzy membership calculation.
result Significant accuracy improvements and robustness against monotonic noise.
We examine gradient descent on unregularized logistic regression problems, with homogeneous linear predictors on linearly separable datasets. We show the predictor converges to the direction of the max-margin (hard margin SVM) solution. The result also generalizes to other monotone decreasing loss functions with an inf…
Study examines explainable machine learning for monotonic models, finding Integrated gradients better for strong monotonicity.
problem Applying explainable machine learning to science-informed models.
method Proposed axioms for monotonicity, tested Shapley value and Integrated gradients methods.
result Integrated gradients provides better explanations for strong monotonicity.
Formula proves monotonicity for anisotropic minimal hypersurfaces.
problem Understanding anisotropic minimal hypersurfaces.
method Proved a monotonicity formula under a sign assumption on the Minkowski norm.
result Monotonicity formula for anisotropic minimal hypersurfaces.
The paper develops algorithms to restore monotonicity in non-monotone functions.
problem Non-monotone solutions from heuristic algorithms need to be corrected.
method Develops algorithms to restore monotonicity with limited queries.
result Restores monotonicity while degrading the function value by at most ε.
New example of manifolds with monotonic heat kernels found.
problem Understanding monotonicity of heat kernels on manifolds.
method Analyzing new examples and classifying flat tori.
result Generic metrics fail monotonicity at large times.
Nonnegative matrix factorization (NMF) factorizes a non-negative matrix into product of two non-negative matrices, namely a signal matrix and a mixing matrix. NMF suffers from the scale and ordering ambiguities. Often, the source signals can be monotonous in nature. For example, in source separation problem, the source…
In [S. Basu, A. Gabrielov, N. Vorobjov, Semi-monotone sets. arXiv:1004.5047v2 (2011)] we defined semi-monotone sets, as open bounded sets, definable in an o-minimal structure over the reals, and having connected intersections with all translated coordinate cones in R^n. In this paper we develop this theory further by d…
COMET learns monotonic neural networks by incorporating counterexamples.
problem Enforcing monotonicity constraints in neural networks for real-world tasks.
method Counterexample-guided learning technique for ReLU neural networks.
result COMET achieves state-of-the-art results and improves model quality.
A new method for fitting mixture models using Boltzmann exploration.
problem Challenges in learning mixture models, especially with good initialisation.
method Boltzmann exploration expectation-maximisation (BEEM) algorithm.
result BEEM can escape local optima and is insensitive to parameter initialisation.
Monotonic differentiable sorting networks improve upon previous methods.
problem Non-monotonicity in differentiable sorting networks.
method Relaxation of conditional swap operations using sigmoid functions to ensure monotonicity.
result Monotonic differentiable sorting networks improve upon previous methods.
We prove three new monotonicity formulas for manifolds with a lower Ricci curvature bound and show that they are connected to rate of convergence to tangent cones. In fact, we show that the derivative of each of these three monotone quantities is bounded from below in terms of the Gromov-Hausdorff distance to the neare…
Proves monotonicity of parabolic frequency on all manifolds without curvature assumptions.
problem Monotonicity of parabolic frequency on manifolds.
method Analyzes parabolic frequency function on manifolds, proving monotonicity without curvature assumptions.
result Monotonicity of parabolic frequency on all manifolds, no curvature assumption needed.
Paper introduces algorithms for explaining monotonic classifiers.
problem Need for explanations of monotonic classifiers.
method Polynomial algorithms for formal explanations of monotonic classifiers.
result Efficient model-agnostic algorithm for enumerating explanations.
Develops monotone tree-based GAMI models using XGBoost.
problem Incorporating monotonicity into GAMI models based on boosted trees.
method Adapting XGBoost algorithm to fit monotone GAMI-Tree models, filtering interactions, and parsing results.
result Fits monotone GAMI-Tree models that are directly interpretable and visualizable.