Partial soft-matching distance improves neural representation comparison by allowing some neurons to remain unmatched.
problem Neural representations are noisy and contain outliers, making traditional matching methods unreliable.
method Extends soft-matching distance to a partial optimal transport setting, allowing some neurons to remain unmatched.
result Partial soft-matching provides robust correspondences that are more reliable under noise and outliers.
End-to-end text-to-speech (TTS) synthesis is a method that directly converts input text to output acoustic features using a single network. A recent advance of end-to-end TTS is due to a key technique called attention mechanisms, and all successful methods proposed so far have been based on soft attention mechanisms. H…
Comparing data defined over space and time is notoriously hard, because it involves quantifying both spatial and temporal variability, while at the same time taking into account the chronological structure of data. Dynamic Time Warping (DTW) computes an optimal alignment between time series in agreement with the chrono…
We propose in this paper a differentiable learning loss between time series, building upon the celebrated dynamic time warping (DTW) discrepancy. Unlike the Euclidean distance, DTW can compare time series of variable size and is robust to shifts or dilatations across the time dimension. To compute DTW, one typically so…
Heterogeneous domain adaptation (HDA) aims to facilitate the learning task in a target domain by borrowing knowledge from a heterogeneous source domain. In this paper, we propose a Soft Transfer Network (STN), which jointly learns a domain-shared classifier and a domain-invariant subspace in an end-to-end manner, for a…
A new framework for averaging spatio-temporal signals using optimal transport and soft alignments.
problem Averaging complex datasets with time and spatial components.
method Inspired by DTW, OT, and UOT, a new loss function is proposed to address shifts in time, space, and population size.
result The proposed loss function can be used to compute spatio-temporal barycenters efficiently.
GSI improves efficiency of large language model inference.
problem Efficiently guiding test-time alignment in large language models.
method Combines soft best-of-n scaling with a reward model and speculative samples. result Achieves higher accuracy and reduced latency compared to standard methods.
EAST aligns neural network classifiers with user-defined evaluation metrics.
problem Mismatch between neural network training and evaluation metrics leads to suboptimal performance.
method EAST uses dynamic thresholding, soft-set confusion matrix, and annealing to align neural network predictions with target evaluation metrics.
result EAST improves alignment between training objectives and evaluation metrics, outperforming existing methods.
New method improves stability of soft FQI for offline RL.
problem Stability issues in soft FQI under function approximation.
method Stationary reweighting to align operator norms.
result Local linear convergence proved under certain conditions.
Soft geometric bias improves physical dynamics predictions.
problem Learning physical dynamics with exact group equivariance can degrade performance.
method Object-centric world models using geometric algebra neural networks.
result Soft geometric inductive bias leads to better physical fidelity predictions.
A new method for unsupervised domain adaptation using manifold learning.
problem Leveraging rich source domain information to target domain without labeled data.
method Discriminative Manifold Embedding and Alignment framework.
result Consistent transferability and discriminability achieved through manifold metric alignment.
Sequence-to-sequence text-to-speech (TTS) is dominated by soft-attention-based methods. Recently, hard-attention-based methods have been proposed to prevent fatal alignment errors, but their sampling method of discrete alignment is poorly investigated. This research investigates various combinations of sampling methods…
Existing deep multitask learning (MTL) approaches align layers shared between tasks in a parallel ordering. Such an organization significantly constricts the types of shared structure that can be learned. The necessity of parallel ordering for deep MTL is first tested by comparing it with permuted ordering of shared la…
Sequence-to-sequence models with soft attention have been successfully applied to a wide variety of problems, but their decoding process incurs a quadratic time and space cost and is inapplicable to real-time sequence transduction. To address these issues, we propose Monotonic Chunkwise Attention (MoChA), which adaptiv…
Deep Neural Network (DNN) acoustic models have yielded many state-of-the-art results in Automatic Speech Recognition (ASR) tasks. More recently, Recurrent Neural Network (RNN) models have been shown to outperform DNNs counterparts. However, state-of-the-art DNN and RNN models tend to be impractical to deploy on embedde…
A novel method for comparing graphs of different sizes using Wasserstein distance.
problem Comparing non-aligned graphs of varying sizes.
method Optimal transport in graph comparison framework, solving a one-to-many assignment problem.
result Significant improvements in graph alignment and classification tasks.
Proposes a framework to improve domain adaptation without labeled data.
problem Improving adaptability and preserving intrinsic data structure in unsupervised domain adaptation.
method Discriminative Manifold Propagation framework using soft labels and manifold metric alignment.
result The method achieves better transferability and discriminability compared to existing approaches.
The abstract explores connections between reinforcement learning, scaling, and diffusion.
problem Aligning reinforcement learning with human feedback and scaling techniques.
method Clarifying connections between reinforcement learning, scaling, and diffusion.
result Introducing a resampling approach for alignment and reward-directed diffusion models.
Fusion of transformer networks using optimal transport for improved performance.
problem Improving performance of transformer-based models through fusion.
method Exploiting optimal transport for soft alignment of transformer components.
result Consistently outperforms vanilla fusion and individual parent models.
Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural ma…
Despite some empirical success at correcting exposure bias in machine translation, scheduled sampling algorithms suffer from a major drawback: they incorrectly assume that words in the reference translations and in sampled sequences are aligned at each time step. Our new differentiable sampling algorithm addresses this…
Neural attention has become central to many state-of-the-art models in natural language processing and related domains. Attention networks are an easy-to-train and effective method for softly simulating alignment; however, the approach does not marginalize over latent alignments in a probabilistic sense. This property …
New method generates plausible counterfactuals for time series classification.
problem Generating realistic counterfactuals for time series data.
method Gradient-based optimization with soft-DTW alignment and multi-faceted loss function.
result Our method outperforms existing approaches in temporal realism and distributional alignment.
Improved inference-time alignment using Best-of-N and smoothing.
problem Reward overoptimization in Best-of-N (BoN) due to poor proxy reward models.
method Introduced Soft Best-of-N (SBoN) and analyzed its performance through KL divergence and regret analysis.
result Smoothing helps SBoN mitigate reward overoptimization, especially when proxy reward quality is low.
A neural architecture learns and refines graph correspondences.
problem Learning structural correspondences between graphs.
method Two-stage neural architecture: node embeddings followed by message passing networks.
result Message passing scheme computes well-founded consensus for corresponding neighborhoods.
New method distills cloud models into edge-friendly ones.
problem Cloud-to-edge model compression with limited data exchange.
method Two-step workflow of deprivatization and distillation.
result Outperforms previous state-of-the-art approaches on various benchmarks.
New metric captures individual neuron tuning across neural networks.
problem Need a metric that respects individual neuron tuning across different neural networks.
method Derived a 'soft' permutation-based metric using optimal transport theory.
result Metric avoids counter-intuitive outcomes and captures geometric insights.
Proposes DFDG for robust domain generalization without source domain labels.
problem Robustness of deep learning models in real-world applications where train and test distributions differ.
method Model-agnostic, class-aware alignment of class relationships through saliency maps.
result Competitive performance on time series sensor and image classification datasets.
SelectMix improves deep learning robustness against noisy labels.
problem Deep neural networks memorize noisy labels, degrading performance.
method Confidence-guided targeted sample mixing with soft labels.
result SelectMix consistently outperforms baseline methods on noisy label datasets.
Monte Carlo methods are widely used in particle physics to integrate and sample probability distributions (differential cross sections or decay rates) on multi-dimensional phase spaces. We present a Neural Network (NN) algorithm optimized to perform this task. The algorithm has been applied to several examples of direc…
Employers actively look for talents having not only specific hard skills but also various soft skills. To analyze the soft skill demands on the job market, it is important to be able to detect soft skill phrases from job advertisements automatically. However, a naive matching of soft skill phrases can lead to false pos…
Optimal control of reserve assets for stablecoins to maintain peg stability.
problem Balancing immediate liquidity and yield on reserve assets for stablecoin peg maintenance.
method Developed a stochastic model predictive control framework with moment closure for event intensities, incorporating a soft-thresholding structure for rebalancing.
result Optimal policy shifts predictably toward cash as expected outflows intensify or windows lengthen, preserving most bill carry in calm markets and quickly building cash during stress.
This study improves knowledge distillation for RNN-T models with noisy labels.
problem Challenges in distilling knowledge from RNN-T models with variable quality teachers.
method Full-sum distillation and sequence-level knowledge distillation.
result Full-sum distillation outperforms other methods for RNN-T models, especially for bad teachers.
Soft labeling impacts OOD detection in neural networks.
problem Impact of soft labeling on OOD detection in deep neural networks.
method Empirical analysis of how soft labeling affects OOD detection performance.
result Soft labeling can either improve or deteriorate OOD detection performance.
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
In this paper, we present Neural Phrase-based Machine Translation (NPMT). Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a recently proposed segmentation-based sequence modeling method. To mitigate the monotonic alignment requirement of SWAN, we introduce a new …
Soft cells fill space without gaps, derived from minimal surfaces and deformed using edge bending.
problem Creating space-filling shapes without sharp corners.
method Edge bending algorithm to deform polyhedral tilings into soft tilings.
result Soft tilings derived from minimal surfaces can be continuously transformed into one another.
ASBART accelerates Soft BART for faster Bayesian regression.
problem Slow computation in Soft BART.
method Proposed ASBART, a variant of Soft BART.
result ASBART is about 10 times faster than Soft BART with similar accuracy.
Maximum entropy deep reinforcement learning (RL) methods have been demonstrated on a range of challenging continuous tasks. However, existing methods either suffer from severe instability when training on large off-policy data or cannot scale to tasks with very high state and action dimensionality such as 3D humanoid l…
We propose JECL, a method for clustering image-caption pairs by training parallel encoders with regularized clustering and alignment objectives, simultaneously learning both representations and cluster assignments. These image-caption pairs arise frequently in high-value applications where structured training data is e…
Generalizes soft noncommutative schemes to flag varieties.
problem Applying soft noncommutative schemes to flag varieties.
method Generalization via toric geometry and distinguished affine charts.
result Soft noncommutative schemes can be applied to flag varieties.
Paper proposes a new loss function for conditional models using soft targets.
problem Improving generalization performance of deep neural networks on supervised classification tasks.
method Introduces a new loss function compatible with soft targets, based on noise contrastive estimation.
result Soft target InfoNCE loss performs on par with cross-entropy baselines and outperforms other losses.
Tutorial on optimizing diffusion model samples for specific metrics.
problem Optimizing diffusion model samples for specific downstream metrics.
method Review and exploration of inference-time guidance and alignment methods.
result Unified perspective on inference-time algorithms and novel methods.
Introduces Soft-SVM for binary classification bridging logistic and SVM.
problem Data separability issues in binary classification.
method Soft-SVM regression using convex relaxation of hinge loss with softness and class-separation parameters.
result Soft-SVM performs well in classification and prediction errors.
Deep model learns from labeled and unlabeled data for industrial soft sensing.
problem Lack of data limits soft sensor development in industrial processes.
method Hierarchical, generative deep latent variable model for semi-supervised multi-unit soft sensing.
result Model outperforms current methods for soft sensing in industrial processes.
New method constructs Floer homologies without hard analysis.
problem Constructing homological invariants in infinite dimensions.
method Soft, finite-dimensional tools to avoid hard analysis.
result Demonstrates construction of Floer homologies without PDE.
This paper introduces TNTK to study infinite soft tree ensembles.
problem Understanding the behavior of infinite soft tree ensembles.
method Introduced Tree Neural Tangent Kernel (TNTK) to analyze infinite soft tree ensembles.
result Identified several non-trivial properties of infinite soft tree ensembles.
Improves deep neural networks using soft labels through alternating minimization.
problem Improving deep neural networks training with soft labels.
method Co-Learns DNNs and soft labels via Alternating Minimization of two objectives.
result COLAM achieves improved performance on many tasks with better testing classification accuracy.