Proposes FCNs for text classification with size-invariant inputs.
problem Classifying text of varying sizes.
method Uses fully convolutional networks with modifications to attention mechanisms.
result Suboptimal results on ITAmoji task, proposes fixes.
We make policy optimization algorithms batch size-invariant by decoupling proximal and behavior policies.
problem Some policy optimization algorithms do not have batch size-invariance, leading to inefficiencies.
method We decouple the proximal policy from the behavior policy to achieve batch size-invariance.
result Our approach makes policy optimization algorithms more efficient and allows them to use stale data more effectively.
Paper analyzes FTPL's effectiveness in combinatorial semi-bandit problems.
problem Optimizing FTPL policy in combinatorial semi-bandit problems.
method Geometric resampling (GR) and conditional geometric resampling (CGR) for FTPL in semi-bandit setting.
result FTPL achieves optimal regret bounds in both Fréchet and Pareto distributions.
Develops a new model to predict training dynamics of large language models.
problem Lack of mechanistic understanding of training dynamics in large language models.
method A first-principles reduced-order model of training dynamics, predicting group-size invariance and stability thresholds.
result Closed-form model predicts training dynamics with high accuracy and provides new diagnostics.
The contour maps of the error of historical resp. parametric estimates for large random portfolios optimized under the risk measure Expected Shortfall (ES) are constructed. Similar maps for the sensitivity of the portfolio weights to small changes in the returns as well as the VaR of the ES-optimized portfolio are also…
ETC improves Transformer models for long and structured inputs.
problem Scaling input length and encoding structured inputs in Transformers.
method Introduces global-local attention, relative position encodings, and CPC pre-training.
result Achieves state-of-the-art results on four natural language datasets.
Unified method for input, data, and model uncertainty in neural networks.
problem Uncertainty in neural network inputs and outputs.
method Propagating uncertainty through inputs using a unified formulation.
result More stable decision boundaries with input noise, and propagation of input uncertainty to model outputs.
New model evaluates how well models handle input faults.
problem Fault tolerance of models to input variations.
method Evaluates fault tolerance using information-based characteristic for arbitrary valid inputs.
result Proposes a new way to measure model robustness.
Improves sample efficiency in reinforcement learning with input representation.
problem Poor sample efficiency in reinforcement learning.
method Attention-based method to project inputs into an invariant representation space.
result Representation space is m! smaller for inputs of m objects, improving sample efficiency. Partial-input models fail to detect dataset artifacts, even when they perform poorly.
problem The effectiveness of partial-input models in detecting dataset artifacts is questionable.
method Design artificial datasets and identify trivial patterns in the SNLI dataset.
result Partial-input models can solve examples previously considered hard, indicating potential dataset artifacts.
New methods handle uncertainty in identifying input regions for a black-box function.
problem Handling uncertainty in identifying input regions for a black-box function.
method Introduce a basic framework and efficient methods for LSE under input uncertainty.
result Proposed methods can be applied to various LSE challenges under input uncertainty.
Detects unusual inputs to neural networks to prevent flawed predictions.
problem Erratic predictions from neural networks on unexpected inputs.
method Evaluates input unusualness by comparing its content to learned parameters.
result Simple, effective method for comparing input metrics across different scales.
Enhances construction input modeling with Bayesian deep neural networks.
problem Deriving reliable simulation input models from construction data.
method Bayesian deep neural networks integrated with multi-source construction data.
result Derives detailed input models for construction operations.
TVS-FNNs can approximate any continuous function on expanded input spaces.
problem Processing a broader range of inputs like sequences and matrices.
method Proving a universal approximation theorem for TVS-FNNs.
result TVS-FNNs can approximate any continuous function on expanded input spaces.
We consider reinforcement learning in input-driven environments, where an exogenous, stochastic input process affects the dynamics of the system. Input processes arise in many applications, including queuing systems, robotics control with disturbances, and object tracking. Since the state dynamics and rewards depend on…
A new method builds sparse polynomial chaos expansions for models with dependent inputs.
problem Quantifying uncertainty in models with dependent inputs.
method Data-driven approach to construct orthonormal polynomials recursively based on input correlations.
result Reduces the number of observations and improves numerical stability and computational efficiency.
Improves AI system's understanding of human inputs by creating better examples.
problem AI systems misinterpret human inputs, leading to inefficiencies.
method Developed a conditional convolutional autoencoder (CCAE) to generate better examples.
result Generated examples lead to lower error rates and require less effort to create.
Validates neural networks inputs to protect against adversarial examples.
problem Ensuring neural networks robustness against adversarial attacks.
method Runtime local robustness verification based on normal distribution of robustness radii.
result Improves neural network accuracy and protects against adversarial examples.
Centroid Transformers reduce memory and computation by summarizing inputs into centroids.
problem Efficiently summarize inputs with reduced memory and computation.
method Generalizes self-attention to map N inputs to M centroids (M ≤ N), reducing complexity.
result Centroid Transformers reduce memory and computation while preserving key information.
MINs learn inverse mappings for high-dimensional optimization problems.
problem Data-driven optimization with high-dimensional inputs and valid subsets.
method Model Inversion Networks (MINs) learn an inverse mapping from scores to inputs.
result MINs can scale to high-dimensional input spaces and handle both offline and active data.
Methods for combining predictions from different models in a supervised learning setting must somehow estimate/predict the quality of a model's predictions at unknown future inputs. Many of these methods (often implicitly) make the assumption that the test inputs are identical to the training inputs, which is seldom re…
IA-BMA adapts model weights to inputs for better predictions.
problem Predicting with multiple models in heterogeneous settings.
method Input adaptive Bayesian Model Averaging (IA-BMA) with an input adaptive prior and amortized variational inference.
result IA-BMA consistently delivers more accurate and better-calibrated predictions.
Paper tackles machine performance testing under uncertain inputs.
problem Guarantee machine performance under input uncertainty.
method Formulates as IU-rLSE problem, proposes active learning method.
result Efficient algorithm for reliable level set estimation.
Framework verifies global correctness of neural networks for perception tasks.
problem Verifying robustness of neural networks is insufficient; global correctness needs to be ensured.
method Specified a state space and observation process to define the target input space. Tiled the spaces and compared ground truth and network output bounds to deliver error bounds.
result Framework can verify error bounds globally over the target input space and detect illegal inputs.
AIRBO optimizes robustly under uncertain inputs.
problem Challenges in Bayesian Optimization due to input uncertainty.
method AIRBO models arbitrary input uncertainty with MMD and accelerates posterior inference.
result AIRBO achieves state-of-the-art performance under various input uncertainties.
When simulating a complex stochastic system, the behavior of output response depends on input parameters estimated from finite real-world data, and the finiteness of data brings input uncertainty into the system. The quantification of the impact of input uncertainty on output response has been extensively studied. Most…
Generalizes memory and forecasting capacities for nonlinear recurrent networks with dependent inputs.
problem Understanding memory and forecasting capabilities in networks with dependent inputs.
method Formulated bounds for memory and forecasting capacities in terms of network size and input properties.
result Proved that memory capacity for linear recurrent networks with independent inputs is given by the rank of the controllability matrix.
New algorithm learns two-layer neural networks under symmetric input distributions.
problem Learning two-layer neural networks with symmetric inputs.
method Method-of-moments framework, spectral algorithms.
result Guaranteed to recover parameters of ground-truth network under certain conditions.
Transforms input design for probabilistic models into optimal control of a Hamiltonian system.
problem Designing inputs for probabilistic models with intractable posterior distributions.
method Representing posterior as Hamiltonian system trajectories, solving optimal control problem.
result Parameter posterior concentrates around true parameter values.
SpinSVAR estimates SVAR models with sparse input, improving accuracy and scalability.
problem Estimating SVAR models with sparse input assumptions.
method SpinSVAR models input as independent Laplacian variables, enforcing sparsity and using least absolute error regression.
result SpinSVAR outperforms state-of-the-art methods in accuracy and runtime, identifying significant structural shocks.
ScieNet improves deep learning resilience to input perturbations.
problem Deep learning's poor resilience to input perturbations in real-world scenarios.
method Hybrid architecture combining SNN for contextual info extraction and DNN for classification.
result Significant improvement in accuracy on noisy and rainy images without prior training.
This research analyzes how input and output layers affect deep neural networks' resistance to adversarial attacks.
problem The vulnerability of deep neural networks to adversarial inputs, especially non-gradient based attacks.
method Analysis of three different fully connected dense network classes with manipulated input and output layers.
result Manipulating input and output layers can significantly enhance a deep neural network's robustness against adversarial attacks.
New method distinguishes predictive distribution estimators in high-dimensional inputs.
problem Difficulty in evaluating predictive distributions for high-dimensional inputs.
method Introduces dyadic sampling to focus on predictive distributions associated with pairs of inputs.
result Demonstrates efficient distinction of predictive distribution estimators in high-dimensional examples.
Input-cell attention improves RNN saliency detection over time.
problem Vanishing saliency in RNNs makes them unreliable for detecting important features at arbitrary time intervals.
method Proposes input-cell attention, a novel RNN cell structure that uses a fixed-size matrix embedding to attend to different inputs from current or previous time steps.
result Input-cell attention produces a saliency map that can detect important features regardless of their occurrence in time.
New method certifies neural network robustness under random input noise.
problem Certifying neural network robustness against random input noise.
method Chance-constrained optimization problem reformulated with input-output samples, convex conditions developed.
result Proposed method certifies robustness against various input noise regimes over larger uncertainty regions.
Dropout improves GAN outputs by preserving input constraints.
problem Generating multiple outputs from one input.
method Using Dropout in GANs to maintain input constraints.
result Enhanced generation of diverse outputs from a single input.
FANNet analyzes noise tolerance and training bias in neural networks.
problem Low noise tolerance and input sensitivity in neural networks lead to failures on unseen inputs.
method Formal analysis using model checking under different noise ranges.
result Noise tolerance of ±11% for the trained network, sensitive input nodes identified, and biasness confirmed. MIMONets speed up neural network inference by processing multiple inputs in parallel.
problem Reducing computational cost in neural network inference for large datasets.
method Proposes MIMONets, which augment neural network architectures with variable binding mechanisms to handle multiple inputs in superposition.
result Achieves significant speedups (2-4x) with minimal accuracy loss, demonstrating adaptability across different architectures.
Enhances sensitivity analysis for correlated inputs.
problem Estimating sensitivity indices in models with correlated inputs.
method Proposes an extension of Sobol' estimator using a linear correlation model.
result Improves accuracy in variance-based sensitivity analysis.
Adaptive algorithm generates unrestricted adversarial inputs, defeating robust classifiers.
problem Vulnerability of neural networks to unrestricted adversarial inputs.
method Adaptive algorithm for generating unrestricted adversarial inputs.
result Adversarial inputs defeat robust classifiers.
Recurrent neural networks (RNNs) have drawn interest from machine learning researchers because of their effectiveness at preserving past inputs for time-varying data processing tasks. To understand the success and limitations of RNNs, it is critical that we advance our analysis of their fundamental memory properties. W…
VAIOM models financial returns using continuous input and categorical output.
problem Modeling continuous, noisy, and heterogeneous financial data.
method VAIOM is a decoder-only Transformer that separates input representation from output likelihood.
result VAIOM models outperform fixed single-bar LightGBM baseline in both Test halves.
Paper proposes an OOD detection method using input complexity estimates.
problem Excessive influence of input complexity on likelihoods from generative models.
method Estimate input complexity and use it to derive an OOD score.
result The derived OOD score performs comparably to or better than existing methods.
Model captures system input variations in latent space for actionable dynamics.
problem Learning dynamical systems from data without prescribing a mathematical model.
method Structured latent ODE model with stochastic factors of variation for each input.
result Improves generation of time-series data and inference of system inputs over baselines.
DeformRS certifies deep networks against various input deformations.
problem Vulnerability of deep networks to input deformations.
method Randomized smoothing reformulation for general deformations.
result Certifies rich deformations including translations, rotations, scaling, and affine.
This research investigates reliable local explanations for machine listening models.
problem Generating reliable local explanations for machine listening models.
method Investigates the sensitivity of SoundLIME explanations to input perturbations and proposes a novel method for identifying suitable content types.
result SoundLIME explanations are sensitive to the content in occluded input regions, and the average magnitude of input mel-spectrogram bins is the most suitable content type for temporal explanations.
Bayesian neural networks simplified with input augmentation.
problem Uncertainty in deep learning models.
method Layer-wise input augmentation to induce uncertainty distributions.
result State-of-the-art performance in uncertainty representation.
Develops HDNNs for mixed geoscience data inputs.
problem Lack of multisource, multi-scale information in deep learning studies.
method Hybrid architecture combining feature and target learning.
result HDNNs achieve higher accuracy and better generalization in reservoir prediction.