Simple GBRT model improved by window-based input transformation outperforms state-of-the-art deep learning models.
problem Improving performance of traditional forecasting models for time series data.
method Transformed GBRT model input structure to include target values and external features, forming one input instance per training window.
result Simple GBRT model with window-based input transformation outperformed state-of-the-art deep learning models on nine datasets.
RPE detects anomalies robustly in time-series data.
problem Detecting anomalies in time-series data efficiently and robustly.
method Window-based, robust projection step, closed-form algorithm.
result RPE can identify anomalies in time-stamp level and outperforms existing methods.
We continue to develop our neural network (NN) based forecasting approach to anomaly detection (AD) using the Secure Water Treatment (SWaT) industrial control system (ICS) testbed dataset. We propose genetic algorithms (GA) to find the best NN architecture for a given dataset, using the NAB metric to assess the quality…
Paper proposes a new neural machine translation method for wave data.
problem Limited real-world sensor data for continuous signal waves.
method Introduces window-based representation and iterative back-translation for wave data.
result Significant performance improvements in wave translation tasks.
In this work, we propose new objective functions to train deep neural network based density ratio estimators and apply it to a change point detection problem. Existing methods use linear combinations of kernels to approximate the density ratio function by solving a convex constrained minimization problem. Approximating…
The goal of the IARAI competition traffic4cast was to predict the city-wide traffic status within a 15-minute time window, based on information from the previous hour. The traffic status was given as multi-channel images (one pixel roughly corresponds to 100x100 meters), where one channel indicated the traffic volume, …
Novel deep learning model for multivariate time series prediction.
problem Challenges in multivariate time series prediction with correlations and complex temporal patterns.
method Temporal Tensor Transformation Network (TTNT) that transforms multivariate time series into tensors for improved feature extraction.
result TTNT outperforms state-of-the-art methods in window-based predictions across various tasks.
PROPO tackles non-stationary MDPs with efficient policy optimization.
problem Non-stationary MDPs with varying reward and transition kernels.
method PROPO, a periodic restarted optimistic policy optimization algorithm with sliding-window-based policy evaluation and improvement.
result PROPO achieves near-optimal performance in non-stationary MDPs.
Gait is a person's natural walking style and a complex biological process that is unique to each person. Recently, the channel state information (CSI) of WiFi devices have been exploited to capture human gait biometrics for user identification. However, the performance of existing CSI-based gait identification systems …
Paper finds linear laws in Bitcoin price changes, aiding anomaly detection.
problem Detecting anomalies in Bitcoin price changes.
method Time embedding of autocorrelation function, binary series generation, stepped time windows.
result Linear laws became more complex before major market events, suggesting price manipulation.
Method identifies causal interactions between time series using extreme eigenvalue variability.
problem Detecting causal interactions between time series.
method Largest eigenvalue of lagged correlation matrices, measuring causal interactions through variability.
result The method outperforms traditional Granger causality tests in detecting structural changes.
ETC improves Transformer models for long and structured inputs.
problem Scaling input length and encoding structured inputs in Transformers.
method Introduces global-local attention, relative position encodings, and CPC pre-training.
result Achieves state-of-the-art results on four natural language datasets.
Unified method for input, data, and model uncertainty in neural networks.
problem Uncertainty in neural network inputs and outputs.
method Propagating uncertainty through inputs using a unified formulation.
result More stable decision boundaries with input noise, and propagation of input uncertainty to model outputs.
Improves sample efficiency in reinforcement learning with input representation.
problem Poor sample efficiency in reinforcement learning.
method Attention-based method to project inputs into an invariant representation space.
result Representation space is m! smaller for inputs of m objects, improving sample efficiency. Detects unusual inputs to neural networks to prevent flawed predictions.
problem Erratic predictions from neural networks on unexpected inputs.
method Evaluates input unusualness by comparing its content to learned parameters.
result Simple, effective method for comparing input metrics across different scales.
TVS-FNNs can approximate any continuous function on expanded input spaces.
problem Processing a broader range of inputs like sequences and matrices.
method Proving a universal approximation theorem for TVS-FNNs.
result TVS-FNNs can approximate any continuous function on expanded input spaces.
We consider reinforcement learning in input-driven environments, where an exogenous, stochastic input process affects the dynamics of the system. Input processes arise in many applications, including queuing systems, robotics control with disturbances, and object tracking. Since the state dynamics and rewards depend on…
A new method builds sparse polynomial chaos expansions for models with dependent inputs.
problem Quantifying uncertainty in models with dependent inputs.
method Data-driven approach to construct orthonormal polynomials recursively based on input correlations.
result Reduces the number of observations and improves numerical stability and computational efficiency.
Validates neural networks inputs to protect against adversarial examples.
problem Ensuring neural networks robustness against adversarial attacks.
method Runtime local robustness verification based on normal distribution of robustness radii.
result Improves neural network accuracy and protects against adversarial examples.
Centroid Transformers reduce memory and computation by summarizing inputs into centroids.
problem Efficiently summarize inputs with reduced memory and computation.
method Generalizes self-attention to map N inputs to M centroids (M ≤ N), reducing complexity.
result Centroid Transformers reduce memory and computation while preserving key information.
MINs learn inverse mappings for high-dimensional optimization problems.
problem Data-driven optimization with high-dimensional inputs and valid subsets.
method Model Inversion Networks (MINs) learn an inverse mapping from scores to inputs.
result MINs can scale to high-dimensional input spaces and handle both offline and active data.
Methods for combining predictions from different models in a supervised learning setting must somehow estimate/predict the quality of a model's predictions at unknown future inputs. Many of these methods (often implicitly) make the assumption that the test inputs are identical to the training inputs, which is seldom re…
Testing under what conditions the product satisfies the desired properties is a fundamental problem in manufacturing industry. If the condition and the property are respectively regarded as the input and the output of a black-box function, this task can be interpreted as the problem called Level Set Estimation (LSE) --…
IA-BMA adapts model weights to inputs for better predictions.
problem Predicting with multiple models in heterogeneous settings.
method Input adaptive Bayesian Model Averaging (IA-BMA) with an input adaptive prior and amortized variational inference.
result IA-BMA consistently delivers more accurate and better-calibrated predictions.
Recent work establishes dataset difficulty and removes annotation artifacts via partial-input baselines (e.g., hypothesis-only models for SNLI or question-only models for VQA). When a partial-input baseline gets high accuracy, a dataset is cheatable. However, the converse is not necessarily true: the failure of a parti…
We analyze the adversarial examples problem in terms of a model's fault tolerance with respect to its input. Whereas previous work focuses on arbitrarily strict threat models, i.e., ε-perturbations, we consider arbitrary valid inputs and propose an information-based characteristic for evaluating tolerance to diverse …
AIRBO optimizes robustly under uncertain inputs.
problem Challenges in Bayesian Optimization due to input uncertainty.
method AIRBO models arbitrary input uncertainty with MMD and accelerates posterior inference.
result AIRBO achieves state-of-the-art performance under various input uncertainties.
When simulating a complex stochastic system, the behavior of output response depends on input parameters estimated from finite real-world data, and the finiteness of data brings input uncertainty into the system. The quantification of the impact of input uncertainty on output response has been extensively studied. Most…
We present a novel framework for specifying and verifying correctness globally for neural networks on perception tasks. Most previous works on neural network verification for perception tasks focus on robustness verification. Unlike robustness verification, which aims to verify that the prediction of a network is stabl…
Generalizes memory and forecasting capacities for nonlinear recurrent networks with dependent inputs.
problem Understanding memory and forecasting capabilities in networks with dependent inputs.
method Formulated bounds for memory and forecasting capacities in terms of network size and input properties.
result Proved that memory capacity for linear recurrent networks with independent inputs is given by the rank of the controllability matrix.
This paper aims to propose a novel deep learning-integrated framework for deriving reliable simulation input models through incorporating multi-source information. The framework sources and extracts multisource data generated from construction operations, which provides rich information for input modeling. The framewor…
SpinSVAR estimates SVAR models with sparse input, improving accuracy and scalability.
problem Estimating SVAR models with sparse input assumptions.
method SpinSVAR models input as independent Laplacian variables, enforcing sparsity and using least absolute error regression.
result SpinSVAR outperforms state-of-the-art methods in accuracy and runtime, identifying significant structural shocks.
New method distinguishes predictive distribution estimators in high-dimensional inputs.
problem Difficulty in evaluating predictive distributions for high-dimensional inputs.
method Introduces dyadic sampling to focus on predictive distributions associated with pairs of inputs.
result Demonstrates efficient distinction of predictive distribution estimators in high-dimensional examples.
This research analyzes how input and output layers affect deep neural networks' resistance to adversarial attacks.
problem The vulnerability of deep neural networks to adversarial inputs, especially non-gradient based attacks.
method Analysis of three different fully connected dense network classes with manipulated input and output layers.
result Manipulating input and output layers can significantly enhance a deep neural network's robustness against adversarial attacks.
Humans increasingly interact with Artificial intelligence(AI) systems. AI systems are optimized for objectives such as minimum computation or minimum error rate in recognizing and interpreting inputs from humans. In contrast, inputs created by humans are often treated as a given. We investigate how inputs of humans can…
New method certifies neural network robustness under random input noise.
problem Certifying neural network robustness against random input noise.
method Chance-constrained optimization problem reformulated with input-output samples, convex conditions developed.
result Proposed method certifies robustness against various input noise regimes over larger uncertainty regions.
Enhances sensitivity analysis for correlated inputs.
problem Estimating sensitivity indices in models with correlated inputs.
method Proposes an extension of Sobol' estimator using a linear correlation model.
result Improves accuracy in variance-based sensitivity analysis.
MIMONets speed up neural network inference by processing multiple inputs in parallel.
problem Reducing computational cost in neural network inference for large datasets.
method Proposes MIMONets, which augment neural network architectures with variable binding mechanisms to handle multiple inputs in superposition.
result Achieves significant speedups (2-4x) with minimal accuracy loss, demonstrating adaptability across different architectures.
Recurrent neural networks (RNNs) have drawn interest from machine learning researchers because of their effectiveness at preserving past inputs for time-varying data processing tasks. To understand the success and limitations of RNNs, it is critical that we advance our analysis of their fundamental memory properties. W…
VAIOM models financial returns using continuous input and categorical output.
problem Modeling continuous, noisy, and heterogeneous financial data.
method VAIOM is a decoder-only Transformer that separates input representation from output likelihood.
result VAIOM models outperform fixed single-bar LightGBM baseline in both Test halves.
Model captures system input variations in latent space for actionable dynamics.
problem Learning dynamical systems from data without prescribing a mathematical model.
method Structured latent ODE model with stochastic factors of variation for each input.
result Improves generation of time-series data and inference of system inputs over baselines.
DeformRS certifies deep networks against various input deformations.
problem Vulnerability of deep networks to input deformations.
method Randomized smoothing reformulation for general deformations.
result Certifies rich deformations including translations, rotations, scaling, and affine.
In the manufacturing industry, it is often necessary to repeat expensive operational testing of machine in order to identify the range of input conditions under which the machine operates properly. Since it is often difficult to accurately control the input conditions during the actual usage of the machine, there is a …
This research investigates reliable local explanations for machine listening models.
problem Generating reliable local explanations for machine listening models.
method Investigates the sensitivity of SoundLIME explanations to input perturbations and proposes a novel method for identifying suitable content types.
result SoundLIME explanations are sensitive to the content in occluded input regions, and the average magnitude of input mel-spectrogram bins is the most suitable content type for temporal explanations.
With a constant improvement in the network architectures and training methodologies, Neural Networks (NNs) are increasingly being deployed in real-world Machine Learning systems. However, despite their impressive performance on "known inputs", these NNs can fail absurdly on the "unseen inputs", especially if these real…
Bayesian neural networks simplified with input augmentation.
problem Uncertainty in deep learning models.
method Layer-wise input augmentation to induce uncertainty distributions.
result State-of-the-art performance in uncertainty representation.
Develops HDNNs for mixed geoscience data inputs.
problem Lack of multisource, multi-scale information in deep learning studies.
method Hybrid architecture combining feature and target learning.
result HDNNs achieve higher accuracy and better generalization in reservoir prediction.
Humans are able to imagine a person's voice from the person's appearance and imagine the person's appearance from his/her voice. In this paper, we make the first attempt to develop a method that can convert speech into a voice that matches an input face image and generate a face image that matches the voice of the inpu…