Improved highway networks by allowing direct information flow through time.
problem Performance degradation of RHNs with increased layer depth.
method Introducing Highway State Gating (HSG) to allow selective information flow and gradient propagation.
result Improvement in performance with increased layer depth, demonstrated on PTB dataset.
This paper proposes a semi-tied unit approach to improve efficiency in LSTM and highway networks.
problem Efficiency issues in gating mechanisms of LSTM and highway networks.
method Semi-tied units (STUs) using one shared weight matrix with extra scaling factors.
result Reduced calculation and storage costs by a factor of three for highway networks and four for LSTMs.
A major contributing factor to the recent advances in deep neural networks is structural units that let sensory information and gradients to propagate easily. Gating is one such structure that acts as a flow control. Gates are employed in many recent state-of-the-art recurrent models such as LSTM and GRU, and feedforwa…
GHNet improves graph learning by balancing homogeneity and heterogeneity.
problem Over-smoothing in GCN leads to similar node representations.
method GHNet uses gating units to balance homogeneity and heterogeneity in feature propagation.
result GHNet achieves larger receptive fields without over-smoothing.
Transfer learning improves highway traffic forecasting using graph neural networks.
problem Lack of historical data for traffic forecasting on large highway networks.
method Developed a transfer learning approach for DCRNN, a graph neural network for highway forecasting.
result TL-DCRNN can forecast traffic on unseen regions of the highway network with high accuracy.
Graph-partitioning-based DCRNN improves traffic forecasting for large highways.
problem Challenges in accurately forecasting traffic on large highway networks.
method Graph-partitioning method to decompose large networks into smaller, independent networks.
result Demonstrated improved traffic forecasting on a large California highway network.
The paper studies 4-qubit Clifford states and their properties.
problem Understanding the set and properties of 4-qubit Clifford states.
method Analyzing the 293760 4-qubit Clifford states, splitting them into 18 groups, and studying the action of CNOT gates and local gates.
result There are 293760 4-qubit Clifford states with specific entanglement entropies, and any pair can be connected with local gates and at most 3 CNOT gates.
New method prepares 3-qubit states using local gates and controlled-Z gates.
problem Preparation of 3-qubit states using quantum gates.
method Uses Ry(θ) gates and controlled-Z gates, with an optimal number of controlled-Z gates. result Optimal number of controlled-Z gates for preparing 3-qubit states is four. This paper uses machine learning to estimate how different types of crashes affect highway traffic.
problem Estimating the heterogeneous causal effects of crashes on highway traffic.
method Neyman-Rubin Causal Model, Conditional Shapley Value Index, Structural Causal Model, Doubly Robust Learning.
result Different types of crashes have varying impacts on traffic, with rear-end crashes causing the most severe congestion.
Batch-normalized RHN improves gradient control in recurrent networks.
problem Gradient vanishing or exploding in recurrent networks.
method Batch normalization applied at each recurrence loop in RHN.
result Batch-normalized RHN converges faster and performs better.
Deep RL for autonomous highway driving avoids unexpected scenarios.
problem Unexpected scenarios in AV operation space lead to poor decision-making.
method Deep reinforcement learning with safety checks for decision-making.
result Enhanced learning efficiency and safe behavior in highway driving.
A new LSTM model reduces state updates and improves convergence for long sequences.
problem Vanishing gradient problem in RNNs and slow convergence on long sequences.
method Gaussian-gated LSTM (g-LSTM) with a time gate to control neuron updates.
result The g-LSTM model reduces state updates and computes by at least 10x compared to an equivalent LSTM.
HRHN predicts time series by integrating exogenous data and temporal dynamics.
problem Challenges in predicting time series with exogenous data and temporal dynamics.
method Hierarchical attention-based Recurrent Highway Network (HRHN) that considers interactions among exogenous variables and temporal dynamics.
result HRHN outperforms state-of-the-art methods in time series prediction, especially in capturing sudden changes and oscillations.
Proposes a new LSTM gate structure using bivariate Beta distribution.
problem Inflexibility of sigmoid gates in modeling multi-modality and skewness, and lack of modeling correlation between gates.
method Introduces a bivariate Beta distribution gate structure within LSTM cells.
result Empirically shows higher gradient values and improved model performance.
Study uses vehicle trajectory data to predict traffic incidents on highways.
problem Early detection of traffic incidents to reduce secondary crashes.
method Machine learning algorithms (Logistic Regression, Random Forest, Extreme Gradient Boost, Artificial Neural Network) applied to vehicle trajectory data.
result Random Forest model performs best for incident prediction.
Paper describes a video surveillance system for highway traffic events.
problem Detecting specific sequences of situations in highway traffic videos.
method Compares RNN and CNN architectures for analyzing video frames and sequences.
result Best architecture performs well in real conditions.
Paper proposes a statistical approach for predicting lane changes in highway scenarios.
problem Early recognition of lane changes for automated driving.
method Prototype trajectories generated from real data using Agglomerative Hierarchical Clustering. Maneuver prediction via Boosted Decision Trees and mixture model.
result Improved performance for lane change and trajectory prediction compared to a reference approach.
Study stability of selective SSMs with discontinuous gating.
problem Challenges in stability analysis of selective SSMs with discontinuous gating.
method Passivity and Input-to-State Stability (ISS) analysis of continuous-time selective SSMs.
result Derivation of sufficient conditions for global ISS with respect to the port input.
HighD dataset captures naturalistic vehicle behavior on German highways for automated driving validation.
problem Current measurement methods fail to meet requirements for scenario-based validation of highly automated vehicles.
method Aerial perspective data collection fulfilling naturalistic behavior, quantity, and variety requirements.
result 16.5 hours of measurements from six locations, 110,000 vehicles, 45,000 km driven, 5600 lane changes.
Gating units in GRUs and LSTMs create slow modes and control phase-space complexity.
problem Training challenges in RNNs due to exploding or vanishing gradients.
method Random matrix theory and mean-field theory applied to GRUs and LSTMs.
result Gates in GRUs and LSTMs lead to accumulation of slow modes and control phase-space complexity.
Deep network improves NILM by distinguishing on/off states of appliances.
problem Break down household aggregate electricity consumption into individual appliance usages.
method Subtask gated network combining regression and classification subtasks.
result Surpasses state-of-the-art performance for most benchmark cases.
Researchers create an exact entangling gate using braiding and measurement of Fibonacci anyons.
problem No known leakage-free entangling gate using braiding of Fibonacci anyons.
method Supplement braiding with measurement operations to produce an exact controlled rotation gate.
result Exact entangling gate on two qubits created using Fibonacci anyons and measurement.
This study identifies RwD crash patterns on rural two-lane highways under different lighting conditions.
problem Insufficient investigation of RwD crashes under varying lighting conditions.
method Data mining using association rules mining (ARM) on crash database.
result Interesting crash patterns and risk factors identified under different lighting conditions.
GLCB uses Gated Linear Networks for online contextual bandits.
problem Online learning in contextual bandits with uncertainty estimation.
method Gated Linear Networks (GLNs) for prediction and uncertainty estimation.
result GLCB outperforms state-of-the-art methods in online contextual bandits.
Paper introduces a simpler gated RNN structure to better capture long-term dependencies.
problem Difficulty in learning long-term dependencies in RNNs.
method Proposes a grouped distributor unit (GDU) with partitioned hidden states and adaptive update rates.
result GDU outperforms LSTM and GRU on various tasks, including pathological and natural data.
Gradient Gating improves deep GNNs by modulating message passing updates.
problem Oversmoothing and performance degradation in deep GNNs.
method Gradient gating mechanism for multi-rate message passing.
result G2 framework alleviates oversmoothing and achieves state-of-the-art performance. New interpretation of RNN forget gate improves learnability for long-term sequential data.
problem Improving learnability of recurrent neural networks for long-term temporal dependencies.
method Generalized theory of gated RNNs, focusing on gradient behavior over time.
result Existing RNNs satisfy the gradient condition for initial training, suggesting validity of forget gate interpretation.
Paper proposes binary-valued gates for better LSTM training.
problem LSTMs struggle with soft gates, leading to unclear information flow.
method Introduces binary-valued gates to control information flow more clearly.
result Binary-valued gates improve LSTM performance and generalization.
DAGR improves navigation by refining goal representations conditioned on the current state.
problem Goal-conditioned reinforcement learning lacks state awareness, leading to inefficient policy recovery.
method DAGR refines static goal embeddings into state-conditioned ones using gated cross-attention with a state-goal discrepancy map.
result DAGR improves navigation tasks on OGBench, matching or outperforming base methods.
Topological theory for qLDPC codes enables non-Clifford gates and magic state injection.
problem Fault-tolerant quantum computation in qLDPC codes with non-Clifford gates and magic state resources.
method Developed a topological theory using simplicial or CW complex structures and deformation retraction.
result Achieved non-Clifford gates and magic state injection in qLDPC codes with constant rate and polynomial distance.
Quantum computing uses Bianchi groups to build gates.
problem Building universal quantum gates from Bianchi groups.
method Using subgroups of Bianchi groups to derive quantum gate generators.
result Demonstrated the use of Bianchi groups for quantum computing.
In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pai…
Channel gating reduces CNN computation cost by skipping ineffective feature regions.
problem Reducing computation cost in CNNs while maintaining accuracy.
method Dynamic, fine-grained pruning scheme that identifies and skips computation on ineffective feature regions.
result 2.7-8.0x reduction in FLOPs and 2.0-4.4x reduction in memory accesses with minimal accuracy loss.
FastGRNN improves RNN accuracy while drastically reducing model size.
problem Inaccurate training and inefficient prediction in RNNs.
method FastGRNN uses a residual connection and gate to achieve state-of-the-art accuracy with a much smaller model.
result FastGRNN achieves state-of-the-art accuracy with models up to 35x smaller than existing RNNs.
Neural Machine Translation (MT) has reached state-of-the-art results. However, one of the main challenges that neural MT still faces is dealing with very large vocabularies and morphologically rich languages. In this paper, we propose a neural MT system using character-based embeddings in combination with convolutional…
New quantum code breaks distance barrier with transversal non-Clifford gates.
problem Breaking the sqrt(N) distance barrier for quantum LDPC codes.
method Combining three qLDPC codes, Freedman-Hastings mapping, and triple cup product.
result Achieves Ω(N^(2/3)) distance and Θ(N^(2/3)) dimension, enabling fault-tolerant magic state preparation.
We propose a novel adaptive approximation approach for test-time resource-constrained prediction. Given an input instance at test-time, a gating function identifies a prediction model for the input among a collection of models. Our objective is to minimize overall average cost without sacrificing accuracy. We learn gat…
VGE provides a practical approach to uncertainty estimation in ensemble models.
problem Uncertainty estimation in ensemble models using additive decomposition breaks down.
method Variance-Gated Ensembles (VGE) introduces a differentiable framework with a signal-to-noise gate.
result VGE provides a Variance-Gated Margin Uncertainty (VGMU) score and Variance-Gated Normalization (VGN) layer.
Novel complex RNN improves stability and performance in sequence tasks.
problem Lack of complex representations in deep learning for sequence tasks.
method Developed a complex gated recurrent cell combining complex-valued and norm-preserving state transitions with a gating mechanism.
result Improves stability and convergence properties, performs competitively on various tasks.
The paper analyzes the role of ReLU gates in deep learning networks.
problem Understanding the role of gates in deep learning networks.
method Developed neural path features (NPF) and neural path values (NPV) to characterize the active sub-networks during training.
result The neural path kernel associated with NPFs is a fundamental quantity that characterizes the information stored in the gates of a DNN.
Improved GRU model with weighted time-delay feedback for long-term dependencies.
problem Modeling long-term dependencies in sequential data.
method Introducing a gated recurrent unit (GRU) with a weighted time-delay feedback mechanism.
result τ-GRU outperforms state-of-the-art models on various tasks.
DeepESNs outperform ESN and GRUs in multivariate time-series prediction.
problem Comparing DeepESNs and gated RNNs for multivariate time-series prediction.
method Experimental comparison of DeepESNs and gated RNNs (Gated Recurrent Units, Long Short-Term Memory) on 4 polyphonic music tasks.
result DeepESNs outperform ESN and GRUs in terms of prediction accuracy and efficiency.
We present a dynamic model selection approach for resource-constrained prediction. Given an input instance at test-time, a gating function identifies a prediction model for the input among a collection of models. Our objective is to minimize overall average cost without sacrificing accuracy. We learn gating and predict…
ABF-T-GLCP forecasts and calibrates uncertainty for multivariate nonstationary time series.
problem Forecasting and uncertainty quantification in nonstationary multivariate time series.
method Adaptive multi-scale forecasting and Gate-Localized Conformal Prediction.
result Consistent gains in point forecasting accuracy and narrower prediction intervals with close empirical coverage.
Extended LSTM improves language modeling performance.
problem Improving LSTM for large-scale language modeling.
method Exponential gating, modified memory structure, and residual stacking.
result xLSTM outperforms state-of-the-art models in performance and scaling.
DSelect-k improves MoE models for multi-task learning with better performance and smoother training.
problem Smoothness and convergence issues in sparse gate selection for MoE models.
method Developed DSelect-k, a differentiable and sparse gate for MoE models.
result DSelect-k achieves statistically significant improvements in prediction and expert selection over Top-k.
Faster ZSL with continual learning and self-gating.
problem Generalizing models to unseen categories and handling sequential data.
method Meta-continual zero-shot learning (MCZSL) with self-gating and scaled class normalization.
result Outperforms state-of-the-art results with faster training (>100imes). MGpi model predicts social actions in group interactions.
problem Social interaction among multiple agents and groups.
method Deep neural network with Kinesic-Proxemic-Message gate for social signal gating.
result Achieves state-of-the-art performance in group identification.