LSTM-FCN and ALSTM-FCN improve time series classification performance.
problem Improving time series classification performance.
method Ablation tests on LSTM-FCN and ALSTM-FCN, comparing z-normalizing techniques, dimension shuffle impact, and GRU replacement.
result LSTM and FCN blocks perform better together, and z-normalizing the whole dataset is more effective.
New dataset tests mental rotation from single images, improving model understanding of 3D scenes.
problem Understanding how a scene looks from a different viewpoint using a single image.
method Created CLEVR-MRT dataset, explored neural architectures for volumetric scene representations.
result Demonstrated the effectiveness of volumetric representations in answering mental rotation questions.
Paper proposes forecast-necessity testing for accurate causal interpretation in nonlinear time-series models.
problem Misinterpretation of causal scores from nonlinear models as regression coefficients.
method Systematic edge ablation and forecast comparison to evaluate causal necessity.
result Causal relationships with similar scores can differ in their necessity for accurate prediction.
In this work, we investigate the use of three information-theoretic quantities -- entropy, mutual information with the class variable, and a class selectivity measure based on Kullback-Leibler divergence -- to understand and study the behavior of already trained fully-connected feed-forward neural networks. We analyze …
Study ablated data augmentation techniques and their mathematical equivalence to penalties.
problem Lack of mathematical understanding of differences between ablated data augmentation techniques.
method Formal model of mean ablated data augmentation and inverted dropout for linear regression; empirical validation for deep networks.
result Ablated data augmentation and inverted dropout are mathematically equivalent to penalties in optimization.
Double descent phenomenon explained in simple terms.
problem Understanding the surprising drop in test error in overparameterized models.
method Informal explanation using linear algebra and probability, visual intuition with polynomial regression, mathematical analysis with ordinary linear regression.
result Three factors create double descent: data undersampling, model size, and parameter count. Ablating any one of these factors prevents double descent.
Ablation studies show BCF model's propensity score is not essential for treatment effect estimation.
problem Understanding the necessity of propensity score in nonparametric treatment effect estimation.
method Partial ablation studies of Bayesian Causal Forest (BCF) model.
result Excluding estimated propensity score does not affect treatment effect estimation or uncertainty quantification.
The study measures feature importance in models by randomly ablating features and averaging loss changes.
problem Measuring feature importance in machine learning models.
method Randomly ablating features and averaging loss changes over many points and replacements.
result Uncertainty measures of feature importance are presented.
CIT and CIF improve feature selection for downstream prediction.
problem Feature selection bias in machine learning models.
method Conditional inference trees and forests with Bonferroni correction.
result CIF ranks top 3 among 18 regression methods and top 4 among 17 classification methods.
Embedding-based Knowledge Base Completion models have so far mostly combined distributed representations of individual entities or relations to compute truth scores of missing links. Facts can however also be represented using pairwise embeddings, i.e. embeddings for pairs of entities and relations. In this paper we ex…
ABC method analyzes diffusion model dependence without retraining.
problem Difficulties in characterizing diffusion models' dependence on training data.
method Ablation-based counterfactuals using model components trained on different splits.
result Demonstrates limits of training data attribution and existence of unattributable samples.
In microsurgery, lasers have emerged as precise tools for bone ablation. A challenge is automatic control of laser bone ablation with 4D optical coherence tomography (OCT). OCT as high resolution imaging modality provides volumetric images of tissue and foresees information of bone position and orientation (pose) as we…
MixBoost generates synthetic instances to balance imbalanced datasets.
problem Training models on imbalanced datasets.
method Iterative data augmentation method that selects and combines instances from majority and minority classes.
result MixBoost outperforms existing approaches on 20 benchmark datasets.
A new method combines multiple bounds and betting strategies for selective prediction, improving risk coverage in data-scarce settings.
problem Selective prediction with risk control in data-scarce domains.
method Combines concentration inequalities, multiple-testing corrections, and betting-based confidence sequences.
result Transfer-Informed Betting achieves tighter bounds and better coverage in data-scarce settings.
Rotationally equivariant convolutions improve molecular property prediction.
problem Predicting molecular properties using graph neural networks.
method Ablation study with rotationally equivariant and invariant convolutions on QM9 data set.
result Rotationally equivariant layers decrease test error by an average of 23%.
The paper aims to develop a methodology to test memory systems in reinforcement learning agents.
problem Understanding and evaluating memory systems in reinforcement learning agents and their generalization.
method Construct diverse memory tasks, develop an agent architecture combining multiple memory systems, and perform ablations.
result The agent architecture can apply learned knowledge to holdout data that differs from the training data.
New method reduces errors in causal discovery from data.
problem Errors in causal discovery from limited data.
method Hierarchical wrapper for constraint-based algorithms.
result Significantly fewer tests, more accurate graphs, shorter run-times.
In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked …
Model identifies order splitting and liquidity replenishment as necessary for the square-root law of market impact.
problem Quantifying the square-root law of market impact and identifying its underlying mechanisms.
method Minimal limit-order-book model with heterogeneous interacting agents calibrated against real data. Counterfactual ablation to isolate mechanisms.
result Order splitting and liquidity replenishment are necessary for the square-root law of market impact.
DAD++ improves test-time adversarial defense without access to training data.
problem Improving adversarial robustness of pre-trained models without access to training data.
method Test-time Data-free Adversarial Defense (DAD++) with detection and correction schemes.
result DAD++ significantly improves adversarial robustness with minimal clean accuracy drop.
New method certifies robustness to sparse adversarial attacks.
problem Ensuring robustness of classifiers to sparse adversarial attacks.
method Randomized ablation of input features to provide robustness guarantees.
result Certifies robustness to sparse adversarial attacks with up to 31 pixels of distortion.
FraudTransformer detects payment fraud by preserving event order and time gaps.
problem Detecting payment fraud in real-world banking streams with irregular time gaps.
method Augments a GPT-style architecture with a dedicated time encoder and a learned positional encoder.
result FraudTransformer outperforms classical and transformer baselines, achieving highest AUROC and PRAUC on held-out test set.
Paper proposes a modified uncertainty sampling method to speed up preference learning from noisy humans.
problem Learning preferences from humans with limited queries and noisy responses.
method Modified uncertainty sampling using expected output value to speed up preference learning.
result The modified method outperforms the baseline uncertainty sampling in preference learning.
Bayesian Quadrature improves ensembling for neural networks with dispersed likelihood peaks.
problem Ensembling neural networks struggles with dispersed, narrow peaks in likelihood surfaces.
method Uses Bayesian Quadrature to construct weighted ensembles of architectures.
result Empirically outperforms state-of-the-art baselines in test likelihood, accuracy, and expected calibration error.
Paper analyzes biases in video QA datasets, showing models can answer 37-48% questions correctly without multimodal context.
problem Question answering biases in video QA datasets can lead to model overfitting and poor generalization.
method Analyzed popular video question answering datasets, conducted ablation studies on biases from annotators and question types.
result Pretrained language models can answer 37-48% questions correctly without multimodal context, far exceeding random guess baseline.
RNNs solve modular addition tasks using low rank and sparse Fourier structures.
problem Solving modular addition tasks with recurrent neural networks.
method Identified low rank structures and sparse Fourier representations in RNN weights.
result RNNs robust to removing individual frequencies but degrade with more ablation.
Generates realistic stock market order streams using GANs.
problem Creating high-fidelity stock market data.
method Conditional Wasserstein GAN with auction mechanism and order-book augmentation.
result Generated data is close to real market data.
TSFMs show redundant components in layers, affecting their performance.
problem Redundant components in TSFMs layers impact their predictive accuracy.
method Mechanistic interpretability tools, ablations, logit attribution, theoretical framework.
result TSFMs are robust to ablations of entire layers and specific heads.
In-Place TTT enhances LLMs with dynamic parameter updates at inference time.
problem Static training limits LLMs from adapting to new information.
method In-Place TTT updates a subset of model parameters (fast weights) at inference time.
result In-Place TTT enables 4B-parameter models to outperform on tasks with up to 128k contexts.
AL2 progressively penalizes network activations to prevent overfitting.
problem Avoiding overfitting in neural networks with limited training data.
method Progressive Activation Loss (PAL) method to regularize network representations.
result AL2 outperforms traditional regularization methods on benchmark datasets.
EXAGREE selects a stakeholder-aligned model to reduce conflicting explanations in machine learning.
problem Conflicting explanations from different attribution methods limit the adoption of machine learning models in safety-critical domains.
method EXAGREE is a two-stage framework that selects a Stakeholder-Aligned Explanation Model (SAEM) from a set of similar-performing models, maximizing Stakeholder-Machine Agreement (SMA).
result EXAGREE achieves simultaneous gains in faithfulness, plausibility, and fairness over baselines while preserving task accuracy.
FairVIC improves fairness in neural networks without sacrificing accuracy.
problem Mitigating bias in automated decision-making systems, particularly in deep learning models.
method Integrates variance, invariance, and covariance terms into the loss function during training to abstract fairness concepts.
result Significant improvements in fairness across all tested metrics without compromising accuracy.
LLM embeddings improve adaptation to tabular Y∣X-shifts with few labeled examples.
problem Improving robustness to Y∣X-shifts in tabular data. method Serializing tabular data to LLM embeddings and fine-tuning for adaptation.
result LLM embeddings can be adapted to target domains with minimal labeled data.
A2MT learns agents to select which modalities to acquire at test time.
problem Learning agents to select modalities for multimodal temporal data acquisition.
method Perceiver IO architecture for active acquisition of multimodal temporal data.
result Agents successfully learn cost-reactive acquisition behavior on real-world datasets.
Efficient neural network optimization reduces costs and improves model performance.
problem High computational costs in optimizing neural networks, especially at scale.
method Introduces self-attentive feed-forward neural units (SAFFU) for efficient optimization.
result Explicit solutions outperform models optimized by backpropagation alone, and further training with backpropagation leads to better optima from smaller data sets.
Code generation maps a program description to executable source code in a programming language. Existing approaches mainly rely on a recurrent neural network (RNN) as the decoder. However, we find that a program contains significantly more tokens than a natural language sentence, and thus it may be inappropriate for RN…
New framework improves offline RL performance without complex methods.
problem Limited access to online interactions in real-world RL applications.
method Behavior Regularized Actor Critic (BRAC) framework.
result Many recent technical complexities are not necessary for strong offline RL performance.
ETGL-DDPG improves DDPG for sparse reward control with new exploration and replay techniques.
problem Sparse reward continuous control in reinforcement learning.
method Introduces εt-greedy search and GDRB framework for efficient exploration and reward use. result ETGL-DDPG outperforms DDPG and other methods on sparse-reward continuous benchmarks.
Proposes a simpler method for quantifying uncertainty in time-series with volatility clustering.
problem Uncertainty quantification for time-series with volatility clustering.
method Proposes a Scale Mixture Distribution to quantify return forecast uncertainty in neural networks.
result The proposed method provides a favorable complexity-accuracy trade-off and separates model parameters into subnetworks.
A new method improves maximum inner product search by locally decomposing residual vectors.
problem Maximum inner product search efficiency and accuracy.
method Local Orthogonal Decomposition (LOD) combined with multiscale quantization.
result LOD consistently achieves higher recall than previous methods under the same bitrates.
Paper tackles cross-granularity few-shot learning with meta-embedder.
problem Few-shot learning with coarse labels and fine-grained testing.
method Meta-embedder that optimizes visual and semantic discrimination across coarse and fine classes.
result Meta-embedder achieves effective cross-granularity few-shot classification.
Hydra boosts efficiency for long-context reasoning in resource-constrained settings.
problem Quadratic complexity of transformers limits long-context reasoning in resource-constrained systems.
method Hydra uses a modular architecture with adaptive routing between sparse global attention, mixture-of-experts, and dual memories.
result Hydra achieves significant throughput and accuracy improvements for long-context reasoning.
Dynamic meta-learning improves multi-agent communication with natural language.
problem Static population-based approaches struggle with adapting to human partners using natural language.
method Dynamic population-based meta-learning approach.
result Agents outperform all prior work in communicating with seen partners and humans.
Transformer-based multi-scale model outperforms traditional methods in solving PDEs on irregular domains.
problem Solving partial differential equations on irregular domains using deep learning.
method Introduces Multi-Scale Attention Transformer (\msat{}) for solving PDEs.
result Achieves state-of-the-art generalization on complex geometry problems with significant speedup.
Neural waveform models such as the WaveNet are used in many recent text-to-speech systems, but the original WaveNet is quite slow in waveform generation because of its autoregressive (AR) structure. Although faster non-AR models were recently reported, they may be prohibitively complicated due to the use of a distillin…
Second-order estimator improves continuous-time policy evaluation.
problem Estimating value surfaces from discrete data with time-inhomogeneous dynamics.
method Moment-matching coefficients for high-order generator regression.
result Second-order estimator consistently outperforms Bellman baseline.
ECP optimizes expensive functions without knowing Lipschitz constant.
problem Optimizing expensive, non-convex functions with unknown Lipschitz constants.
method ECP minimizes evaluations by focusing on potentially optimal regions, eliminating Lipschitz constant estimation.
result Guaranteed no-regret performance and minimax-optimal regret bounds.
Triangulation filters spurious circuits in multilingual models.
problem Unreliable explanations of multilingual models across languages.
method Formalizes reference families and introduces triangulation as a causal acceptance rule.
result Triangulation provides a falsifiable standard for mechanistic claims.