Ablation studies show BCF model's propensity score is not essential for treatment effect estimation.
problem Understanding the necessity of propensity score in nonparametric treatment effect estimation.
method Partial ablation studies of Bayesian Causal Forest (BCF) model.
result Excluding estimated propensity score does not affect treatment effect estimation or uncertainty quantification.
Study ablated data augmentation techniques and their mathematical equivalence to penalties.
problem Lack of mathematical understanding of differences between ablated data augmentation techniques.
method Formal model of mean ablated data augmentation and inverted dropout for linear regression; empirical validation for deep networks.
result Ablated data augmentation and inverted dropout are mathematically equivalent to penalties in optimization.
ABC method analyzes diffusion model dependence without retraining.
problem Difficulties in characterizing diffusion models' dependence on training data.
method Ablation-based counterfactuals using model components trained on different splits.
result Demonstrates limits of training data attribution and existence of unattributable samples.
Given a model f that predicts a target y from a vector of input features x=x1,x2,…,xM, we seek to measure the importance of each feature with respect to the model's ability to make a good prediction. To this end, we consider how (on average) some measure of goodness or badness of prediction (wh…
In microsurgery, lasers have emerged as precise tools for bone ablation. A challenge is automatic control of laser bone ablation with 4D optical coherence tomography (OCT). OCT as high resolution imaging modality provides volumetric images of tissue and foresees information of bone position and orientation (pose) as we…
TSFMs show redundant components in layers, affecting their performance.
problem Redundant components in TSFMs layers impact their predictive accuracy.
method Mechanistic interpretability tools, ablations, logit attribution, theoretical framework.
result TSFMs are robust to ablations of entire layers and specific heads.
In this work, we investigate the use of three information-theoretic quantities -- entropy, mutual information with the class variable, and a class selectivity measure based on Kullback-Leibler divergence -- to understand and study the behavior of already trained fully-connected feed-forward neural networks. We analyze …
Recently, techniques have been developed to provably guarantee the robustness of a classifier to adversarial perturbations of bounded L_1 and L_2 magnitudes by using randomized smoothing: the robust classification is a consensus of base classifications on randomly noised samples where the noise is additive. In this pap…
New dataset tests mental rotation from single images, improving model understanding of 3D scenes.
problem Understanding how a scene looks from a different viewpoint using a single image.
method Created CLEVR-MRT dataset, explored neural architectures for volumetric scene representations.
result Demonstrated the effectiveness of volumetric representations in answering mental rotation questions.
Hydra boosts efficiency for long-context reasoning in resource-constrained settings.
problem Quadratic complexity of transformers limits long-context reasoning in resource-constrained systems.
method Hydra uses a modular architecture with adaptive routing between sparse global attention, mixture-of-experts, and dual memories.
result Hydra achieves significant throughput and accuracy improvements for long-context reasoning.
This paper presents a supervised learning algorithm, namely, the Synaptic Efficacy Function with Meta-neuron based learning algorithm (SEF-M) for a spiking neural network with a time-varying weight model. For a given pattern, SEF-M uses the learning algorithm derived from meta-neuron based learning algorithm to determi…
Predicts stock volatility using Twitter data and random forests.
problem Predicting stock implied volatility using Twitter data.
method Random forests with ablation study on different predictors, including Twitter attention and sentiment features.
result Certain sectors like Consumer Discretionary, Technology, Real Estate, and Utilities are easier to predict.
We review some of the latest approaches to analysing cardiac electrophysiology data using machine learning and predictive modelling. Cardiac arrhythmias, particularly atrial fibrillation, are a major global healthcare challenge. Treatment is often through catheter ablation, which involves the targeted localized destruc…
Method generates anatomically-controllable medical images with segmentation guidance.
problem Challenging to enforce anatomical constraints in generated medical images.
method Segmentation-guided diffusion models with random mask ablation training.
result New state-of-the-art in faithfulness to input anatomical masks.
Double descent phenomenon explained in simple terms.
problem Understanding the surprising drop in test error in overparameterized models.
method Informal explanation using linear algebra and probability, visual intuition with polynomial regression, mathematical analysis with ordinary linear regression.
result Three factors create double descent: data undersampling, model size, and parameter count. Ablating any one of these factors prevents double descent.
Paper proposes forecast-necessity testing for accurate causal interpretation in nonlinear time-series models.
problem Misinterpretation of causal scores from nonlinear models as regression coefficients.
method Systematic edge ablation and forecast comparison to evaluate causal necessity.
result Causal relationships with similar scores can differ in their necessity for accurate prediction.
Embedding-based Knowledge Base Completion models have so far mostly combined distributed representations of individual entities or relations to compute truth scores of missing links. Facts can however also be represented using pairwise embeddings, i.e. embeddings for pairs of entities and relations. In this paper we ex…
Long Short Term Memory Fully Convolutional Neural Networks (LSTM-FCN) and Attention LSTM-FCN (ALSTM-FCN) have shown to achieve state-of-the-art performance on the task of classifying time series signals on the old University of California-Riverside (UCR) time series repository. However, there has been no study on why L…
A novel gradient boosting framework is proposed where shallow neural networks are employed as ``weak learners''. General loss functions are considered under this unified framework with specific examples presented for classification, regression, and learning to rank. A fully corrective step is incorporated to remedy the…
Model separates overall uncertainty into aleatoric and epistemic components for active learning.
problem Active learning with uncertainty quantification.
method Non-stationary Heteroscedastic Gaussian process model.
result Model separates overall uncertainty into aleatoric and epistemic components.
Model identifies order splitting and liquidity replenishment as necessary for the square-root law of market impact.
problem Quantifying the square-root law of market impact and identifying its underlying mechanisms.
method Minimal limit-order-book model with heterogeneous interacting agents calibrated against real data. Counterfactual ablation to isolate mechanisms.
result Order splitting and liquidity replenishment are necessary for the square-root law of market impact.
ECP optimizes expensive functions without knowing Lipschitz constant.
problem Optimizing expensive, non-convex functions with unknown Lipschitz constants.
method ECP minimizes evaluations by focusing on potentially optimal regions, eliminating Lipschitz constant estimation.
result Guaranteed no-regret performance and minimax-optimal regret bounds.
Aims to create a world model without baggage, achieving good performance.
problem Creating world models without baggage and limitations.
method Self-supervised representation learning, frame and action stacking, data augmentation.
result Good performance on the Atari 100k benchmark.
Deep learning improves forensic matching of casings.
problem Identifying if two casings came from the same firearm.
method Contrastive neural networks trained on casings dataset.
result Contrastive learning achieved better ROC AUC than CMC.
Review of integrating Bayesian methods with neural network-based MPC.
problem Lack of standardized benchmarks and reliable analyses in Bayesian MPC.
method Systematic analysis of Bayesian methods in neural-network-based MPC.
result Need for standardized benchmarks, ablation studies, and transparent reporting.
Efficient neural network optimization reduces costs and improves model performance.
problem High computational costs in optimizing neural networks, especially at scale.
method Introduces self-attentive feed-forward neural units (SAFFU) for efficient optimization.
result Explicit solutions outperform models optimized by backpropagation alone, and further training with backpropagation leads to better optima from smaller data sets.
INFERS PDEs from data samples using learned context.
problem Inferring explicit PDEs from unseen dynamics.
method Contextual Finite Differences (CFD) method integrating PDE form and differential scheme.
result Yields a PDE fitting the data sample for signal prediction and explanation.
Improved electrical load forecasting model using Fourier-enhanced RNN.
problem Electrical load time series downscaling with high accuracy and low error.
method Combines recurrent neural network with Fourier seasonal embeddings and self-attention.
result Significantly reduces RMSE across different time horizons compared to existing methods.
Community detection is a fundamental problem in machine learning. While deep learning has shown great promise in many graphrelated tasks, developing neural models for community detection has received surprisingly little attention. The few existing approaches focus on detecting disjoint communities, even though communit…
IGSD separates task-specific content channels in transformer components by comparing activation replacement with zero ablation.
problem Mechanistic interpretability of transformer components
method IGSD: paired-intervention framework for comparing activation replacement with zero ablation
result IGSD identifies an early-layer content channel in transformer components that standard importance methods underestimate.
CIT and CIF improve feature selection for downstream prediction.
problem Feature selection bias in machine learning models.
method Conditional inference trees and forests with Bonferroni correction.
result CIF ranks top 3 among 18 regression methods and top 4 among 17 classification methods.
Optimizes neural networks by removing unnecessary layers, improving performance and speed.
problem Finding the optimal depth of neural networks to improve performance and speed.
method Develops a fast end-to-end method for training lightweight neural networks with multiple classifier heads, allowing the model to determine the importance of each head and choosing a single shallow classifier.
result Significantly reduces the number of parameters and accelerates inference, outperforming many standard pruning methods.
Behavioral cloning reduces policy learning to supervised learning by training a discriminative model to predict expert actions given observations. Such discriminative models are non-causal: the training procedure is unaware of the causal structure of the interaction between the expert and the environment. We point out …
Neural Architecture Search remains a very challenging meta-learning problem. Several recent techniques based on parameter-sharing idea have focused on reducing the NAS running time by leveraging proxy models, leading to architectures with competitive performance compared to those with hand-crafted designs. In this pape…
Sparse point observations can provide useful local constraints, but their benefit for radar-like fields depends on the training loss, uncertainty representation, and how observation support is encoded in the model.
problem Improving dense radar-field forecasts with sparse point observations
method A multimodal graph neural network nowcasting system over the Nordic radar domain
result Each source improves a different part of the forecast problem
New insights into training deep networks with low rank layers.
problem Efficiency in training deep neural networks.
method Analysis of techniques for training in low rank space.
result Falsified common beliefs in training deep networks.
A novel nonstationary permanental process relaxes kernel constraints and captures complex data patterns.
problem Limitations of existing permanental processes in terms of kernel types and stationarity.
method Sparse spectral representation of nonstationary kernels and hierarchical stacking of spectral feature mappings.
result Enhanced model expressiveness and reduced computational complexity.
FraudTransformer detects payment fraud by preserving event order and time gaps.
problem Detecting payment fraud in real-world banking streams with irregular time gaps.
method Augments a GPT-style architecture with a dedicated time encoder and a learned positional encoder.
result FraudTransformer outperforms classical and transformer baselines, achieving highest AUROC and PRAUC on held-out test set.
In this paper, we introduce a new model for leveraging unlabeled data to improve generalization performances of image classifiers: a two-branch encoder-decoder architecture called HybridNet. The first branch receives supervision signal and is dedicated to the extraction of invariant class-related representations. The s…
LSTMs were introduced to combat vanishing gradients in simple RNNs by augmenting them with gated additive recurrent connections. We present an alternative view to explain the success of LSTMs: the gates themselves are versatile recurrent models that provide more representational power than previously appreciated. We do…
Improved neural topic model for semi-supervised learning.
problem Representing textual data in an interpretable manner with limited labeled data.
method Label-Indexed Neural Topic Model (LI-NTM) that combines deep generative models with semi-supervised learning.
result LI-NTM outperforms existing models in document reconstruction and classifier performance.
Deep neural networks have achieved impressive performance and become the de-facto standard in many tasks. However, troubling phenomena such as adversarial and fooling examples suggest that the generalization they make is flawed. I argue that among the roots of the phenomena are two geometric properties of common deep l…
EXAGREE selects a stakeholder-aligned model to reduce conflicting explanations in machine learning.
problem Conflicting explanations from different attribution methods limit the adoption of machine learning models in safety-critical domains.
method EXAGREE is a two-stage framework that selects a Stakeholder-Aligned Explanation Model (SAEM) from a set of similar-performing models, maximizing Stakeholder-Machine Agreement (SMA).
result EXAGREE achieves simultaneous gains in faithfulness, plausibility, and fairness over baselines while preserving task accuracy.
We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation. FiLM layers influence neural network computation via a simple, feature-wise affine transformation based on conditioning information. We show that FiLM layers are highly effective for visual reasoning - an…
RNNs solve modular addition tasks using low rank and sparse Fourier structures.
problem Solving modular addition tasks with recurrent neural networks.
method Identified low rank structures and sparse Fourier representations in RNN weights.
result RNNs robust to removing individual frequencies but degrade with more ablation.
Neural framework for conditional OT maps learns from categorical and continuous variables.
problem Learning conditional optimal transport maps between complex distributions.
method Hypernetwork generates adaptive transport layer parameters based on conditioning variables.
result Our method outperforms simpler conditioning methods in comprehensive ablation studies.
MixBoost generates synthetic instances to balance imbalanced datasets.
problem Training models on imbalanced datasets.
method Iterative data augmentation method that selects and combines instances from majority and minority classes.
result MixBoost outperforms existing approaches on 20 benchmark datasets.
The AlphaGo, AlphaGo Zero, and AlphaZero series of algorithms are remarkable demonstrations of deep reinforcement learning's capabilities, achieving superhuman performance in the complex game of Go with progressively increasing autonomy. However, many obstacles remain in the understanding of and usability of these prom…