Study ablated data augmentation techniques and their mathematical equivalence to penalties.
problem Lack of mathematical understanding of differences between ablated data augmentation techniques.
method Formal model of mean ablated data augmentation and inverted dropout for linear regression; empirical validation for deep networks.
result Ablated data augmentation and inverted dropout are mathematically equivalent to penalties in optimization.
New method certifies robustness to sparse adversarial attacks.
problem Ensuring robustness of classifiers to sparse adversarial attacks.
method Randomized ablation of input features to provide robustness guarantees.
result Certifies robustness to sparse adversarial attacks with up to 31 pixels of distortion.
Given a model f that predicts a target y from a vector of input features x=x1,x2,…,xM, we seek to measure the importance of each feature with respect to the model's ability to make a good prediction. To this end, we consider how (on average) some measure of goodness or badness of prediction (wh…
Predicts stock volatility using Twitter data and random forests.
problem Predicting stock implied volatility using Twitter data.
method Random forests with ablation study on different predictors, including Twitter attention and sentiment features.
result Certain sectors like Consumer Discretionary, Technology, Real Estate, and Utilities are easier to predict.
Ablation studies show BCF model's propensity score is not essential for treatment effect estimation.
problem Understanding the necessity of propensity score in nonparametric treatment effect estimation.
method Partial ablation studies of Bayesian Causal Forest (BCF) model.
result Excluding estimated propensity score does not affect treatment effect estimation or uncertainty quantification.
Method generates anatomically-controllable medical images with segmentation guidance.
problem Challenging to enforce anatomical constraints in generated medical images.
method Segmentation-guided diffusion models with random mask ablation training.
result New state-of-the-art in faithfulness to input anatomical masks.
ABC method analyzes diffusion model dependence without retraining.
problem Difficulties in characterizing diffusion models' dependence on training data.
method Ablation-based counterfactuals using model components trained on different splits.
result Demonstrates limits of training data attribution and existence of unattributable samples.
In microsurgery, lasers have emerged as precise tools for bone ablation. A challenge is automatic control of laser bone ablation with 4D optical coherence tomography (OCT). OCT as high resolution imaging modality provides volumetric images of tissue and foresees information of bone position and orientation (pose) as we…
Recent advances in deep learning-based object detection techniques have revolutionized their applicability in several fields. However, since these methods rely on unwieldy and large amounts of data, a common practice is to download models pre-trained on standard datasets and fine-tune them for specific application doma…
MLDL preserves manifold geometry in vector transformations.
problem Geometric deterioration in neural network transformations.
method Locally isometric smoothness (LIS) and Markov random field (MRF) encoding.
result Enhanced vector transformations into well-behaved metric homeomorphisms.
Off-policy reinforcement learning (RL) using a fixed offline dataset of logged interactions is an important consideration in real world applications. This paper studies offline RL using the DQN replay dataset comprising the entire replay experience of a DQN agent on 60 Atari 2600 games. We demonstrate that recent off-p…
RFFNet scales kernel methods to large datasets by learning kernel relevance.
problem Scaling kernel methods to large datasets while maintaining interpretability.
method Designs random Fourier features for ARD kernels and uses first-order stochastic optimization for learning kernel relevances.
result RFFNet achieves low prediction error and identifies relevant features, leading to more interpretable solutions.
In this work, we investigate the use of three information-theoretic quantities -- entropy, mutual information with the class variable, and a class selectivity measure based on Kullback-Leibler divergence -- to understand and study the behavior of already trained fully-connected feed-forward neural networks. We analyze …
TSFMs show redundant components in layers, affecting their performance.
problem Redundant components in TSFMs layers impact their predictive accuracy.
method Mechanistic interpretability tools, ablations, logit attribution, theoretical framework.
result TSFMs are robust to ablations of entire layers and specific heads.
ECPv2 optimizes Lipschitz functions efficiently and scalably.
problem Global optimization of Lipschitz-continuous functions with unknown Lipschitz constants.
method Adapting the Every Call is Precious (ECP) framework, ECPv2 introduces adaptive lower bounds, Worst-m memory, and random projections to reduce computational cost and improve acceptance regions.
result ECPv2 retains ECP's no-regret guarantees with optimal finite-time bounds and expands the acceptance region with high probability.
RIFLE improves deep transfer learning by reinitializing fully-connected layers.
problem Limited improvement in transfer learning accuracy with pre-trained models on small datasets.
method Re-Initializing fully-connected layers with random scratch during fine-tuning.
result Significant improvement in deep transfer learning accuracy across various datasets.
AL2 progressively penalizes network activations to prevent overfitting.
problem Avoiding overfitting in neural networks with limited training data.
method Progressive Activation Loss (PAL) method to regularize network representations.
result AL2 outperforms traditional regularization methods on benchmark datasets.
DeRisk improves credit risk prediction using deep learning.
problem Challenges in training deep neural networks with real-world financial data.
method DeRisk, an effective deep learning framework for credit risk prediction.
result DeRisk outperforms statistical learning methods in credit risk prediction.
Local MDI+ improves feature importance for tree-based models, enhancing interpretability and performance.
problem Lack of instance-specific feature importance for tree-based models.
method Local MDI+ extends MDI+ framework to provide instance-specific feature importances.
result Local MDI+ outperforms existing baselines, improving predictive performance by 10%.
New framework improves offline RL performance without complex methods.
problem Limited access to online interactions in real-world RL applications.
method Behavior Regularized Actor Critic (BRAC) framework.
result Many recent technical complexities are not necessary for strong offline RL performance.
Curriculum learning and imitation learning improve control over financial time-series data.
problem Improving control performance over complex financial time-series data.
method Data augmentation for curriculum learning and policy distillation for imitation learning.
result Curriculum learning shows significant improvement over time-series control tasks.
New method creates diverse neural ensembles for better uncertainty estimation and robustness.
problem Creating more robust neural networks for uncertainty estimation and dataset shift.
method Automatically constructing ensembles with varying architectures.
result Ensembles with varying architectures outperform deep ensembles in accuracy, uncertainty calibration, and robustness.
We analyze the dynamics of training deep ReLU networks and their implications on generalization capability. Using a teacher-student setting, we discovered a novel relationship between the gradient received by hidden student nodes and the activations of teacher nodes for deep ReLU networks. With this relationship and th…
Hydra boosts efficiency for long-context reasoning in resource-constrained settings.
problem Quadratic complexity of transformers limits long-context reasoning in resource-constrained systems.
method Hydra uses a modular architecture with adaptive routing between sparse global attention, mixture-of-experts, and dual memories.
result Hydra achieves significant throughput and accuracy improvements for long-context reasoning.
iGCL preserves graph semantics in latent space augmentations.
problem Manual tuning of augmentation ratios and unexpected graph changes.
method iGCL uses a Variational Graph Auto-Encoder to learn augmentations in the latent space, optimizing an upper bound for contrastive loss.
result iGCL achieves state-of-the-art performance on graph-level and node-level tasks.
New dataset tests mental rotation from single images, improving model understanding of 3D scenes.
problem Understanding how a scene looks from a different viewpoint using a single image.
method Created CLEVR-MRT dataset, explored neural architectures for volumetric scene representations.
result Demonstrated the effectiveness of volumetric representations in answering mental rotation questions.
Paper analyzes biases in video QA datasets, showing models can answer 37-48% questions correctly without multimodal context.
problem Question answering biases in video QA datasets can lead to model overfitting and poor generalization.
method Analyzed popular video question answering datasets, conducted ablation studies on biases from annotators and question types.
result Pretrained language models can answer 37-48% questions correctly without multimodal context, far exceeding random guess baseline.
ECP optimizes expensive functions without knowing Lipschitz constant.
problem Optimizing expensive, non-convex functions with unknown Lipschitz constants.
method ECP minimizes evaluations by focusing on potentially optimal regions, eliminating Lipschitz constant estimation.
result Guaranteed no-regret performance and minimax-optimal regret bounds.
Deep learning improves forensic matching of casings.
problem Identifying if two casings came from the same firearm.
method Contrastive neural networks trained on casings dataset.
result Contrastive learning achieved better ROC AUC than CMC.
GCN model fails to match KG entities as expected.
problem Entity alignment in Knowledge Graphs using GCN.
method Used Graph Convolutional Network (GCN) for entity alignment in KGs.
result Implementation of GCN model differs from original paper, requiring additional tricks.
Locally-contextual CRFs improve sequence labeling performance.
problem Improving sequence labeling with contextual embeddings.
method Locally-contextual nonlinear CRFs using deep neural networks.
result Consistently outperforms linear chain CRF and previous state of the art.
Long Short Term Memory Fully Convolutional Neural Networks (LSTM-FCN) and Attention LSTM-FCN (ALSTM-FCN) have shown to achieve state-of-the-art performance on the task of classifying time series signals on the old University of California-Riverside (UCR) time series repository. However, there has been no study on why L…
GCNs help in diagnosing label scarcity and feature quality on graphs.
problem Understanding when GCNs improve node classification.
method Simulated label scarcity, feature ablation, and per-class analysis.
result GCNs provide largest gains under extreme label scarcity, matching original performance with noisy features, but hurt when homophily is low and features are strong.
This paper presents a supervised learning algorithm, namely, the Synaptic Efficacy Function with Meta-neuron based learning algorithm (SEF-M) for a spiking neural network with a time-varying weight model. For a given pattern, SEF-M uses the learning algorithm derived from meta-neuron based learning algorithm to determi…
We review some of the latest approaches to analysing cardiac electrophysiology data using machine learning and predictive modelling. Cardiac arrhythmias, particularly atrial fibrillation, are a major global healthcare challenge. Treatment is often through catheter ablation, which involves the targeted localized destruc…
This paper proposes a deep learning model combining CNN and Transformer for improved credit default prediction.
problem Traditional machine learning models struggle with complex financial data and risk patterns.
method Combines CNN for local feature extraction and Transformer for global dependency modeling.
result The CNN+Transformer model outperforms traditional models in accuracy, AUC, and KS value.
Double descent phenomenon explained in simple terms.
problem Understanding the surprising drop in test error in overparameterized models.
method Informal explanation using linear algebra and probability, visual intuition with polynomial regression, mathematical analysis with ordinary linear regression.
result Three factors create double descent: data undersampling, model size, and parameter count. Ablating any one of these factors prevents double descent.
IGSD separates task-specific content channels in transformer components by comparing activation replacement with zero ablation.
problem Mechanistic interpretability of transformer components
method IGSD: paired-intervention framework for comparing activation replacement with zero ablation
result IGSD identifies an early-layer content channel in transformer components that standard importance methods underestimate.
CIT and CIF improve feature selection for downstream prediction.
problem Feature selection bias in machine learning models.
method Conditional inference trees and forests with Bonferroni correction.
result CIF ranks top 3 among 18 regression methods and top 4 among 17 classification methods.
Study improves reliability of neural models for virtual screening.
problem Reliability issues in neural models for molecular property prediction.
method Investigated model architectures, regularization, and loss functions.
result Correct choice of regularization and inference methods improves reliability.
Paper proposes forecast-necessity testing for accurate causal interpretation in nonlinear time-series models.
problem Misinterpretation of causal scores from nonlinear models as regression coefficients.
method Systematic edge ablation and forecast comparison to evaluate causal necessity.
result Causal relationships with similar scores can differ in their necessity for accurate prediction.
Embedding-based Knowledge Base Completion models have so far mostly combined distributed representations of individual entities or relations to compute truth scores of missing links. Facts can however also be represented using pairwise embeddings, i.e. embeddings for pairs of entities and relations. In this paper we ex…
Model identifies order splitting and liquidity replenishment as necessary for the square-root law of market impact.
problem Quantifying the square-root law of market impact and identifying its underlying mechanisms.
method Minimal limit-order-book model with heterogeneous interacting agents calibrated against real data. Counterfactual ablation to isolate mechanisms.
result Order splitting and liquidity replenishment are necessary for the square-root law of market impact.
INFERS PDEs from data samples using learned context.
problem Inferring explicit PDEs from unseen dynamics.
method Contextual Finite Differences (CFD) method integrating PDE form and differential scheme.
result Yields a PDE fitting the data sample for signal prediction and explanation.
Model separates overall uncertainty into aleatoric and epistemic components for active learning.
problem Active learning with uncertainty quantification.
method Non-stationary Heteroscedastic Gaussian process model.
result Model separates overall uncertainty into aleatoric and epistemic components.
Paper analyzes and improves GPSP algorithm for block sparse signal recovery.
problem Recovering block sparse signals from noisy data.
method Group Projected Subspace Pursuit (GPSP) with convergence analysis and feature selection criteria.
result GPSP exactly recovers true block sparse signals under certain conditions.
Improved electrical load forecasting model using Fourier-enhanced RNN.
problem Electrical load time series downscaling with high accuracy and low error.
method Combines recurrent neural network with Fourier seasonal embeddings and self-attention.
result Significantly reduces RMSE across different time horizons compared to existing methods.
Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem. In this paper, we give a formalization of this shared pattern, which we call GIMLI, prove its general requirements, and derive…