Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

183366548731 · Jun 202019922001200920172026
48 results for Ablation studies

Ablation studies show BCF model's propensity score is not essential for treatment effect estimation.

problem Understanding the necessity of propensity score in nonparametric treatment effect estimation.
method Partial ablation studies of Bayesian Causal Forest (BCF) model.
result Excluding estimated propensity score does not affect treatment effect estimation or uncertainty quantification.

Study ablated data augmentation techniques and their mathematical equivalence to penalties.

problem Lack of mathematical understanding of differences between ablated data augmentation techniques.
method Formal model of mean ablated data augmentation and inverted dropout for linear regression; empirical validation for deep networks.
result Ablated data augmentation and inverted dropout are mathematically equivalent to penalties in optimization.

Given a model ff that predicts a target yy from a vector of input features x=x1,x2,,xM\pmb{x} = x_1, x_2, \ldots, x_M, we seek to measure the importance of each feature with respect to the model's ability to make a good prediction. To this end, we consider how (on average) some measure of goodness or badness of prediction (wh…

2019-10-01abs ↗pdf ↗

Predicts stock volatility using Twitter data and random forests.

problem Predicting stock implied volatility using Twitter data.
method Random forests with ablation study on different predictors, including Twitter attention and sentiment features.
result Certain sectors like Consumer Discretionary, Technology, Real Estate, and Utilities are easier to predict.

ECP optimizes expensive functions without knowing Lipschitz constant.

problem Optimizing expensive, non-convex functions with unknown Lipschitz constants.
method ECP minimizes evaluations by focusing on potentially optimal regions, eliminating Lipschitz constant estimation.
result Guaranteed no-regret performance and minimax-optimal regret bounds.

Hydra boosts efficiency for long-context reasoning in resource-constrained settings.

problem Quadratic complexity of transformers limits long-context reasoning in resource-constrained systems.
method Hydra uses a modular architecture with adaptive routing between sparse global attention, mixture-of-experts, and dual memories.
result Hydra achieves significant throughput and accuracy improvements for long-context reasoning.

Paper proposes forecast-necessity testing for accurate causal interpretation in nonlinear time-series models.

problem Misinterpretation of causal scores from nonlinear models as regression coefficients.
method Systematic edge ablation and forecast comparison to evaluate causal necessity.
result Causal relationships with similar scores can differ in their necessity for accurate prediction.

A novel gradient boosting framework is proposed where shallow neural networks are employed as ``weak learners''. General loss functions are considered under this unified framework with specific examples presented for classification, regression, and learning to rank. A fully corrective step is incorporated to remedy the…

2020-02-19abs ↗pdf ↗

In this paper we study the problem of acoustic scene classification, i.e., categorization of audio sequences into mutually exclusive classes based on their spectral content. We describe the methods and results discovered during a competition organized in the context of a graduate machine learning course; both by the st…

2018-08-02abs ↗pdf ↗

Review of integrating Bayesian methods with neural network-based MPC.

problem Lack of standardized benchmarks and reliable analyses in Bayesian MPC.
method Systematic analysis of Bayesian methods in neural-network-based MPC.
result Need for standardized benchmarks, ablation studies, and transparent reporting.

New dataset tests mental rotation from single images, improving model understanding of 3D scenes.

problem Understanding how a scene looks from a different viewpoint using a single image.
method Created CLEVR-MRT dataset, explored neural architectures for volumetric scene representations.
result Demonstrated the effectiveness of volumetric representations in answering mental rotation questions.

Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem. In this paper, we give a formalization of this shared pattern, which we call GIMLI, prove its general requirements, and derive…

2019-10-03abs ↗pdf ↗

In this work, we introduce a new method for imitation learning from video demonstrations. Our method, Relational Mimic (RM), improves on previous visual imitation learning methods by combining generative adversarial networks and relational learning. RM is flexible and can be used in conjunction with other recent advanc…

2019-12-18abs ↗pdf ↗

Neural Architecture Search remains a very challenging meta-learning problem. Several recent techniques based on parameter-sharing idea have focused on reducing the NAS running time by leveraging proxy models, leading to architectures with competitive performance compared to those with hand-crafted designs. In this pape…

2019-06-07abs ↗pdf ↗

Enhances neural forecasting for hierarchically organized time series data.

problem Probabilistic coherent forecasting of time series data across different levels of aggregation.
method Proposes a coherent multivariate mixture output for neural forecasting architectures, optimizing with a composite likelihood objective.
result 13.2% average accuracy improvements on most datasets compared to state-of-the-art baselines.

The problem of attributing a deep network's prediction to its \emph{input/base} features is well-studied. We introduce the notion of \emph{conductance} to extend the notion of attribution to the understanding the importance of \emph{hidden} units. Informally, the conductance of a hidden unit of a deep network is the \e…

2018-05-30abs ↗pdf ↗

Bayesian optimization with preference learning using monotonic neural networks.

problem Optimizing complex systems with multiple conflicting objectives.
method Proposes a neural network ensemble for utility surrogate modeling, leveraging monotonicity.
result Demonstrates superior performance compared to existing methods.

New mechanisms from primate vision improve neural network robustness.

problem Demonstrating robust neural networks to small adversarial perturbations.
method Investigated two biologically plausible mechanisms: non-uniform retina sampling and receptive field diversity.
result Non-uniform retina sampling and receptive field diversity improve adversarial robustness.

Sparse point observations can provide useful local constraints, but their benefit for radar-like fields depends on the training loss, uncertainty representation, and how observation support is encoded in the model.

problem Improving dense radar-field forecasts with sparse point observations
method A multimodal graph neural network nowcasting system over the Nordic radar domain
result Each source improves a different part of the forecast problem

Towards the challenging problem of semi-supervised node classification, there have been extensive studies. As a frontier, Graph Neural Networks (GNNs) have aroused great interest recently, which update the representation of each node by aggregating information of its neighbors. However, most GNNs have shallow layers wi…

2019-10-07abs ↗pdf ↗

Optimizes neural networks by removing unnecessary layers, improving performance and speed.

problem Finding the optimal depth of neural networks to improve performance and speed.
method Develops a fast end-to-end method for training lightweight neural networks with multiple classifier heads, allowing the model to determine the importance of each head and choosing a single shallow classifier.
result Significantly reduces the number of parameters and accelerates inference, outperforming many standard pruning methods.

Community detection is a fundamental problem in machine learning. While deep learning has shown great promise in many graphrelated tasks, developing neural models for community detection has received surprisingly little attention. The few existing approaches focus on detecting disjoint communities, even though communit…

2019-09-26abs ↗pdf ↗

Method generates anatomically-controllable medical images with segmentation guidance.

problem Challenging to enforce anatomical constraints in generated medical images.
method Segmentation-guided diffusion models with random mask ablation training.
result New state-of-the-art in faithfulness to input anatomical masks.

Double descent phenomenon explained in simple terms.

problem Understanding the surprising drop in test error in overparameterized models.
method Informal explanation using linear algebra and probability, visual intuition with polynomial regression, mathematical analysis with ordinary linear regression.
result Three factors create double descent: data undersampling, model size, and parameter count. Ablating any one of these factors prevents double descent.

Sparse reward is one of the biggest challenges in reinforcement learning (RL). In this paper, we propose a novel method called Generative Exploration and Exploitation (GENE) to overcome sparse reward. GENE automatically generates start states to encourage the agent to explore the environment and to exploit received rew…

2019-04-21abs ↗pdf ↗

IGSD separates task-specific content channels in transformer components by comparing activation replacement with zero ablation.

problem Mechanistic interpretability of transformer components
method IGSD: paired-intervention framework for comparing activation replacement with zero ablation
result IGSD identifies an early-layer content channel in transformer components that standard importance methods underestimate.

Colored noise improves neural network robustness against adversarial attacks.

problem Vulnerability of neural networks to adversarial perturbations.
method Injection of colored noise into network weights and activations during adversarial training.
result Our approach outperforms previous methods in terms of adversarial accuracy on CIFAR-10 and CIFAR-100 datasets.

EXAGREE selects a stakeholder-aligned model to reduce conflicting explanations in machine learning.

problem Conflicting explanations from different attribution methods limit the adoption of machine learning models in safety-critical domains.
method EXAGREE is a two-stage framework that selects a Stakeholder-Aligned Explanation Model (SAEM) from a set of similar-performing models, maximizing Stakeholder-Machine Agreement (SMA).
result EXAGREE achieves simultaneous gains in faithfulness, plausibility, and fairness over baselines while preserving task accuracy.

Researchers propose better probabilistic models for deep learning.

problem Using cross-entropy loss for non-categorical data.
method Introducing continuous-categorical distribution and proposing probabilistic alternatives.
result Potential for outperformance in deep learning models with proper probabilistic treatment.