Adversarial examples have become one of the largest challenges that machine learning models, especially neural network classifiers, face. These adversarial examples break the assumption of attack-free scenario and fool state-of-the-art (SOTA) classifiers with insignificant perturbations to human. So far, researchers ac…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Single-step Computation”
A new method reduces adversarial training time without overfitting.
problem Catastrophic overfitting in single-step adversarial training.
method FGSMPR: FGSM with PGD Regularization.
result Reduces the gap to multi-step adversarial training.
Paper tackles catastrophic overfitting in single-step adversarial training.
problem Catastrophic overfitting leads to sudden drop in robust accuracy.
method Proposes a method to prevent overfitting by using all adversarial examples.
result Demonstrates prevention of catastrophic overfitting and improves robustness.
CARV reduces compute cost for downstream pipelines using diffusion models.
problem High variance in Monte Carlo estimators from diffusion models limits compute efficiency.
method CARV uses hierarchical MC estimation with amortized upstream computation and stratified-inverse-CDF.
result CARV delivers 2-3x effective compute multipliers without changing the objective.
Lazy neural networks are vulnerable to adversarial attacks.
problem Adversarial robustness issues in lazy training models.
method Extending recent work on adversarial examples to lazy training models.
result Over-parametrized neural networks that generalize well remain vulnerable to single-step gradient ascent attacks.
Single-step samplers generate high-quality samples efficiently.
problem Sampling from unnormalized distributions is computationally expensive.
method Developed consistent diffusion samplers that generate samples in a single step.
result Single-step samplers produce high-fidelity samples with less than 1% of traditional samplers' evaluations.
Efficient regularization mitigates catastrophic overfitting in single-step adversarial training.
problem Catastrophic overfitting in single-step adversarial training.
method ELLE regularization term to enforce local linearity of the loss function.
result Our regularization term effectively mitigates catastrophic overfitting without the drawbacks of previous methods.
New method improves adversarial training efficiency and robustness.
problem High computational costs and lack of stability in adversarial training.
method Backward smoothing for randomized smoothing of random initialization.
result Our method achieves similar model robustness as state-of-the-art methods but with significantly less training time.
This paper establishes a theoretical foundation for consistency training in diffusion models.
problem Lack of a comprehensive theoretical understanding of consistency training in diffusion models.
method Demonstrates the necessity of a number of steps in consistency learning exceeding for generating samples within proximity to the target distribution.
result Establishes rigorous insights into the validity and efficacy of consistency models, offering theoretical underpinnings for their utility.
We present an extension of our Molecular Transformer architecture combined with a hyper-graph exploration strategy for automatic retrosynthesis route planning without human intervention. The single-step retrosynthetic model sets a new state of the art for predicting reactants as well as reagents, solvents and catalysts…
Adversarial examples are malicious inputs designed to fool machine learning models. They often transfer from one model to another, allowing attackers to mount black box attacks without knowledge of the target model's parameters. Adversarial training is the process of explicitly training a model on adversarial examples,…
TKFT models computation via smooth vector fields, simulating functions in a single dynamical step.
problem Modeling computation in a single step.
method Established Topological Kleene Field Theory (TKFT) as a new model of computation.
result Any computable function can be simulated in a single go of a dynamical system.
Maximum Entropy Model Rollouts: Fast Model Based Policy Optimization without Compounding Errorscs.LG
A new method reduces compounding errors in model-based reinforcement learning.
problem Compounding errors in long horizon predictions from model-based reinforcement learning.
method Maximum Entropy Model Rollouts (MEMR) with non-uniform sampling and prioritized experience replay.
result Significantly reduces computation requirements compared to other model-based methods.
Certified training improves robustness against adversarial attacks.
problem Certified training's gap with empirical robustness limits its practical utility.
method Combining adversarial attacks with network over-approximations.
result Certified training can prevent catastrophic overfitting and bridge the gap to multi-step baselines.
Adversarial examples are perturbed inputs designed to fool machine learning models. Adversarial training injects such examples into training data to increase robustness. To scale this technique to large datasets, perturbations are crafted using fast single-step methods that maximize a linear approximation of the model'…
A novel optimisation framework through quadratic nonlinear projection is introduced for credit portfolio when the portfolio risk is measured by Conditional Value-at-Risk (CVaR). The whole optimisation procedure to search toward the optimal portfolio state is conducted by a series of single-step optimisations under the …
Closed-form pricing method for multi-asset options.
problem Pricing multi-asset contingent claims in an incomplete market.
method Proving extremal martingale measures and constructing algorithms for bounds and hedging.
result Closed-form formulas for no-arbitrage price intervals and hedging strategies.
Improved chemical reaction prediction using augmented NLP models.
problem Predicting chemical reactions from text representations.
method Data augmentation and Transformer architecture for SMILES representation.
result Significantly improved accuracy in predicting chemical reactions.
BONG optimizes Bayesian inference online with natural gradient descent.
problem Sequential Bayesian inference in online settings.
method Bayesian online natural gradient (BONG) approach based on variational Bayes.
result BONG outperforms other online VB methods in non-conjugate settings.
Unified model improves sampling speed and quality.
problem Difficult to balance sampling speed and quality.
method Multistep Consistency Models combining consistency and diffusion models.
result Improved sampling quality with reduced steps.
The problem of automatic and accurate forecasting of time-series data has always been an interesting challenge for the machine learning and forecasting community. A majority of the real-world time-series problems have non-stationary characteristics that make the understanding of trend and seasonality difficult. Our int…
Interaction-Grounded Learning for Contextual Markov Decision Processes with Personalized Feedbackcs.LG
Efficient algorithm for learning from indirect feedback in complex decision-making scenarios.
problem Learning from indirect feedback in realistic scenarios with personalized mechanisms.
method IGW algorithm for policy optimization, extending reward-estimator construction from single-step to multi-step.
result Achieves sublinear regret guarantee for contextual episodic MDPs with personalized feedback.
Study optimizes compute usage for LLM web agents, improving performance.
problem High compute costs and narrow focus on single-step tasks limit LLM web agents.
method Two-stage pipeline: SFT followed by on-policy RL, with hyperparameter optimization.
result Combining SFT and on-policy RL requires 55% less compute to match peak SFT performance.
Gradient descent memorizes many Gaussians efficiently.
problem Memorizing many Gaussians with minimal parameters.
method Gradient descent on a depth-two neural network.
result One step of gradient descent memorizes $Ω\left(\frac{dq}{\log^4(d)}
ight)$ Gaussians.
Deep neural networks excel at learning the training data, but often provide incorrect and confident predictions when evaluated on slightly different test examples. This includes distribution shifts, outliers, and adversarial examples. To address these issues, we propose Manifold Mixup, a simple regularizer that encoura…
Hopfield networks improve reaction template prediction for few/zero-shot scenarios.
problem Predicting reaction templates for new molecules in CASP.
method Adapted Hopfield networks to associate reaction templates, molecules, and structural information.
result Significantly improved performance for templates with few or zero training examples.
QBSD optimizes KPI forecasting for RAN networks with fast runtime and accuracy.
problem Efficiently forecasting KPIs for RAN networks with dynamic operating ranges.
method Quartile-Based Seasonality Decomposition (QBSD) for live single-step forecasting.
result QBSD outperforms other methods in runtime efficiency and forecast accuracy.
We show that the lack of arbitrage in a model with both fixed and proportional transaction costs is equivalent to the existence of a family of absolutely continuous single-step probability measures, together with an adapted process with values between the bid-ask spreads that satisfies the martingale property with resp…
Bounded rationality, that is, decision-making and planning under resource limitations, is widely regarded as an important open problem in artificial intelligence, reinforcement learning, computational neuroscience and economics. This paper offers a consolidated presentation of a theory of bounded rationality based on i…
Due to the surprisingly good representation power of complex distributions, neural network (NN) classifiers are widely used in many tasks which include natural language processing, computer vision and cyber security. In recent works, people noticed the existence of adversarial examples. These adversarial examples break…
A major drawback of backpropagation through time (BPTT) is the difficulty of learning long-term dependencies, coming from having to propagate credit information backwards through every single step of the forward computation. This makes BPTT both computationally impractical and biologically implausible. For this reason,…
Advances in deep neural networks (DNN) greatly bolster real-time detection of anomalous IoT data. However, IoT devices can barely afford complex DNN models due to limited computational power and energy supply. While one can offload anomaly detection tasks to the cloud, it incurs long delay and requires large bandwidth …
Study matches two noisy point clouds with geometric transformations and relabeling.
problem Matching two noisy point clouds with orthogonal transformations and relabeling.
method Information-theoretic results and Ping-Pong algorithm for computational alignment.
result The Ping-Pong algorithm retrieves the planted signal after one step.
A new method improves stochastic gradient descent for faster and more efficient estimation.
problem Efficient and fast parametric estimation methods.
method Projected stochastic gradient descent corrected by Fisher scoring.
result The method is faster and more efficient than traditional methods.
We study an exploration method for model-free RL that generalizes the counter-based exploration bonus methods and takes into account long term exploratory value of actions rather than a single step look-ahead. We propose a model-free RL method that modifies Delayed Q-learning and utilizes the long-term exploration bonu…
In this work, we describe practical lessons we have learned from successfully using contextual bandits (CBs) to improve key business metrics of the Microsoft Virtual Agent for customer support. While our current use cases focus on single step einforcement learning (RL) and mostly in the domain of natural language proce…
When environmental interaction is expensive, model-based reinforcement learning offers a solution by planning ahead and avoiding costly mistakes. Model-based agents typically learn a single-step transition model. In this paper, we propose a multi-step model that predicts the outcome of an action sequence with variable …
Paper proposes a new method to speed up diffusion models.
problem High computational cost of sampling from diffusion models.
method Stochastic Runge-Kutta method for acceleration.
result Provable acceleration with reduced score function evaluations.
Generative model emulates climate model for 100-year forecasts.
problem Challenges in accurately simulating long-term climate data.
method Integrates DYffusion with SFNO for stable, accurate climate simulations.
result Achieves near gold-standard performance for climate model emulation.
This paper optimizes high-dimensional oblique splits for decision trees, enhancing performance and computational efficiency.
problem Enhancing decision tree performance and computational efficiency in high-dimensional data.
method Established Sufficient Impurity Decrease (SID) convergence for -sparse oblique splits, proposing progressive trees for iterative refinement.
result Demonstrated that SID function class expands with -sparsity, enabling capture of complex data-generating processes.
CTM improves diffusion model sampling quality with efficient ODE traversal.
problem Lack of natural trade-off between sample quality and speed in consistency models.
method CTM trains a neural network to output scores and traverse ODE trajectories efficiently.
result CTM achieves state-of-the-art FIDs and improves sample quality with increased computational budget.
Machine Learning for Pricing American Options in High-Dimensional Markovian and non-Markovian modelsq-fin.CP
In this paper we propose two efficient techniques which allow one to compute the price of American basket options. In particular, we consider a basket of assets that follow a multi-dimensional Black-Scholes dynamics. The proposed techniques, called GPR Tree (GRP-Tree) and GPR Exact Integration (GPR-EI), are both based …
Although there has been a rapid development of practical applications, theoretical explanations of deep learning are in their infancy. Deep learning performs a sophisticated coarse graining. Since coarse graining is a key ingredient of the renormalization group (RG), RG may provide a useful theoretical framework direct…
Efficiently clusters data sequences with varying distributions.
problem Sequentially grouping data sequences with fixed error probability.
method Proposes Efficient Bandit Clustering (EBC) algorithm for asymptotic optimality and computational efficiency.
result EBC and EBC-H achieve better performance and computational efficiency compared to existing methods.
Adaptive anomaly detection for IoT data reduces delay without sacrificing accuracy.
problem Real-time anomaly detection for IoT data in distributed edge computing systems.
method Adaptive anomaly detection approach using contextual bandit and reinforcement learning.
result Significantly reduces detection delay (e.g., 71.4% for univariate data) without sacrificing accuracy.
Learning with a primary objective, such as softmax cross entropy for classification and sequence generation, has been the norm for training deep neural networks for years. Although being a widely-adopted approach, using cross entropy as the primary objective exploits mostly the information from the ground-truth class f…
Unified framework improves meta-learning generalization bounds.
problem Limited sharpness of existing meta-generalization bounds.
method Unified information-theoretic derivation for single-step bounds.
result Unified bounds exhibit tighter scaling and computational advantages.
Shampoo optimizes preconditioners for faster convergence in machine learning.
problem Improving convergence speed in machine learning optimization.
method Explicit connection between Shampoo's Kronecker product approximation and optimal matrix approximations.
result The square of Shampoo's approximation is equivalent to a single power iteration step for optimal Kronecker product approximation.