The study examines when MAML's objective has a benign landscape.
problem Understanding when MAML's objective landscape is benign.
method Analyzing the landscape of MAML objective on LQR tasks.
result The benign landscape of the MAML objective depends on task similarities.
Deep RL policies share adversarial features across different MDPs.
problem Understanding decision boundaries and loss landscapes in neural policies.
method Investigating similarities in high sensitivity directions across MDPs using Arcade Learning Environment.
result High sensitivity directions for neural policies are correlated across MDPs, suggesting shared non-robust features.
Less-than-one-shot learning tackles few-shot learning with minimal data.
problem Training models on very small datasets while maintaining accuracy.
method Soft-label k-Nearest Neighbors classifier and theoretical lower bounds analysis.
result Achieving learning of multiple classes with fewer than the required samples.
Machine learning algorithms can be fooled by small well-designed adversarial perturbations. This is reminiscent of cellular decision-making where ligands (called antagonists) prevent correct signalling, like in early immune recognition. We draw a formal analogy between neural networks used in machine learning and model…
Policy shifts between Trump and Biden impact ESG investments, creating volatility.
problem Dramatic policy shifts between Trump and Biden administrations affect ESG investments.
method Analyzes contrasting policies of Trump and Biden administrations and their impacts on ESG investments.
result Policy changes significantly influence ESG investments, leading to volatility and portfolio reassessment.
A barrier to the wider adoption of neural networks is their lack of interpretability. While local explanation methods exist for one prediction, most global attributions still reduce neural network decisions to a single set of features. In response, we present an approach for generating global attributions called GAM, w…
We apply a simple trading strategy for various time series of real and artificial stock prices to understand the origin of fractality observed in the resulting profit landscapes. The strategy contains only two parameters p and q, and the sell (buy) decision is made when the log return is larger (smaller) than p (…
Clarifies the various fairness definitions in ML.
problem Addressing fairness in ML with different definitions.
method Analyzes and clarifies the differences between fairness definitions.
result Provides a clearer understanding of fairness definitions in ML.
Flexible Kernels for Protein Property Prediction
problem Predicting protein properties from sparse experimental data
method Sequence kernels using evolutionary substitution matrices and local linearity
result Data-efficient models of protein property landscapes
Research examines motivations and factors influencing retailers' payment method choices.
problem Understanding motivations and factors affecting retailers' payment method choices.
method Qualitative and quantitative analysis of various factors including regulatory constraints, merchant service providers, and demographic variables.
result Lower interchange fees and regulatory constraints make card payment adoption financially feasible for merchants.
Rectified decision trees improve machine learning interpretability and effectiveness.
problem Combining interpretability and effectiveness in machine learning models.
method Knowledge distillation and modified decision tree splitting criteria.
result Soft labels improve model performance and reduce model size.
Study uses AI to predict changes in international public finances based on US markets.
problem Understanding correlations between US and international public finances.
method Artificial intelligence and neural networks to model and predict changes.
result Neural network model achieved MSE of 2.79, indicating significant correlation and impact of US market volatility on international markets.
Persistence landscapes map persistence diagrams into a function space, which may often be taken to be a Banach space or even a Hilbert space. In the latter case, it is a feature map and there is an associated kernel. The main advantage of this summary is that it allows one to apply tools from statistics and machine lea…
Reviews recent findings on neural network landscapes.
problem Non-convexity of loss functions causing bad landscapes.
method Rigorous geometric analysis and empirical exploration.
result Wide neural nets may have sub-optimal local minima.
SimStock learns stock similarities for better investment management.
problem Challenges in identifying similar stocks due to non-stationary financial markets.
method Temporal self-supervised learning framework combining SSL and temporal domain generalization.
result SimStock outperforms existing methods in finding similar stocks.
Dimensionality reduction is ubiquitous in analysis of complex dynamics. The conventional dimensionality reduction techniques, however, focus on reproducing the underlying configuration space, rather than the dynamics itself. The constructed low-dimensional space does not provide complete and accurate description of the…
This work proposes xGEMs or manifold guided exemplars, a framework to understand black-box classifier behavior by exploring the landscape of the underlying data manifold as data points cross decision boundaries. To do so, we train an unsupervised implicit generative model -- treated as a proxy to the data manifold. We …
We characterize value functions in partially observable MDPs as semi-algebraic sets.
problem Understanding feasible value functions in partially observable Markov decision processes.
method Characterization of feasible value functions as semi-algebraic sets defined by polynomial inequalities.
result The feasible set of value functions in POMDPs is a semi-algebraic set, not a polytope as in MDPs.
Black holes offer insights into machine learning's loss landscapes.
problem Understanding the loss landscape in machine learning.
method Comparing machine learning loss landscapes to black hole entropy.
result Black holes provide an infinite family of potential landscapes with known minima.
Adversarial training makes logistic regression weight loss landscapes sharper.
problem Understanding why adversarial training sharpens the weight loss landscape in logistic regression.
method Theoretical analysis of linear logistic regression model with L2 norm constraints, and experiments on ResNet18.
result Adversarial training sharpens the weight loss landscape in linear logistic regression models.
AWP improves robustness by flattening weight loss landscape.
problem Improving robustness of deep neural networks against adversarial examples.
method Explicitly regularizes the flatness of weight loss landscape through adversarial weight perturbation.
result AWP forms a double-perturbation mechanism in adversarial training, leading to flatter weight loss landscape.
Efficiently infers graph edges from genetic similarity data in landscape genetics.
problem Inferring unknown graph edges from genetic similarity data in a heterogeneous landscape.
method Developed an efficient first-order optimization method to solve the inverse landscape genetics problem.
result Our method provides fast and reliable convergence, significantly outperforming existing heuristics.
Smoothed fitness landscape improves protein optimization.
problem Infeasibility of combinatorially large protein sequence space.
method Formulate protein fitness as a graph signal, smooth using Tikunov regularization, and optimize with Gibbs sampling.
result 2.5 fold fitness improvement over training set.
Contextual linear optimization shows naive plug-in methods can outperform direct optimization.
problem Optimizing decisions with side observations to reduce uncertainty.
method Using off-the-shelf machine learning methods to learn a predictive model and plug it in for optimization.
result The naive plug-in approach achieves faster regret convergence rates than direct optimization methods.
Researchers improve visualization of neural network loss landscapes.
problem Understanding neural network generalization performance.
method Novel 'jump and retrain' procedure, non-linear dimensionality reduction (PHATE), computational homology.
result Improved visualization and quantification of neural network generalization performance.
B-cos transformers explain Vision Transformers' decisions.
problem Lack of holistic explanations for transformer outputs.
method Formulate each component as dynamic linear, allowing a single linear transform for summarization.
result Bcos-ViTs are highly interpretable and competitive on ImageNet.
Deeper models have a more favorable optimization landscape, making them more robust to noise.
problem Characterizing the effect of depth on the optimization landscape of linear regression models.
method Robust and over-parameterized setting, simple sub-gradient method.
result A simple sub-gradient method converges to a balanced solution that is close to the ground truth and enjoys a flat local landscape.
Machine learning techniques are being increasingly used as flexible non-linear fitting and prediction tools in the physical sciences. Fitting functions that exhibit multiple solutions as local minima can be analysed in terms of the corresponding machine learning landscape. Methods to explore and visualise molecular pot…
New sampler tackles complex discrete energy landscapes efficiently.
problem Stagnation in gradient-based discrete samplers for non-convex settings.
method DREXEL sampler with Replica Exchange and Adjusted Metropolis.
result Proves samplers satisfy detailed balance and converge to target distribution.
Regularization is typically understood as improving generalization by altering the landscape of local extrema to which the model eventually converges. Deep neural networks (DNNs), however, challenge this view: We show that removing regularization after an initial transient period has little effect on generalization, ev…
A central challenge of adversarial learning is to interpret the resulting hardened model. In this contribution, we ask how robust generalization can be visually discerned and whether a concise view of the interactions between a hardened decision map and input samples is possible. We first provide a means of visually co…
We analyze the optimization landscape of α-loss in logistic models.
problem Optimization landscape of α-loss in logistic models.
method Tools from strictly-locally-quasi-convex functions and geometric techniques.
result Evolution of optimization landscape with respect to α.
Proposes a method for explaining ranking decisions in learning systems.
problem Limited work on interpreting ranking decisions from learning systems.
method Model agnostic local explanation method using optimization to maximize validity.
result Approach outperforms other methods in validity across different LTR models.
Gradient-free optimizers are ineffective on barren plateaus in quantum computing.
problem Effect of barren plateaus on gradient-free optimization in quantum computing.
method Numerical simulations and theoretical analysis of gradient-free optimization algorithms.
result Gradient-free optimizers are not effective in barren plateau landscapes due to exponentially suppressed cost function differences.
Study reveals sharp characterisation of local minima in neural network loss landscapes.
problem Characterizing local minima in high-dimensional two-layer ReLU neural networks.
method Exact low-dimensional representation of local minima using summary statistics and link with one-pass SGD dynamics.
result Local minima in overparameterized neural networks form discrete families with varying stability and reachability.
AI models assess psychological risks in currency trading.
problem Identifying psychological risks in currency traders.
method Developed a decision tree model to identify patterns in historical data.
result Enhanced decision-making through real-time alerts.
Experimental fractal landscape dynamics observed in emulsions.
problem Understanding anomalous motions in soft glassy materials.
method Quantitative analysis of oil droplet trajectories in dense emulsions.
result Experimental fractal geometry matches computational model of soft glassy dynamics.
SGD vs quasi-Newton optimization in neural networks: different landscapes, different generalizability.
problem Understanding neural network optimization and generalizability.
method Comparison of stochastic gradient descent (SGD) and quasi-Newton optimization methods using computational tools.
result SGD solutions are separated by lower barriers than quasi-Newton solutions, but quasi-Newton solutions are deeper and more isolated.
Study visualizes actor-critic loss landscapes for inventory optimization.
problem Difficulties in solving multi-store dynamic inventory control problems.
method Low-dimensional visualizations of actor loss function.
result Loss landscapes favor optimal policies in reinforcement learning.
We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gaussian functions, and on the spiked-tensor model and generalizations. We thoroughly analyze the stati…
In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures. We propose deep structured energy based models (DSEBMs), where the energy function is the output of a deterministic deep neural network with structure. We develop novel model architectures to integ…
Considering the creation of persistence landscape on a parametrized curve and structure of sampling, there exists a random process for which a finite mixture model of persistence landscape (FMMPL) can provide a better description for a given dataset. In this paper, a nonparametric approach for computing integrated mean…
Analyzes adversarial training's impact on loss landscape, proposing PAS to improve model performance.
problem Challenges in optimizing models under adversarial training due to loss landscape properties.
method Analytical studies of adversarial loss functions, numerical analyses, PAS strategy.
result Adversarial training impairs optimization, but PAS strategy improves model performance.
LSAM optimizes deep learning training with improved efficiency.
problem Inefficiency in distributed large-batch training with Sharpness-Aware Minimization (SAM).
method Integrates SAM's adversarial steps with an asynchronous distributed sampling strategy.
result Higher final accuracy compared to data-parallel SAM.
Paper characterizes optimization landscape of Tucker decomposition.
problem Finding exact Tucker decomposition is a nonconvex optimization problem.
method Characterized the optimization landscape and provided a local search algorithm.
result All local minima are globally optimal if tensor has an exact Tucker decomposition.
Researchers improve NCE by addressing its flat loss landscape issues.
problem NCE's poor performance due to an ill-behaved loss landscape.
method Introduced eNCE with an exponential loss and normalized gradient descent.
result Proven that landscape issues arise from inappropriate noise distribution.
Embedding principle explains loss landscape of deep neural networks.
problem Understanding the structure of loss landscapes in deep neural networks.
method Proposed an embedding principle that critical points of narrower DNNs can be embedded to critical points of wider DNNs.
result Wide DNNs are often attracted by highly-degenerate critical points embedded from narrower DNNs.
There are many surprising and perhaps counter-intuitive properties of optimization of deep neural networks. We propose and experimentally verify a unified phenomenological model of the loss landscape that incorporates many of them. High dimensionality plays a key role in our model. Our core idea is to model the loss la…