In semi-supervised learning, virtual adversarial training (VAT) approach is one of the most attractive method due to its intuitional simplicity and powerful performances. VAT finds a classifier which is robust to data perturbation toward the adversarial direction. In this study, we provide a fundamental explanation why…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “bad samples”
In this paper, we study a simple and generic framework to tackle the problem of learning model parameters when a fraction of the training samples are corrupted. We first make a simple observation: in a variety of such settings, the evolution of training accuracy (as a function of training epochs) is different for clean…
In this work, we investigate semi-supervised learning (SSL) for image classification using adversarial training. Previous results have illustrated that generative adversarial networks (GANs) can be used for multiple purposes. Triple-GAN, which aims to jointly optimize model components by incorporating three players, ge…
A simple modification improves GAN performance by discarding bad samples.
problem Improving GAN performance with minimal computational cost.
method Top-k update procedure: zero out gradient contributions from least realistic elements.
result Significant improvement in FID score for conditional generation on CIFAR-10.
We provide two fundamental results on the population (infinite-sample) likelihood function of Gaussian mixture models with components. Our first main result shows that the population likelihood function has bad local maxima even in the special case of equally-weighted mixtures of well-separated and spherical…
Betting Against (Bad) Betaq-fin.PM
Proposes a new factor to improve BAB strategies by recognizing bad-beta assets.
problem Investors often misprice assets based on beta, ignoring bad-beta.
method Double-sorting on beta and bad-beta to create a new factor.
result The Betting Against Bad Beta factor improves BAB strategies.
New technique improves imitation learning by preventing local minima and exploring states.
problem Behavioral cloning gets stuck in local minima and lacks effective exploration.
method Two-phase model with sampling mechanisms and self-attention modules.
result Significantly outperforms previous state-of-the-art in various environments.
Bad models can teach well by replicating noise.
problem Overparameterized models can replicate noise in training data.
method Knowledge distillation from noisy samplers.
result Distillation from samplers approximates Bayes optimal classifier.
The abstract constructs a set of bad 3-orbifolds and shows how any bad 3-orbifold can be transformed into a good one.
problem Characterizing and transforming bad 3-orbifolds into good ones.
method Explicit construction of bad 3-orbifolds and a method of cutting-and-capping to transform them.
result Any bad 3-orbifold can be transformed into a good 3-orbifold through a finite number of operations.
Recent work has noted that all bad local minima can be removed from neural network loss landscapes, by adding a single unit with a particular parameterization. We show that the core technique from these papers can be used to remove all bad local minima from any loss landscape, so long as the global minimum has a loss o…
New attacks reduce bad queries in black-box classifiers, improving effectiveness.
problem Evasion attacks often issue many bad queries, which are costly.
method Design new attacks that minimize bad queries while increasing non-bad queries.
result Reduces the number of bad queries by 1.5-7.3 times, improving attack effectiveness.
The study analyzes local minima in ReLU networks and finds low probability of bad local minima.
problem Understanding the existence and probability of local minima in ReLU networks.
method Theoretical analysis combined with linear programming and experiments on MNIST and CIFAR-10 datasets.
result No bad differentiable local minima found almost everywhere in weight space.
New findings suggest non-contrastive learning has many bad minima, not just collapsed ones.
problem The effectiveness of non-contrastive learning in unsupervised feature learning.
method Theoretical analysis and controlled experiments on simple data models.
result Non-contrastive losses have a preponderance of non-collapsed bad minima, and these minima are not avoided during training.
The empirically successful Thompson Sampling algorithm for stochastic bandits has drawn much interest in understanding its theoretical properties. One important benefit of the algorithm is that it allows domain knowledge to be conveniently encoded as a prior distribution to balance exploration and exploitation more eff…
Study finds stocks with common firm fears earn lower returns.
problem Identifying and quantifying firm-level investor fears.
method Analysis of equity options to identify common firm-level fears and their impact on stock returns.
result Stocks with exposure to common bad fears earn lower returns and require higher compensation.
A Gradient Analysis Framework for Rewarding Good and Penalizing Bad Examples in Language Modelscs.CL
This paper introduces a gradient analysis framework to improve language model performance by rewarding good examples and penalizing bad ones.
problem Improving language model output quality by penalizing bad examples.
method Gradient analysis of loss functions to reward good examples and penalize bad ones.
result ExMATE is superior to MLE and combining DPO with ExMATE enhances performance.
New proof shows knot Floer thickness limits bad domains in diagrams.
problem Understanding knot thickness and dealternating number.
method Modified Stipsicz-Szabo approach using Kauffman states.
result Knot Floer thickness is a lower bound on dealternating number.
In this paper, we consider a dynamic asset pricing model in an approximate fractional economy to address empirical regularities related to both investor protection and past information. Our newly developed model features not only in terms with a controlling shareholder who diverts a fraction of the output, but also goo…
For research to go in the right direction, it is essential to be able to compare and quantify performance of different algorithms focused on the same problem. Choosing a suitable evaluation metric requires deep understanding of the pursued task along with all of its characteristics. We argue that in the case of applied…
Due to the insufficient measurements in the distribution system state estimation (DSSE), full observability and redundant measurements are difficult to achieve without using the pseudo measurements. The matrix completion state estimation (MCSE) combines the matrix completion and power system model to estimate voltage b…
Gaussians as noise in NCE lead to exponentially bad conditioning, hindering its efficiency.
problem Exponential conditioning of Hessian in NCE with Gaussian noise.
method Using Gaussian as the noise distribution in NCE.
result Gaussian noise in NCE leads to exponentially bad conditioning of the loss Hessian.
NICE learns a representation to avoid bad controls in causal inference.
problem Avoiding bad controls in causal inference from observational data.
method Uses invariant risk minimization (IRM) to learn a representation of covariates that avoids bad controls.
result NICE outperforms adjusting for all covariates in cases with unknown collider variables and bad controls.
In deep learning, \textit{depth}, as well as \textit{nonlinearity}, create non-convex loss surfaces. Then, does depth alone create bad local minima? In this paper, we prove that without nonlinearity, depth alone does not create bad local minima, although it induces non-convex loss surface. Using this insight, we greatl…
We show that every bad orbifold vector bundle can be realized as the restriction of a good orbifold vector bundle to a suborbifold of the base space. We give an explicit construction of this result in which the Chen-Ruan orbifold cohomology of the two base spaces are isomorphic (as additive groups). This construction i…
Thompson Sampling is at most twice as bad as any other policy in Bayesian bandit models.
problem Optimizing selection of the best arm in Bayesian bandit models with independent latent processes.
method Thompson Sampling approach applied to models with independent latent arm processes.
result Thompson Sampling makes at most twice the expected number of mistakes compared to any other policy.
Computational models in fields such as computational neuroscience are often evaluated via stochastic simulation or numerical approximation. Fitting these models implies a difficult optimization problem over complex, possibly noisy parameter landscapes. Bayesian optimization (BO) has been successfully applied to solving…
Graph attacks can be successful with just a few bad nodes.
problem Adversarial attacks on graph neural networks.
method Identifying and exploiting anchor nodes to compromise graph models.
result A few bad nodes can significantly degrade graph model performance.
We propose to improve trust region policy search with normalizing flows policy. We illustrate that when the trust region is constructed by KL divergence constraints, normalizing flows policy generates samples far from the 'center' of the previous policy iterate, which potentially enables better exploration and helps av…
Asymmetries in volatility spillovers are highly relevant to risk valuation and portfolio diversification strategies in financial markets. Yet, the large literature studying information transmission mechanisms ignores the fact that bad and good volatility may spill over at different magnitudes. This paper fills this gap…
The model is aimed to discriminate the 'good' and the 'bad' companies in Russian corporate sector based on their financial statements data based on Russian Accounting Standards. The data sample consists of 126 Russian public companies- issuers of Ruble bonds which represent about 36% of total number of corporate bonds …
One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima. In this paper, we study the landscape of neural networks for binary classification tasks. Under mild assumptions, we prove that after adding one special neuron with a skip connection…
New framework removes harmful momentum effect for long-tailed classification.
problem Challenges in maintaining balanced datasets with long-tailed data.
method Causal inference framework to disentangle and remove harmful effects of momentum.
result Achieves state-of-the-art performance on long-tailed visual recognition benchmarks.
Deep neural networks (DNNs) are known for their vulnerability to adversarial examples. These are examples that have undergone small, carefully crafted perturbations, and which can easily fool a DNN into making misclassifications at test time. Thus far, the field of adversarial research has mainly focused on image model…
We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that from any point in parameter space there exists a continuous path on which the cross-entropy loss is non-increasing and gets arbitrarily clos…
We detect and quantify asymmetries in volatility spillovers using the realized semivariances of petroleum commodities: crude oil, gasoline, and heating oil. During the 1987--2014 period we document increasing spillovers from volatility among petroleum commodities that substantially change after the 2008 financial crisi…
Highly connected orbifolds are rare but exist.
problem Understanding the maximum connectivity of orbifolds.
method Provided examples and improved bounds for n-connected n-orbifolds.
result Improved bounds for n-connected n-orbifolds, up to dimension 5.
PyBADS optimizes complex functions quickly and reliably.
problem Optimizing rough, noisy, and expensive functions with unknown gradients.
method Bayesian Adaptive Direct Search (BADS) algorithm.
result PyBADS performs well on both artificial and real-world problems.
Reviews recent findings on neural network landscapes.
problem Non-convexity of loss functions causing bad landscapes.
method Rigorous geometric analysis and empirical exploration.
result Wide neural nets may have sub-optimal local minima.
Hamiltonian Monte Carlo is a powerful algorithm for sampling from difficult-to-normalize posterior distributions. However, when the geometry of the posterior is unfavorable, it may take many expensive evaluations of the target distribution and its gradient to converge and mix. We propose neural transport (NeuTra) HMC, …
This research secures deployed sentiment analysis models by identifying and defending against attack vectors.
problem Securing deployed machine learning models, particularly sentiment analysis systems, from adversarial attacks.
method BAD (Build, Attack, Defend) Architecture, evaluating two implementations.
result Demonstrated a viable methodology for securing machine learning models in production.
We introduce a new method for sparse principal component analysis, based on the aggregation of eigenvector information from carefully-selected axis-aligned random projections of the sample covariance matrix. Unlike most alternative approaches, our algorithm is non-iterative, so is not vulnerable to a bad choice of init…
Proposes a method to simulate data for testing credit risk scorecard stability.
problem Ensuring credit risk scorecards remain representative of the population over time.
method Specification of bad ratios to generate parameter values for scorecards.
result Simulated data adheres closely to specified bad ratios.
MCN improves deep neural networks by bettering local minima and generalizing well.
problem Bad local minima and poor generalization in deep neural networks.
method Introducing Maximum-and-Concatenation Networks (MCN) to eliminate bad local minima and improve generalization.
result MCN can autonomously improve local minima's goodness by increasing network depth.
Great year, bad Sharpe? A note on the joint distribution of performance and risk-adjusted returnq-fin.ST
Investments with best performance are not associated with best Sharpe ratios.
problem The relationship between performance and risk-adjusted return (Sharpe ratio) is counterintuitive for heavy-tailed distributions.
method Synthetic and real data analysis of returns distributions.
result The best-performing investments are not the best in terms of Sharpe ratio, and vice versa.
Mixed linear regression involves the recovery of two (or more) unknown vectors from unlabeled linear measurements; that is, where each sample comes from exactly one of the vectors, but we do not know which one. It is a classic problem, and the natural and empirically most popular approach to its solution has been the E…
To Bag is to Prunestat.ML
Random Forests automatically prune a latent 'true' tree, explaining their overfitting without tuning.
problem Difficulty in building bad Random Forests and overfitting without apparent consequences.
method Bootstrap aggregation and model perturbation in Random Forests.
result Randomized ensembles implicitly perform optimal early stopping out-of-sample, explaining overfitting.
Affective computing has become a very important research area in human-machine interaction. However, affects are subjective, subtle, and uncertain. So, it is very difficult to obtain a large number of labeled training samples, compared with the number of possible features we could extract. Thus, dimensionality reductio…
In this paper, we prove that depth with nonlinearity creates no bad local minima in a type of arbitrarily deep ResNets with arbitrary nonlinear activation functions, in the sense that the values of all local minima are no worse than the global minimum value of corresponding classical machine-learning models, and are gu…