arXiv research

Alternative approach to regularize time-dependent singular Lagrangian systems.

Lowered regularity assumption for a phase-dependent Helfrich energy equation.

Manifold regularization is a commonly used technique in semi-supervised learning. It enforces the classification rule to be smooth with respect to the data-manifold. Here, we derive sample complexity bounds based on pseudo-dimension for models that add a convex data dependent regularization term to a supervised learnin…

We determine regularity results for energy minimizing maps from an $n$-dimensional Riemannian polyhedral complex $X$ into a CAT(1) space. Provided that the metric on $X$ is Lipschitz regular, we prove Hölder regularity with Hölder constant and exponent dependent on the total energy of the map and the metric on the doma…

This paper proposes a new method to improve domain adaptation by distinguishing between marginal and dependence structure differences.

Tuning-free OR-PCA improves scalability for large datasets.

Regularization and data augmentation can be class-dependent, leading to poor performance on some classes.

This paper analyzes RMR under Markov-dependent samples, improving understanding of its generalization error.

Novel model captures high-dimensional copulas with spectral dynamics and regularization.

We establish continuous maximal regularity results for parabolic differential operators acting on sections of tensor bundles on Riemannian manifolds. As an application, we show that solutions to the Yamabe flow instantaneously regularize and become real analytic in space and time. The regularity result is obtained by i…

Study online learning in RKHS with dependent processes, focusing on \(β\)- and \(φ\)-mixing.

The paper analyzes how re-weighting helps in reducing variance in high-dimensional kernel methods under covariate shifts.

Two new regularization methods improve neural network performance and complexity control.

In this work, we develop a novel regularizer to improve the learning of long-range dependency of sequence data. Applied on language modelling, our regularizer expresses the inductive bias that sequence variables should have high mutual information even though the model might not see abundant observations for complex lo…

The regularity of weak solutions of a two-dimensional nonlinear sigma model with coarse gravitino is shown. Here the gravitino is only assumed to be in $L^p$ for some $p>4$. The precise regularity results depend on the value of $p$.

Regularizes RNNs to be invariant to input order.

The paper analyzes high-dimensional kernel regression, showing different risk curves based on data and regularization.

This paper explains how batch normalization auto-tunes the regularization parameter based on data statistics.

New method estimates spatial weights matrix for lattice data, improving prediction accuracy.

Study LASSO for high-dimensional VAR models with weakly dependent innovations.

We establish a theoretical link between adversarial training and operator norm regularization for deep neural networks. Specifically, we prove that $\ell_p$-norm constrained projected gradient ascent based adversarial training with an $\ell_q$-norm loss on the logits of clean and perturbed inputs is equivalent to data-…

Regularizers change the geometric properties of loss functions in neural networks.

Dropout technique is analyzed using information geometry.

Current adoption of machine learning in industrial, societal and economical activities has raised concerns about the fairness, equity and ethics of automated decisions. Predictive models are often developed using biased datasets and thus retain or even exacerbate biases in their decisions and recommendations. Removing …

Regular variation provides a convenient theoretical framework to study large events. In the multivariate setting, the dependence structure of the positive extremes is characterized by a measure - the spectral measure - defined on the positive orthant of the unit sphere. This measure gathers information on the localizat…

We propose a novel data-dependent structured gradient regularizer to increase the robustness of neural networks vis-a-vis adversarial perturbations. Our regularizer can be derived as a controlled approximation from first principles, leveraging the fundamental link between training with noise and regularization. It adds…

This study explores star-shaped regularizers learned from critic-based losses.

New algorithms reduce regret in online MDPs by adapting to data and variance.

Estimating the dependences between random variables, and ranking them accordingly, is a prevalent problem in machine learning. Pursuing frequentist and information-theoretic approaches, we first show that the p-value and the mutual information can fail even in simplistic situations. We then propose two conditions for r…

We establish regularity results for critical points to energies of immersed surfaces depending on the first and the second fundamental form exclusively. These results hold for a large class of intrinsic elliptic Lagrangians which are sub-critical or critical. They are derived using uniform $ε-$regularity estimates whic…

This paper proposes a simple approach to derive efficient error bounds for learning multiple components with sparsity-inducing regularization. We show that for such regularization schemes, known decompositions of the Rademacher complexity over the components can be used in a more efficient manner to result in tighter b…

Multi-label text classification is a popular machine learning task where each document is assigned with multiple relevant labels. This task is challenging due to high dimensional features and correlated labels. Multi-label text classifiers need to be carefully regularized to prevent the severe over-fitting in the high …

ARO overfits by making constraints dependent on uncertainty, leading to brittleness.

The purpose of this present paper is to investigate the geometric structure of regular overdetermined systems of second order with two independent and one dependent variables from the point of view of rank 2 prolongations. Utilizing this notion of prolongations, we characterize the type of these overdetermined systems.…

A new model integrates LSTM and copulas for high-dimensional financial data.

Deep neural nets can estimate regression with dependent data without the curse of dimensionality.

The paper examines how small positive dependence can lead to correlated tail risks.

Batch Normalization is a commonly used trick to improve the training of deep neural networks. These neural networks use L2 regularization, also called weight decay, ostensibly to prevent overfitting. However, we show that L2 regularization has no regularizing effect when combined with normalization. Instead, regulariza…

Curve shortening flow's regularity depends on initial conditions after a certain time.

This paper suggests a learning-theoretic perspective on how synaptic plasticity benefits global brain functioning. We introduce a model, the selectron, that (i) arises as the fast time constant limit of leaky integrate-and-fire neurons equipped with spiking timing dependent plasticity (STDP) and (ii) is amenable to the…

Sharp analysis improves RLHF sample complexity with KL-regularization.

Paper introduces SLS to improve label smoothing regularization.

TATD predicts missing entries in time-evolving tensors by exploiting temporal dependency and sparsity.

We present an exploration of the rich theoretical connections between several classes of regularized models, network flows, and recent results in submodular function theory. This work unifies key aspects of these problems under a common theory, leading to novel methods for working with several important models of inter…

New method for hedging path-dependent options with price impact using probabilistic arguments.

Study on linear regression with dependent covariates, proving universality and error characterization.

Generalizing the notion of domains of dependence in the Minkowski space, we define and study regular domains in the affine space with respect to a proper convex cone. In dimension three, we show that every proper regular domain is uniquely foliated by a particular kind of surfaces with constant affine Gaussian curvatur…

Entropy regularization improves MFG learning efficiency and stability.