DONUT improves treatment effect estimation by enforcing orthogonality constraints.
problem Estimating treatment effects from observational data is challenging due to unobserved outcomes.
method DONUT uses a regularization framework that formalizes unconfoundedness as orthogonality, leading to deep orthogonal networks.
result DONUT outperforms state-of-the-art methods in estimating average treatment effects.
L1-orthogonal regularization improves decision tree explainability of deep neural networks.
problem Lack of explainability in deep neural networks.
method L1-orthogonal regularization during training of decision trees.
result Decision trees closely approximate trained deep neural networks with improved accuracy and fidelity.
The paper proposes new ways to enforce orthogonality in deep CNNs training.
problem Enforcing orthogonality in deep CNNs training.
method Developed novel orthogonality regularizations using analytical tools like mutual coherence and restricted isometry property.
result Consistent performance gains in final accuracies and convergence speed after applying regularizations.
A l1-norm penalized orthogonal forward regression (l1-POFR) algorithm is proposed based on the concept of leaveone- out mean square error (LOOMSE). Firstly, a new l1-norm penalized cost function is defined in the constructed orthogonal space, and each orthogonal basis is associated with an individually tunable regulari…
Orthogonal Wasserstein GANs improve image quality without gradient norm regularization.
problem Wasserstein-GANs' gradient norm regularization limits the distribution's fidelity.
method Substituted gradient norm regularization with orthogonality constraints on weight matrices.
result Orthogonal Wasserstein GANs achieve better image quality and generalization.
Paper introduces regularization for multi-head attention to spot keywords.
problem Redundancy in multi-head attention leads to lack of rich information.
method Regularization technique to enforce orthogonality between attention heads.
result Significant improvement in keyword spotting performance.
Study spherical images of modified vector fields on a unit sphere.
problem Understanding spherical images of modified vector fields.
method Analysis of spherical images using modified orthogonal vector fields and Darboux vector.
result Characterization of spherical indicatrices with modified orthogonal frame.
Orthogonal deep models defend against black-box attacks by ensuring internal representations are nearly orthogonal.
problem Vulnerability of deep learning models to black-box adversarial attacks.
method Introduce a gradient regularization scheme to encourage deep models' internal representations to be orthogonal to another model's.
result Orthogonal deep models significantly boost robustness against transferable black-box adversarial attacks.
Study of surfaces meeting a plane orthogonally or contained in a line, deriving weak forms and proving regularity.
problem Critical points of the Willmore functional with boundary constraints.
method Weak forms of free boundary conditions derived by reflection.
result Proved regularity of the surfaces.
Regularizes deep multi-task networks to prevent task interference.
problem Interfering tasks in deep neural networks reduce overall performance.
method Proposes a gradient regularization term to minimize task interference.
result Models with orthogonal gradients perform better on various datasets.
Chevalley theorems extended to isotropic functions on matrix spaces.
problem Extending Chevalley theorems to isotropic functions on matrix spaces.
method Proving ultradifferentiable Chevalley restriction theorems for various ultradifferentiable classes.
result Isotropic functions on symmetric matrices have ultradifferentiable regularity if and only if their diagonal restrictions do.
TANGOS improves neural network performance on tabular data by encouraging neuron specialization.
problem Improving neural network performance on tabular data.
method Gradient orthogonality and specialization of latent units.
result TANGOS leads to improved out-of-sample generalization performance.
We study conditions for the integrability of the distribution defined on a regular Poisson manifold as the orthogonal complement (with respect to some (pseudo)-Riemannian metric) to the tangent spaces of the leaves of a symplectic foliation. Examples of integrability and non-integrability of this distribution are provi…
A parametric manifold can be viewed as the manifold of orbits of a (regular) foliation of a manifold by means of a family of curves. If the foliation is hypersurface orthogonal, the parametric manifold is equivalent to the 1-parameter family of hypersurfaces orthogonal to the curves, each of which inherits a metric and…
OMP improves text classification accuracy with sparse models.
problem Overfitting in text classification due to high dimensionality.
method Greedy variable selection algorithm (OMP) and overlapping Group OMP.
result OMP and overlapping GOMP produce effective and very sparse models.
Study spectral flow on a warped cylinder with special boundary conditions.
problem Analyzing spectral flow on a warped cylinder with specific boundary conditions.
method Complexifying the twisting bundle, diagonalizing the orthogonal twist, and regrouping conjugate and reflection-paired blocks.
result Explicit formula for RO(O(2))-valued spectral flow, refining ordinary spectral flow. The paper proves smoothness of almost-minimizers' boundaries near the free boundary.
problem Minimizing degenerate area functionals with weighted boundary conditions.
method Epsilon-regularity theorem applied to almost-minimizers.
result Almost-minimizers' boundaries are C1,γ0-smooth, orthogonal to the boundary Ω. New method relaxes PCA orthogonality constraints using explained variance of correlated components.
problem Difficulty in using PCA for sparse design due to orthogonality constraints and non-differentiable penalty.
method Introduce expvar(Y) to measure variance explained by correlated components, relax orthogonality constraints.
result Two expvar(Y) definitions suitable for block PCA formulations without orthogonality constraints.
New method for sampling orthogonal matrices using Hamiltonian Monte-Carlo.
problem Sampling from posterior distributions of orthogonal matrices in Bayesian models.
method Proposes a new sampling scheme based on Hamiltonian Monte-Carlo and Riemannian optimization.
result New method is comparable or faster in time per iteration and more sample-efficient than conventional methods.
Through Cayley and Langlands type correspondences, we give a geometric description of the moduli spaces of real orthogonal and symplectic Higgs bundles of any signature in the regular fibres of the Hitchin fibration. As applications of our methods, we complete the concrete abelianization of real slices corresponding to…
A recent theoretical analysis shows the equivalence between non-negative matrix factorization (NMF) and spectral clustering based approach to subspace clustering. As NMF and many of its variants are essentially linear, we introduce a nonlinear NMF with explicit orthogonality and derive general kernel-based orthogonal m…
New filter bank regularization improves DCNNs by incorporating image priors.
problem Improving DCNNs' robustness and generality.
method Structured filter bank regularization of DCNN kernels.
result Filter bank regularization leads to faster convergence and better generalization.
The theory of slice regular functions of a quaternion variable is applied to the study of orthogonal complex structures on domains Ω of R^4. When Ω is a symmetric slice domain, the twistor transform of such a function is a holomorphic curve in the Klein quadric. The case in which Ω is the complement of a parabola is st…
OMIC improves matrix completion with orthonormal side information and nuclear-norm regularization.
problem Matrix completion with improved interpretability and adaptability.
method OMIC combines orthonormal side information and nuclear-norm regularization, optimized by a converging algorithm.
result OMIC outperforms state-of-the-art methods in synthetic and real-world datasets.
PROD method improves high-dimensional regression by handling strong correlations.
problem Violation of Irrepresentable Condition in LASSO for high-dimensional data.
method PROD procedure based on orthogonal decomposition of design matrix.
result PROD enhances performance of high-dimensional penalized regression.
Stability inequalities for specific solutions in high dimensions.
problem Stability of solutions to the one-phase Bernoulli problem.
method Proving strict stability inequalities for cohomogeneity one solutions with bi-orthogonal symmetry.
result Strict stability for cohomogeneity one solutions in dimensions 7 and above.
Unified framework for rigidity results on (κ,μ)-manifolds.
problem Rigidity of metrics on (κ,μ)-manifolds. method Study of deviations preserving bi-Legendrian structure, orthogonalizing canonical structure.
result Unified rigidity results in both Riemannian and semi-Riemannian categories.
Subspace clustering methods based on ℓ1, ℓ2 or nuclear norm regularization have become very popular due to their simplicity, theoretical guarantees and empirical success. However, the choice of the regularizer can greatly impact both theory and practice. For instance, ℓ1 regularization is guaranteed t…
SVD training reduces DNN rank and computation load without SVD per step.
problem High memory and computational load in deep neural networks.
method Explicitly achieves low-rank DNNs during training without SVD per step, using orthogonality regularization and sparsity-inducing regularizers.
result Significantly reduces DNN rank and computation load compared to existing methods.
Echo state network (ESN) is viewed as a temporal non-orthogonal expansion with pseudo-random parameters. Such expansions naturally give rise to regressors of various relevance to a teacher output. We illustrate that often only a certain amount of the generated echo-regressors effectively explain the variance of the tea…
Study Higgs bundles with non-regular fibers for SO(4,C) and SL(2,C)XSL(2,C).
problem Understanding non-regular fibers in Hitchin fibrations.
method Examined Higgs bundles for non-compact real forms of SO(4,C) and SL(2,C)XSL(2,C).
result Found interesting examples of non-abelian spectral data.
We compute approximate solutions to L0 regularized linear regression using L1 regularization, also known as the Lasso, as an initialization step. Our algorithm, the Lass-0 ("Lass-zero"), uses a computationally efficient stepwise search to determine a locally optimal L0 solution given any L1 regularization solution. We …
Two-layer networks trained on low-dimensional subspaces are vulnerable to adversarial examples.
problem Vulnerability of two-layer neural networks to adversarial examples on low-dimensional subspaces.
method Analysis of gradient behavior and effect of initialization scale and regularization.
result Decreasing initialization scale or adding L2 regularization can improve robustness to adversarial perturbations orthogonal to the data.
OrthDNNs improve DNNs' generalization by ensuring equal singular values.
problem Improving deep learning models' generalization ability.
method Introducing OrthDNNs, proving local isometry, and establishing new generalization bounds.
result OrthDNNs achieve optimal generalization bounds with equal singular values.
The paper shows how gradient flow on over-parametrized tensor decomposition behaves like deflation.
problem Understanding the training dynamics of gradient flow on tensor decomposition.
method Empirical observation and mathematical proof of gradient flow dynamics for orthogonally decomposable tensors.
result Gradient flow dynamics for orthogonally decomposable tensors follows a tensor deflation process, recovering all tensor components.
Study Finsler metrics with vanishing Landsberg curvature.
problem Characterize Finsler metrics with specific curvature properties.
method Derive expressions for curvatures, solve PDEs, construct examples.
result Construct non-regular Landsberg metrics not of Berwald type.
TGCCA analyzes higher-order tensors using orthogonal rank-R CP decomposition.
problem Handling higher-order structures in multi-block data analysis.
method Tensor Generalized Canonical Correlation Analysis (TGCCA) with orthogonal rank-R CP decomposition.
result TGCCA outperforms state-of-the-art methods on simulated and real data.
DFSOS improves sparse discriminant analysis for high-dimensional data.
problem Sparse discriminant analysis in high-dimensional settings with feature selection.
method Deflation-Free Sparse Optimal Scoring (DFSOS) using Bregman iteration and orthogonality-constrained optimization.
result DFSOS achieves comparable or better classification accuracy than deflation-based methods.
Distance metric learning (DML), which learns a distance metric from labeled "similar" and "dissimilar" data pairs, is widely utilized. Recently, several works investigate orthogonality-promoting regularization (OPR), which encourages the projection vectors in DML to be close to being orthogonal, to achieve three effect…
New approach to portfolio optimization shows entropy regularization is ineffective.
problem Entropy regularization in mean-variance portfolio optimization under drift uncertainty.
method Combining Bayesian filtering and stochastic policy optimization.
result Entropy regularization does not accelerate learning about unknown drift.
The paper develops methods to reduce deployment risk under dynamic covariate shifts.
problem Reduction of deployment risk under dynamic covariate shifts.
method Time-domain Poincare inequality and Jacobian-velocity theorem to identify and control directional tangent energy.
result Drift-aligned tangent regularization (DTR) reduces risk volatility and directional gain in low-rank drift regimes.
This paper proposes a Lasso-type estimator for a high-dimensional sparse parameter identified by a single index conditional moment restriction (CMR). In addition to this parameter, the moment function can also depend on a nuisance function, such as the propensity score or the conditional choice probability, which we es…
Multivariate Analysis (MVA) comprises a family of well-known methods for feature extraction that exploit correlations among input variables of the data representation. One important property that is enjoyed by most such methods is uncorrelation among the extracted features. Recently, regularized versions of MVA methods…
Bayesian Markowitz portfolio problem shows entropy regularization is ineffective.
problem Entropy regularization in Bayesian Markowitz portfolio optimization.
method Combines continuous-time Bayesian filtering with stochastic policy optimization.
result Entropy regularization does not accelerate learning of unknown drift.
We define systems of pre-extremals for the energy functional of regular rheonomic Lagrange manifolds and show how they induce well-defined Hamilton orthogonal nets. Such nets have applications in the modelling of e.g. wildfire spread under time- and space-dependent conditions. The time function inherited from such a Ha…
BigGAN achieves state-of-the-art image synthesis on ImageNet.
problem Generating high-fidelity images from complex datasets like ImageNet.
method Trained a large-scale GAN with orthogonal regularization and a truncation trick.
result Improved Inception Score (IS) of 166.5 and Frechet Inception Distance (FID) of 7.4.
Study submanifolds in hyperbolic space, focusing on their boundary and Laplace operator.
problem Understanding the geometry and regularity of submanifolds in hyperbolic space.
method Analyzing asymptotic geometry and regularity properties near the ideal boundary, computing essential spectra.
result Computed essential spectra of the Laplace operator on certain submanifolds.
Proposes nAIPW for robust ATE estimation using neural networks.
problem Estimation of ATE with potential confounders and nonlinear relationships.
method Normalized AIPW (nAIPW) with neural networks and regularization.
result nAIPW maintains double-robustness and orthogonality properties.