Neural networks with depth can't be approximated by shallow ones unless exponentially large.
problem Approximating neural networks with limited depth.
method Proving limitations of shallow neural networks using semi-algebraic gates.
result Neural networks with depth are essential for certain tasks, even for common activation functions.
Triangulates semi-algebraic sets over p-adic fields, proving similar results to reals.
problem Triangulating semi-algebraic sets over p-adic fields.
method Derives from a triangulation theorem for semi-algebraic sets over a p-adically closed field.
result Existence of flexible retractions and splitting for semi-algebraic sets.
This paper provides a stratification of semi-algebraic sets in the plane with finitely many geodesic segments.
problem How to stratify semi-algebraic sets in the plane with finitely many geodesic segments.
method Develops a semi-algebraic stratification of a real semi-algebraic set in the plane with open cells having the finiteness property.
result Provides insights for high-dimensional stratifications of semi-algebraic sets in connection with geodesics.
Classifies semi-algebraic surfaces up to bi-Lipschitz homeomorphisms.
problem Classifying semi-algebraic surfaces with isolated singularities.
method Bi-Lipschitz homeomorphisms with inner distance.
result Complete classifications for Nash surfaces and complex algebraic curves.
We give a survey of algorithms for computing topological invariants of semi-algebraic sets with special emphasis on the more recent developments in designing algorithms for computing the Betti numbers of semi-algebraic sets. Aside from describing these results, we discuss briefly the background as well as the importanc…
We prove a formula that relates the Euler-Poincaré characteristic of a closed semi-algebraic set to its Lipschitz-Killing curvatures
In this thesis, we consider semi-algebraic sets over a real closed field R defined by quadratic polynomials. Semi-algebraic sets of Rk are defined as the smallest family of sets in Rk that contains the algebraic sets as well as the sets defined by polynomial inequalities, and which is also closed under the bool…
In this paper we consider connections between Ricci solitons and Einstein metrics on homogeneous spaces. We show that a semi-algebraic Ricci soliton admits an Einstein one-dimensional extension if the soliton derivation can be chosen to be normal. Using our previous work on warped product Einstein metrics, we show that…
We describe an explicit semi-algebraic partition for the complement of a real hyperplane arrangement such that each piece is contractible and so that the pieces form a basis of Borel-Moore homology. We also give an explicit correspondence between the de Rham cohomology and the Borel-Moore homology.
We characterize value functions in partially observable MDPs as semi-algebraic sets.
problem Understanding feasible value functions in partially observable Markov decision processes.
method Characterization of feasible value functions as semi-algebraic sets defined by polynomial inequalities.
result The feasible set of value functions in POMDPs is a semi-algebraic set, not a polytope as in MDPs.
Characterizes G2-structures on Lie algebras with non-trivial center.
problem Classifying Lie algebras with G2-structures.
method Analyzing Lie algebras with non-trivial center, using contactization and symplectic properties.
result Six unimodular Lie algebras with non-trivial center admit closed G2-structures.
Paper discusses conditions for global injectivity of semi-algebraic local diffeomorphisms.
problem Conditions for global injectivity of semi-algebraic local diffeomorphisms in higher dimensions.
method Analyzes foliations and simply connectedness of leaves, relates to fibrations and Jacobian conjecture.
result Relates simply connectedness of foliation leaves to locally trivial fibrations and provides computable regularity conditions.
Let R be a real closed field, Q⊂R[Y1,...,Yℓ,X1,...,Xk], with $ °_{Y}(Q) \leq 2, °_{X}(Q) \leq d, Q \in {\mathcal Q}, #({\mathcal Q})=m$, and P⊂R[X1,...,Xk] with $°_{X}(P) \leq d, P \in {\mathcal P}, #({\mathcal P})=s$. Let S⊂Rℓ+k be a semi-alg…
We prove a nearly optimal bound on the number of stable homotopy types occurring in a k-parameter semi-algebraic family of sets in Rℓ, each defined in terms of m quadratic inequalities. Our bound is exponential in k and m, but polynomial in ℓ. More precisely, we prove the following. Let R be a real close…
Let R be a real closed field, Q⊂R[Y1,...,Yℓ,X1,...,Xk], with $ °_{Y}(Q) \leq 2, °_{X}(Q) \leq d, Q \in {\mathcal Q}, #({\mathcal Q})=m,$ and P⊂R[X1,...,Xk] with $°_{X}(P) \leq d, P \in {\mathcal P}, #({\mathcal P})=s$, and S⊂Rℓ+k a semi-algebr…
Proves algebraicity of Hodge loci in arithmetic quotients.
problem Proving algebraicity of Hodge loci in arithmetic quotients.
method Real semi-algebraic structure and o-minimal theory.
result Hodge locus is a countable union of algebraic subvarieties.
Study of cut locus in Carnot groups disproves conjectures and finds new sets.
problem Disprove conjectures on the shape of cut loci in Carnot groups.
method Analyzing left-invariant Carnot-Carathéodory metric and semi-algebraic sets.
result Found sets of cut points with codimension 2, disproving previous conjectures.
In this paper we construct a compactification for the parameter space of convex projective structures on a fixed n-manifold M. This parameter space is a closed semi-algebraic subset of the variety of characters of representations of the fundamental group of M in SL_{n+1}(R). The boundary is the inverse limit of an inve…
Machine learning improves searching for polynomial proofs.
problem Automatically searching for proofs of polynomial inequalities.
method Deep reinforcement learning guiding inference rules in semi-algebraic proof systems.
result Reduces the size of linear programs by several orders of magnitude.
Study expanding solitons on complex Lie groups with specific algebraic structures.
problem Investigate expanding solitons on complex Lie groups with left-invariant metrics.
method Analyze the algebraic structure of complex Lie groups and their decompositions.
result Show that the Lie algebras of complex Lie groups decompose into semidirect products.
In this paper we present some bounds of Hausdorff measures of objects definable in o-minimal structures: sets, fibers of maps, inverse images of curves of maps, etc. Moreover, we also give some explicit bounds for semi-algebraic or semi-Pfaffian cases, which depend only on the combinatoric data representing the objects…
In this paper we give an interpretation to the boundary points of the compactification of the parameter space of convex projective structures on an n-manifold M. These spaces are closed semi-algebraic subsets of the variety of characters of representations of the fundamental group of M in SL_{n+1}(R). The boundary was …
The paper studies 4-qubit Clifford states and their properties.
problem Understanding the set and properties of 4-qubit Clifford states.
method Analyzing the 293760 4-qubit Clifford states, splitting them into 18 groups, and studying the action of CNOT gates and local gates.
result There are 293760 4-qubit Clifford states with specific entanglement entropies, and any pair can be connected with local gates and at most 3 CNOT gates.
Improved logistic MoE with sigmoid gate shows better sample efficiency.
problem Improving sample efficiency in logistic MoE models.
method Comprehensive analysis of multinomial logistic MoE with modified sigmoid gate, incorporating temperature parameter and using Euclidean score.
result The sigmoid gate leads to lower sample complexity than softmax gate for both parameter and expert estimation.
New method prepares 3-qubit states using local gates and controlled-Z gates.
problem Preparation of 3-qubit states using quantum gates.
method Uses Ry(θ) gates and controlled-Z gates, with an optimal number of controlled-Z gates. result Optimal number of controlled-Z gates for preparing 3-qubit states is four. Study of curvature flow on specific Lie groups, leading to soliton solutions.
problem Curvature flow on 2-step nilpotent Lie groups with complex structures.
method Left-invariant metrics and complex structures on Lie groups, convergence analysis.
result Existence and convergence of flow to soliton solutions.
A forget-gate-only LSTM outperforms standard LSTM on benchmark datasets.
problem The necessity of all gates in LSTM networks.
method A forget-gate-only LSTM with chrono-initialized biases.
result The forget-gate-only LSTM outperforms standard LSTM on MNIST and pMNIST datasets.
Sigmoid gating is more sample efficient than softmax in mixture of experts.
problem Softmax gating leads to unnecessary competition among experts, causing representation collapse.
method Theoretical analysis of a regression framework with mixture of experts, identifying identifiability conditions and convergence rates.
result Sigmoid gating requires fewer samples to achieve the same expert estimation error as softmax gating.
In quantum computation, series of quantum gates have to be arranged in a predefined sequence that led to a quantum circuit in order to solve a particular problem. What if the sequence of quantum gates is known but both the problem to be solved and the outcome of the so defined quantum circuit remain in the shadow? This…
Proposes a new LSTM gate structure using bivariate Beta distribution.
problem Inflexibility of sigmoid gates in modeling multi-modality and skewness, and lack of modeling correlation between gates.
method Introduces a bivariate Beta distribution gate structure within LSTM cells.
result Empirically shows higher gradient values and improved model performance.
Bayesian method sparsifies gated RNNs, improving speed and interpretability.
problem Sparsifying neural networks to reduce complexity and improve performance.
method Bayesian approach to sparsify weights, neurons, and gates in LSTM architectures.
result Sparsified gated RNNs speed up forward pass and improve compression.
In this paper we investigate how germs of real functions can change under deformation. In particular we look at deformations of germs of isolated singularities from R_n to R_k (n >= k) and the relation with there natural stratification in some tame categorie (algebraic, analytic, semi-algebraic, subanalytic, o-minimal …
We sparsify gated RNNs by simplifying their structure.
problem Improving efficiency of RNNs by reducing their complexity.
method Adjust existing sparsification techniques to gated RNNs, sparsifying preactivations of gates.
result Simplified LSTM structure improves model performance and efficiency.
Gated attention improves model curvature, enhancing performance on nonlinear tasks.
problem Understanding the geometric implications of gating in attention mechanisms.
method Modeling attention outputs as Gaussian distributions and analyzing Fisher--Rao geometry.
result Gated attention enables non-flat geometries, including positively curved manifolds.
Improved RNNs with flexible gates using kernel activation functions.
problem Modeling long-term dependencies in sequential data.
method Designed a more flexible architecture with adaptable parameters using kernel activation functions.
result Improved accuracy with negligible computational cost and speed-up in training iterations.
Three GRU variants reduce parameters in RNNs, improving efficiency.
problem Reducing computational expense in RNNs.
method Three variants of GRU with reduced parameters in update and reset gates.
result Variant models perform similarly to original GRU RNN models.
Gated attention improves performance by using a hierarchical mixture of experts.
problem Improving performance of self-attention mechanisms in Transformers.
method Rigorously show that gated attention can be modeled as a hierarchical mixture of experts, providing a theoretical justification for its benefits.
result Gated attention is more sample-efficient than multi-head self-attention, requiring fewer data points to achieve the same estimation error.
Gating units in GRUs and LSTMs create slow modes and control phase-space complexity.
problem Training challenges in RNNs due to exploding or vanishing gradients.
method Random matrix theory and mean-field theory applied to GRUs and LSTMs.
result Gates in GRUs and LSTMs lead to accumulation of slow modes and control phase-space complexity.
Improved HMoE models using Laplace gating function enhance expert specialization and performance.
problem Improving performance of hierarchical mixture of experts models.
method Used Laplace gating function instead of Softmax in hierarchical mixture of experts models.
result Laplace gating function accelerates expert convergence and enhances specialization.
This paper introduces a flexible p-norm gating scheme to speed up deep neural networks training.
problem Training very deep neural networks is slow and challenging.
method Proposes a flexible p-norm gating scheme to control flow and speed up training. result Significantly improves learning speed in deep neural networks without extra overhead.
This study analyzes communication constraints in MoE architectures using information theory.
problem Communication constraints in Mixture-of-Experts (MoE) architectures.
method Developed a rate-distortion characterization of finite-rate gating in MoE architectures using information theory.
result Yielded capacity-aware limits for communication-constrained MoE systems.
Reduced-gate LSTM improves spatiotemporal prediction with less parameters.
problem Next-frame video prediction in deep learning.
method Predictive coding framework with reduced-gate convolutional LSTM.
result Reduced-gate model achieves equal or better accuracy with fewer parameters.
Paper proposes binary-valued gates for better LSTM training.
problem LSTMs struggle with soft gates, leading to unclear information flow.
method Introduces binary-valued gates to control information flow more clearly.
result Binary-valued gates improve LSTM performance and generalization.
This paper analyzes how gradient descent can learn parameters in mixture-of-experts models with gating.
problem Understanding parameter recovery in mixture-of-experts models with gating.
method Careful analysis of the optimization landscape and design of two distinct loss functions.
result Gradient descent can accurately learn parameters in mixture-of-experts models with gating.
GLCB uses Gated Linear Networks for online contextual bandits.
problem Online learning in contextual bandits with uncertainty estimation.
method Gated Linear Networks (GLNs) for prediction and uncertainty estimation.
result GLCB outperforms state-of-the-art methods in online contextual bandits.
Paper connects MoE and self-attention, proposing active-attention.
problem Improving efficiency and performance of self-attention mechanisms.
method Established connection between MoE and self-attention, analyzed quadratic gating functions, proposed active-attention mechanism.
result Active-attention outperforms standard self-attention in various tasks.
Solves challenges in estimating parameters of softmax gating Gaussian mixture models.
problem Identifiability issues and complex interactions in Gaussian mixture of experts.
method Proposes novel Voronoi loss functions and establishes convergence rates of MLE.
result Connects convergence rate of MLE to a solvability problem of polynomial equations.
A new LSTM model reduces state updates and improves convergence for long sequences.
problem Vanishing gradient problem in RNNs and slow convergence on long sequences.
method Gaussian-gated LSTM (g-LSTM) with a time gate to control neuron updates.
result The g-LSTM model reduces state updates and computes by at least 10x compared to an equivalent LSTM.