Formula found for neural network error with fixed weights.
problem Understanding error in neural networks with fixed weights.
method Provided an explicit formula for approximation error.
result Explicit formula for neural network error with fixed weights.
The conjecture of Kosniowski asserts that if the circle acts on a compact unitary manifold M with a non-empty fixed point set and M does not bound a unitary manifold equivariantly, then the dimension of the manifold is bounded above by a linear function on the number of fixed points. We confirm the conjecture for a…
A recent analysis of a model of iterative neural network in Hilbert spaces established fundamental properties of such networks, such as existence of the fixed points sets, convergence analysis, and Lipschitz continuity. Building on these results, we show that under a single mild condition on the weights of the network,…
New proof for 6D symplectic manifold with 4 fixed points.
problem Classifying the integral cohomology ring and total Chern class for 6D symplectic manifolds with 4 fixed points.
method New different argument using moment map values and weights of fixed points.
result Determined the sets of weights and global invariants for the manifold.
Study connects Gaussian processes and regularization for sequence-function mappings.
problem Understanding and interpreting sequence-function maps in biology.
method Relates Gaussian process priors, regularization, and gauge fixing in overparameterized weight space.
result Established the relationship between regularized regression and Gaussian processes in function space.
We establish a necessary and sufficient condition for pairs of integers to arise as the weights at the fixed points of an effective circle action on a compact almost complex 4-manifold with a discrete fixed point set. As an application, we provide a necessary and sufficient condition for a pair of integers to arise as …
Study circle actions on unitary manifolds with discrete fixed points.
problem Understanding circle actions on compact unitary manifolds with discrete fixed points.
method Prove relationships between weights at fixed points and derive results regarding the first equivariant Chern class and Hirzebruch χy-genus. result Derive a multigraph encoding fixed point data, leading to new insights into unitary S1-manifolds. Belief propagation (BP) is an iterative method to perform approximate inference on arbitrary graphical models. Whether BP converges and if the solution is a unique fixed point depends on both the structure and the parametrization of the model. To understand this dependence it is interesting to find \emph{all} fixed poi…
Quantized neural networks can represent all fixed-point functions under certain conditions.
problem Expressive power of quantized neural networks under fixed-point arithmetic.
method Analyzing necessary and sufficient conditions for quantized networks to represent all fixed-point functions.
result Various popular activation functions satisfy the sufficient condition for representing all fixed-point functions.
Fixed points of nonnegative neural networks are analyzed using fixed point theory.
problem Analyzing fixed points in nonnegative neural networks.
method Fixed point theory, nonlinear Perron-Frobenius theory, monotonic and scalable mappings.
result Conditions for the existence of fixed points in nonnegative neural networks are provided.
We obtain general formulae expressing Hirzebruch genera of a manifold with Z/p-action in terms of invariants of this action (the sets of weights of fixed points). As an illustration, we consider numerous particular cases of well-known genera, in particular, the elliptic genus. We also describe the connection with the s…
Stability of weighted extremal manifolds proven through blowups.
problem Stability of weighted extremal manifolds.
method Blowup technique to analyze weighted extremal Kähler manifolds.
result Proves weighted extremal manifolds are relatively weighted K-polystable.
Propose an XMSE-aware mixed estimator for EB that interpolates between ML and EB shrinkage.
problem Kernel-based EB estimation may be worse than ML when the kernel is poorly aligned with the true parameter.
method An XMSE-aware mixed estimator that interpolates between ML and EB shrinkage.
result Fixed-weight XMSE is a scalar quadratic, yielding a closed-form oracle mixing weight that is no worse than both ML and the base EB estimator at the XMSE scale.
OTSS learns personalized decision weights from logged decisions and outputs.
problem Learning context-specific decision weights from logged decisions and outputs.
method Output-targeted soft-segmentation model that deploys personalized decision-ready weight vectors.
result OTSS achieves the lowest mean regret in benchmark settings.
Classifies circle actions on 6D manifolds with 4 fixed points.
problem Classifying circle actions on 6D manifolds with specific fixed points.
method Analyzes fixed point data and proves agreement with known actions.
result Agrees with actions on 6-spheres or CP3. Optimal Euclidean structure minimizes energy in weighted toroidal graphs.
problem Finding the optimal Euclidean structure for weighted toroidal graphs.
method Minimizing Dirichlet energy over all possible Euclidean structures and realizations within a fixed homotopy class.
result The optimal Euclidean structure induces a weighted Delaunay decomposition.
Spectral Adaptive Conformal Prediction for Structured Non-Exchangeable Data
problem Improving prediction intervals for non-exchangeable time-indexed datasets
method Spectral adaptive conformal prediction
result Improves on fixed spectral weighting while monitoring uncertainty changes
Study a 10D symplectic manifold with 6 fixed points, linking to G2 orbit.
problem Understanding fixed points and Chern classes in Hamiltonian S1 actions. method Analyzing manifold data, comparing to G2 orbit. result Certain data uniquely determine others, showing similarities to G2 orbit. The high computational and parameter complexity of neural networks makes their training very slow and difficult to deploy on energy and storage-constrained computing systems. Many network complexity reduction techniques have been proposed including fixed-point implementation. However, a systematic approach for designin…
In this paper, we study a circle action on a compact oriented manifold with a discrete fixed point set. The fixed point data consists of the weights of the S1-representations at the fixed points. We prove various results and properties of the action, in terms of the fixed point data. We show that the manifold can be…
We construct the first examples of families of bad Riemannian orbifolds which are isospectral with respect to the Laplacian but not isometric. In our case these are particular fixed weighted projective spaces equipped with isospectral metrics obtained by a generalization of Schüth's version of the torus method.
Optimization algorithms with momentum, e.g., (ADAM), have been widely used for building deep learning models due to the faster convergence rates compared with stochastic gradient descent (SGD). Momentum helps accelerate SGD in the relevant directions in parameter updating, which can minify the oscillations of parameter…
The paper classifies circle actions on 6D manifolds with isolated fixed points.
problem Classifying circle actions on 6D manifolds with isolated fixed points.
method Performing equivariant connected sums at fixed points with specific manifolds.
result A sequence of operations can reduce the fixed point data to the empty collection.
The paper studies almost complex torus manifolds using graphs and Hirzebruch genera, proving properties of their fixed points and cohomology.
problem Understanding the fixed points and cohomology of almost complex torus manifolds.
method Using directed labeled multigraphs and Hirzebruch genera to encode and analyze the manifolds.
result Almost complex torus manifolds have positive Todd genus and at least n+1 fixed points.
Neural networks with integer weights approximate continuous functions efficiently.
problem Approximating continuous functions using neural networks with integer weights.
method Integrates superexpressive activation functions and integer weights.
result Convergence rate of order n2β+d−2βlog2n for neural network regression. We announce the following result and give several applications: A Hamiltonian T-space (for T a torus) with isolated fixed points is cobordant to a disjoint union of weighted projective spaces which are constructed from its fixed point data. The applications concern the Duistermaat-Heckman formula, the topological J…
Positive weights improve kernel quadrature's accuracy.
problem Improving kernel quadrature weights to be positive and stable.
method Using convex geometry to approximate the kernel mean embedding with positive weights.
result Positive weights lead to improved kernel quadrature bounds with Monte-Carlo-beating rates.
Paper converts deep networks to flat, equivalent kernel machines.
problem Capacity control and uniform convergence in deep learning.
method Push-forward transformation from deep networks to indefinite kernel machines.
result Flat network weights are Lp-norm regularized (0<p<1).
Study circle actions on manifolds with 3 fixed points, finding dimension constraints and unique structures.
problem Characterize circle actions on oriented manifolds with exactly 3 fixed points.
method Analyzes manifold dimensions, isotropy submanifolds, and uses quaternionic projective space as a reference.
result For a manifold with three fixed points, its dimension must be a multiple of 4, and specific weights are unique.
Probabilistic Autoencoder learns latent space weights' distribution.
problem Nonlinear model reconstruction error and sample quality.
method Normalizing flow for latent space weights' probability distribution.
result PAE achieves small reconstruction errors, high sample quality, and good performance.
Model compression techniques, such as pruning and quantization, are becoming increasingly important to reduce the memory footprints and the amount of computations. Despite model size reduction, achieving performance enhancement on devices is, however, still challenging mainly due to the irregular representations of spa…
Blowups of Kähler manifolds can inherit extremal metrics.
problem Extending extremal metrics to blowups of Kähler manifolds.
method Analyzing the action of a torus on blowups and weighted extremal metrics.
result Blowups of Kähler manifolds can inherit weighted extremal metrics.
The study examines stable regions in weighted manifolds with boundary properties.
problem Studying stable regions in weighted manifolds with boundary properties.
method Using deformations constructed from parallel vector fields tangent to the boundary, the study deduces rigidity properties for stable sets.
result The classification of stable sets in some Riemannian cylinders and uniqueness results for minimizers.
The study introduces anytime learning schedules for large language models without fixed horizons.
problem Training large language models without knowing the total training horizon.
method Theoretical analysis and weight averaging to create anytime learning schedules.
result Theoretical and empirical evidence shows that weight averaging with simple step sizes can achieve comparable final loss to well-tuned cosine schedules.
We consider the problem of deep neural net compression by quantization: given a large, reference net, we want to quantize its real-valued weights using a codebook with K entries so that the training loss of the quantized net is minimal. The codebook can be optimally learned jointly with the net, or fixed, as for bina…
KANs replace fixed MLP weights with learnable edge functions, improving accuracy and interpretability.
problem Lack of interpretability and scalability in MLPs.
method KANs use learnable activation functions on edges instead of fixed weights, replacing weights with spline functions.
result KANs outperform MLPs in accuracy and interpretability with smaller models.
Optimizer memory affects learning rate sensitivity in shuffle order, impacting fine-tuning noise.
problem Optimizer memory affects the learning rate sensitivity in shuffle order, leading to fine-tuning noise.
method Isolated the mechanism of fixed-clock optimizer memory affecting the learning rate sensitivity in shuffle order, deriving a fit-free way to size the noise.
result Fixed-clock optimizers like AdamW produce a larger first-order noise channel compared to memoryless optimizers, affecting fine-tuning comparisons.
In this work we study some problems related with algebraic hypersurfaces invariant by foliations on weighted projective spaces PC(ϖ0,...,ϖn) generalizing some results known for $\p$, as for example: the number of singularities, with multiplicities, contained in the invariant quasi-smo…
Transformers can emulate various algorithms by prompting, proving universality.
problem How to emulate algorithms using fixed-weight Transformers.
method Two modes of in-context algorithm emulation: task-specific and prompt-programmable. Constructing prompts that encode algorithm parameters into token representations.
result Fixed-weight Transformers can emulate a broad class of algorithms via prompts.
Paper improves DNN accelerator robustness against bit errors with energy savings.
problem Bit errors in quantized DNN weights reduce energy efficiency.
method Combines robust fixed-point quantization, weight clipping, and random bit error training.
result Significantly improves robustness against random bit errors with high energy savings.
Optimal strategy found for identifying best arm in bandits with small gap.
problem Best arm identification in two-armed bandits with a fixed budget and small gap.
method Neyman allocation rule augmented with inverse probability weighting.
result Proposed strategy is asymptotically optimal when gap is small.
Let T be a torus of dimension at least k and M a T-manifold. M is a GKM_k-manifold if the action is equivariantly formal, has only isolated fixed points, and any k weights of the isotropy representation in the fixed points are linearly independent. In this paper we compute the cohomology rings with real and integer coe…
Following the idea of Lusztig, Atiyah-Hirzebruch and Kosniowski, we note that the Dolbeault-type operators on compact, almost-complex manifolds are rigid. When the circle action has isolated fixed points, this rigidity result will produce many identities concerning the weights on the fixed points. In particular, it giv…
Optimizes sample weights for representative data averages.
problem Achieving sample averages close to prescribed values.
method Formulates as an optimization problem, often convex and efficiently solvable.
result Heuristic methods based on convex optimization perform well.
Fixed-parameter tractability of private synthetic data generation
problem Generating synthetic data under differential privacy
method Linear programming and subsampled private multiplicative weights method
result Optimal error rates across all regimes
Proposes a sliding window method for better portfolio trading.
problem Log-optimal portfolio problem with time-varying weights.
method Data-driven sliding window approach to solve log-optimal portfolio problem.
result Trading strategy outperforms classical log-optimal portfolio in cumulative returns.
We describe a Markov latent state space (MLSS) model, where the latent state distribution is a decaying mixture over multiple past states. We present a simple sampling algorithm that allows to approximate such high-order MLSS with fixed time and memory costs.
The paper develops concentration inequalities for structured random data, extending beyond independent terms.
problem Developing concentration inequalities for structured weighted sums of random data, including tensors and matrix-valued data.
method The paper develops Hoeffding and Bernstein bounds for structured weighted sums under exchangeability, extending beyond the classical framework of independent terms.
result The paper develops a sharper concentration bound for combinatorial sums of matrix arrays.