Two retraining techniques outperform fine-tuning in neural network pruning.
problem Improving accuracy and compression in neural network pruning.
method Weight rewinding and learning rate rewinding compared to fine-tuning.
result Rewinding techniques outperform fine-tuning in accuracy and compression.
The paper proves finite topological type theorems for open manifolds with non-negative Ricci curvature and almost maximal local rewinding volume.
problem Proving finite topological type theorems for open manifolds with non-negative Ricci curvature and almost maximal local rewinding volume.
method The approach removes constraints of sectional curvature or conjugate radius and extends to previous related studies.
result Theorems are proven for open manifolds with non-negative Ricci curvature and almost maximal local rewinding volume, without the need for triangle comparison of Toponogov type.
We examine how recently documented, fundamental phenomena in deep learning models subject to pruning are affected by changes in the pruning procedure. Specifically, we analyze differences in the connectivity structure and learning dynamics of pruned models found through a set of common iterative pruning techniques, to …
The paper proves fibration theorems for manifolds with almost nonnegative Ricci curvature.
problem Proving fibration theorems for manifolds with specific curvature conditions.
method Using equivariant regularity theorems and Gromov-Hausdorff convergence.
result Closed manifolds with certain curvature conditions fiber over a b1-torus. The paper proves rigidity results for Einstein manifolds with specific geometric constraints.
problem Understanding the rigidity of Einstein manifolds under bounded covering geometry.
method Analyzing Einstein manifolds with bounded covering geometry to prove rigidity results.
result Compact Einstein manifolds with specific geometric properties are isometric to space forms.
We study the phenomenon that some modules of deep neural networks (DNNs) are more critical than others. Meaning that rewinding their parameter values back to initialization, while keeping other modules fixed at the trained parameters, results in a large drop in the network's performance. Our analysis reveals interestin…
If a document is about travel, we may expect that short snippets of the document should also be about travel. We introduce a general framework for incorporating these types of invariances into a discriminative classifier. The framework imagines data as being drawn from a slice of a Levy process. If we slice the Levy pr…
This paper studies limits of aspherical manifolds with specific curvature conditions.
problem Understanding the Gromov-Hausdorff limits of aspherical manifolds with given curvature constraints.
method Analyzing sequences of compact manifolds with Ricci curvature or sectional curvature conditions, and using diffeomorphism or homeomorphism properties.
result If the manifolds are diffeomorphic or homeomorphic to nilmanifolds, their limits are also diffeomorphic or homeomorphic to nilmanifolds.
IMP finds sparse subnetworks that match full networks, revealing geometric insights.
problem Finding sparse subnetworks that match full, overparameterized networks.
method Iterative magnitude pruning (IMP) algorithm, analyzing error landscape geometry.
result IMP masks found at end of training convey useful information for rewound networks.
ICE-Pruning accelerates deep neural network pruning by 9.61x.
problem Efficiently pruning deep neural networks while maintaining accuracy.
method Iterative pruning with automatic fine-tuning steps, freezing strategy, and custom learning rate scheduler.
result Significantly reduces pruning time by up to 9.61x.
The sample mean is among the most well studied estimators in statistics, having many desirable properties such as unbiasedness and consistency. However, when analyzing data collected using a multi-armed bandit (MAB) experiment, the sample mean is biased and much remains to be understood about its properties. For exampl…
Generative models need per-sample confidence scores to improve quality and stability.
problem Generative models produce unreliable outputs and lack confidence measures.
method Flow Matching with Confidence (FMwC) injects noise and integrates it through the network, providing per-sample confidence scores.
result The confidence score correlates with the velocity field's divergence, offering insights into generative processes.
A new weighted MCC measure improves classifier performance evaluation.
problem Lack of measures sensitive to observation weights in multiclass classification.
method Proposes weighted versions of Pearson-Matthews Correlation Coefficient (MCC) for binary and multiclass classification.
result Weighted MCC values are higher for classifiers that perform better on highly weighted observations.
Stability of weighted extremal manifolds proven through blowups.
problem Stability of weighted extremal manifolds.
method Blowup technique to analyze weighted extremal Kähler manifolds.
result Proves weighted extremal manifolds are relatively weighted K-polystable.
Develops theory of weightings for Lie groupoids and algebroids.
problem Understanding differential geometry of weightings for Lie groupoids and algebroids.
method Extending work on weighted manifolds, defining weighted submanifolds, and developing theories of linear weightings and multiplicative weightings.
result Characterizes infinitesimally multiplicative weightings for Lie algebroids and classifies multiplicative weightings of Lie groupoids.
The paper extends spin geometry to weighted manifolds and defines a new mass for Ricci flow.
problem Generalizing spin geometry to weighted manifolds and defining a new mass.
method Investigates spectral properties of the weighted Dirac operator and defines a new mass.
result Defines a new mass for weighted asymptotically Euclidean manifolds and shows its monotonicity under Ricci flow.
The study explores weightings on submanifolds and their geometric properties.
problem Understanding weightings on submanifolds and their geometric implications.
method Detailed exploration of weighted normal bundles, weighted deformation spaces, and weighted blow-ups.
result A description of weightings in terms of subbundles of higher tangent bundles, leading to new concepts for Lie algebroids and groupoids.
New mass and staticity concepts derived from weighted curvature maps.
problem Deriving mass and staticity concepts for weighted manifolds.
method Developed a weighted curvature map and its adjoint, leading to weighted mass and static metrics.
result Equivalence and uniqueness theorems for weighted static manifolds and Penrose inequality.
Proves existence and uniqueness of weighted metrics for smooth spaces.
problem Existence and uniqueness of weighted metrics for smooth metric measure spaces.
method Proves existence and uniqueness using weighted ambient metrics and Poincaré metrics.
result Existence and uniqueness of weighted metrics for smooth metric measure spaces.
Defines and proves properties of weighted renormalized volume coefficients.
problem None explicitly stated; focuses on mathematical definitions and proofs.
method Defines weighted renormalized volume coefficients and proves their variational nature and polynomial representation.
result Weighted renormalized volume coefficients are variational and can be expressed as polynomials of specific tensors.
A new weighted FDA method improves face recognition accuracy.
problem Equal treatment of all class pairs in FDA leads to suboptimal performance.
method Cosine-weighted and automatically weighted FDA methods are proposed.
result Improved face recognition accuracy through weighted FDA.
Proves positive mass theorem for non-spin weighted manifolds.
problem Proving the positive mass theorem for non-spin weighted manifolds.
method Establishing density theorem and generalizing Geroch conjecture.
result Proves positive weighted mass theorem for non-spin weighted manifolds.
New invariants help solve existence of weighted cscK metrics.
problem Existence of weighted cscK metrics in K-stability.
method Introduced weighted analytic delta invariant and beta invariant.
result Sufficient condition for existence of weighted cscK metrics.
Derives integral formulae on weighted manifolds.
problem No specific problem stated; focuses on mathematical derivations.
method Introduces weighted mean sigma-r curvature and uses weighted Newton transformations.
result Derives integral formulae generalizing previous work.
Method measures weight similarity in neural networks using normalization and statistical inference.
problem Quantifying weight similarity in non-convex neural networks.
method Chain normalization rule and hypothesis-training-testing statistical inference.
result Weights of identical neural networks converge to similar local solutions.
The paper studies weighted Ricci curvatures and characterizes Randers metrics.
problem Characterizing Randers metrics with weighted Ricci curvatures.
method General weighted Ricci curvatures and characterization of Randers metrics.
result Characterization of Randers metrics with almost isotropic weighted Ricci curvatures.
Study on stable minimal hypersurfaces under Ricci curvature constraints.
problem Stability of weighted minimal hypersurfaces under Ricci curvature bounds.
method Derive geometric consequences and prove a Schoen-Yau type criterion.
result Structure theorem for three-dimensional weighted manifolds of non-negative Ricci curvature.
Explains weightings along submanifolds, focusing on Lie groupoids.
problem None explicitly stated; focuses on theory review.
method Reviews basic notions and emphasizes multiplicative weightings.
result Provides a comprehensive overview of weightings along submanifolds.
The study establishes comparison theorems for weighted Finsler manifolds and spacetimes.
problem Analyzing weighted Finsler manifolds and spacetimes with curvature conditions.
method Using weight function and ε-range, the Bonnet-Myers theorem, Laplacian comparison theorem, and Bishop-Gromov volume comparison theorem are formulated. result New comparison theorems for weighted Finsler manifolds and spacetimes are derived, including those for weighted Riemannian manifolds.
The study analyzes weighted manifolds with curvature bounds, proving eigenvalue estimates and inequalities.
problem Analyzing geometric properties of weighted manifolds under Ricci curvature bounds.
method Develops geometric analysis techniques on weighted Riemannian manifolds with lower 0-weighted Ricci curvature bounds. result Proves eigenvalue estimates for Steklov and ABP inequalities on weighted manifolds.
The paper generalizes K-stability results to singular and weighted settings.
problem Generalizing K-stability to singular and weighted settings.
method Generalization of results in \cite{Li22a} to singular and weighted settings.
result The \(\mathbb{G}\)-uniform weighted K-stability for models implies \(\mathbb{G}\)-coercivity of the weighted Mabuchi functional.
Study on deformation of weighted scalar curvature, proving geometric results and stability.
problem Deformation of weighted scalar curvature and related geometric properties.
method Linearization of weighted scalar curvature, studying kernel of formal adjoint.
result Definition and study of weighted vacuum static spaces, stability results on flat spaces.
Paper extends trigonometric summation formula with weights.
problem Trigonometric summation formula by Grigor'yan, Lin and Yau.
method Weighted trigonometric summation formula derivation.
result Extension of trigonometric summation formula.
The Penrose theorem and Hawking's topology theorem are extended to weighted spacetimes.
problem Extending Penrose's singularity theorem and Hawking's topology theorem to weighted spacetimes.
method Using weighted null energy condition and synthetic dimension to generalize the theorems.
result Generalized versions of the Penrose and Hawking theorems hold under a weighted null energy condition.
A new method trains deep networks by separating weight locations from values.
problem Training deep networks efficiently and effectively.
method Lookahead Permutation (LaPerm) to train DNNs by reconnecting weights.
result LaPerm can train DNNs with random and dense, sparse, or single-valued initial weights.
A new method to improve deep neural networks using weight rescaling.
problem Overfitting and sensitivity to hyperparameters in weight decay.
method Weight rescaling (WRS) to control weight norm and prevent overfitting.
result WRS outperforms weight decay and other methods in various applications.
The paper predicts edge weights in weighted directed networks using metric geometry.
problem Predicting edge weights in weighted directed networks.
method Introducing new types of weighted directed networks (AWDNs), constructing metrics, and proposing modified kNN and SVM methods.
result The proposed methods outperform traditional approaches in predicting edge weights.
Optimal weight windows are found by projecting the origin onto a convex polytope.
problem Finding the best weight windows for a weighted moving average smoother.
method Formulated as a quadratic program and projection onto a convex polytope.
result Optimal weight windows are symmetrical and decrease in weight away from the center.
Optimizes weights for better model performance in shifting data.
problem Improper importance weighting leads to poor model performance in data shifts.
method Interprets weights as a bias-variance trade-off and optimizes them simultaneously with model parameters.
result Optimizing weights significantly improves model generalization performance.
Study of weighted nonlinear flags in symplectic geometry.
problem Understanding the geometry of weighted nonlinear flags.
method Generalizing weighted nonlinear Grassmannians to Frechet manifolds and using them to describe coadjoint orbits.
result Description of coadjoint orbits of Hamiltonian diffeomorphisms using weighted isotropic nonlinear flags.
Due to a resource-constrained environment, network compression has become an important part of deep neural networks research. In this paper, we propose a new compression method, \textit{Inter-Layer Weight Prediction} (ILWP) and quantization method which quantize the predicted residuals between the weights in all convol…
Study on convergence rate of weighted Yamabe flow.
problem Weighted Yamabe problem on smooth metric measure spaces.
method Weighted Yamabe flow and its convergence rate analysis.
result Study and analysis of convergence rate of the weighted Yamabe flow.
Survey on importance weighting in machine learning applications.
problem Distribution shift in supervised learning.
method Weighting objective function or probability distribution based on instance importance.
result Importance weighting can guarantee desirable statistical properties in distribution shift scenarios.
Smooth superspace with special weights has a Fubini-Study form.
problem Describing a new smooth superspace with a special structure.
method Construction of weighted projective superspace and description of its structure.
result Smooth superspace with weights +1,−1 has an analog of the Fubini-Study form. Study weakly weighted Einstein-Finsler metrics, showing specific curvature properties and characterizing them.
problem Characterizing weakly weighted Einstein-Finsler metrics.
method Showed isotropic S-curvature under certain conditions. Characterized via navigation expressions and α and β. result Weakly weighted Einstein-Kropina metrics have isotropic S-curvature and can be completely characterized.
AWP improves robustness by flattening weight loss landscape.
problem Improving robustness of deep neural networks against adversarial examples.
method Explicitly regularizes the flatness of weight loss landscape through adversarial weight perturbation.
result AWP forms a double-perturbation mechanism in adversarial training, leading to flatter weight loss landscape.
We study the inverse spectral problem for weighted projective spaces using wave-trace methods. We show that in many cases one can "hear" the weights of a weighted projective space.
Study distortion risk measures for step-weighted distributions.
problem Analyzing risk measures for specific distribution types.
method Investigate distortion risk measures of step-weighted distributions.
result Developed methods for calculating risk measures.