Fine shape of local compacta represented by ordinary maps.
problem Representing fine shape of local compacta.
method Constructing a space ∣X∣ for each local compactum X such that fine shape classes correspond to homotopy classes of maps to ∣X∣. result Fine shape classes from any locally compact metrizable space Y to X bijectively correspond to homotopy classes of maps from Y to ∣X∣. The paper shows how to use fine shape to understand infinite-dimensional spaces.
problem Understanding infinite-dimensional metrizable spaces and their homology theories.
method Obtained results indicating fine shape is tractable and can be used for Polish spaces.
result Every Polish space is fine shape equivalent to the limit of an inverse sequence of simplicial maps.
Fine shape theory extends strong shape to noncompact metrizable spaces.
problem Computational complexity in extending strong shape to noncompact spaces.
method Introducing FDR-embeddings and mapping cylinders to extend SSDR-maps to noncompact spaces.
result Fine shape category can be represented as a left fraction localization.
We introduce and develop fine shape, which has a very simple definition and aims to supersede all previously known shape theories for metrizable spaces. The problem with known shape theories of metrizable spaces is illustrated by the following bizarre situation. Čech cohomology is an invariant of shape, and a fortiori …
The paper proves a Whitehead theorem for fine shape spaces.
problem Proving a Whitehead theorem for fine shape spaces.
method Using Steenrod-Sitnikov homotopy groups and ind-groups.
result Fine shape morphisms are equivalences if they induce isomorphisms on π_i.
Space mapping speeds up shape optimization for PDEs.
problem Efficiently solving shape optimization problems constrained by PDEs.
method Combines fine and coarse model optimizations using Riemannian metrics.
result Space mapping methods are highly efficient for complex shape optimization problems.
The paper develops a theory linking pretraining and fine-tuning in neural networks.
problem Understanding how initialization choices impact feature learning and generalization in neural networks.
method Analytical theory of diagonal linear networks, deriving generalization error as a function of initialization parameters and task statistics.
result Different initialization choices place networks into four fine-tuning regimes with varying abilities to support feature learning and generalization.
PointGMM learns hGMMs from point clouds for 3D shape representation.
problem Lack of shape priors and non-local information in point cloud representations.
method Neural network that learns hierarchical Gaussian mixture models (hGMMs) for 3D shapes.
result Generative model learns meaningful latent space for interpolations and novel shape synthesis.
TempBalance boosts model performance with low data.
problem Low-data training and fine-tuning in model alignment.
method Inspired by HT-SR theory, TempBalance balances training quality across layers.
result TempBalance improves model performance as data decreases.
Emergent misalignment is influenced by training dynamics, model priors, and data.
problem Emergent misalignment in models
method Exploring training dynamics, model priors, and data
result Activation deltas before and after narrow fine-tuning correlate with their similarities when measured with the last prompt-token activations.
Identifies most probable flows for Kunita SDEs in fluid dynamics.
problem Modeling stochastic processes with Eulerian noise and deterministic drifts.
method Equipping the domain with a Riemannian metric from the noise, solving the resulting PDEs.
result Most probable flows differ from deterministic flows, especially under noise.
New model generates images by reversing heat equation, revealing disentanglement.
problem Image generation without considering image structure.
method Stochastically reverses the heat equation to generate images, using variational approximation.
result Emergent disentanglement of overall colour and shape in images.
Paper presents a method for efficient robot adaptation using fine-tuning.
problem Continuous adaptation of robot learning systems in real-world scenarios.
method Fine-tuning previously learned policies using off-policy reinforcement learning.
result Fine-tuning leads to substantial performance gains and adaptation to new conditions.
Various semigroups of noninvertible supermatrices of the special (antitriangle) shape having nilpotent Berezinian which appear in supersymmetric theories are defined and investigated. A subset of them continuously represents left and right zero semigroups and rectangular bands. The ideal properties of higher order rect…
We present a method that trains large capacity neural networks with significantly improved accuracy and lower dynamic computational cost. We achieve this by gating the deep-learning architecture on a fine-grained-level. Individual convolutional maps are turned on/off conditionally on features in the network. To achieve…
We introduce SADs to reveal how network architecture shapes score-based generative models.
problem Understanding and predicting the inductive biases of score-based generative models.
method Introducing Score Anisotropy Directions (SADs) to analyze network architecture.
result SADs reliably capture model behavior and correlate with performance.
JEPA fails to improve language model performance when fine-tuning.
problem Improving language model performance through latent representation learning.
method Testing various training-time auxiliaries on natural-language-to-regex generation.
result No auxiliary improves cross-entropy gradient cosine with decoder visibility.
The paper reveals surprising star-shaped connectivity in neural networks.
problem Understanding mode connectivity in neural network landscapes.
method Fine-grained analysis of connectivity in overparameterized and finite minima cases.
result Star-shaped connectivity exists in neural network landscapes, suggesting near convexity.
New method uses LLMs to generate detailed scientific hypotheses.
problem Generating detailed, actionable scientific hypotheses from coarse initial directions.
method Hierarchical search method that incrementally adds details to hypotheses.
result Hierarchical search method consistently outperforms strong baselines on expert-annotated hypotheses.
Territorial control is a key aspect shaping the dynamics of civil war. Despite its importance, we lack data on territorial control that are fine-grained enough to account for subnational spatio-temporal variation and that cover a large set of conflicts. To resolve this issue, we propose a theoretical model of the relat…
Early diagnosis, playing an important role in preventing progress and treating the Alzheimer\{'}s disease (AD), is based on classification of features extracted from brain images. The features have to accurately capture main AD-related variations of anatomical brain structures, such as, e.g., ventricles size, hippocamp…
Early diagnosis, playing an important role in preventing progress and treating the Alzheimer's disease (AD), is based on classification of features extracted from brain images. The features have to accurately capture main AD-related variations of anatomical brain structures, such as, e.g., ventricles size, hippocampus …
New mechanism protects neural network weights from privacy attacks during self-supervised learning.
problem Privacy risks during fine-tuning stage of self-supervised learning.
method Proposes a novel differential privacy mechanism using additive logistic noise.
result Reduces membership inference attack accuracy to 50% while maintaining below 5% performance loss.
The paper introduces a method to learn and apply value envelopes for faster online reinforcement learning.
problem Accelerating online reinforcement learning using offline data with theoretical grounding.
method A two-stage framework: offline data for learning value bounds, online algorithms for applying them.
result Substantial regret reductions in empirical tests on tabular MDPs.
QABBA improves time series storage efficiency while preserving shape information.
problem Efficient storage and shape preservation of time series data.
method Quantized symbolic time series approximation (QABBA) using ABBA technique.
result QABBA achieves a new state-of-the-art on Monash regression dataset.
Medical ultrasound technology is widely used in routine clinical applications such as disease diagnosis and treatment as well as other applications like real-time monitoring of human tongue shapes and motions as visual feedback in second language training. Due to the low-contrast characteristic and noisy nature of ultr…
We propose a novel automatic method for accurate segmentation of the prostate in T2-weighted magnetic resonance imaging (MRI). Our method is based on convolutional neural networks (CNNs). Because of the large variability in the shape, size, and appearance of the prostate and the scarcity of annotated training data, we …
Proposes a framework for energy-efficient AIGC workload scheduling in cloud data centers.
problem Challenges of scheduling AIGC workloads for energy efficiency and quality control.
method Joint energy management and coordinated AIGC workload scheduling framework with diffusion model-aided reward shaping.
result Effective learning of scheduling policies under sparse environmental feedback.
New measure FTC quantifies how much a ReLU network can fine-tune.
problem Analyzing memorization capacity in fine-tuned neural networks.
method Defined Fine-Tuning Capacity (FTC) for additive fine-tuning of ReLU networks.
result Upper and lower bounds on FTC for 2 and 3-layer ReLU networks.
This paper re-evaluates hyperparameters for fine-tuning pre-trained models.
problem Current hyperparameter settings for fine-tuning are often ad-hoc and fixed.
method Empirical evaluation of learning rate, batch size, and momentum for fine-tuning.
result Optimal hyperparameters are not only dataset-dependent but also sensitive to domain similarity.
We use high-frequency data of 1364 Chinese A-share stocks traded on the Shanghai Stock Exchange and Shenzhen Stock Exchange to investigate the intraday patterns in the bid-ask spreads. The daily periodicity in the spread time series is confirmed by Lomb analysis and the intraday bid-ask spreads are found to exhibit L…
New method quantifies uncertainty in fine-tuned LLMs using LoRA ensembles.
problem Uncertainty in fine-tuned LLMs and how to trust their predictions.
method Posterior approximations using low-rank adaptation ensembles.
result Unexpected retention of acquired knowledge during fine-tuning in overfitting regime.
BERT fine-tuning is unstable due to optimization issues, not forgetting or dataset size.
problem Stability of fine-tuning BERT-based models across different random seeds.
method Analysis of BERT, RoBERTa, and ALBERT fine-tuned on GLUE datasets, identifying optimization difficulties as the cause of instability.
result Fine-tuning instability is due to optimization difficulties leading to vanishing gradients, not forgetting or dataset size.
Neural networks are known to be a class of highly expressive functions able to fit even random input-output mappings with 100% accuracy. In this work, we present properties of neural networks that complement this aspect of expressivity. By using tools from Fourier analysis, we show that deep ReLU networks are biased…
Improved code translation by preserving structure with composed fine-tuning.
problem Improving code translation accuracy with unlabeled code outputs.
method Pre-trained denoiser to capture output structure, composed fine-tuning to fine-tune predictor.
result Composed fine-tuning significantly improves generalization over standard fine-tuning.
New findings on hyperbolicity of fine curve graphs and their subgraphs.
problem Investigating hyperbolicity of fine curve graphs and their subgraphs.
method Analyzing large subgraphs of fine curve graphs and computing distances in specific cases.
result Large subgraphs of fine curve graphs contain flats of every finite dimension, indicating they are not hyperbolic.
Geomstats introduces shape module for analyzing shapes of objects.
problem Analyzing shapes of objects represented as landmarks, curves, and surfaces.
method Implementing shape spaces, group actions, fiber bundles, quotient spaces, and Riemannian metrics.
result Users can compare, average, and interpolate shapes inside shape spaces.
The paper introduces a Hessian-based method to improve generalization in fine-tuned deep neural networks.
problem Improving generalization in fine-tuned deep neural networks, especially in noisy conditions.
method PAC-Bayesian analysis to identify a Hessian-based distance measure, proving generalization bounds, and developing an algorithm with a generalization error guarantee.
result Hessian-based distance measure correlates well with observed generalization gaps and can match the scale of these gaps in practice.
Homotopy types of curve and arc complexes are studied.
problem Understanding the homotopy types of curve and arc complexes.
method Proving homotopy equivalence and contractibility of complexes.
result Fine curve complex is homotopy equivalent to curve complex, fine arc complex is contractible.
Fine-tuning LLMs improves capability but harms safety, study finds.
problem Balancing capability and safety in LLM fine-tuning.
method Theoretical framework and numerical experiments for two safety-aware fine-tuning strategies.
result Characterization of fundamental limits of safety-capability trade-off in LLM fine-tuning.
Optimizes sparse fine-tuning for privacy in neural networks.
problem Performance gap between DP-SGD and non-private fine-tuning.
method Optimization-based approach using private gradient information for selecting trainable weights.
result Our selection method leads to better prediction accuracy compared to existing approaches.
A new shape space allows optimization of non-smooth shapes in fluid mechanics.
problem Optimizing non-smooth shapes in fluid mechanics.
method Constructing a product manifold to include piecewise-smooth shapes.
result Numerical results show applicability in minimizing viscous energy dissipation.
Fine-tuning harms in-context learning, but restricting updates to the value matrix improves zero-shot performance.
problem Fine-tuning harms in-context learning, reducing zero-shot performance on unseen tasks.
method Theoretical analysis of linear attention models, identifying conditions for degraded few-shot performance.
result Restricting updates to the value matrix improves zero-shot performance while preserving in-context learning.
Introduces Star-Shaped deviation measures for risk analysis.
problem Risk measurement and analysis in finance.
method Characterizes Star-Shaped deviation measures through acceptance sets and convex deviation measures.
result Exposes the relationship between Star-Shaped risk measures and deviation measures.
Measures consistency of tabular LLM predictions under fine-tuning multiplicity.
problem Conflicting predictions from fine-tuned tabular LLMs.
method Local stability measure in embedding space.
result Probabilistic guarantees on prediction consistency under multiplicity.
Self-play fine-tuning improves diffusion models for text-to-image generation.
problem Plateauing performance of diffusion models after data saturation.
method Self-play fine-tuning (SPIN-Diffusion) using competition among model versions.
result Significantly improved model performance and human preference alignment.
Optimizes shapes in uncertain Navier-Stokes flow problems.
problem Optimizing shapes with geometric constraints and physical uncertainty.
method Multi-shape calculus and stochastic augmented Lagrangian method.
result Successfully optimized shapes in uncertain Navier-Stokes flow.
Automorphisms of fine 1-curve graph linked to surface homeomorphisms.
problem Understanding automorphisms of fine 1-curve graphs.
method Isomorphic mapping to surface homeomorphisms.
result Automorphism group is isomorphic to homeomorphism group of a surface.