Model shows cascading failures are more severe in multiplex networks than single-layer networks.
problem Underestimation of risks in single-layer network analyses due to overlooked impact of weak layers.
method Simple model of cascading failure on multiplex networks of weight-heterogeneous layers.
result Multiplex model produces more catastrophic cascading failures than single-layer model.
Random feature models can outperform a weak teacher with early stopping.
problem Generalization from a weak to a strong model in random feature networks.
method Random feature models, early stopping, proving weak-to-strong generalization.
result Random feature models can outperform a weak teacher with early stopping.
Gradient descent converges to minimum Bayes risk for two-layer ReLU networks in mean field regime.
problem Training two-layer ReLU networks using gradient descent in the mean field regime.
method Describes a condition for convergence to minimum Bayes risk, extending previous results to ReLU-activated networks.
result The condition for convergence does not depend on initialization and concerns weak convergence of network realization.
Proves a central limit theorem for neural networks with hidden layers.
problem Understanding the statistical behavior of neural networks with large numbers of hidden units and training iterations.
method Rigorous mathematical proof using weak convergence methods and stochastic analysis.
result Neural network fluctuations around mean-field limit follow a Gaussian distribution and satisfy a stochastic partial differential equation.
Study shows how a strong model can learn a task's feature while retaining other capabilities.
problem How to align superhuman AI systems using weak-to-strong generalization.
method Two-layer neural networks, reward-model learning, multi-step SGD, feature learning.
result The strong model efficiently learns task features while retaining general capabilities.
Paper analyzes weak-to-strong generalization in CNNs, identifying data-scarce and data-abundant regimes.
problem Weak-to-strong generalization in CNNs trained on weak models.
method Formal analysis of gradient descent dynamics in data-scarce and data-abundant regimes.
result Identifies two regimes and distinct mechanisms of generalization in each.
CLAPS improves conformal regression by adaptively scaling interval widths based on last-layer Laplace uncertainty.
problem Lack of adaptive interval width scaling in conformal regression for heterogeneous inputs.
method CLAPS uses heteroscedastic last-layer Laplace uncertainty to adaptively scale interval widths, combining aleatoric and epistemic uncertainties.
result CLAPS provides competitive interval efficiency with nominal-level coverage, reducing to aleatoric scaling as epistemic uncertainty decreases.
New framework explains how deep networks reduce complexity and maintain weak correlations.
problem Understanding the interpretability of deep neural networks.
method Mean-field framework to analyze deterministic and generative deep networks.
result Deep computation reduces dimensionality while keeping weak neuron correlations.
Dense neural networks can't approximate all functions.
problem Approximation capabilities of dense neural networks.
method Model compression approach combining weak regularity lemma and graph neural networks.
result Existence of Lipschitz continuous functions not approximable by dense neural networks.
Paper analyzes why deeper layers of ViTs perform worse on out-of-distribution tasks.
problem Performance degradation of intermediate layers in ViTs under distribution shift.
method Extensive linear probing experiments across various benchmarks and fine-grained analysis of transformer modules.
result Probing feedforward network activations yields best performance under significant distribution shift.
DCFS predicts stock indices using deep learning and fuzzy systems.
problem Predicting stock indices with high-dimensional inputs.
method Bottom-up layer-by-layer design of DCFS, parameter sharing.
result DCFS models outperform weak estimators and real-world data.
The paper analyzes deep learning representations using function space theory.
problem Understanding the geometry and smoothness of feature maps in deep learning.
method Function space analysis, specifically Besov smoothness index.
result The Besov smoothness of feature maps increases from layer to layer in well-performing networks.
WeLa-VAE learns interpretable disentangled representations with weak supervision.
problem Learning disentangled representations without strong supervision.
method Variational inference framework with shared latent variables and modified variational lower bound.
result WeLa-VAE learns alternative disentangled representations (polar) from weak labels (distance and angle) without refined supervision.
We analyze deep neural networks in the large size and iteration limit, revealing a deterministic system of equations.
problem Understanding the behavior of deep neural networks in the asymptotic regime of large network sizes and iterations.
method Sequential limit of each hidden layer and characterization of parameter evolution, using weak convergence and stochastic analysis.
result The limit neural network recovers a global minimum with zero loss for the objective function.
New method detects communities in complex hypergraphs, matching theoretical limits.
problem Detecting communities in non-uniform hypergraphs with varying hyperedge sizes.
method Developed a spectral theory for weighted non-backtracking operators on non-uniform hypergraphs.
result Achieved the Kesten-Stigum bound for weak recovery in a general class of non-uniform HSBMs.
Neural networks trained with actor-critic algorithms converge to ODEs under weak convergence analysis.
problem Challenges in convergence analysis due to changing data distributions in online learning.
method Geometric ergodicity of data samples, Poisson equation, weak convergence techniques.
result Actor and critic networks converge to solutions of ODEs with random initial conditions.
Transformers show strengths and weaknesses in complexity analysis.
problem Understanding the strengths and limitations of attention layers in transformers.
method Analysis of representation power through complexity parameters and task-specific constructions.
result Transformers can solve sparse averaging tasks with logarithmic complexity, but triple detection tasks require linear complexity.
Improved NER performance on imbalanced data.
problem Highly unbalanced training data in NER tasks.
method Adapted a neural architecture with CRF and BI-LSTM layers, using pre-trained embeddings. Introduced a two-class split to optimize performance.
result Significant improvement in performance for weak classes with minimal training data.
Attention mechanism learns to focus on sparse tokens efficiently.
problem Detecting weak, rare, and sparsely located features in long sequences.
method Theoretical analysis and training of a single-layer attention classifier in a sparse-token classification model.
result A single-layer attention classifier can achieve vanishing test error with logarithmic signal strength growth, unlike linear classifiers requiring linear growth.
Bayesian U-Net exploits epistemic uncertainty for anomaly detection in retinal OCT images.
problem Anomaly detection in retinal OCT images using weak labels of healthy anatomy.
method Bayesian U-Net trained on weak labels of healthy anatomy, using Monte Carlo dropout for uncertainty estimation, and post-processing to transfer uncertainty to anomaly segmentations.
result Achieved a Dice index of 0.789 in an independent test set of AMD cases.
Method learns feature maps from deep CNN layers for weakly supervised chest pathology localization.
problem Localization of chest pathologies in X-ray images is challenging due to varying sizes and appearances.
method Class-aware deep multiscale feature learning using intermediate feature maps from CNN layers.
result Improves localization performance of small pathologies like nodules and masses.
Study uses weak labels and visual attention networks to detect lung nodules in chest radiographs.
problem Automated detection of lung nodules in chest radiographs requires large amounts of manually annotated images.
method Proposes two network architectures: one using saliency maps and the other a recurrent attention model trained with reinforcement learning.
result Demonstrates promising nodule detection performance using weak labels and visual attention mechanisms.
LAGS-SGD optimizes deep learning training by sparsifying gradients layer-wise.
problem Reduces long training times in large deep neural networks with distributed S-SGD.
method Layer-wise adaptive gradient sparsification combined with S-SGD.
result LAGS-SGD achieves convergence guarantees and outperforms vanilla S-SGD.
Spotlight method finds hidden errors in deep learning models.
problem Systematic errors in deep learning models on rare data subsets.
method Shining a spotlight on hidden layer representations to find poor performance areas.
result Identifies semantically meaningful areas of weakness in various models.
RaNNDy uses randomized neural networks to learn transfer operators efficiently.
problem Efficiently learning transfer operators from data.
method Randomized neural network approach with randomly initialized hidden layers and trained output layer.
result Significant reduction in training time and resources with improved stability.
Gradient descent amplifies random features in neural networks to useful ones.
problem Generalization in neural networks trained on corrupted data.
method Characterization of feature-learning process in two-layer ReLU networks trained by gradient descent.
result Gradient descent amplifies random features to useful ones, achieving near optimal generalization error.
Deep learning's performance is analyzed using information theory.
problem Understanding why deep networks generalize better and whether deeper networks always improve performance.
method An upper bound on the expected generalization error is derived for deep neural networks.
result The expected generalization error decreases exponentially with the number of layers, but this doesn't guarantee small test error.
Snorkel automates training data creation using weak supervision.
problem Labeling training data is a bottleneck in deploying machine learning systems.
method Snorkel uses labeling functions written by users, which are denoised without ground truth.
result Snorkel increases predictive performance by 45.5% on average.
Random ReLU features are shown to be a universally consistent learning algorithm but struggle with complex functions.
problem Approximating complex functions with random ReLU features.
method Study of random ReLU features through their RKHS and composition of functions.
result Random ReLU features can efficiently approximate complex functions but not as well as multi-layer ReLU networks.
Supervised learning alone can be effective for offline RL, revealing essential elements.
problem Understanding when and how supervised learning alone can be effective for offline RL.
method Extensive experiments to identify essential elements for offline RL via supervised learning.
result Maximizing likelihood with a two-layer feedforward MLP is competitive with more complex methods.
Designing effective and efficient classifier for pattern analysis is a key problem in machine learning and computer vision. Many the solutions to the problem require to perform logic operations such as `and', `or', and `not'. Classification and regression tree (CART) include these operations explicitly. Other methods s…
Study shows how neural networks generalize with minimal training data.
problem Understanding how neural networks generalize with limited data.
method Mean-field analysis of KL-regularized empirical risk minimization.
result Generalization error rate is O(1/n) for large n. Improves deep networks' robustness to adversarial attacks.
problem Deep networks' failure to perform well on data different from training distribution.
method Fortifies hidden layers by mapping them back to parts of the data manifold where the network performs well.
result Improves robustness to adversarial attacks in both black-box and white-box threat models.
Deep neural networks generate dance steps from music with minimal labeled data.
problem Generating dance steps from music with little labeled data and maintaining timing accuracy.
method Weakly supervised deep recurrent neural network with convolutional and LSTM layers.
result Model generates dance steps with low cross entropy and maintains timing accuracy.
We discuss general notions of metrics and of Finsler structures which we call weak metrics and weak Finsler structures. Any convex domain carries a canonical weak Finsler structure, which we call its tautological weak Finsler structure. We compute distances in the tautological weak Finsler structure of a domain and we …
PortBench benchmarks LLMs for PM, revealing their weaknesses in diversification and robustness.
problem Lack of benchmarks for LLM-driven portfolio management, especially in diversification and robustness.
method Developed a comprehensive benchmark with a static QA dataset and a dynamic allocation pipeline, introducing metrics to evaluate correlation and robustness.
result 90% of LLMs fail to outperform a basic equal-weight allocation, highlighting their limitations in diversification and robustness.
New structures defined for studying contact foliations and their geometry.
problem Understanding dynamics of contact foliations and their applications.
method Define and study weak nearly S- and weak nearly C-structures.
result Characterize weak nearly S- and weak nearly C- submanifolds in weak nearly Kähler manifolds.
Tensoring p-weak differentiable structures preserves their properties.
problem Tensorization of p-weak differentiable structures. method Proving the product of p-weak charts is a p-weak chart, and showing isometric embeddings. result Tensorization of p-weak differentiable structures is possible under certain conditions. Deep neural networks undergo hierarchical free-energy landscape transitions with increasing data size.
problem Understanding the design space and dynamics of deep neural networks.
method Statistical mechanical approach based on replica method.
result Hierarchical free-energy landscape transitions with ultrametricity, leading to simpler configurations in deeper layers.
Graphs on surfaces have a 2-dimensional large scale structure.
problem Understanding the large scale structure of graphs on surfaces.
method Proving asymptotic dimension for specific graph classes and surfaces.
result Graphs on surfaces have an asymptotic dimension of 2.
New method prevents forgetting in learning new tasks.
problem Poor ability of models to solve new problems without forgetting.
method Task-agnostic hierarchical information-theoretic optimality principle with Mixture-of-Variational-Experts layer.
result Demonstrated competitive performance in continual supervised and reinforcement learning.
Paper shows attacks on transfer learning models without target-specific info.
problem Security vulnerability in transfer learning models.
method Brute force attack on pre-trained models without target-specific data.
result Demonstrates effective and efficient attacks on transfer learning models.
The study examines conditions for weak nearly cosymplectic manifolds to split into products.
problem Understanding the curvature and topology of weak nearly cosymplectic manifolds.
method Analyzes the conditions for splitting and characterizes specific manifolds.
result Conditions for weak nearly cosymplectic manifolds to become Riemannian products are identified.
Subset selection improves weak supervision performance.
problem Optimizing the use of weakly-labeled data.
method Combining pretrained data representations with the cut statistic for subset selection.
result Subset selection improves weak supervision performance by up to 19%.
Defines weak geodesics on specific subsets of manifolds.
problem Characterizing geodesics on prox-regular subsets of Riemannian manifolds.
method Defining weak geodesics as continuous curves with weak regularities, and characterizing them as viscosity critical points of the energy functional.
result Characterizes weak geodesics on prox-regular subsets of Riemannian manifolds.
New model shows weak teachers can help strong students learn even with imperfect labels.
problem Improving strong student's performance with weak teacher's imperfect pseudolabels.
method Stylized overparameterized spiked covariance model with Gaussian covariates, proving two phases of generalization.
result Provable successful and random guessing phases of strong student's generalization.
Study weak conjugacy in surface homeomorphisms.
problem Understanding weak conjugacy in homeomorphisms of surfaces.
method Exploring the group of homeomorphisms isotopic to the identity.
result New insights into weak conjugacy relations.
Paper improves Native ads CTR prediction using event embeddings and recurrent networks.
problem Hard CTR prediction for Native ads due to lack of direct query intent.
method Proposes a large-scale event embedding scheme and a recurrent neural network model.
result Significantly outperforms baseline and variants in CTR prediction.