A new method trains deep networks by separating weight locations from values.
problem Training deep networks efficiently and effectively.
method Lookahead Permutation (LaPerm) to train DNNs by reconnecting weights.
result LaPerm can train DNNs with random and dense, sparse, or single-valued initial weights.
Due to a resource-constrained environment, network compression has become an important part of deep neural networks research. In this paper, we propose a new compression method, \textit{Inter-Layer Weight Prediction} (ILWP) and quantization method which quantize the predicted residuals between the weights in all convol…
Proposes volumization for neural networks to control bias-variance tradeoff.
problem Improving generalization and preventing memorization in neural networks.
method Defines a physical volume for weights, interpolating between L2 and L∞ regularization.
result Volumization interpolates between weight decay and clipping, improving generalization.
Method measures weight similarity in neural networks using normalization and statistical inference.
problem Quantifying weight similarity in non-convex neural networks.
method Chain normalization rule and hypothesis-training-testing statistical inference.
result Weights of identical neural networks converge to similar local solutions.
Current deep neural networks (DNNs) can easily overfit to biased training data with corrupted labels or class imbalance. Sample re-weighting strategy is commonly used to alleviate this issue by designing a weighting function mapping from training loss to sample weight, and then iterating between weight recalculating an…
Weight Squeezing transfers knowledge from large models to smaller ones, improving performance and speed.
problem Transfer learning and model compression for faster and more efficient training.
method Reparameterization of weights from a large model to a smaller one.
result Weight Squeezing outperforms other methods on GLUE benchmark with faster training.
The paper predicts edge weights in weighted directed networks using metric geometry.
problem Predicting edge weights in weighted directed networks.
method Introducing new types of weighted directed networks (AWDNs), constructing metrics, and proposing modified kNN and SVM methods.
result The proposed methods outperform traditional approaches in predicting edge weights.
A new weighted FDA method improves face recognition accuracy.
problem Equal treatment of all class pairs in FDA leads to suboptimal performance.
method Cosine-weighted and automatically weighted FDA methods are proposed.
result Improved face recognition accuracy through weighted FDA.
New method for initializing RBM weights without datasets.
problem No dataset-free weight-initialization for RBMs.
method Statistical mechanical analysis to derive Gaussian distribution with optimized standard deviation.
result Optimal weight initialization improves learning efficiency in RBMs.
S-SGD adds symmetrical noise to weights to avoid sharp minima in deep learning.
problem SGD does not always converge to a flat minimum, leading to poor generalization.
method Symmetrical weight noise injection in SGD.
result S-SGD outperforms conventional SGD and weight-noise injection methods in large batch training.
A new method corrects weight values to improve treatment effect estimation.
problem Estimating heterogeneous treatment effects in high-dimensional data with sample selection bias.
method Differentiable Pareto-Smoothed Weighting (DPSW) framework.
result Our method outperforms existing methods in treatment effect estimation.
A new method uses nearest neighbors for importance weighting.
problem Data covariate shift problems in machine learning.
method Nearest neighbor classification scheme for determining importance weights.
result Demonstrated effectiveness through comparative experiments on various classification tasks.
Method identifies change points in high-dimensional models using sample weights.
problem Identifying change points in high-dimensional generalized linear models.
method Sample-weighted empirical risk minimization (Weighted ERM).
result Weighted ERM yields precise asymptotic performance characterization for Gaussian designs.
We study the inverse spectral problem for weighted projective spaces using wave-trace methods. We show that in many cases one can "hear" the weights of a weighted projective space.
Estimates Gaussian mixtures from weighted samples efficiently.
problem Estimating Gaussian mixtures from weighted samples with correct weight treatment.
method Density interpretation and expectation-maximization method considering weights.
result Correctly estimates Gaussian mixtures with weighted samples.
New method estimates Nishimori temperature for node classification in weighted graphs.
problem Estimating Nishimori temperature for Bayesian inference.
method Spectral method using eigenvalues of Bethe Hessian matrix.
result Spectral method outperforms existing approaches in node classification.
New method clusters hypergraphs using weighted random walks and Laplacians.
problem Clustering hypergraph data with edge-dependent weights.
method Random walks with edge-dependent vertex weights, constructing hypergraph Laplacians for clustering.
result Proposed methods outperform existing hypergraph clustering algorithms.
Novel Adam-family method with decoupled weight decay for training neural networks.
problem Training nonsmooth neural networks with weight decay.
method Proposes a novel Adam-family method with decoupled weight decay, establishing convergence properties and demonstrating superior performance.
result Asymptotically approximates SGD and enhances generalization performance.
AWNN improves matrix completion by adaptively weighting nearest neighbors.
problem Matrix completion with optimal nearest neighbor weights and radii selection.
method Adaptively weighted nearest neighbor method for matrix completion.
result Theoretical guarantees and synthetic experiments support the effectiveness of AWNN.
Develops a weighting framework to generalize ITRs from source to target populations.
problem Challenges in generalizing ITRs from a source population to a target population with differing characteristics.
method A robust sample weighting framework using a reproducing kernel Hilbert space to balance covariates and improve ITR learning methods.
result Improves ITR estimation for the target population compared to other weighting methods.
A new accelerated method with simpler momentum update rules.
problem Optimizing parameters in machine learning models.
method Proposes a novel accelerated stochastic gradient method with simpler momentum update rules.
result The method outperforms Sgdm and Adam in practical problems.
Proves positive mass theorem for non-spin weighted manifolds.
problem Proving the positive mass theorem for non-spin weighted manifolds.
method Establishing density theorem and generalizing Geroch conjecture.
result Proves positive weighted mass theorem for non-spin weighted manifolds.
A new stock selection strategy uses combined machine learning with dynamic weighting methods.
problem Improving stock selection accuracy and performance.
method Combined machine learning algorithms with static and dynamic weighting methods.
result IC-based dynamic weighting outperforms static evaluation metrics in backtested returns and predictive performance.
New method improves causal effect estimation by addressing imbalance in training data.
problem Imbalance between treatment and control groups in training data.
method Combines distributionally robust optimization and weight regularization.
result Consistent improvements over existing methods in experiments.
Bayesian method improves neural net convergence for character recognition.
problem Improving convergence rate of neural network training algorithms.
method Customization of Kalman filter into Bayesian statistics for initialization of weights.
result Improved convergence rate for backpropagation training algorithm.
This paper proposes a new evaluation metric and boosting method for weight separability in neural network design. In contrast to general visual recognition methods designed to encourage both intra-class compactness and inter-class separability of latent features, we focus on estimating linear independence of column vec…
TSC improves causal effect estimation in panel data.
problem Estimating causal effects in panel data with a single treated unit.
method Targeted synthetic control method that refines initial weights through a one-dimensional targeted update.
result TSC consistently improves estimation accuracy over state-of-the-art SCM baselines.
We present an alternative to the pseudo-inverse method for determining the hidden to output weight values for Extreme Learning Machines performing classification tasks. The method is based on linear discriminant analysis and provides Bayes optimal single point estimates for the weight values.
Reintroduces straight-through estimators for binary neural networks.
problem Training neural networks with binary weights and activations is challenging due to gradient issues and discrete weight optimization.
method Derives ST methods as estimators in the SBN model, analyzes properties and estimation accuracy, explains latent weights and mirror descent method.
result Reintroduces ST methods as sound approximations and provides clearer application and improvements.
Adversarial robustness improved by sparsity in network weights.
problem How sparsity affects adversarial robustness in neural networks.
method Theoretical proof and experimental validation of adversarial pruning methods.
result Weights sparsity improves adversarial robustness, especially through inheritance from smaller networks.
New graph kernel for weighted directed networks using functor homology.
problem Studying weighted directed networks with functor homology.
method Proposes a new homological method to define graph kernels for weighted directed graphs.
result Defined a new graph kernel for weighted directed graphs.
A new method estimates treatment effects without strong assumptions.
problem Treatment effect estimation with strong model assumptions.
method Distribution learning-based weighting method.
result Our method outperforms existing methods in estimating ATT.
Improves feature selection in high-dimensional data using LLM-generated weights.
problem Inaccurate LLM-generated weights degrade feature selection performance.
method Integrates LLM-generated weights into prior inclusion probabilities using LLM Sparsity Prior (LSP).
result Improves prediction accuracy and identifies clinically relevant features.
Reverse-weighted portfolios outperform in commodity futures markets.
problem Efficiency of commodity futures markets.
method Permutation-weighted portfolios, rank-based methods.
result Reverse-weighted portfolio outperforms price-weighted portfolio.
The paper analyzes constrained optimal portfolios in high dimensions using novel statistical learning techniques.
problem Forming optimal portfolios with constraints in high-dimensional asset spaces.
method CROWN method integrating factor models with nodewise regression for estimation in large dimensions.
result Demonstrates estimation consistency and convergence rates for constrained portfolio weights, risk, and Sharpe Ratio.
Method selects number of communities in weighted networks.
problem Selecting the number of communities in weighted networks.
method Proposes a novel weighted DCSBM and uses a sequential testing framework with spectral clustering and matrix scaling.
result Method is consistent in estimating the true number of communities under mild conditions.
Independent Component Analysis (ICA) - one of the basic tools in data analysis - aims to find a coordinate system in which the components of the data are independent. In this paper we present Multiple-weighted Independent Component Analysis (MWeICA) algorithm, a new ICA method which is based on approximate diagonalizat…
New algorithm detects communities in weighted networks, improving on binary ones.
problem Few methods exist for detecting communities in weighted networks.
method Pseudo-likelihood approach for weighted stochastic block model.
result The method is consistent and works well for both homogeneous and heterogeneous networks.
Efficient policy learning from observational data using weighted classification reductions.
problem Efficient policy evaluation does not necessarily lead to efficient estimation of policy parameters.
method Proposed an estimation approach based on generalized method of moments, efficient for policy parameters.
result Demonstrated empirical efficiency and regret benefits of a proposed method.
New method learns to weight unlabeled data in semi-supervised learning.
problem Equal weighting of all unlabeled data in semi-supervised learning.
method Adjust weights for each unlabeled example using influence function.
result Technique outperforms state-of-the-art methods on image and language classification tasks.
The paper proposes a new method for covariate balancing using IPM to improve causal inference.
problem Covariate imbalance in causal inference weighting methods, especially when models are not correctly specified.
method The integral probability metric (IPM) is used to determine optimal weights for treated and control groups.
result The proposed method can be consistent without specifying either the propensity score or outcome regression model.
Dropout increases the generalization of neural networks by expanding the weight space.
problem Understanding and improving the generalization of neural networks.
method Introducing weight expansion and showing that dropout leads to it.
result Dropout increases the generalization of neural networks by expanding the weight space.
Proposes tracking DNN weights for uncertainty estimation.
problem Discarded information during DNN training.
method Tracks weights' trajectory during optimization.
result Competitive uncertainty estimation results.
Develops method to construct Lie algebra weight system kernel using Vogel algebra.
problem Detecting correlators and distinguishing knots in 3D Chern-Simons theory.
method Uses Vogel's Λ algebra and Jacobi diagrams.
result Explicitly provides Jacobi diagrams in the kernel of sl_N weight system.
New methods train neural networks without changing weights, achieving similar or higher performance.
problem Training neural networks efficiently with randomly initialized weights.
method Switching connections on and off, flipping weights' signs, minimizing changed connections.
result Achieves similar or higher performance with less computational cost than training all weights.
The Importance Weighted Auto Encoder (IWAE) objective has been shown to improve the training of generative models over the standard Variational Auto Encoder (VAE) objective. Here, we derive importance weighted extensions to AVB and AAE. These latent variable models use implicitly defined inference networks whose approx…
In this paper, we develop a novel weighted Laplacian method, which is partially inspired by the theory of graph Laplacian, to study recent popular graph problems, such as multilevel graph partitioning and balanced minimum cut problem, in a more convenient manner. Since the weighted Laplacian strategy inherits the virtu…
A new method for offline RL using diffusion models and self-weighted guidance.
problem Challenges in computing scores for offline RL using weight functions.
method Constructing a diffusion over actions and weights, using the diffusion model for guidance.
result Performs on par with state-of-the-art methods on challenging environments.