arXiv research

New summary measures reveal geometric structure in weighted measures on manifolds.

New training method for ReLU networks achieves optimal weight size for memorization.

Adam's generalization performance is improved by batch size and weight decay in neural networks.

The classical $k$-means algorithm for partitioning $n$ points in $\mathbb{R}^d$ into $k$ clusters is one of the most popular and widely spread clustering methods. The need to respect prescribed lower bounds on the cluster sizes has been observed in many scientific and business applications. In this paper, we present an…

New concept of mixture complexity helps detect gradual clustering changes.

The paper proves barriers to approximating functions with small weights and depth in neural networks.

Paper proposes a cost-sensitive conformal training method with provably controllable learning bounds.

In this study, classification problems based on feedforward neural networks in a data-imbalanced environment are considered. Learning from an imbalanced dataset is one of the most important practical problems in the field of machine learning. A weighted loss function (WLF) based on a cost-sensitive approach is a well-k…

We compress large neural networks for quick adaptation to specific contexts.

A new method for averaging model predictions using minimum divergence.

SCS identifies a range of plausible equally weighted portfolios, quantifying selection uncertainty.

Deep networks run with low precision operations at inference time offer power and space advantages over high precision alternatives, but need to overcome the challenge of maintaining high accuracy as precision decreases. Here, we present a method for training such networks, Learned Step Size Quantization, that achieves…

DeepWeightFlow generates diverse neural network weights efficiently.

Combining logic and probability has been a long stand- ing goal of AI research. Markov Logic Networks (MLNs) achieve this by attaching weights to formulas in first-order logic, and can be seen as templates for constructing features for ground Markov networks. Most techniques for learning weights of MLNs are domain-size…

In the past years, Deep convolution neural network has achieved great success in many artificial intelligence applications. However, its enormous model size and massive computation cost have become the main obstacle for deployment of such powerful algorithm in the low power and resource-limited mobile systems. As the c…

IMPACT optimizes LLM compression by focusing on activation importance, reducing model size up to 55.4%.

As a consequence of the strong and usually violated conditional independence assumption (CIA) of naive Bayes (NB) classifier, the performance of NB becomes less and less favorable compared to sophisticated classifiers when the sample size increases. We learn from this phenomenon that when the size of the training data …

AdamP optimizes momentum-based optimizers for scale-invariant weights, improving model performance.

In this study, we consider classification problems based on neural networks in data-imbalanced environment. Learning from an imbalanced data set is one of the most important and practical problems in the field of machine learning. A weighted loss function based on cost-sensitive approach is a well-known effective metho…

The effect of proportional transaction costs on systematically generated portfolios is studied empirically. The performance of several portfolios (the index tracking portfolio, the equally-weighted portfolio, the entropy-weighted portfolio, and the diversity-weighted portfolio) in the presence of dividends and transact…

In this short report, we discuss how coordinate-wise descent algorithms can be used to solve minimum variance portfolio (MVP) problems in which the portfolio weights are constrained by $l_{q}$ norms, where $1\leq q \leq 2$. A portfolio which weights are regularised by such norms is called a sparse portfolio (Brodie et …

As a model problem for clustering, we consider the densest k-disjoint-clique problem of partitioning a weighted complete graph into k disjoint subgraphs such that the sum of the densities of these subgraphs is maximized. We establish that such subgraphs can be recovered from the solution of a particular semidefinite re…

An algorithm was recently introduced by INTECH for the purposes of estimating the trading-profit contribution of systematic rebalancing to the relative return of rules-based investment strategies. We apply this methodology to analyze the size factor through the use of equal-weighted portfolios. These strategies combine…

The Tsetlin Machine (TM) is an interpretable mechanism for pattern recognition that constructs conjunctive clauses from data. The clauses capture frequent patterns with high discriminating power, providing increasing expression power with each additional clause. However, the resulting accuracy gain comes at the cost of…

New sketches for weighted $\ell_p$ sampling without replacement improve accuracy and efficiency.

Modern deep neural network training is typically based on mini-batch stochastic gradient optimization. While the use of large mini-batches increases the available computational parallelism, small batch training has been shown to provide improved generalization performance and allows a significantly smaller memory footp…

A new framework SIMBA improves graph classification performance on size-imbalanced datasets.

New algorithm improves hypergraph clustering for unbalanced communities.

A new RNN architecture reduces model size and improves performance.

Study shows attention-style models learn pairwise interactions efficiently.

New duality found linking neural network weights and activities for better generalization.

We study binary classification in the setting where the learner is presented with multiple corrupted training samples, with possibly different sample sizes and degrees of corruption, and introduce an approach based on minimizing a weighted combination of corruption-corrected empirical risks. We establish a generalizati…

Analyzes why neural networks generalize beyond training data.

Optimizes weights for better model performance in shifting data.

This paper shows neural networks can solve complex graph problems efficiently.

This paper investigates compression techniques for deep neural networks to reduce their size without sacrificing performance.

This paper optimizes binary linear classifiers by tuning their weight vectors.

Study of linear classifiers in infinite imbalance scenarios.

Unrolled neural networks emerged recently as an effective model for learning inverse maps appearing in image restoration tasks. However, their generalization risk (i.e., test mean-squared-error) and its link to network design and train sample size remains mysterious. Leveraging the Stein's Unbiased Risk Estimator (SURE…

We extend quantization-aware training to extreme model compression.

Importance-weighting is a popular and well-researched technique for dealing with sample selection bias and covariate shift. It has desirable characteristics such as unbiasedness, consistency and low computational complexity. However, weighting can have a detrimental effect on an estimator as well. In this work, we empi…

Meta-learning predicts optimal ensemble size and methods for time series forecasting.

We develop a simple theoretical framework for the evolution of weighted networks that is consistent with a number of stylized features of real-world data. In our framework, the Barabasi-Albert model of network evolution is extended by assuming that link weights evolve according to a geometric Brownian motion. Our model…

SGD and weight decay encourage neural networks to learn low-rank weight matrices.

The paper analyzes prediction error in nonstationary settings using weighted risk minimization.

We speed up marginal inference by ignoring factors that do not significantly contribute to overall accuracy. In order to pick a suitable subset of factors to ignore, we propose three schemes: minimizing the number of model factors under a bound on the KL divergence between pruned and full models; minimizing the KL dive…

Extend CPS to non-exchangeable settings with observation-specific permutation weights

With the development of deep neural networks, the size of network models becomes larger and larger. Model compression has become an urgent need for deploying these network models to mobile or embedded devices. Model quantization is a representative model compression technique. Although a lot of quantization methods hav…