Optimal microlending group size is 5 people.
problem Determining the best group size for microlending to minimize default risk.
method Mathematical modeling with interacting forces and precise hypotheses.
result The optimal microlending group size is 5 people.
The paper analyzes how contagion affects the survival probability of investment groups in microfinance.
problem The impact of contagion on the survival probability of investment groups in microfinance.
method A probabilistic approach to compute group survival probability with and without contagion effects.
result In homogeneous groups, including more members increases the probability of eventual default to 1.
Proposes a differentiable hypergeometric distribution for learning group importance.
problem Learning the sizes of subsets in applications like clustering and weakly-supervised learning.
method Introduces a reparameterizable hypergeometric distribution to model group sizes and learn their relative importance.
result Outperforms previous methods in weakly-supervised learning and clustering.
Multi-group learners suffer a penalty in transductive learning.
problem The penalty on multi-group learners in transductive learning.
method Analyzing the relationship between the number of groups and the error rate.
result The penalty can increase linearly with the number of groups, up to the square-root of the sample size.
Unified framework for understanding GRPO as U-statistic.
problem Theoretical properties of GRPO remain less studied.
method Unified framework through classical U-statistics.
result GRPO is asymptotically equivalent to an oracle policy gradient algorithm.
Develops a new model to predict training dynamics of large language models.
problem Lack of mechanistic understanding of training dynamics in large language models.
method A first-principles reduced-order model of training dynamics, predicting group-size invariance and stability thresholds.
result Closed-form model predicts training dynamics with high accuracy and provides new diagnostics.
E2GC optimizes energy efficiency in DNNs by balancing computational and data movement costs.
problem Imbalance between computational complexity and data reuse in GConv leads to suboptimal energy efficiency.
method Developed an optimum group size model and proposed E2GC module with constant group size.
result E2GC modules improve energy efficiency by 10.8% and 4.73% on P100 and P4000 GPUs, respectively.
We study a generalized framework for structured sparsity. It extends the well-known methods of Lasso and Group Lasso by incorporating additional constraints on the variables as part of a convex optimization problem. This framework provides a straightforward way of favouring prescribed sparsity patterns, such as orderin…
Optimal scaling found to depend on operator norm across large models and datasets.
problem Lack of unifying principle for optimal hyperparameter scaling across models and datasets.
method Discovered that optimal scaling is conditioned on the operator norm of the output layer.
result The optimal learning rate/batch size pair (η∗,B∗) consistently has the same operator norm value. This article considers the problem of multi-group classification in the setting where the number of variables p is larger than the number of observations n. Several methods have been proposed in the literature that address this problem, however their variable selection performance is either unknown or suboptimal to…
New theorem bounds group quotient size to subgroups index.
problem Understanding subgroup structure in hyperbolic groups.
method Proved quotient size bounds on Eilenberg-MacLane spaces.
result One-ended hyperbolic groups cannot have isomorphic finite-index subgroups.
New mathematical framework proves the effectiveness of reducing neural network sizes.
problem Selecting optimal neural network sizes to avoid overfitting.
method Adaptive group Lasso applied to one-hidden-layer feedforward networks.
result Adaptive group Lasso is consistent and can accurately reconstruct network sizes.
Bayesian optimization improves efficiency with semi-supervised learning.
problem Efficiently find global optima of expensive functions.
method Density ratio estimation combined with semi-supervised learning.
result Improved accuracy in identifying global optima with unlabeled data.
We leverage recent advances in high-dimensional statistics to derive new L2 estimation upper bounds for Lasso and Group Lasso in high-dimensions. For Lasso, our bounds scale as (k∗/n)log(p/k∗)---n×p is the size of the design matrix and k∗ the dimension of the ground truth β∗---and match t…
Proposes a group-splicing algorithm for efficient BSGS in high-dimensional settings.
problem Efficiently selecting a small part of non-overlapping groups for best interpretability in high-dimensional settings.
method Iteratively detects relevant groups and excludes irrelevant ones using a novel group information criterion.
result Certifiable polynomial-time algorithm for identifying the optimal subset of groups with high probability.
Paper classifies totally symmetric sets in groups and bounds their sizes.
problem Understanding homomorphisms between groups using totally symmetric sets.
method Full classifications and size bounds for totally symmetric sets in various groups.
result Derives restrictions on homomorphisms between certain groups.
In data summarization we want to choose k prototypes in order to summarize a data set. We study a setting where the data set comprises several demographic groups and we are restricted to choose ki prototypes belonging to group i. A common approach to the problem without the fairness constraint is to optimize a c…
Ensembles of neural networks improve training dynamics and performance.
problem Improving neural network performance through model size increase.
method Defining collegial ensembles (CE) as multiple independent models trained as a single model, and using theoretical results on NTK to optimize architecture search.
result CE dynamics simplify and scale favorably, resembling wide models, and can be efficiently implemented using group convolutions and block diagonal layers.
The paper improves A/B testing for non-Gaussian data, ensuring reliable results with large sample sizes.
problem Inaccurate A/B testing results due to non-normal data and unequal sample sizes.
method Derives explicit formulas for minimum sample size and introduces an Edgeworth-based correction.
result Corrected method improves reliability of A/B testing in real-world conditions.
Optimizes survey design for private mean estimation with reduced variance.
problem Minimizing variance in private mean estimation with privacy constraints.
method Formulates optimal survey design as an optimization problem, determining optimal subsampling sizes to minimize variance.
result Identifies the first privacy-aware stratified sampling scheme that minimizes variance under different privacy mechanisms.
Adaptive SGD learns optimal batch size for strong convex functions.
problem Finding optimal batch size for SGD in practice.
method Adaptive SGD method that learns optimal batch size.
result Adaptive SGD exhibits nearly optimal performance in experiments.
We prove that the rank (that is, the minimal size of a generating set) of lattices in a general connected Lie group is bounded by the co-volume of the projection of the lattice to the semi-simple part of the group. This was proved by Gelander for semi-simple Lie groups and by Mostow for solvable Lie groups. Here we con…
Standard optimizers perform as well as LARS and LAMB at large batch sizes.
problem Comparing optimizers for neural network training at large batch sizes.
method Used standard optimizers like Nesterov momentum and Adam to match or exceed LARS and LAMB results.
result Standard optimizers can match or exceed LARS and LAMB at large batch sizes.
Study optimizes data collection from biased, costly sources to minimize risk.
problem Estimating population means and group-conditional means from multiple sources with varying costs and biases.
method Develops a sampling plan that maximizes effective sample size, paired with a post-stratification estimator.
result Achieves budgeted minimax optimal risk for estimating population means and group-conditional means.
SGD's performance improves with critical batch size, minimizing SFO complexity.
problem Optimizing SGD's performance with batch size and learning rate.
method Analysis of SGD using constant and decaying learning rates, focusing on batch size effects.
result SGD with critical batch size minimizes SFO complexity.
New algorithm improves stability of optimization algorithms by adapting step-size.
problem Optimization algorithms' effectiveness is sensitive to step-size hyperparameters.
method Adapts NGN step-size method with momentum to enhance stability.
result Achieves convergence rate of O(1/√K) without restrictive assumptions.
A new scaling law predicts optimal batch size for training models.
problem Finding the optimal batch size for training models efficiently.
method Proposed a three-term scaling law that considers model size, training data, training steps, and batch size.
result The three-term law accurately recovers the optimal batch size and can be robustly fit with fewer training runs.
Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.
problem Optimizing non-convex functions on Riemannian manifolds.
method Batch size adaptation in R-SVRG, R-SRG, and R-SPIDER.
result Achieves lower total complexities for various non-convex functions.
We make policy optimization algorithms batch size-invariant by decoupling proximal and behavior policies.
problem Some policy optimization algorithms do not have batch size-invariance, leading to inefficiencies.
method We decouple the proximal policy from the behavior policy to achieve batch size-invariance.
result Our approach makes policy optimization algorithms more efficient and allows them to use stale data more effectively.
We study the problem of learning with label proportions in which the training data is provided in groups and only the proportion of each class in each group is known. We propose a new method called proportion-SVM, or ∝SVM, which explicitly models the latent unknown instance labels together with the known group …
With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Often we want to identify an influential group of training samples in a particular test prediction for a given machine learning model. Existing influence functions tackle this…
Ensembles of random-feature models can't outperform a single large model.
problem Finding the optimal balance between model size and ensemble size.
method Deterministic equivalent risk estimates and scaling laws analysis.
result Ensembles of random-feature models achieve near-optimal performance only under specific conditions.
For n at least 7 and n equal to 5, we give generating sets of size 2 for the commutator subgroup of the braid group on n strands. These generating sets are of the smallest possible cardinality. For n equal to 4 or 6, we give generating sets of size three. We also prove that the commutator subgroup of the braid …
Reinforcement Patching optimizes dynamic sequence patching for efficient time series forecasting.
problem Efficiently learning data-adaptive representations for long-horizon sequence data, especially continuous sequences.
method Reinforcement Patching (ReinPatch) uses reinforcement learning to optimize dynamic patching policies and sequence backbones.
result ReinPatch achieves compelling performance in time-series forecasting compared to state-of-the-art methods.
We propose an adaptive optimization method for deep learning that dynamically adjusts batch size.
problem Optimizing deep learning models with varying sensitivity to batch size selection.
method Adaptive regularization with dynamically determined stochastic batch size based on gradient norms.
result Our method outperforms state-of-the-art optimization algorithms in generalization and robustness.
sGPO optimizes RLVR training by balancing inference FLOPs and training efficiency.
problem RLVR training allocates rollout budget without considering query difficulty.
method sGPO trades inference FLOPs for reduced training FLOPs.
result sGPO matches or exceeds baseline performance while reducing training compute by a factor of three.
The sparse group lasso optimization problem is solved using a coordinate gradient descent algorithm. The algorithm is applicable to a broad class of convex loss functions. Convergence of the algorithm is established, and the algorithm is used to investigate the performance of the multinomial sparse group lasso classifi…
Increasing the batch size is a popular way to speed up neural network training, but beyond some critical batch size, larger batch sizes yield diminishing returns. In this work, we study how the critical batch size changes based on properties of the optimization algorithm, including acceleration and preconditioning, thr…
We present the Integrated Size and Price Optimization Problem (ISPO) for a fashion discounter with many branches. Based on a two-stage stochastic programming model with recourse, we develop an exact algorithm and a production-compliant heuristic that produces small optimality gaps. In a field study we show that a distr…
The paper characterizes simply connected quandles using cocycles with prime values.
problem Characterizing simply connected quandles.
method Using cocycles with values in abelian groups of prime size.
result Classification of simply connected quandles of sizes p2 and p3 for p>3. We present a novel approach which is able to explore the configuration of grouped convolutions within neural networks. Group-size Series (GroSS) decomposition is a mathematical formulation of tensor factorisation into a series of approximations of increasing rank terms. GroSS allows for dynamic and differentiable selec…
Adaptive step-size improves optimization in complex geometries.
problem Optimizing functions with non-Euclidean geometries.
method Adaptive step-size strategy for optimization algorithms.
result Guaranteed convergence for Adaptive Conditional Gradient Descent.
A simple block configures optimal kernel sizes for time series classification.
problem Choosing the right kernel size for time series classification.
method Proposes Omni-Scale block (OS-block) with kernel sizes determined by prime numbers.
result Models with OS-block achieve state-of-the-art performance on time series benchmarks.
New adaptive step-size method for convex optimization without tuning.
problem Optimizing convex functions efficiently with stochastic gradients.
method Adapted Adaptive Gradient Descent Without Descent to stochastic setting.
result Stochastic gradient descent converges under various assumptions.
New method selects variables in groups with few nonzeros, improving support recovery.
problem Structured variable selection with sparse patterns across groups.
method Composite norm and proximal algorithm for exclusive group sparsity.
result Asymptotic consistency in signed support recovery under conventional assumptions.
We compress large neural networks for quick adaptation to specific contexts.
problem How to quickly adapt a pretrained large neural network to specific contexts.
method Propose a Bayesian hypernetwork framework to compress the network and encourage sparsity.
result Generated compressed networks are significantly smaller than baseline methods.
We find the optimal error for a constrained regression model under a linear model.
problem Minimizing error while adhering to demographic parity constraints.
method Proposed a minimax optimal error analysis for a demographic parity-constrained regression problem within a linear model.
result The minimax optimal error is characterized by $Θ( rac{dM}{n})$.
New preconditioners speed up SGD on Lie groups.
problem Accelerating SGD convergence on Lie groups.
method Online updating of low-rank and matrix-free preconditioners.
result Preconditioners maintain symmetry and invariance.