Greedy training of recursive partitioning estimators faces a computational barrier when the true function doesn't satisfy a specific property.
problem Computational inefficiency of greedy training for recursive partitioning estimators.
method Analysis of greedy training for sparse regression functions over binary features.
result Greedy training requires exponential samples when the true function doesn't satisfy a specific property (MSP), but only logarithmic samples when it does.
New property helps SGD learn sparse functions efficiently in neural networks.
problem Characterizing functions learnable by SGD in non-linear neural networks.
method Mean-field analysis, hierarchical merged-staircase property, dimension-free dynamics approximation.
result Merged-staircase property is necessary and nearly sufficient for SGD learnability.
The staircase property aids deep learning by guiding hierarchical feature learning.
problem Understanding how hierarchical structure influences deep learning performance.
method Defined and proved the staircase property for Boolean hypercube functions, and showed its learnability by layerwise stochastic coordinate descent.
result Staircase functions can be learned in polynomial time using layerwise stochastic coordinate descent on regular neural networks.
The straight-line flow on almost every staircase and on almost every square tiled staircase is recurrent. For almost every square tiled staircase the set of periodic orbits is dense in the phase space.
Optimal DP mechanisms for vector queries are found to be staircase distributions.
problem Designing optimal additive mechanisms for vector-valued queries under differential privacy.
method Reduction to radially symmetric distributions and convex rearrangement theory.
result Staircase mechanisms are optimal for any norm and cost function.
Study Markov staircases in symplectic embeddings of rational homology ellipsoids.
problem Symplectic embeddings of rational homology ellipsoids into the complex projective plane.
method Analysis of almost toric fibrations and Hamiltonian isotopies.
result Existence of an infinite staircase for each Markov triple.
In this paper, a similarity-driven cluster merging method is proposed for unsuper-vised fuzzy clustering. The cluster merging method is used to resolve the problem of cluster validation. Starting with an overspecified number of clusters in the data, pairs of similar clusters are merged based on the proposed similarity-…
The hierarchical Dirichlet process (HDP) has become an important Bayesian nonparametric model for grouped data, such as document collections. The HDP is used to construct a flexible mixed-membership model where the number of components is determined by the data. As for most Bayesian nonparametric models, exact posterio…
SGD learns neural networks with a complexity measure called leap.
problem Time complexity of SGD learning on neural networks.
method Introduced a complexity measure called leap, proved conjecture for Gaussian data, and showed saddle-to-saddle dynamics.
result Proved a conjecture about the time complexity of learning functions with low-dimensional support.
Efficient algorithms find optimal monotone transforms for calibration under strictly convex losses.
problem Calibrating estimations to improve performance with monotone transforms.
method Proposed linear-time and space algorithm for finding optimal monotone transforms for specific loss functions. Also proposed an anytime algorithm with linear space and pseudo-linearithmic time complexity.
result Optimal monotone transforms are unique and can be found efficiently for various strictly convex loss functions.
Neural networks outperform kernels by learning features better.
problem Current theories of feature learning do not adequately assess feature quality.
method Introduced feature quality metric and examined existing theories empirically.
result Current theories of feature learning do not provide a sufficient foundation for neural network generalization.
DPSM clusters nodes in data and graph spaces via density propagation and subcluster merging.
problem Automatic clustering of nodes in data and graph spaces.
method Density-based node clustering with propagation process and spectral clustering on subclusters.
result DPSM effectively clusters nodes in both data and graph spaces.
Study on connection points on double regular polygons, providing coordinates and proving non-connection points.
problem Identifying connection points on double regular polygons.
method Examined coordinates in trace field, provided constructive proof for prime n. result For n=7, conjectured all remaining points are connection points; for n≥7 prime, provided explicit separatrix. A critical decision point when training predictors using multiple studies is whether studies should be combined or treated separately. We compare two multi-study prediction approaches in the presence of potential heterogeneity in predictor-outcome relationships across datasets: 1) merging all of the datasets and traini…
Hierarchical clustering is a popular method for analyzing data which associates a tree to a dataset. Hartigan consistency has been used extensively as a framework to analyze such clustering algorithms from a statistical point of view. Still, as we show in the paper, a tree which is Hartigan consistent with a given dens…
Split-Merge MCMC (Monte Carlo Markov Chain) is one of the essential and popular variants of MCMC for problems when an MCMC state consists of an unknown number of components. It is well known that state-of-the-art methods for split-merge MCMC do not scale well. Strategies for rapid mixing requires smart and informative …
LEWIS merges LLMs without training, improving performance on specific tasks.
problem Limited performance improvement of merged models on specific benchmarks.
method Guided model merging using layer-wise sparsity and task-vector pruning.
result Improved model performance by up to 11.3% on math-solving tasks.
Study shows training duration impacts model merging quality, suggesting joint selection of duration and method.
problem Impact of expert training duration on model merging quality for large language models (LLMs).
method Systematically fine-tuned experts on five domains across three model sizes, evaluating five merging methods at each duration.
result Training duration affects merging quality, with simple averaging degrading sharply and sparsification-based methods performing well past the validation optimum.
What are the possible shapes of various things and why? For instance, when a closed wire or a frame is dipped into a soap solution and is raised up from the solution, the surface spanning the wire is a soap film. What are the possible shapes of soap films and why? Or, for instance, why is DNA like a double spiral stair…
Study shows training duration affects model merging quality, suggesting joint selection of duration and method.
problem Impact of expert training duration on model merging quality for large language models (LLMs).
method Systematically fine-tuned experts on five domains across three model sizes, evaluated five merging methods at each duration.
result Training duration and merging method should be chosen jointly, not independently.
EpiMer merges models by solving Fréchet mean on a Riemannian manifold.
problem Integrating knowledge from multiple models without retraining.
method EpiMer casts model merging as solving the Fréchet mean on a Riemannian manifold, restricting computation to a low-rank subspace.
result EpiMer outperforms flat-geometry methods on image classification tasks.
New Bayesian method improves Pareto front estimation in multitask finetuning.
problem Efficiently estimating Pareto fronts for multitask finetuning.
method Variational Model Merging using non-Gaussian posteriors.
result More flexible posteriors lead to better Pareto front estimates.
Deep reinforcement learning (DRL) on Markov decision processes (MDPs) with continuous action spaces is often approached by directly training parametric policies along the direction of estimated policy gradients (PGs). Previous research revealed that the performance of these PG algorithms depends heavily on the bias-var…
Study on merging predictors in causal and anticausal directions using CMAXENT.
problem Comparing merging predictors in causal and anticausal directions.
method Using CMAXENT as inductive bias, study differences in merging predictors.
result CMAXENT solution reduces to logistic regression in causal direction and LDA in anticausal direction.
New method merges MCMC samples without distributional assumptions.
problem Efficiently merging MCMC samples from disjoint subsets.
method Diffusion generative modelling for density approximation.
result Outperforms existing methods on high-dimensional problems.
Securely evaluates the benefits of merging datasets for causal estimation.
problem Challenges in assessing the value of merging datasets for causal treatment effect estimation.
method Cryptographically secure multi-party computation to evaluate Expected Information Gain (EIG) while ensuring privacy.
result Demonstrates the first privacy-preserving method for dataset acquisition tailored to causal estimation.
A new method merges neural networks using CCA to improve model performance.
problem Improving model accuracy through ensembling while reducing computational and storage costs.
method CCA Merge, a new model merging algorithm based on Canonical Correlation Analysis.
result CCA Merge leads to better model performance than past methods, especially in merging more than two models.
This paper reverses a construction by merging boundary critical points into an interior one.
problem Pushing interior critical points to the boundary and splitting them into two boundary points.
method Specific assumptions allow merging two boundary critical points into one interior critical point.
result Merging two boundary critical points into a single interior critical point.
Budgeted Stochastic Gradient Descent (BSGD) is a state-of-the-art technique for training large-scale kernelized support vector machines. The budget constraint is maintained incrementally by merging two points whenever the pre-defined budget is exceeded. The process of finding suitable merge partners is costly; it can a…
NAMEx merges experts using Nash bargaining for improved performance.
problem Sparse Mixture of Experts merging strategies lack a principled weighting mechanism.
method Reinterpreting expert merging through game theory, introducing Nash Merging and complex momentum.
result NAMEx consistently outperforms competing methods across various tasks and system sizes.
Efficiently estimates longitudinal networks by merging sparse networks.
problem Estimating longitudinal networks with sparse and temporal data.
method Adaptive network merging, tensor decomposition, point process.
result Significantly reduces estimation error and provides guidance for network merging.
Study of 13,456 hot stellar systems reveals multi-layered grouping.
problem Understanding physical and evolutionary properties of Hot Stellar Systems.
method Used stellar mass, effective radius, and mass-to-luminosity ratio to group HSS into eight homogeneous ellipsoidal groups, then merged them through a multi-phased syncytial algorithm.
result Identified two complex-structured groups of HSS, one older and smaller, the other brighter and younger.
A new method reduces task interference in model merging.
problem Model merging overlooks structural information and is susceptible to task interference.
method Task Singular Vectors (TSV) and TSV-Compress for compression and interference reduction.
result TSV-Merge significantly outperforms existing methods.
Discrete knot theory models use lattice-filtered graphs to detect merging knot components.
problem Detecting merging knot components in discrete models.
method Lattice-filtered move graphs to model knot types, identifying connected components and merge scales.
result Merge scale defined by connected components of lattice-filtered move graphs, with specific examples for the figure-eight knot.
Boosting strategies for merging vs. ensembling studies analyzed.
problem Deciding between merging and ensembling studies for boosting.
method Analytical transition point and bias-variance decomposition for boosting with linear learners.
result Theoretical guidelines for merging vs. ensembling studies.
Method constructs finance LLMs without instruction data using pretraining and model merging.
problem Developing domain-specific LLMs for finance is resource-intensive.
method Continual pretraining on financial data + model merging of instruction-tuned and domain-specific pretrained vectors.
result Successfully constructs instruction-tuned LLMs for finance without additional instruction data.
Characterizes a subset of links using quasipositive and homogeneous properties.
problem Understanding the properties of T-positive links.
method Characterization through strongly quasipositive and T-homogeneous braids.
result T-positive links are precisely the strongly quasipositive links that are closures of T-homogeneous braids.
Deep learning algorithms for connectomics rely upon localized classification, rather than overall morphology. This leads to a high incidence of erroneously merged objects. Humans, by contrast, can easily detect such errors by acquiring intuition for the correct morphology of objects. Biological neurons have complicated…
LEARN-SAM improves RL from sub-optimal demonstrations by localizing expert policies and selectively using demonstrations.
problem Improving RL from sub-optimal or sparse demonstrations.
method Local Ensemble and Reparameterization with Split and Merge of expert policies (LEARN-SAM).
result LEARN-SAM boosts learning speed and accuracy by selectively using demonstrations.
Single global merging boosts decentralized learning performance.
problem Limited communication in decentralized learning hinders performance.
method Scheduled communication, focusing on final step with global merging.
result Single global merging improves global test performance.
Gradient boosted decision trees (GBDT) is the leading algorithm for many commercial and academic data applications. We give a deep analysis of this algorithm, especially the histogram technique, which is a basis for the regulized distribution with compact support. We present three new modifications. 1) Share memory tec…
Bayesian Federated Inference improves survival model analysis without merging data.
problem Accurately estimating survival model parameters requires sufficient data and events, which is often lacking in practice.
method Bayesian Federated Inference (BFI) for survival models, where local centers perform analyses and combine results.
result Results from BFI are similar to those from merged data analyses, demonstrating excellent performance.
In order to drive safely and efficiently under merging scenarios, autonomous vehicles should be aware of their surroundings and make decisions by interacting with other road participants. Moreover, different strategies should be made when the autonomous vehicle is interacting with drivers having different level of coop…
New methods merge discrete gradient fields from patches to correct errors.
problem Correctly merging partially defined discrete gradient fields from patches.
method Developed general and lean merging procedures for specific covering patterns.
result Corrected errors in merging discrete gradient fields from patches.
The article improves the display of acceptable exchange ratios for merging companies.
problem Determining feasible exchange ratios for merging companies.
method Exploits a diagrammatic approach to display the bargaining region.
result Shares face upper and lower bounds for acceptable exchange ratios.
Language recognition system is typically trained directly to optimize classification error on the target language labels, without using the external, or meta-information in the estimation of the model parameters. However labels are not independent of each other, there is a dependency enforced by, for example, the langu…
The paper uses Seshadri constants to construct symplectic ellipsoid embeddings.
problem Constructing symplectic embeddings of ellipsoids.
method Exploring weighted blow-ups and Seshadri constants to demonstrate symplectic embeddings.
result Illustrates constructions of ellipsoid fillings and embeddings.
We introduce a new algorithm, called CDER, for supervised machine learning that merges the multi-scale geometric properties of Cover Trees with the information-theoretic properties of entropy. CDER applies to a training set of labeled pointclouds embedded in a common Euclidean space. If typical pointclouds correspondin…