Securely evaluates the benefits of merging datasets for causal estimation.
problem Challenges in assessing the value of merging datasets for causal treatment effect estimation.
method Cryptographically secure multi-party computation to evaluate Expected Information Gain (EIG) while ensuring privacy.
result Demonstrates the first privacy-preserving method for dataset acquisition tailored to causal estimation.
LEWIS merges LLMs without training, improving performance on specific tasks.
problem Limited performance improvement of merged models on specific benchmarks.
method Guided model merging using layer-wise sparsity and task-vector pruning.
result Improved model performance by up to 11.3% on math-solving tasks.
LiteMORT reduces memory usage for GBDT models by 30% with improved accuracy.
problem Memory inefficiency in GBDT models, especially with adaptive compact distributions.
method Share memory, implicit merging, adaptive histogram resizing.
result Significant reduction in memory usage with improved accuracy.
Deep learning algorithms for connectomics rely upon localized classification, rather than overall morphology. This leads to a high incidence of erroneously merged objects. Humans, by contrast, can easily detect such errors by acquiring intuition for the correct morphology of objects. Biological neurons have complicated…
New method merges MCMC samples without distributional assumptions.
problem Efficiently merging MCMC samples from disjoint subsets.
method Diffusion generative modelling for density approximation.
result Outperforms existing methods on high-dimensional problems.
Efficiently estimates longitudinal networks by merging sparse networks.
problem Estimating longitudinal networks with sparse and temporal data.
method Adaptive network merging, tensor decomposition, point process.
result Significantly reduces estimation error and provides guidance for network merging.
The MAXENT principle helps merge datasets to infer causal effects.
problem Inferring causal effects from unobserved variables.
method Using the maximum entropy principle with causal sufficiency and faithfulness assumptions.
result Identifies causal edges among variables from merged datasets.
A critical decision point when training predictors using multiple studies is whether studies should be combined or treated separately. We compare two multi-study prediction approaches in the presence of potential heterogeneity in predictor-outcome relationships across datasets: 1) merging all of the datasets and traini…
Study merges datasets to improve AI model performance.
problem Improving machine learning models with diverse datasets.
method Developed an algorithm using oracle inequality and data-driven estimators.
result Algorithm reduces population loss with high probability.
Federated learning for Bayesian clustering of large datasets.
problem Bayesian model-based clustering of large-scale binary and categorical data.
method Federated variational inference with local merge and delete moves in parallel batches, followed by global merge moves.
result Empirical validation shows superior performance compared to existing algorithms.
Merging datasets is a key operation for data analytics. A frequent requirement for merging is joining across columns that have different surface forms for the same entity (e.g., the name of a person might be represented as "Douglas Adams" or "Adams, Douglas"). Similarly, ontology alignment can require recognizing disti…
Split-Merge MCMC (Monte Carlo Markov Chain) is one of the essential and popular variants of MCMC for problems when an MCMC state consists of an unknown number of components. It is well known that state-of-the-art methods for split-merge MCMC do not scale well. Strategies for rapid mixing requires smart and informative …
Method constructs finance LLMs without instruction data using pretraining and model merging.
problem Developing domain-specific LLMs for finance is resource-intensive.
method Continual pretraining on financial data + model merging of instruction-tuned and domain-specific pretrained vectors.
result Successfully constructs instruction-tuned LLMs for finance without additional instruction data.
We consider the initial situation where a dataset has been over-partitioned into k clusters and seek a domain independent way to merge those initial clusters. We identify the total variation distance (TVD) as suitable for this goal. By exploiting the relation of the TVD to the Bayes accuracy we show how neural networ…
Clustering partitions a dataset such that observations placed together in a group are similar but different from those in other groups. Hierarchical and K-means clustering are two approaches but have different strengths and weaknesses. For instance, hierarchical clustering identifies groups in a tree-like structure b…
Generative model tackles inconsistent attributes across datasets by enabling precise conditional generation.
problem Inconsistent attributes across merged datasets limit controllability in conditional generative modeling.
method Diffusion Model with Double Guidance, maintaining control over multiple conditions without joint annotations.
result Outperforms baselines in molecular and image generation tasks, aligning with target distributions and controlling missing conditions.
In big data image/video analytics, we encounter the problem of learning an overcomplete dictionary for sparse representation from a large training dataset, which can not be processed at once because of storage and computational constraints. To tackle the problem of dictionary learning in such scenarios, we propose an a…
Study shows training duration impacts model merging quality, suggesting joint selection of duration and method.
problem Impact of expert training duration on model merging quality for large language models (LLMs).
method Systematically fine-tuned experts on five domains across three model sizes, evaluating five merging methods at each duration.
result Training duration affects merging quality, with simple averaging degrading sharply and sparsification-based methods performing well past the validation optimum.
Study shows training duration affects model merging quality, suggesting joint selection of duration and method.
problem Impact of expert training duration on model merging quality for large language models (LLMs).
method Systematically fine-tuned experts on five domains across three model sizes, evaluated five merging methods at each duration.
result Training duration and merging method should be chosen jointly, not independently.
EpiMer merges models by solving Fréchet mean on a Riemannian manifold.
problem Integrating knowledge from multiple models without retraining.
method EpiMer casts model merging as solving the Fréchet mean on a Riemannian manifold, restricting computation to a low-rank subspace.
result EpiMer outperforms flat-geometry methods on image classification tasks.
New Bayesian method improves Pareto front estimation in multitask finetuning.
problem Efficiently estimating Pareto fronts for multitask finetuning.
method Variational Model Merging using non-Gaussian posteriors.
result More flexible posteriors lead to better Pareto front estimates.
Deep reinforcement learning (DRL) on Markov decision processes (MDPs) with continuous action spaces is often approached by directly training parametric policies along the direction of estimated policy gradients (PGs). Previous research revealed that the performance of these PG algorithms depends heavily on the bias-var…
Study on merging predictors in causal and anticausal directions using CMAXENT.
problem Comparing merging predictors in causal and anticausal directions.
method Using CMAXENT as inductive bias, study differences in merging predictors.
result CMAXENT solution reduces to logistic regression in causal direction and LDA in anticausal direction.
In this paper, a similarity-driven cluster merging method is proposed for unsuper-vised fuzzy clustering. The cluster merging method is used to resolve the problem of cluster validation. Starting with an overspecified number of clusters in the data, pairs of similar clusters are merged based on the proposed similarity-…
Hierarchical clustering is a popular method for analyzing data which associates a tree to a dataset. Hartigan consistency has been used extensively as a framework to analyze such clustering algorithms from a statistical point of view. Still, as we show in the paper, a tree which is Hartigan consistent with a given dens…
CQL (ReDS) learns from varied driving behaviors, improving offline RL performance.
problem Learning from datasets with non-uniform variability in behavior policies.
method Reweighting the data distribution to allow per-state flexibility in following the behavior policy.
result CQL (ReDS) improves performance in various offline RL tasks.
A new method merges neural networks using CCA to improve model performance.
problem Improving model accuracy through ensembling while reducing computational and storage costs.
method CCA Merge, a new model merging algorithm based on Canonical Correlation Analysis.
result CCA Merge leads to better model performance than past methods, especially in merging more than two models.
This paper reverses a construction by merging boundary critical points into an interior one.
problem Pushing interior critical points to the boundary and splitting them into two boundary points.
method Specific assumptions allow merging two boundary critical points into one interior critical point.
result Merging two boundary critical points into a single interior critical point.
Budgeted Stochastic Gradient Descent (BSGD) is a state-of-the-art technique for training large-scale kernelized support vector machines. The budget constraint is maintained incrementally by merging two points whenever the pre-defined budget is exceeded. The process of finding suitable merge partners is costly; it can a…
NAMEx merges experts using Nash bargaining for improved performance.
problem Sparse Mixture of Experts merging strategies lack a principled weighting mechanism.
method Reinterpreting expert merging through game theory, introducing Nash Merging and complex momentum.
result NAMEx consistently outperforms competing methods across various tasks and system sizes.
Constructs models to meet any DP requirement without retraining.
problem Evolving privacy requirements at inference time.
method Two post-processing techniques: random selection and linear combination.
result Final private models satisfying any target privacy parameter.
Study improves deep learning chest X-ray models by incorporating lateral views.
problem Lack of lateral views in training datasets limits deep learning performance.
method Used PadChest dataset with multiple views to explore merging methods.
result Incorporating lateral views increases model performance for 32 labels.
A new method reduces task interference in model merging.
problem Model merging overlooks structural information and is susceptible to task interference.
method Task Singular Vectors (TSV) and TSV-Compress for compression and interference reduction.
result TSV-Merge significantly outperforms existing methods.
Discrete knot theory models use lattice-filtered graphs to detect merging knot components.
problem Detecting merging knot components in discrete models.
method Lattice-filtered move graphs to model knot types, identifying connected components and merge scales.
result Merge scale defined by connected components of lattice-filtered move graphs, with specific examples for the figure-eight knot.
Boosting strategies for merging vs. ensembling studies analyzed.
problem Deciding between merging and ensembling studies for boosting.
method Analytical transition point and bias-variance decomposition for boosting with linear learners.
result Theoretical guidelines for merging vs. ensembling studies.
Single global merging boosts decentralized learning performance.
problem Limited communication in decentralized learning hinders performance.
method Scheduled communication, focusing on final step with global merging.
result Single global merging improves global test performance.
Paper proposes a new method for designing materials using deep learning.
problem Designing high-performance material distributions from given distributions.
method Iterative process of selecting, generating, and merging material distributions using a deep generative model.
result The method improves material performance through iterative refinement.
Efficiently recovers piecewise linear functions from noisy samples.
problem Recovering a piecewise linear function from noisy samples with unknown segmentation.
method Iterative merging approach for multidimensional segmented regression.
result First sample and computationally efficient algorithm in any fixed dimension.
Motivation: With the development of droplet based systems, massive single cell transcriptome data has become available, which enables analysis of cellular and molecular processes at single cell resolution and is instrumental to understanding many biological processes. While state-of-the-art clustering methods have been…
It has always been a great challenge for clustering algorithms to automatically determine the cluster numbers according to the distribution of datasets. Several approaches have been proposed to address this issue, including the recent promising work which incorporate Bayesian Nonparametrics into the k-means clusterin…
DPSM clusters nodes in data and graph spaces via density propagation and subcluster merging.
problem Automatic clustering of nodes in data and graph spaces.
method Density-based node clustering with propagation process and spectral clustering on subclusters.
result DPSM effectively clusters nodes in both data and graph spaces.
Commonly-used clustering algorithms usually find ellipsoidal, spherical or other regular-structured clusters, but are more challenged when the underlying groups lack formal structure or definition. Syncytial clustering is the name that we introduce for methods that merge groups obtained from standard clustering algorit…
Bayesian Federated Inference improves survival model analysis without merging data.
problem Accurately estimating survival model parameters requires sufficient data and events, which is often lacking in practice.
method Bayesian Federated Inference (BFI) for survival models, where local centers perform analyses and combine results.
result Results from BFI are similar to those from merged data analyses, demonstrating excellent performance.
In order to drive safely and efficiently under merging scenarios, autonomous vehicles should be aware of their surroundings and make decisions by interacting with other road participants. Moreover, different strategies should be made when the autonomous vehicle is interacting with drivers having different level of coop…
PNN-smoothing improves k-means clustering by merging subsets' clusterings.
problem Improving k-means clustering initialization efficiency and effectiveness. method Split dataset into subsets, cluster each subset, merge with PNN method.
result PNN-smoothing enhances k-means++ seeding, reducing costs. New methods merge discrete gradient fields from patches to correct errors.
problem Correctly merging partially defined discrete gradient fields from patches.
method Developed general and lean merging procedures for specific covering patterns.
result Corrected errors in merging discrete gradient fields from patches.
The hierarchical Dirichlet process (HDP) has become an important Bayesian nonparametric model for grouped data, such as document collections. The HDP is used to construct a flexible mixed-membership model where the number of components is determined by the data. As for most Bayesian nonparametric models, exact posterio…
The article improves the display of acceptable exchange ratios for merging companies.
problem Determining feasible exchange ratios for merging companies.
method Exploits a diagrammatic approach to display the bargaining region.
result Shares face upper and lower bounds for acceptable exchange ratios.