Sparse Bayesian Optimization (SEBO) finds interpretable configurations.
problem Optimizing black-box functions for recommendation systems while maintaining interpretability.
method Regularization-based approaches, including a differentiable relaxation for L0 regularization, and a hyperparameter-free method SEBO. result SEBO efficiently optimizes for sparsity without hyperparameters.
Entropy regularization improves sparse model discovery in federated learning.
problem Sparse model discovery in federated learning with limited data.
method Entropy regularization of gate distributions for probabilistic sparse model exploration.
result Entropy regularization leads to better sparse model recovery and performance.
DreamerV3 learns diverse tasks with a single configuration.
problem Generalizing reinforcement learning across multiple domains.
method World model and farsighted strategies.
result Outperforms specialized methods across 150 diverse tasks.
We propose a method for solving statistical mechanics problems defined on sparse graphs. It extracts a small Feedback Vertex Set (FVS) from the sparse graph, converting the sparse system to a much smaller system with many-body and dense interactions with an effective energy on every configuration of the FVS, then learn…
BONSAI optimizes parameters while respecting a default configuration, reducing unnecessary changes.
problem Standard BO pushes weakly relevant parameters to the boundary, making it hard to distinguish between important and spurious changes.
method BONSAI is a default-aware BO policy that prunes low-impact deviations from a default configuration while controlling acquisition value loss.
result BONSAI matches the GP-UCB regret rate while recovering the minimal-ℓ0 solution, reducing the number of non-default parameters in recommended configurations. Campfire simplifies sparse training for CNNs, achieving high accuracy with minimal pruning.
problem Training CNNs with sparse weights to maintain high accuracy and robustness.
method Gradual pruning technique with simplified sparsity structure, exploring various tradeoffs.
result Sparse ResNet-50 and ResNet-50 v1.5 achieve 75% top-1 accuracy with 70% sparsity, <1% accuracy loss.
Greedy algorithm performs well in online matching despite non-i.i.d. connections.
problem Online matching in sparse random graphs with fixed degree distributions.
method Approximating stochastic processes with partial differential equations.
result GREEDY algorithm can outperform RANKING in certain configurations.
High entropy alloys (HEAs) have been increasingly attractive as promising next-generation materials due to their various excellent properties. It's necessary to essentially characterize the degree of chemical ordering and identify order-disorder transitions through efficient simulation and modeling of thermodynamics. I…
Neuromorphic hardware tends to pose limits on the connectivity of deep networks that one can run on them. But also generic hardware and software implementations of deep learning run more efficiently for sparse networks. Several methods exist for pruning connections of a neural network after it was trained without conne…
In this article, we consider the sparse tensor singular value decomposition, which aims for dimension reduction on high-dimensional high-order data with certain sparsity structure. A method named Sparse Tensor Alternating Thresholding for Singular Value Decomposition (STAT-SVD) is proposed. The proposed procedure featu…
Sparse random networks reduce communication in federated learning.
problem Large communication cost in federated learning.
method Freeze random weights, train stochastic binary mask to sparsify.
result Improves accuracy, reduces communication, speeds convergence.
Sparse point observations can provide useful local constraints, but their benefit for radar-like fields depends on the training loss, uncertainty representation, and how observation support is encoded in the model.
problem Improving dense radar-field forecasts with sparse point observations
method A multimodal graph neural network nowcasting system over the Nordic radar domain
result Each source improves a different part of the forecast problem
Modern deep neural networks are typically highly overparameterized. Pruning techniques are able to remove a significant fraction of network parameters with little loss in accuracy. Recently, techniques based on dynamic reallocation of non-zero parameters have emerged, allowing direct training of sparse networks without…
Standard compressive sensing results state that to exactly recover an s sparse signal in R^p, one requires O(s. log(p)) measurements. While this bound is extremely useful in practice, often real world signals are not only sparse, but also exhibit structure in the sparsity pattern. We focus on group-structured patterns …
3BASiL-TM decomposes LLMs into sparse and low-rank matrices for efficient compression.
problem Efficiently compressing large language models without significant performance loss.
method 3-Block ADMM method and transformer-matching refinement step for sparse plus low-rank decomposition.
result 3BASiL-TM reduces perplexity gap by over 30% and speeds up compression by 2.5x.
NACs learn modular neural architectures without domain knowledge.
problem Jointly learn module configuration and execution without domain knowledge.
method Jointly trains two systems: module configuration and execution.
result Improves low-shot adaptation and OOD robustness.
New scheme for sparse feature selection in networked data.
problem Sparse feature selection in distributed, communication-restricted networks.
method Distributed sparse linear regression and feature selection method.
result True causal features can be reliably recovered with minimal bandwidth usage.
Generative model learns wireless channel distributions efficiently.
problem Learning precise wireless channel distributions for optimal communication.
method Physics-informed sparse Bayesian generative modeling (SBGM) with compressed data.
result Model learns channel parameters from compressed AP observations, is physically interpretable, and generalizes across different systems.
Entropy regularization improves interpretability of probabilistic clustering models.
problem Bayesian nonparametric mixture models often produce unbalanced cluster frequencies.
method Interpreting the posterior as penalized likelihood, entropy regularization reduces sparsely-populated clusters.
result The proposed entropy-regularized estimator enhances interpretability without sacrificing computational convenience.
A new method for training generative models with sparse supervision.
problem Training deep generative models with sparse and varying supervision.
method Caffeinated Wake-Sleep (CWS) method, combining reweighted wake-sleep and teacher-forcing.
result The CWS method is robust to variable length supervision and performs well on various datasets.
Proposes a robust 3D classification method for sparse point clouds.
problem Invariance to rotation, positional shift, scaling, and robustness to point sparsity in point cloud classification.
method Introduces a graph-based feature learning approach with an end-to-end neural network.
result Significantly improves 3D object classification and retrieval tasks with sparse point clouds.
This paper presents a spermwhale' localization architecture using jointly a bag-of-features (BoF) approach and machine learning framework. BoF methods are known, especially in computer vision, to produce from a collection of local features a global representation invariant to principal signal transformations. Our idea …
Computes fundamental groups of restricted configuration spaces.
problem Understanding the structure of configuration spaces after removing hypersurfaces.
method Fibration over unordered configuration spaces of n−1 points, computation of fundamental groups. result Fundamental groups of restricted configuration spaces computed in small dimensions.
We study the configuration space of equilateral and equiangular spatial hexagons for any bond angle by giving explicit expressions of all the possible shapes. We show that the chair configuration is isolated, whereas the boat configuration allows one-dimensional deformations which form a circle in the configuration spa…
DCRL learns causal relationships from mixed-type discrete data.
problem Challenges in learning causal relationships from discrete, mixed-type data.
method Generative framework modeling directed acyclic graph and sparse bipartite graph, flexible measurement models for different types of data.
result Consistent recovery of latent causal structure from observed data distribution.
Study shows configuration spaces' homological dimension increases monotonically.
problem Understanding the homological properties of configuration spaces of manifolds.
method Analyzing the homological monotonicity of unordered configuration spaces of manifolds.
result Homological dimension of configuration spaces increases monotonically in each degree.
A new Bayesian model improves forecasting for intermittent demand.
problem Sparse observations, cold-start items, and obsolescence in intermittent demand forecasting.
method Hierarchical Bayesian TSB model with partial pooling and calibrated probabilistic configuration.
result TSB-HB achieves the lowest RMSE and RMSSE on the UCI Online Retail dataset.
Researchers create a model for surface point configurations.
problem No existing models for surface configuration spaces.
method Constructed a discrete combinatorial model for an oriented surface.
result Homotopy equivalence between configuration space and subcomplex of cube complex.
Transformers exhibit sparse activation maps, reducing computational load and improving robustness.
problem Sparse activation in Transformer models.
method Extensive experiments on various Transformer architectures and tasks.
result Sparsity in Transformers is a prevalent phenomenon, reducing FLOP count and improving model robustness.
Tripod configurations of plane curves, formed by certain triples of normal lines coinciding at a point, were introduced by Tabachnikov, who showed that C2 closed convex curves possess at least two tripod configurations. Later, Kao and Wang established the existence of tripod configurations for C2 closed locally c…
Proposes a thermodynamic work minimization framework for guiding generative models.
problem Guiding generative models in sparse-data regimes with limited target samples or constraints.
method Regularization framework inspired by thermodynamic work, introducing Path Guidance and Observable Guidance.
result Improves sample efficiency and reduces bias in molecular simulations.
This paper extends homological stability results for configuration spaces of manifolds.
problem Homological stability of configuration spaces of manifolds.
method Analyzing the cohomology of configuration spaces of manifolds, focusing on stability in odd and even degrees.
result The stable range for homology groups of configuration spaces depends on the dimension of the manifold and the number of configuration points.
We study the Orchard relation for generic configurations of points in the plane (also called order types). We introduce infinitesimally-close points and analyse the relation of this notion with the Orchard relation. The second part of the paper deals with monochromatic configurations (for the Orchard relation). We give…
Quantum groups created from disk configuration space homologies.
problem Creating quantum groups from algebraic structures.
method Reconstructing quantum groups from homologies of configuration spaces of disks.
result New combinatorics and actual submanifolds of configuration spaces.
We study configurations of immersed curves in surfaces and surfaces in 3-manifolds. Among other results, we show that primitive curves have only finitely many configurations which minimize the number of double points. We give examples of minimal configurations not realized by geodesics in any hyperbolic metric.
A modern aircraft may require on the order of thousands of custom shims to fill gaps between structural components in the airframe that arise due to manufacturing tolerances adding up across large structures. These shims are necessary to eliminate gaps, maintain structural performance, and minimize pull-down forces req…
A machine learning configuration refers to a combination of preprocessor, learner, and hyperparameters. Given a set of configurations and a large dataset randomly split into training and testing set, we study how to efficiently select the best configuration with approximately the highest testing accuracy when trained f…
Study orders of canonical bundles over graph configuration spaces.
problem Determining bundle orders for planar and nonplanar graphs.
method Analyzing configuration spaces of graphs to find bundle orders.
result Bundle orders are 2 for planar and 4 for nonplanar graphs.
Low bit-width integer weights and activations are very important for efficient inference, especially with respect to lower power consumption. We propose Monte Carlo methods to quantize the weights and activations of pre-trained neural networks without any re-training. By performing importance sampling we obtain quantiz…
Solves Plateau-Douglas problem for singular configurations in general metric spaces.
problem Existence of minimal surfaces for singular configurations.
method Generalized approach via minimal sequences in metric spaces.
result Existence of minimal surfaces for singular configurations in general metric spaces.
Ensuring that all supposedly valid configurations of a software product line (SPL) lead to well-formed and acceptable products is challenging since it is most of the time impractical to enumerate and test all individual products of an SPL. Machine learning classifiers have been recently used to predict the acceptabilit…
Study on disk configurations in strips shows stability patterns.
problem Understanding stability patterns in disk configurations in strips.
method Finite presentation of rational homology groups, representation stability.
result Disk configuration space exhibits first-order representation stability.
Study maps surface configurations to Heisenberg homologies for mapping class groups.
problem Understanding Mapping Class Groups of punctured surfaces.
method Action of mapping classes on Heisenberg homologies of surface configurations.
result Representations of Mapping Class Groups derived from Heisenberg homologies.
Sharp upper bound for quasi polynomial degree of manifold configuration spaces.
problem Determining the exact degree of quasi-polynomial homology groups of configuration spaces.
method Analyzing extremal homology groups of unordered configuration spaces of manifolds.
result The upper bound for the degree of quasi-polynomials is sharp for every manifold.
We prove a conjecture of Crapo and Penne which characterizes isotopy classes of skew configurations with spindle-structure. We use this result in order to define an invariant, spindle-genus, for spindle-configurations. We also slightly simplify the exposition of some known invariants for configurations of skew lines an…
We consider the problem of configuring general-purpose solvers to run efficiently on problem instances drawn from an unknown distribution. The goal of the configurator is to find a configuration that runs fast on average on most instances, and do so with the least amount of total work. It can run a chosen solver on a r…
Optimizes glmnet configuration for better accuracy and efficiency.
problem Inappropriate glmnet configuration leads to inaccurate solutions and increased computation time.
method Data-driven framework using neural networks to predict accuracy and computation time from dataset characteristics and configuration.
result Automatic selection of optimal configuration maximizing accuracy under a time constraint.
The square-peg problem is solved using configuration spaces and multijet transversality.
problem Proving that every simple closed curve in the plane has an odd number of inscribed squares.
method Using the multijet transversality theorem and configuration spaces, we find a dense set of smooth embeddings for which the configuration space of points is transverse to any submanifold.
result A dense family of smoothly embedded circles in the plane and in Rn have an odd number of inscribed square-like quadrilaterals.