New LNS framework improves integer program solving.
problem Solving large-scale integer linear programs efficiently.
method Large neighborhood search with imitation and reinforcement learning.
result Framework significantly outperforms commercial solvers.
Learning how to automatically solve optimization problems has the potential to provide the next big leap in optimization technology. The performance of automatically learned heuristics on routing problems has been steadily improving in recent years, but approaches based purely on machine learning are still outperformed…
LES optimizes designs by sampling descent sequences, achieving strong sample efficiency.
problem Optimizing large, complex design spaces is infeasible and unnecessary.
method LES uses Bayesian optimization to target solutions reachable by iterative optimizers.
result LES achieves strong sample efficiency compared to existing methods.
The paper analyzes how much data points can be altered to change their rank in nearest neighbor searches.
problem Vulnerability of nearest neighbor search in high-dimensional data.
method Statistical analysis of perturbation needed to change neighbor rank.
result Derived statistical distribution of perturbation needed to modify neighbor rank.
In this paper, we propose the use of a black-box optimization method called deterministic Mesh Adaptive Direct Search (MADS) algorithm with orthogonal directions (Ortho-MADS) for the selection of hyperparameters of Support Vector Machines with a Gaussian kernel. Different from most of the methods in the literature that…
Chordal graphs can be used to encode dependency models that are representable by both directed acyclic and undirected graphs. This paper discusses a very simple and efficient algorithm to learn the chordal structure of a probabilistic model from data. The algorithm is a greedy hill-climbing search algorithm that uses t…
A metaheuristic approach solves portfolio optimization with constraints.
problem Portfolio Optimization Problem with cardinality and quantity constraints.
method Combination of TabuSearch and TokenRing Search with three neighborhood relations.
result The proposed techniques perform well on public benchmarks.
The k-NN graph has played a central role in increasingly popular data-driven techniques for various learning and vision tasks; yet, finding an efficient and effective way to construct k-NN graphs remains a challenge, especially for large-scale high-dimensional data. In this paper, we propose a new approach to const…
Tubes in manifolds require wide spaces.
problem Embedding constraints in Riemannian manifolds.
method Analyzing uniformly thick tubular neighborhoods.
result Conditions for manifold embeddings with wide tubes.
Local search improves GFlowNets' ability to generate high-reward samples.
problem GFlowNets struggle with over-exploration in high-reward space.
method Local search focusing on high-reward samples via backtracking and reconstruction.
result Significant performance improvement in biochemical tasks.
Markov networks are widely studied and used throughout multivariate statistics and computer science. In particular, the problem of learning the structure of Markov networks from data without invoking chordality assumptions in order to retain expressiveness of the model class has been given a considerable attention in t…
Glitches cause unreliable AI decisions with steep boundaries.
problem Glitches impair the reliability of AI models with steep decision boundaries.
method Formal definition of glitches, algorithmic search using MILP encoding.
result Glitches are widespread and indicate potential model inconsistencies.
An important linear algebra routine, GEneral Matrix Multiplication (GEMM), is a fundamental operator in deep learning. Compilers need to translate these routines into low-level code optimized for specific hardware. Compiler-level optimization of GEMM has significant performance impact on training and executing deep lea…
We propose a new metaheuristic training scheme that combines Stochastic Gradient Descent (SGD) and Discrete Optimization in an unconventional way. Our idea is to define a discrete neighborhood of the current SGD point containing a number of "potentially good moves" that exploit gradient information, and to search this …
New proof shows how to identify DAGs with weakly increasing errors.
problem Identifying the true DAG in models with weakly increasing error variances.
method Minimum-trace DAG method and hill climbing algorithm with R2R neighborhood.
result Hill climbing algorithm without strict local optima under weakly increasing error variances.
Neighborhood sampling affects graph neural network training outcomes.
problem Understanding the impact of neighborhood sampling on graph neural network training.
method Theoretical analysis using neural tangent kernels and Gaussian processes.
result Posterior covariance differs for different neighborhood sampling approaches, indicating no dominant approach.
This letter presents a new spectral-clustering-based approach to the subspace clustering problem. Underpinning the proposed method is a convex program for optimal direction search, which for each data point d finds an optimal direction in the span of the data that has minimum projection on the other data points and non…
New method speeds up k-means clustering for large k by improving nearest-neighbor search.
problem Efficiently clustering large datasets with high-dimensional points.
method Seeded Approximate Nearest-Neighbor Search methods to improve Lloyd's algorithm.
result Significantly faster k-means clustering for large k values.
A new framework scales active search for large datasets.
problem Scaling active search for large, high-dimensional data sets.
method Hierarchical Batch Bandit Search (HBBS) framework.
result HBBS improves performance and scalability for batch search.
Enhances clustering performance with a novel high-order Laplacian matrix.
problem Limited representation capability and insufficient information exploitation in multi-view spectral clustering.
method Proposes a multi-view spectral clustering algorithm that learns a high-order optimal neighborhood Laplacian matrix.
result Improves clustering performance through enhanced representation capacity of the learned optimal Laplacian matrix.
New adaptive stepsize method for stochastic approximation converges to target point.
problem Finding optimal step sizes for stochastic approximation algorithms.
method Adaptive block-coordinate stepsizes using online estimates of second moment.
result New method converges almost surely to a small neighborhood of the target point.
The study proves Strichartz and spectral projection theorems on specific types of curved surfaces.
problem Proving Strichartz and spectral projection theorems on curved surfaces.
method Using large negative curvature neighborhoods, the study proves theorems on asymptotically conic and Euclidean ends surfaces.
result The study proves theorems without loss of interval on specific types of curved surfaces.
A simple geometrical proof shows that any target function can be found in a random network's neighborhood.
problem Finding any target function in a random network's neighborhood.
method Geometrical proof using a simple model of a high-dimensional sphere projected onto a low-dimensional subspace.
result High-dimensional geometry ensures that a uniform distribution over a sphere reduces to a Gaussian distribution with negligible covariances, enabling the presence of any target function in a random network's neighborhood.
Biclustering techniques have been widely used to identify homogeneous subgroups within large data matrices, such as subsets of genes similarly expressed across subsets of patients. Mining a max-sum sub-matrix is a related but distinct problem for which one looks for a (non-necessarily contiguous) rectangular sub-matrix…
Given a compact Riemannian manifold M without boundary, we show that large isoperimetric regions in M×Rk are tubular neighborhoods of M×{x}, with x∈Rk.
Paper uses CMAB to improve NAS efficiency and accuracy.
problem Improving efficiency and accuracy of NAS for DNNs.
method Formulated NAS as CMAB, used Nested Monte-Carlo Search.
result Discovered cell structure achieves comparable accuracy to state-of-the-art, 20x faster.
In iterative supervised learning algorithms it is common to reach a point in the search where no further induction seems to be possible with the available data. If the search is continued beyond this point, the risk of overfitting increases significantly. Following the recent developments in inductive semantic stochast…
This work proposes searching for optimal operation distribution in neural architecture search.
problem Finding optimal neural architecture with specific operations and connections.
method Search for the optimal operation distribution, providing a stochastic and approximate solution.
result Operation distribution holds enough discriminating power to reliably identify a solution and is easier to optimise than traditional encodings.
Adapts large transformer model for search query intent understanding.
problem Understanding and predicting user intents from search queries.
method Adapts BERT-like architecture for search queries, accounts for query noisiness and sparseness.
result Builds a shareable deep learning model for query intent identification.
Efficiently selects nearest neighbors for labeling to speed up active learning.
problem Intractable active learning and search for large-scale unlabeled data.
method Restricts candidate pool to nearest neighbors of labeled set.
result Achieved similar performance to global approach but reduced computational cost by up to 3 orders of magnitude.
New method pools labels from similar data items to improve learning from small samples.
problem Learning from small, human-annotated samples with potential disagreement among annotators.
method Proposes neighborhood-based pooling for sharing labels across similar data items.
result Improves learning from small, noisy samples by pooling labels from similar items.
Learning Bayesian networks is often cast as an optimization problem, where the computational task is to find a structure that maximizes a statistically motivated score. By and large, existing learning tools address this optimization problem using standard heuristic search techniques. Since the search space is extremely…
We investigate active learning with access to two distinct oracles: Label (which is standard) and Search (which is not). The Search oracle models the situation where a human searches a database to seed or counterexample an existing solution. Search is stronger than Label while being natural to implement in many situati…
MOBO-OSD optimizes multi-objective functions using orthogonal search directions.
problem Challenging multi-objective optimization problem.
method Solves multiple constrained optimization problems along orthogonal search directions.
result Consistently outperforms state-of-the-art algorithms.
GLSearch uses GNN to learn efficient search strategies for finding large common subgraphs.
problem Finding the Maximum Common Subgraph (MCS) between two graphs is NP-hard and hard to solve efficiently.
method GLSearch combines GNN and DQN to learn optimal node pairs for expansion in a branch and bound algorithm.
result GLSearch finds significantly larger common subgraphs than heuristic search methods given the same computation budget.
DrNAS improves neural architecture search with Dirichlet distribution and progressive learning.
problem Efficiently search for neural architectures with improved generalization and exploration.
method Formulates architecture search as a distribution learning problem using Dirichlet distribution and gradient-based optimization. Introduces a progressive learning scheme to handle large-scale tasks.
result Achieves state-of-the-art results on CIFAR-10 and ImageNet, demonstrating improved generalization and exploration.
MICO uses mutual information co-training to improve selective search efficiency.
problem Efficiently search and route unseen queries in large-scale search systems.
method Mutual Information Co-training framework for selective search with minimal supervision.
result Significantly improves performance on multiple metrics of selective search.
Resolves conjecture on cylindrical mean curvature flows in all dimensions.
problem Mean Convex Neighborhood Conjecture for cylindrical singularities.
method Complete classification of ancient, asymptotically cylindrical flows; refined asymptotic analysis; leading mode condition; induction over thresholds.
result Establishes mean-convex neighborhood for cylindrical singularities; provides local models and canonical families.
Machine learning boosts physics research, especially at high energy experiments.
problem Finding new fundamental physics in high energy experiments.
method Review of machine learning methods and applications in high energy physics.
result Modern machine learning techniques have expanded the scope of physics research.
A real-time federated neural architecture search approach reduces costs and improves performance.
problem High communication and computational demands in federated learning for large models.
method Evolutionary approach with double-sampling technique to optimize model performance and reduce costs.
result Effective real-time federated neural architecture search for deep models on edge devices.
Improved causal discovery methods for large graphs without strict assumptions.
problem Sub-optimal solutions due to faithfulness assumption violations.
method Super-structure estimation and local search strategies.
result The proposed method scales to hundreds of nodes with high accuracy.
Efficient search methods can outperform random search on challenging tasks.
problem Comparing the performance of efficient and random search methods in neural architecture search.
method Comparison of weight sharing and random search methods on progressively larger search spaces for image classification and detection.
result Efficient search methods can provide substantial gains over random search on large, realistic tasks.
A new L-BFGS method tackles large-scale optimization with fewer evaluations.
problem Efficiently solving large-scale unconstrained optimization problems.
method Proposes a regularized L-BFGS method with line search techniques.
result Shows global convergence and robust performance in numerical tests.
A new framework generates large hierarchical search spaces for neural architectures.
problem Discovering neural architectures from simple blocks is hard.
method Context-free grammars for a unified, scalable search space.
result Efficiently searches over complete architectures, outperforming existing methods.
Many engineering problems require identifying feasible domains under implicit constraints. One example is finding acceptable car body styling designs based on constraints like aesthetics and functionality. Current active-learning based methods learn feasible domains for bounded input spaces. However, we usually lack pr…
In the 'Big Data' era, many real-world applications like search involve the ranking problem for a large number of items. It is important to obtain effective ranking results and at the same time obtain the results efficiently in a timely manner for providing good user experience and saving computational costs. Valuable …
In this paper, we compare the performances of FAISS and FENSHSES on nearest neighbor search in Hamming space--a fundamental task with ubiquitous applications in nowadays eCommerce. Comprehensive evaluations are made in terms of indexing speed, search latency and RAM consumption. This comparison is conducted towards a b…
Scalable NAS by factorizing operators into subspaces.
problem Scaling up NAS search space while avoiding operator competition.
method Factorizing a large set of candidate operators into smaller subspaces.
result Achieved state-of-the-art performance on CIFAR10 and ImageNet.