Optimizes wide low-rank neural networks for reduced parameters and cost.
problem Reducing the number of learnable parameters in wide neural networks.
method Analyzed edge-of-chaos dynamics and derived formulae for optimal weight and bias variances.
result Optimal weight and bias variances for low-rank networks follow from multiplicative scaling.
FLAML automates model selection and hyperparameter tuning with low resource cost.
problem Automating model selection and hyperparameter tuning for ad-hoc datasets and metrics.
method Conducts trials of different configurations on training data, optimizing for low computational cost.
result Significantly outperforms top-ranked AutoML libraries under smaller budget constraints.
MUMBO optimizes multiple tasks efficiently, even with low-cost related functions.
problem Efficiently optimizing multiple related functions with low-cost evaluations.
method Derives a novel multi-task version of entropy search.
result Robust performance with low computational overhead across various optimization challenges.
The multimodal web elements such as text and images are associated with inherent memory costs to store and transfer over the Internet. With the limited network connectivity in developing countries, webpage rendering gets delayed in the presence of high-memory demanding elements such as images (relative to text). To ove…
Label tree-based algorithms are widely used to tackle multi-class and multi-label problems with a large number of labels. We focus on a particular subclass of these algorithms that use probabilistic classifiers in the tree nodes. Examples of such algorithms are hierarchical softmax (HSM), designed for multi-class class…
In this work, a new approach for Sun tracking systems is presented. Due to the current system limitations regarding costs and operational problems, a new approach based on low cost, computer vision open hardware and deep learning has been developed. The preliminary tests carried out successfully in Plataforma solar de …
Sparse nearest-centroid classifiers detect relevant features for classification.
problem Classifying data with low computational cost and feature selection.
method Proposes ℓ1 and ℓ2 sparse variants of nearest-centroid classifiers. result Training sparse classifiers can be done exactly and at quasi-linear cost.
Researchers develop neural optimal transport with Lagrangian costs for efficient computation.
problem Optimal transport between measures with Lagrangian costs for systems with geometric constraints.
method Neural network approach to compute geodesics and optimal transport maps efficiently.
result Efficient computation of geodesics and optimal transport maps without ODE solvers.
Low rank tensor learning, such as tensor completion and multilinear multitask learning, has received much attention in recent years. In this paper, we propose higher order matching pursuit for low rank tensor learning problems with a convex or a nonconvex cost function, which is a generalization of the matching pursuit…
Solving symmetric positive definite linear problems is a fundamental computational task in machine learning. The exact solution, famously, is cubicly expensive in the size of the matrix. To alleviate this problem, several linear-time approximations, such as spectral and inducing-point methods, have been suggested and a…
New method reduces computational cost for nonnegative low rank matrix approximation.
problem Efficiently compute nonnegative low rank matrix approximation for nonnegative matrices.
method Alternating projections onto tangent spaces of fixed rank matrices manifold and nonnegative matrix manifold.
result Sequence converges linearly to optimal solutions, showing better performance in terms of computational time and accuracy.
New GPnn method achieves scalable regression with low computational cost.
problem Inefficient Gaussian Process (GP) regression for large datasets.
method GP nearest-neighbour (GPnn) prediction with robustness and limiting behaviour exploration.
result GPnn achieves high MSE accuracy with minimal parameter estimation effort, even in gross misspecification.
New framework reduces cost of financial option pricing simulations on FPGAs.
problem Efficiently simulate financial option pricing with reduced computational cost.
method Nested MLMC framework with low precision calculations on FPGAs.
result Higher computational savings compared to existing mixed-precision MLMC frameworks.
A new method reduces inference cost for FwFM by allowing it to scale with item fields only.
problem High computational cost in FwFM for large field counts.
method Low-rank diagonal plus symmetric decomposition for field-wise interactions.
result Aggressive rank reduction outperforms pruning in accuracy and speed.
Low-rank modeling plays a pivotal role in signal processing and machine learning, with applications ranging from collaborative filtering, video surveillance, medical imaging, to dimensionality reduction and adaptive filtering. Many modern high-dimensional data and interactions thereof can be modeled as lying approximat…
Low-cost sensors improve air quality prediction accuracy significantly.
problem Improving air quality monitoring networks with affordable sensors.
method Developed a high-resolution air quality prediction engine using low-cost sensors and official data.
result The use of low-cost sensors improves prediction accuracy by 25% and 15% for PM2.5 and PM10 respectively in densely monitored areas.
New method detects if data points were used in training models with low cost and high power.
problem Detecting if a particular data point was used in training a model.
method Fine-grained modeling of null hypothesis in likelihood ratio tests, leveraging reference models and population data.
result RMIA has superior test power compared to prior methods, even at extremely low false positive rates.
StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.
problem Optimal quantization parameters selection for deep neural networks with diverse data distributions.
method Statistical error analysis framework for uniform and floating-point quantization, iterative and analytic quantizers designed for arbitrary and Gaussian-like distributions.
result Improved accuracy and stability in training low-precision neural networks.
Recurrent neural networks are a powerful tool, but they are very sensitive to their hyper-parameter configuration. Moreover, training properly a recurrent neural network is a tough task, therefore selecting an appropriate configuration is critical. Varied strategies have been proposed to tackle this issue. However, mos…
We study the approximate nearest neighbour method for cost-sensitive classification on low-dimensional manifolds embedded within a high-dimensional feature space. We determine the minimax learning rates for distributions on a smooth manifold, in a cost-sensitive setting. This generalises a classic result of Audibert an…
oPoW proposes a new PoW algorithm to reduce mining costs and environmental impact.
problem Scalability issues, environmental concerns, and systemic risks in Bitcoin PoW.
method oPoW is a novel PoW algorithm that shifts mining costs from electricity to hardware (CAPEX).
result oPoW reduces mining costs and improves network scalability, decentralization, and issuance.
Improves ML efficiency for vast, rapidly growing data.
problem Low latency and cost in ML with distributed, growing data.
method Designs ML systems exploiting ML characteristics, data structures, and data distribution.
result Improves ML latency and cost by 1-2 orders of magnitude.
Implementing large-scale deep neural networks with high computational complexity on low-cost IoT devices may inevitably be constrained by limited computation resource, making the devices hard to respond in real-time. This disjunction makes the state-of-art deep learning algorithms, i.e. CNN (Convolutional Neural Networ…
ROAD-EnKFs use learned low-dimensional models to improve state reconstruction and forecasting.
problem Reconstructing and forecasting states of unknown or expensive systems.
method Learned low-dimensional surrogate models and ensemble Kalman filter integration.
result ROAD-EnKFs achieve higher accuracy at lower computational cost than existing methods.
A new parallel algorithm for learning optimal policies in MDPs with low communication costs.
problem Learning optimal policies for infinite-horizon MDPs.
method Primal-Dual Stochastic Mirror Descent for convex programming problems with inexact constraints.
result First parallel algorithm for average-reward MDPs with generative model and low communication costs.
A deep learning algorithm designs low-cost SPP films.
problem Designing SPP films with high accuracy and low cost.
method Guided deep learning algorithm with low-cost sample replacement.
result Deep CNN learns to replace precious metals with ordinary metals, reducing cost.
This work introduces a novel nonparametric density index defined on graphs, the Sum-over-Forests (SoF) density index. It is based on a clear and intuitive idea: high-density regions in a graph are characterized by the fact that they contain a large amount of low-cost trees with high outdegrees while low-density regions…
E2CM uses class means for efficient early exits in neural networks.
problem Efficient early exits in neural networks with low computational cost.
method Early Exit Class Means (E2CM) based on class means of samples, without gradient-based training. result E2CM achieves higher accuracy with fixed training time budget and boosts existing early exit schemes. Kernel matrices (e.g. Gram or similarity matrices) are essential for many state-of-the-art approaches to classification, clustering, and dimensionality reduction. For large datasets, the cost of forming and factoring such kernel matrices becomes intractable. To address this challenge, we introduce a new adaptive sampli…
New method quantizes neural networks for mobile devices.
problem High computational and memory costs of deep neural networks.
method Formulates quantization as a differentiable function.
result Quantization networks outperform state-of-the-art methods.
This work reduces computation cost for on-device CNN training.
problem High computation cost during on-device CNN training.
method Self-supervised instance filtering and error map pruning.
result Substantial computation saving without significant accuracy loss.
This letter proposes a low-computational Bayesian algorithm for noisy sparse recovery in the context of one bit compressed sensing with sensing matrix perturbation. The proposed algorithm which is called BHT-MLE comprises a sparse support detector and an amplitude estimator. The support detector utilizes Bayesian hypot…
Faster GW alignment for incomparable point clouds via low-rank couplings.
problem Aligning points across incomparable point clouds.
method Low-rank couplings and costs to solve Gromov-Wasserstein framework in linear time.
result Linear-time computation of Gromov-Wasserstein distances.
The successive projection algorithm (SPA) can quickly solve a nonnegative matrix factorization problem under a separability assumption. Even if noise is added to the problem, SPA is robust as long as the perturbations caused by the noise are small. In particular, robustness against noise should be high when handling th…
Unified framework reduces NFEs for inverse problems.
problem High computational costs and degraded reconstruction quality in existing LDM-based inverse solvers.
method Consistency Regularised Gradient Flows for posterior sampling and prompt optimization.
result Significantly reduced computational cost with state-of-the-art performance.
This paper develops a method to train compact neural networks with reduced memory and computational costs.
problem Training large neural networks consumes excessive resources and energy.
method End-to-end training framework using Bayesian tensor decomposition with automatic rank determination.
result The method achieves significant parameter reduction and maintains or improves accuracy.
New algorithm improves deep learning models' robustness without sacrificing accuracy.
problem Low-rank methods compromise model robustness against adversarial perturbations.
method Robust low-rank training via approximate orthonormal constraints.
result Ensures well-conditioning and better adversarial robustness without sacrificing model accuracy.
Study assesses CNN model robustness to noise in low-cost CT scans.
problem Evaluate CNN model performance on noisy, artifact-prone low-cost CT images.
method Developed and tested a CNN model for head CT triage, varying tube current and projections.
result Model remains robust to reduced tube current and fewer projections, maintaining AUROC close to original.
Using Intel's Loihi neuromorphic research chip and ABR's Nengo Deep Learning toolkit, we analyze the inference speed, dynamic power consumption, and energy cost per inference of a two-layer neural network keyword spotter trained to recognize a single phrase. We perform comparative analyses of this keyword spotter runni…
Researchers develop methods to reduce simulation costs for cardiovascular modeling.
problem High computational cost of high-fidelity simulations in cardiovascular modeling.
method Use low-fidelity approximations, neural networks, and normalizing flows to construct surrogates.
result Validated methods reduce computational cost while maintaining accuracy.
CNNs show sensitivity to low-frequency signals due to image frequency distribution.
problem Understanding why CNNs are sensitive to low-frequency signals.
method Theoretical analysis of CNN representations in frequency space.
result CNNs sensitivity to low-frequency signals is due to the frequency distribution of natural images.
New method improves model explainability and accuracy with low computational cost.
problem Improving model explainability and accuracy in classification models.
method Distributionally robust optimization to learn sparse ensembles of rule sets.
result Improves model performance on various metrics compared to competing methods.
Paper presents a low-cost algorithm for bipartite ranking with improved sample size requirements.
problem Bipartite ranking's quadratic dependence on sample size makes it computationally expensive.
method Uses a novel uniform risk bound based on matrix and vector concentration inequalities to achieve low cost and competitive performance.
result Shows that the sample size required for competitive performance is not quadratic, improving efficiency.
Optimizes optimal transport distances using low-dimensional embeddings.
problem High computational cost of optimal transport distances in high dimensions.
method Approximate OT distances using 1-Lipschitz maps in a lower-dimensional space.
result Efficiently approximates optimal transport distances with lower computational cost.
Improved machine learning with reduced tensor rank constraints and dropout.
problem Efficiently approximating large tensors in machine learning.
method Tree tensor networks with CP rank constraints and tensor dropout.
result Low-rank TTN classifier achieves 90.3% accuracy in Fashion-MNIST.
Low-precision training reduces computational cost and produces efficient models. Recent research in developing new low-precision training algorithms often relies on simulation to empirically evaluate the statistical effects of quantization while avoiding the substantial overhead of building specific hardware. To suppor…
A new BO framework reduces costs by using low-fidelity data.
problem Optimizing expensive experiments with low-fidelity data.
method Developed a multi-fidelity cost-aware Bayesian optimization framework.
result Significantly outperforms state-of-the-art BO methods.
We develop and analyze a procedure for gradient-based optimization that we refer to as stochastically controlled stochastic gradient (SCSG). As a member of the SVRG family of algorithms, SCSG makes use of gradient estimates at two scales, with the number of updates at the faster scale being governed by a geometric rand…