New activation functions achieve arbitrary-accuracy Sobolev approximation by fixed-size neural networks.
problem Approximation of Sobolev functions by neural networks
method Elementary Universal Activation Function and Differentiable Universal Activation Functions
result Arbitrary-accuracy Sobolev approximation by fixed-size neural networks
Belief propagation (BP) is an iterative method to perform approximate inference on arbitrary graphical models. Whether BP converges and if the solution is a unique fixed point depends on both the structure and the parametrization of the model. To understand this dependence it is interesting to find \emph{all} fixed poi…
Improved accuracy in community detection with vertex labels.
problem Efficient inference in stochastic block models with vertex labels.
method Linearized belief propagation algorithm with vertex labels.
result Belief propagation achieves highest accuracy when a function of network parameters has a unique fixed point.
Hybrid BFP-FP improves DNN training accuracy with 8.5x higher throughput.
problem Limited dynamic range of fixed-point arithmetic for DNN training convergence.
method Introducing HBFP, a hybrid BFP-FP approach.
result HBFP matches floating point's accuracy while delivering up to 8.5x higher throughput.
Polynomial-time algorithm for optimal stopping with fixed accuracy.
problem High-dimensional path-dependent optimal stopping problems.
method Efficient simulator of underlying information process, polynomial-time algorithm based on novel expansion.
result Polynomial-time solution for epsilon-optimal stopping policies and values.
Improved decision tree split selection to enhance accuracy in unbalanced datasets.
problem Bias in decision tree split selection, especially for unbalanced datasets.
method Proposed an updated gain ratio to correct bias and improve split selection.
result The updated gain ratio leads to better predictive accuracy in unbalanced datasets.
Deep QMC ansatzes improve variational QMC accuracy.
problem Improving variational QMC accuracy with neural network ansatzes.
method Analysis of deep neural network ansatzes PauliNet and FermiNet convergence to fixed-node limit.
result Deep QMC ansatzes can reach fixed-node limit with large network sizes.
A novel method quantizes Batch Normalization for QNNs, maintaining accuracy and efficiency.
problem Quantization challenges in Batch Normalization for QNNs.
method Converts BN to fixed-point operation with shared scale, suitable for hardware.
result Maintains same outputs through rigorous analysis and experiments.
LUNA improves linear attention for long sequences without sacrificing accuracy.
problem Quadratic computational cost of softmax attention in long-sequence domains.
method LUNA learns a learnable kernel feature map to reduce attention cost to linear while maintaining accuracy.
result LUNA achieves state-of-the-art performance on the LRA and excels at post-hoc conversion.
Fixing the last weight layer in neural networks can improve efficiency.
problem Large number of parameters in the last weight layer of neural networks.
method Training the last weight layer with a global scale constant and initializing it with a Hadamard matrix.
result The last weight layer can be fixed with little loss of accuracy and benefits memory and computational efficiency.
New method improves model accuracy in Byzantine-robust distributed learning by optimizing batch size.
problem Reduces model accuracy drop due to large variance of stochastic gradients in Byzantine-robust distributed learning.
method Proposes ByzSGDnm, a novel BRDL method that uses normalized momentum to mitigate accuracy drop in large batch sizes.
result The optimal batch size increases with the fraction of Byzantine workers, leading to better model accuracy under Byzantine attacks.
KANs replace fixed MLP weights with learnable edge functions, improving accuracy and interpretability.
problem Lack of interpretability and scalability in MLPs.
method KANs use learnable activation functions on edges instead of fixed weights, replacing weights with spline functions.
result KANs outperform MLPs in accuracy and interpretability with smaller models.
Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of low-precision fixed-point computations, we observe the rounding scheme to play a cruc…
Efficiently tunes hyperparameters with dynamic accuracy method.
problem Optimizing machine learning hyperparameters with inexact evaluations.
method Dynamic accuracy derivative-free optimization for hyperparameter tuning.
result Demonstrates robust and efficient hyperparameter tuning compared to fixed accuracy methods.
MARGINATTACK improves zero-confidence adversarial attacks' accuracy and efficiency.
problem Improving zero-confidence adversarial attacks' accuracy and efficiency.
method Proposes MARGINATTACK, a zero-confidence attack framework that computes margin with improved accuracy and efficiency.
result MARGINATTACK computes a smaller margin than state-of-the-art zero-confidence attacks and matches state-of-the-art fix-perturbation attacks.
Method trains sparse neural networks without sacrificing accuracy.
problem Training sparse neural networks limits model size.
method Updates sparse network topology during training.
result Requires fewer FLOPs to achieve accuracy.
A faster method for estimating effects in large data using fixed-point trees.
problem Estimating heterogeneous effects in large dimensions with computational efficiency.
method Fixed-point approximation to eliminate Jacobian estimation and speed up GRFs.
result Significant computational efficiency improvement without sacrificing statistical accuracy.
Paper defines when early exercise of American options is optimal under negative rates.
problem Determining optimal exercise times for American options with negative interest rates.
method Developed a new integral equation to price options and find exercise boundaries under negative rates, using modified fixed point method.
result Successfully developed and validated a new algorithm for pricing American options under negative rates.
Efficient CF approach using fast adaptive PCA for recommender systems.
problem Matrix completion problem in recommender systems.
method Fast adaptive randomized singular value decomposition (SVD) and termination mechanism for latent factors.
result The approach achieves near optimal prediction accuracy with high runtime efficiency.
New definition shows no trade-off between adversarial and standard accuracy.
problem Inexact definition of adversarial perturbation causes confusion.
method Proposed a slight modification to adversarial perturbation definition.
result Existence of classifiers that are robust and achieve high standard accuracy.
Develops DML for nonlinear panel data models with fixed effects.
problem Estimating causal effects in nonlinear panel data models with fixed effects.
method Double machine learning (DML) procedures for approximating nuisance functions.
result First-differencing yields the least constraints on fixed effects distribution.
New method reduces energy consumption of Hoeffding trees by up to 92%.
problem Inefficient energy consumption of Hoeffding trees due to fixed parameters.
method nmin adaptation for Hoeffding trees to adapt nmin parameter dynamically.
result VFDT-nmin consumes up to 92% less energy than CVFDT, trading off a few percent of accuracy.
LinearAPT optimizes decision-making under resource constraints for a linear threshold problem.
problem Optimizing sequential decisions with a linear threshold under resource limitations.
method LinearAPT, an adaptive algorithm for fixed-budget TLB problem.
result LinearAPT achieves theoretical upper bounds and robust performance on various datasets.
GKAN extends KAN to graphs, improving graph-based learning.
problem Graph-based learning tasks, especially semi-supervised.
method Learnable spline-based functions applied to graph data.
result GKAN achieves higher accuracy in graph semi-supervised learning.
Compression method reduces word embedding size for NLP models.
problem Memory constraints in deploying deep learning models for NLP tasks.
method Low rank matrix factorization during training to compress word embeddings.
result 90% compression with minimal accuracy loss for sentence classification tasks.
We present a numerical approach for solving the free boundary problem for the Black-Scholes equation for pricing American style of floating strike Asian options. A fixed domain transformation of the free boundary problem into a parabolic equation defined on a fixed spatial domain is performed. As a result a nonlinear t…
Paper improves DNN accelerator robustness against bit errors with energy savings.
problem Bit errors in quantized DNN weights reduce energy efficiency.
method Combines robust fixed-point quantization, weight clipping, and random bit error training.
result Significantly improves robustness against random bit errors with high energy savings.
4-bit quantization reduces U-Net memory by 8x with minimal accuracy loss.
problem Reducing memory and computation time in deep learning models.
method Fixed-point quantization of U-Net architecture.
result 8x reduction in memory usage with minimal accuracy loss.
A DRL-based strategy improves vehicle tracking accuracy while saving energy.
problem Enhancing vehicle tracking accuracy in WSNs without increasing energy consumption.
method Decentralized strategy with dynamic reinforcement learning to adjust sensing areas.
result Simulation results demonstrate superior performance of DRL-aided design.
Improved accuracy in a commercial assistant by smartly selecting new training data.
problem Expensive and time-consuming to annotate new data for ML systems.
method Automatically identifies new helpful examples suitable for human annotation.
result The proposed method leads to higher accuracy improvements with a fixed annotation budget.
Bounds on VC dimension for 1NN classifiers with fixed prototype sets.
problem No theoretical results for VC dimension of 1NN classifiers with fixed size prototype sets.
method Collected and used relevant theoretical results to provide explicit lower and upper bounds.
result Explicit lower and upper bounds for VC dimension of 1NN classifiers with fixed prototype set size.
A neural program synthesis method with iterative fix operations.
problem Creating correct programs from input-output examples.
method Combines encoder-decoder synthesis with a differentiable fixer.
result Improves synthesis accuracy by reducing discrepancies between outputs and desired outputs.
Deep-MAPS uses machine learning for mobile air pollution sensing in Beijing.
problem Ubiquitous sensing of urban air quality.
method Machine learning framework (Deep-MAPS) based on mobile and fixed sensors.
result Deep-MAPS achieves high spatial-temporal resolution (1km-by-1km and 1 hour) with over 85% accuracy.
TIP-Search optimizes market prediction accuracy and timeliness under uncertain load.
problem Real-time market prediction requires accurate predictions before a deadline.
method Filters feasible models, dispatches workers, trades accuracy for deadline risk.
result Optimized pool achieves 0.991 timely accuracy and 0.994 raw accuracy.
We provide a fast L2-embedding for arbitrary accuracy with applications to regression and L1 tasks.
problem Efficiently embedding high-dimensional data while maintaining accuracy.
method Oblivious L2-embedding with dimension independent of accuracy.
result Achieves arbitrary accuracy with constant embedding dimension.
Paper explores low-precision arithmetic for neural networks, improving efficiency.
problem Training neural networks with high precision and energy efficiency.
method 12-bit fixed-point, 12-bit floating-point, local scaling, Power-of-Two arithmetic.
result 7-bit Power-of-Two arithmetic achieves minimal loss in accuracy with reduced computation.
New method improves accuracy of quantized neural networks.
problem Accuracy drop in quantized neural networks, especially MobileNet family.
method Weight equalizing shift scaler, binary shifting to recover output range.
result Top-1 accuracy improved from 0.1% to 69.78% ~ 70.96% in MobileNets.
Enhances deep neural networks with fixed-mean Gaussian processes for uncertainty estimation.
problem Post-hoc uncertainty estimation of pre-trained deep neural networks.
method Fixed-mean Gaussian processes with variational inference for efficient stochastic optimization.
result FMGP improves uncertainty estimation and computational efficiency compared to state-of-the-art methods.
Improved Nystrom method reduces landmark points for better kernel matrix approximations.
problem Poor performance and lack of theoretical guarantees in standard Nystrom method.
method QR decomposition for efficient rank reduction in fixed-rank Nystrom approximations.
result Improved accuracy in many cases with nearly identical computational complexity.
New framework improves robustness of implicit neural networks.
problem Ill-posedness and convergence instability in implicit neural networks.
method NEMON framework based on contraction theory for ℓ∞ norm, including well-posedness condition, average iteration, and input-output Lipschitz constant regularization. result Improved accuracy and robustness of implicit models with smaller input-output Lipschitz bounds.
Mixed-size training improves CNN accuracy and speed.
problem Training CNNs on fixed image sizes limits their adaptability to various image sizes.
method Mixed-size training: training on multiple image sizes at once.
result Models trained with mixed-size images achieve higher accuracy and faster inference.
MIMONets speed up neural network inference by processing multiple inputs in parallel.
problem Reducing computational cost in neural network inference for large datasets.
method Proposes MIMONets, which augment neural network architectures with variable binding mechanisms to handle multiple inputs in superposition.
result Achieves significant speedups (2-4x) with minimal accuracy loss, demonstrating adaptability across different architectures.
Current OOD benchmarks overestimate model robustness to spurious correlations.
problem Spurious correlations degrade OOD performance, but benchmarks show the opposite.
method Analyze OOD datasets for spurious correlations and derive conditions for robustness.
result Current OOD benchmarks are misspecified and overestimate model robustness.
Crowdsourcing platforms provide marketplaces where task requesters can pay to get labels on their data. Such markets have emerged recently as popular venues for collecting annotations that are crucial in training machine learning models in various applications. However, as jobs are tedious and payments are low, errors …
Improves GBDT accuracy with differential privacy.
problem Balancing privacy and accuracy in GBDT models.
method Adaptive gradient control and novel boosting framework for privacy budget allocation.
result Achieves better model accuracy with differential privacy.
FixyNN splits CNN models into fixed and trainable parts for efficient on-device inference.
problem Energy inefficiency in on-device CNN inference for real-time computer vision.
method Co-designed hardware accelerator platform with transfer learning for training.
result Achieved nearly 2x better energy efficiency than a conventional accelerator.
New findings show score matching's accuracy doesn't ensure numerical stability in diffusion sampling.
problem Numerical stability issues in diffusion sampling despite small forward-marginal error.
method Constructing a smooth score field with arbitrarily small forward-marginal L2 error, showing nonexplosive behavior and moments of every order. result Euler--Maruyama discretizations can converge in probability even when moments diverge, demonstrating failure of weak convergence.
Mean Field Variational Bayes (MFVB) is a popular posterior approximation method due to its fast runtime on large-scale data sets. However, it is well known that a major failing of MFVB is its (sometimes severe) underestimates of the uncertainty of model variables and lack of information about model variable covariance.…