Develops algorithms to optimize machine replacement schedules using operational data.
problem Optimizing machine replacement intervals when the lifetime distribution is unknown.
method Formulates as a stochastic multi-armed bandit problem and proposes Hoeffding- and Bernstein-based algorithms.
result Achieves optimal or near-optimal replacement intervals with minimal regret.
CGRL improves graph neural networks' OOD generalization by blocking spurious correlations.
problem Graph Neural Networks struggle with out-of-distribution data due to learning spurious correlations.
method Formulates a causal graph, uses backdoor adjustment, and introduces a loss replacement strategy.
result Significantly improves OOD generalization of GNNs, stabilizing mutual information learning.
Softmax is found ineffective for NL block, leading to improved performance.
problem Inefficiency of softmax in NL block for global context modeling.
method Empirical analysis and replacement of softmax with scaling factor.
result Improved performance on various datasets with reduced computational cost.
Big data sets must be carefully partitioned into statistically similar data subsets that can be used as representative samples for big data analysis tasks. In this paper, we propose the random sample partition (RSP) data model to represent a big data set as a set of non-overlapping data subsets, called RSP data blocks,…
The desire to map neural networks to varying-capacity devices has led to the development of a wealth of compression techniques, many of which involve replacing standard convolutional blocks in a large network with cheap alternative blocks. However, not all blocks are created equally; for a required compute budget there…
This paper introduces GLT for better input data representation in BNN and proposes a compact topology with block pruning.
problem Improving input data representation for Binary Neural Networks (BNN).
method Generic Learned Thermometer (GLT) for encoding, block pruning and Knowledge Distillation for compact topology.
result Significant accuracy gains and lightweight fully-binarized models with limited accuracy degradation.
Recently, deep neural networks (DNNs) have been regarded as the state-of-the-art classification methods in a wide range of applications, especially in image classification. Despite the success, the huge number of parameters blocks its deployment to situations with light computing resources. Researchers resort to the re…
Training-free looped transformers improve model performance without additional training.
problem Improving model performance without additional training or fine-tuning.
method A lightweight inference-time wrapper loops a contiguous mid-stack block of layers of a frozen checkpoint without additional fine-tuning.
result Our method improves model performance across various model families.
K-bMOM robustly clusters data with outliers, improving on Lloyd-type methods.
problem Outliers in datasets disrupt traditional clustering algorithms.
method Lloyd-type iterations with robust median-of-means estimates.
result K-bMOM outperforms existing robust K-means methods.
The study extends stochastic block models to geometric settings, focusing on community detection and information flow.
problem Generalizing community detection and information flow models to geometric settings.
method Considered a geometric random graph over a homogeneous metric space, defined a geometric counterpart of flow of information on trees.
result Sufficient conditions for recovering locations and for percolation of information in geometric settings.
Stochastic LWTA networks resist adversarial attacks while maintaining accuracy.
problem Adversarial robustness of neural networks.
method Replaced ReLU with stochastic LWTA activations, trained with Variational Bayesian and PGD.
result Stochastic LWTA networks achieve state-of-the-art robustness against adversarial attacks.
Long Short Term Memory Fully Convolutional Neural Networks (LSTM-FCN) and Attention LSTM-FCN (ALSTM-FCN) have shown to achieve state-of-the-art performance on the task of classifying time series signals on the old University of California-Riverside (UCR) time series repository. However, there has been no study on why L…
Self-attention can replace convolutional layers in vision tasks.
problem The supremacy of convolutional layers in vision tasks.
method Analysis and experiments with self-attention layers compared to convolutional layers.
result Self-attention layers can perform as well as convolutional layers and learn to do so.
The study examines MCMC methods for arbitrary objectives and finds likelihood sharpness impacts performance and regularization.
problem Limitations of MCMC methods for arbitrary objective functions.
method Two-block MCMC framework with Metropolis-Hastings and Gibbs sampling, exploring likelihood curvature and sharpness.
result Likelihood sharpness governs in-sample performance and regularization inferred by training data.
Post-hoc explanations improve CNNs by replacing final linear layer with k-means classifier.
problem CNNs lack accurate data representation in their built-in prototypes.
method Introduces k-means-based post-hoc explanations for CNNs, leveraging spatial consistency of convolutional receptive fields.
result Using shallower, less compressed feature activations improves semantic fidelity at the cost of slight predictive performance.
Tiled Squeeze-and-Excite improves channel attention with local spatial context.
problem Improving channel attention mechanisms in neural networks.
method Proposes tiled squeeze-and-excite (TSE) framework for channel attention.
result Local context of 7 rows or columns is sufficient for matching global context performance.
In this paper we present a kinetic model with stochastic game-type interactions, analyzing the relationship between the level of political competition in a society and the degree of economic liberalization. The above issue regards the complex interactions between economy and institutional policies intended to introduce…
This paper extends Median-of-Means to new learning problems involving pairwise comparisons.
problem Learning from pairwise comparisons in machine learning.
method Segmenting data into blocks, comparing pairs of decision rules, and declaring the winner based on majority performance.
result The Median-of-Means approach maintains robustness and performance under various sampling schemes.
This paper optimizes subsampling for large datasets using Poisson distribution.
problem Efficiently subsample large datasets for quasi-likelihood estimation.
method Derives optimal Poisson subsampling probabilities and develops a distributed subsampling framework.
result Consistent and asymptotically normal estimators are obtained.
New method improves scalability of SGD for large datasets.
problem High variance in stochastic gradient descent.
method Adaptive measure reduction with Carathéodory's theorem.
result Improved scalability to high-dimensional spaces.
The wide adoption of DNNs has given birth to unrelenting computing requirements, forcing datacenter operators to adopt domain-specific accelerators to train them. These accelerators typically employ densely packed full precision floating-point arithmetic to maximize performance per area. Ongoing research efforts seek t…
New ARIMA framework improves forecast accuracy for economic and financial time series.
problem Improving forecast accuracy for nonlinear dynamics in time series data.
method Projection-based ARIMA framework using Galerkin basis expansions.
result Galerkin-SARIMA matches or improves forecast accuracy compared to classical ARIMA/SARIMA.
DAM with MRL improves relational reasoning in MANNs.
problem Limited performance of associative memory networks on complex relational reasoning tasks.
method Distributed Associative Memory architecture with Memory Refreshing Loss.
result Enhanced relation reasoning performance of MANNs on long temporal sequence data.
The one-term distributive homology was introduced by J.H.Przytycki as an atomic replacement of rack and quandle homology, which was first introduced and developed by R.Fenn, C.Rourke and B.Sanderson, and J.S.Carter, S.Kamada and M.Saito. This homology was initially suspected to be torsion-free, but we show in this pape…
The traditional approach of hand-crafting priors (such as sparsity) for solving inverse problems is slowly being replaced by the use of richer learned priors (such as those modeled by generative adversarial networks, or GANs). In this work, we study the algorithmic aspects of such a learning-based approach from a theor…
Improved tensor GLM estimation for complex data.
problem Complex tensor data in GLMs leads to high-dimensional, ill-posed estimation.
method Proposed LSRTR-M algorithm using Muon updates for faster convergence and lower errors.
result LSRTR-M converges faster and achieves lower errors than LSRTR.
Paper proves SHB convergence with biased gradients and approximate step sizes.
problem Establishing convergence of SHB with biased gradients and approximate step sizes.
method Generalizes SHB convergence conditions for biased gradients, approximate step sizes, and block updating.
result Proves convergence of SHB with new conditions for biased gradients and approximate step sizes.
Continuous-depth Evoformer reduces protein folding prediction time and resource usage.
problem Efficient protein structure prediction with reduced computational costs.
method Continuous-depth formulation of Evoformer using Neural Ordinary Differential Equations (Neural ODEs).
result The continuous-time Evoformer achieves constant memory cost and improved efficiency.
The paper classifies certain singular projective varieties with specific properties.
problem Classifying projective klt pairs with nef anti-log canonical divisors.
method Establishes a structure theorem using locally trivial rationally connected fibrations.
result Projective klt pairs can be decomposed into rationally connected and Calabi-Yau varieties.
We propose ROI regularization (ROIreg) as a semi-supervised learning method for image classification. ROIreg focuses on the maximum probability of a posterior probability distribution g(x) obtained when inputting an unlabeled data sample x into a convolutional neural network (CNN). ROIreg divides the pixel set of x int…
SGPA calibrates transformer uncertainty for safety-critical tasks.
problem Uncertainty estimation in transformer models for safety-critical domains.
method Bayesian inference in transformer's output space using sparse Gaussian processes.
result SGPA-based Transformers improve in-distribution calibration and out-of-distribution robustness.
This paper presents a novel Block Iterative Bayesian Algorithm (Block-IBA) for reconstructing block-sparse signals with unknown block structures. Unlike the existing algorithms for block sparse signal recovery which assume the cluster structure of the nonzero elements of the unknown signal to be independent and identic…
This letter presents a novel Block Bayesian Hypothesis Testing Algorithm (Block-BHTA) for reconstructing block sparse signals with unknown block structures. The Block-BHTA comprises the detection and recovery of the supports, and the estimation of the amplitudes of the block sparse signal. The support detection and rec…
Proposes a Nested Block Model to unify various network block models.
problem Lack of nested structure and differing parameter complexity among block models.
method Formulates a hierarchy of block models (NBM) that includes SBM, DCBM, and PABM as special cases.
result Allows clustering and estimation without preliminary testing, simplifying model selection.
We examine the recovery of block sparse signals and extend the framework in two important directions; one by exploiting signals' intra-block correlation and the other by generalizing signals' block structure. We propose two families of algorithms based on the framework of block sparse Bayesian learning (BSBL). One fami…
K-means clustering improved for robustness to outliers and distribution shifts.
problem K-means is brittle to outliers, distribution shifts, and limited samples.
method Developed a distributionally robust variant using Wasserstein-2 ball around the empirical distribution.
result Substantial gains in outlier detection and robustness to noise demonstrated.
We theoretically investigate the convergence rate and support consistency (i.e., correctly identifying the subset of non-zero coefficients in the large sample limit) of multiple kernel learning (MKL). We focus on MKL with block-l1 regularization (inducing sparse kernel combination), block-l2 regularization (inducing un…
This work tackles representation learning by introducing stochastic competition-based activations.
problem Learning diversified representations in deep learning models.
method Combining information-theoretic arguments with stochastic competition-based activations, using Stochastic Local Winner-Takes-All (LWTA) units.
result The proposed method yields significant discriminative representation learning abilities and allows for a principled investigation of intermediate network representations.
SympFormer accelerates attention blocks using inertial dynamics on density spaces.
problem Improving the efficiency of self-attention blocks in Transformers.
method Introduced accelerated attention blocks derived from inertial Nesterov dynamics on density spaces.
result Accelerated attention blocks converge faster than classical blocks while preserving oracle calls.
New kernel models multi-output Gaussian processes accurately.
problem Challenges in modelling cross-covariances for multiple-output Gaussian processes.
method Replaced Gaussian components with block components of finite bandwidth in spectral mixture kernel.
result First multi-output generalization of spectral mixture kernel that can approximate any stationary multi-output kernel to arbitrary precision.
We introduce block-tree graphs as a framework for deriving efficient algorithms on graphical models. We define block-tree graphs as a tree-structured graph where each node is a cluster of nodes such that the clusters in the graph are disjoint. This differs from junction-trees, where two clusters connected by an edge al…
We simplify matrix computations for block matrices, especially useful for covariance and correlation matrices.
problem Complex computations for block matrices, especially for covariance and correlation matrices.
method Obtained a canonical representation for block matrices, facilitating computation of various matrix operations.
result Simplified computation of matrix operations for block matrices, particularly useful for covariance and correlation matrices.
Attention mechanism is a hot spot in deep learning field. Using channel attention model is an effective method for improving the performance of the convolutional neural network. Squeeze-and-Excitation block takes advantage of the channel dependence, selectively emphasizing the important channels and compressing the rel…
Proposes BMME for optimizing nonsmooth nonconvex problems with block structure.
problem Optimizing nonsmooth nonconvex problems with block structure.
method Block Alternating Bregman Majorization Minimization with Extrapolation (BMME).
result Subsequential convergence to a first-order stationary point under mild assumptions, global convergence under stronger conditions.
Alternating Direction Method of Multipliers (ADMM) has become a widely used optimization method for convex problems, particularly in the context of data mining in which large optimization problems are often encountered. ADMM has several desirable properties, including the ability to decompose large problems into smalle…
Paper proposes ABDR for convex subspace clustering with adaptive block diagonal representation.
problem Subspace clustering with block diagonal structure for noisy data.
method ABDR explicitly pursues block diagonality without sacrificing convexity, using a specially designed convex regularizer.
result Experimental results show ABDR outperforms state-of-the-arts.
New activation networks improve model efficiency and performance.
problem Creating hardware-efficient deep learning models.
method Restructurable Activation Networks (RANs) with RAN-explicit and RAN-implicit methods.
result RANs achieve state-of-the-art results with improved hardware efficiency.
Higher-order motif structures and multi-vertex interactions are becoming increasingly important in studies that aim to improve our understanding of functionalities and evolution patterns of networks. To elucidate the role of higher-order structures in community detection problems over complex networks, we introduce the…