Quaternion self-attention reduces computational cost and improves performance.
problem Existing quaternion self-attention increases computational cost and diverges attention distributions.
method Proposes a shared-score quaternion self-attention mechanism.
result Reduces score-computation multiplications by 75% and softmax operations from four to one.
The paper develops methods to accurately locate change points in high-dimensional mean shift models.
problem Locating change points in high-dimensional mean shift models.
method Locally refitted least squares estimator, component-wise and simultaneous rates of estimation.
result Asymptotic validity of component-wise and simultaneous confidence intervals for change point parameters.
Boosting method for causal SEMs from observational data.
problem Learning causal order among variables from observational data.
method Boosting-based approach with early stopping and component-wise gradient descent.
result Boosting with early stopping consistently favors the true causal ordering.
Proposes a boosting framework for sparsity in grouped covariates.
problem Sparsity and selection bias in grouped covariates.
method Component-wise and group-wise gradient boosting with adjusted degrees of freedom.
result Reduces bias and improves predictability in variable selection.
A new method improves posterior approximation for complex distributions.
problem Difficulty in capturing multimodal and heavy-tailed posteriors with standard normalizing flows.
method StiCTAF: stick-breaking mixture base with component-wise tail adaptation.
result Improved tail recovery and better mode coverage compared to benchmarks.
Multi-output prediction deals with the prediction of several targets of possibly diverse types. One way to address this problem is the so called problem transformation method. This method is often used in multi-label learning, but can also be used for multi-output prediction due to its generality and simplicity. In thi…
We investigate metric learning in the context of dynamic time warping (DTW), the by far most popular dissimilarity measure used for the comparison and analysis of motion capture data. While metric learning enables a problem-adapted representation of data, the majority of methods has been proposed for vectorial data onl…
Quandle cocycles are constructed from extensions of quandles. The theory is parallel to that of group cohomology and group extensions. An interpretation of quandle cocycle invariants as obstructions to extending knot colorings is given, and is extended to links component-wise.
The study improves prediction regions for hierarchical data using a projection step.
problem Improving prediction regions for hierarchical multivariate data.
method Integrating a projection step in the split conformal prediction (SCP) procedure to reduce prediction regions.
result The resulting prediction regions are globally smaller under both joint and component-wise coverage objectives.
We propose a statistical inference framework for the component-wise functional gradient descent algorithm (CFGD) under normality assumption for model errors, also known as L2-Boosting. The CFGD is one of the most versatile tools to analyze data, because it scales well to high-dimensional data sets, allows for a very…
The paper explores the relationship between joint mixability and negative dependence structures.
problem Understanding the connection between joint mixability and various negative dependence concepts.
method Analyzes the properties of joint mixes and their relation to negative dependence structures.
result Derives necessary and sufficient conditions for a joint mix to be negatively dependent.
A market model with d assets in discrete time is considered where trades are subject to proportional transaction costs given via bid-ask spreads, while the existence of a numèraire is not assumed. It is shown that robust no arbitrage holds if, and only if, there exists a Pareto solution for some vector-valued utility…
GLU variants improve Transformer performance.
problem Improving the performance of Transformer models.
method Tested variations of Gated Linear Units (GLU) in Transformer feed-forward sublayers.
result Some GLU variants yield better performance than ReLU or GELU activations.
Paper proposes a privacy-preserving method for estimating complex models.
problem Lack of flexibility in existing model classes for approximating data-generating processes.
method Privacy-preserving distributed estimation of generalized additive mixed models using component-wise gradient boosting.
result Proposed algorithm yields equivalent model estimates as component-wise gradient boosting on pooled data.
Bayesian active learning improves holistic educational assessments.
problem Gap between holistic CJ and criterion-based rubrics in education.
method Extends Bayesian CJ to handle multiple LO components, using entropy-based active learning.
result Enhanced predictive rankings with uncertainty estimates and quantified assessor agreement.
EPMF factorizes matrices by adjusting their entries to match a specified power.
problem Factorizing matrices with adjusted entries to match a specified power.
method Analyzes the computational complexity of exact and approximate EPMF problems.
result Exact EPMF is strongly NP-hard, but can be solved in polynomial time when rank is fixed.
The link between different psychophysiological measures during emotion episodes is not well understood. To analyse the functional relationship between electroencephalography (EEG) and facial electromyography (EMG), we apply historical function-on-function regression models to EEG and EMG data that were simultaneously r…
Develops conformal Bayes for two-sided censored Gaussian regression under label shift.
problem Prediction under label shift with censored responses.
method Combines posterior predictive tilting with weighted conformal calibration.
result Restores marginal coverage with smaller prediction sets.
New method for sparse data using L1-NMF with improved sparsity control.
problem Sparse data with false zeros and heavy-tailed noise.
method Component-wise L1-NMF with weighted penalization and coordinate descent.
result Effective in handling sparse data with false zeros.
Proposes iVDFM for identifying latent factors in multivariate time series.
problem Identifying latent factors in multivariate time series with structural dynamics.
method Identifiable Variational Dynamic Factor Model (iVDFM) with iVAE-style conditioning.
result Identifiable latent factors up to permutation and component-wise affine transformations.
Although a majority of the theoretical literature in high-dimensional statistics has focused on settings which involve fully-observed data, settings with missing values and corruptions are common in practice. We consider the problems of estimation and of constructing component-wise confidence intervals in a sparse high…
Study of cable links of uniformly thick knots, revealing new isotopy phenomena.
problem Understanding Legendrian isotopy in cable links of uniformly thick knots.
method Introduced new technique of Legendrian surgeries to classify Legendrian knots in negative cables of twist knots.
result Found new phenomena of stabilized Legendrian links that are smoothly isotopic but not Legendrian isotopic.
New convergence bounds for online learning with heavy-tailed noise.
problem Learning on streaming data with heavy-tailed noise.
method Nonlinear stochastic gradient descent (SGD) for non-convex and strongly convex costs.
result Strong convergence rates for various nonlinearities and noise distributions.
Deep learning models have been successfully used in computer vision and many other fields. We propose an unorthodox algorithm for performing quantization of the model parameters. In contrast with popular quantization schemes based on thresholds, we use a novel technique based on periodic functions, such as continuous t…
In clustering we normally output one cluster variable for each datapoint. However it is not necessarily the case that there is only one way to partition a given dataset into cluster components. For example, one could cluster objects by their colour, or by their type. Different attributes form a hierarchy, and we could …
We consider the problem of recovering a common latent source with independent components from multiple views. This applies to settings in which a variable is measured with multiple experimental modalities, and where the goal is to synthesize the disparate measurements into a single unified representation. We consider t…
U-Clip clips gradients to reduce bias without discarding them.
problem Reduces bias in stochastic gradient descent.
method Maintains a buffer of clipped gradients to add to next iteration.
result U-Clip updates are unbiased on average.
New method uses conformal prediction for time series forecasting, accounting for temporal correlation.
problem Uncertainty quantification in temporally correlated time series data.
method Time series decomposition with component-wise conformal prediction.
result The method provides customized prediction intervals for different temporal components.
mfBm models and forecasts volatility with different Hurst exponents and correlations.
problem Modeling and forecasting volatility with varying Hurst exponents and correlations.
method Multivariate fractional Brownian motion (mfBm) with component-wise Hurst exponents, novel estimation method, time-reversibility test.
result mfBm reduces forecasting errors compared to a one-dimensional model and outperforms HAR model.
The paper uses neural networks to learn system dynamics from data with Lipschitz regularization.
problem Learning governing equations from time-sampled data.
method Lipschitz regularized deep neural networks for ODE system identification.
result Lipschitz regularization improves the smoothness and generalization of the learned function.
Flow models recover causal transformations from observational data and a valid ordering.
problem Causal inference with only observational data and a valid causal ordering.
method Flow models that can recover component-wise, invertible transformations of exogenous variables.
result Flow models outperform previous methods and deliver consistent performance across various structural causal models.
Study of panhandle polynomials of torus links with geometric applications.
problem Characterizing the HOMFLY-PT polynomial of torus knots and links.
method Utilizing quantum group representations and the Rosso-Jones formula.
result Established panhandle-like structure of HOMFLY-PT polynomials for torus knots and links.
Boosting strategies for merging vs. ensembling studies analyzed.
problem Deciding between merging and ensembling studies for boosting.
method Analytical transition point and bias-variance decomposition for boosting with linear learners.
result Theoretical guidelines for merging vs. ensembling studies.
We present a new algorithm for boosting generalized additive models for location, scale and shape (GAMLSS) that allows to incorporate stability selection, an increasingly popular way to obtain stable sets of covariates while controlling the per-family error rate (PFER). The model is fitted repeatedly to subsampled data…
TADDAA improves accuracy diagnostics for variational approximations.
problem Challenges in evaluating the accuracy of variational approximations.
method Uses many short parallel MCMC chains to obtain lower bounds on the error of each posterior functional of interest.
result Validates the practical utility and computational efficiency of TADDAA on various models.
Generative models' evaluation scores can be misleading, leading to inflated grades.
problem Misleading evaluation scores for generative models.
method Analyzed and compared various scores for evaluating synthetic vs. ground-truth data.
result The Eden score avoids grade inflation and better aligns with human perception.
New method identifies latent sources from nonlinear mixtures without auxiliary variables.
problem Identifying latent sources from nonlinear mixtures without additional information.
method Structural Sparsity assumptions on the mixing process.
result Latent sources can be identified up to permutation and transformation.
New framework IIA identifies innovations in general nonlinear vector autoregressive processes.
problem Limited generality of NVAR models due to additive innovation assumption.
method Independent Innovation Analysis (IIA) framework, assuming mutual independence and modulation by an auxiliary variable.
result Guarantees identifiability of innovations with arbitrary nonlinearities, up to permutation and component-wise invertible nonlinearities.
In image classification, visual separability between different object categories is highly uneven, and some categories are more difficult to distinguish than others. Such difficult categories demand more dedicated classifiers. However, existing deep convolutional neural networks (CNN) are trained as flat N-way classifi…
Improves score estimation for noised targets using known clean scores.
problem Poor score estimation at low noise levels in Denoising Score Matching.
method Introduces Target Score Identity and Target Score Matching loss.
result Score estimates are more accurate at low noise levels.
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
Unified diffusion framework enhances generative models flexibility.
problem Improving generative models' design freedom and efficiency.
method Unified framework incorporating choice of representation, prior distribution, and noise scheduling.
result Enhanced flexibility leading to more efficient training and data generation.
A central question of representation learning asks under which conditions it is possible to reconstruct the true latent variables of an arbitrarily complex generative process. Recent breakthrough work by Khemakhem et al. (2019) on nonlinear ICA has answered this question for a broad class of conditional generative proc…
Study compares multivariate scoring rules for distribution forecasts.
problem Evaluating the discrimination ability of multivariate scoring rules.
method Simulation study comparing energy and variogram scores using historical data.
result Variogram score with p=0.5 outperforms other scores.
A new method improves data generation quality by correcting score mismatches.
problem Score mismatch issue in conditional score-based data generation methods.
method Denoising Likelihood Score Matching (DLSM) loss for classifier training.
result The proposed method outperforms previous methods on Cifar-10 and Cifar-100 benchmarks.
This paper tackles sequential distribution shifts in representation learning.
problem Learning meaningful representations in a sequence of distribution shifts.
method Nonlinear Independent Component Analysis (ICA) framework for continual causal representation learning.
result The method achieves performance comparable to joint training on multiple offline distributions and shows no benefit from the incoming new distribution on all latent variables.
New scoring rules improve probabilistic classification model evaluation.
problem Traditional scoring rules misalign with the preference for correct classifications.
method Introduces Penalized Brier Score (PBS) and Penalized Logarithmic Loss (PLL) to modify proper scoring rules.
result PBS and PLL better identify optimal checkpoints and early stopping points, leading to superior F1 scores.
ADMM algorithm solves nonlinear matrix decompositions efficiently.
problem Nonlinear matrix decompositions for various applications.
method Alternating Direction Method of Multipliers (ADMM) for nonlinear matrix factorization.
result The method efficiently solves diverse nonlinear matrix decompositions.