We present a multi-task learning formulation for Deep Gaussian processes (DGPs), through non-linear mixtures of latent processes. The latent space is composed of private processes that capture within-task information and shared processes that capture across-task dependencies. We propose two different methods for segmen…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “non-linear tasks”
Paper introduces non-linearity signature to measure deep neural network performance.
problem Difficulty in explaining performance differences among similar DNN architectures.
method Affine Optimal Transport mappings to measure non-linearity.
result Signature provides better understanding of DNN inner workings.
The one-class kernel spectral regression (OC-KSR), the regression-based formulation of the kernel null-space approach has been found to be an effective Fisher criterion-based methodology for one-class classification (OCC), achieving state-of-the-art performance in one-class classification while providing relatively hig…
SSL framework identifies non-linear systems without labeled data.
problem System identification in non-linear environments without labeled data.
method Dynamics contrastive learning framework.
result SSL can identify non-linear dynamics in latent space.
Paper analyzes dataset distillation for efficient encoding of task-relevant information.
problem Efficiently encoding task-relevant information from gradient-based learning of non-linear tasks.
method Theoretical analysis of dataset distillation applied to two-layer neural networks with gradient-based training.
result Low-dimensional structure of the problem is efficiently encoded into distilled data, reproducing a model with high generalization ability.
TaCo: Targeted Concept Erasure Prevents Non-Linear Classifiers From Detecting Protected Attributescs.CL
TaCo prevents non-linear classifiers from detecting sensitive attributes.
problem Ensuring fairness in NLP models by preventing sensitive attribute detection.
method Targeted Concept Erasure (TaCo) removes sensitive information from final latent representations, even against non-linear classifiers.
result TaCo outperforms state-of-the-art methods in reducing sensitive attribute prediction accuracy while preserving overall task performance.
We provide an overview of several non-linear activation functions in a neural network architecture that have proven successful in many machine learning applications. We conduct an empirical analysis on the effectiveness of using these function on the MNIST classification task, with the aim of clarifying which functions…
Unified framework for non-linear attention using modern Hopfield networks.
problem Improving transformer model's understanding of complex relationships and efficiency.
method Proposes an energy functional based on Modern Hopfield Networks (MNH) to unify linear and non-linear attention mechanisms.
result Context wells encapsulate contextual relationships among tokens, offering a richer representation of non-linear data.
NDMs enable non-linear transformations in diffusion models for better generative tasks.
problem Limited to linear transformations, diffusion models struggle with generative tasks.
method Presented NDMs that allow time-dependent non-linear transformations.
result NDMs outperform conventional diffusion models in likelihood and sample quality.
Matrix completion aims to predict missing elements in a partially observed data matrix which in typical applications, such as collaborative filtering, is large and extremely sparsely observed. A standard solution is matrix factorization, which predicts unobserved entries as linear combinations of latent variables. We g…
We consider the task of robust non-linear regression in the presence of both inlier noise and outliers. Assuming that the unknown non-linear function belongs to a Reproducing Kernel Hilbert Space (RKHS), our goal is to estimate the set of the associated unknown parameters. Due to the presence of outliers, common techni…
PatternLocal improves XAI for non-linear models by suppressing suppressor variables.
problem Suppressor variables cause false-positive feature attributions in non-linear models.
method PatternLocal uses locally linear surrogate models and transforms weights into a generative representation.
result PatternLocal reduces false-positive attributions and provides more reliable explanations.
Paper establishes limits for accurately estimating low-rank matrices from noisy, non-linear data.
problem Estimating low-rank matrices from noisy, non-linear observations.
method Proves strong universality result with equivalent Gaussian model and effective prior parameters.
result Signal-to-noise ratio requirement grows as $N^{ rac 12 (1-1/k_F)}$ for accurate reconstruction.
We present a general framework for solving a large class of learning problems with non-linear functions of classification rates. This includes problems where one wishes to optimize a non-decomposable performance metric such as the F-measure or G-mean, and constrained training problems where the classifier needs to sati…
New method disentangles perceptual uncertainty and behavioral costs in partially observable systems.
problem Tackles inverse optimal control for non-linear partially observable systems.
method Probabilistic approach using maximum causal entropy formulations and local linearization.
result Disentangles perceptual factors and behavioral costs in sequential decision-making.
This paper conditions non-linear infinite-dimensional diffusion processes.
problem Conditioning non-linear and infinite-dimensional diffusion processes.
method Infinite-dimensional Girsanov's theorem to condition function-valued stochastic processes.
result Conditioning of non-linear infinite-dimensional diffusion processes is achieved.
Inference-aware meta-alignment of LLMs reduces computational cost.
problem Aligning LLMs to diverse human preferences is challenging due to conflicting criteria.
method IAMA trains a base model to be aligned to multiple tasks via different inference-time alignment algorithms, using non-linear GRPO for optimization.
result IAMA enables effective alignment of LLMs to multiple criteria with limited computational budget.
Graph representation learning is of paramount importance for a variety of graph analytical tasks, ranging from node classification to community detection. Recently, graph convolutional networks (GCNs) have been successfully applied for graph representation learning. These GCNs generate node representation by aggregatin…
Over the past decade a wide spectrum of machine learning models have been developed to model the neurodegenerative diseases, associating biomarkers, especially non-intrusive neuroimaging markers, with key clinical scores measuring the cognitive status of patients. Multi-task learning (MTL) has been commonly utilized by…
Learning by integrating multiple heterogeneous data sources is a common requirement in many tasks. Collective Matrix Factorization (CMF) is a technique to learn shared latent representations from arbitrary collections of matrices. It can be used to simultaneously complete one or more matrices, for predicting the unknow…
Multivariate categorical data occur in many applications of machine learning. One of the main difficulties with these vectors of categorical variables is sparsity. The number of possible observations grows exponentially with vector length, but dataset diversity might be poor in comparison. Recent models have gained sig…
Proposes HSIC-Lasso for selective inference in non-linear data.
problem Detecting influential features in non-linear and high-dimensional data.
method Model-free HSIC-Lasso based on truncated Gaussians and polyhedral lemma.
result Tight control of type-I error even for small sample sizes.
This study compares mtl architectures for renewable power generation forecasting.
problem Forecasting expected power generation for renewable power plants.
method Comparative study of mtl architectures including Hard parameter sharing, cross-stitch network, sluice network, and task embedding.
result Emerging Relation Network (ERN) and task embedding provide significant improvements in forecasting.
Paper tackles intervention extrapolation using identifiable representations.
problem Predicting effects of unseen interventions on outcomes.
method Combines identifiable representation learning with autoencoders to enforce linear invariance.
result Identifiable representations enable non-linear extrapolation of interventions.
We introduce a method for constructing skills capable of solving tasks drawn from a distribution of parameterized reinforcement learning problems. The method draws example tasks from a distribution of interest and uses the corresponding learned policies to estimate the topology of the lower-dimensional piecewise-smooth…
Comparison of non-linear activation functions for deep neural networks on MNIST classification taskcs.LG
Activation functions play a key role in neural networks so it becomes fundamental to understand their advantages and disadvantages in order to achieve better performances. This paper will first introduce common types of non linear activation functions that are alternative to the well known sigmoid function and then eva…
Adequate evaluation of an information retrieval system to estimate future performance is a crucial task. Area under the ROC curve (AUC) is widely used to evaluate the generalization of a retrieval system. However, the objective function optimized in many retrieval systems is the error rate and not the AUC value. This p…
Matrix factorization techniques have been widely used as a method for collaborative filtering for recommender systems. In recent times, different variants of deep learning algorithms have been explored in this setting to improve the task of making a personalized recommendation with user-item interaction data. The idea …
KalmanNet uses neural networks to improve state estimation in systems with unknown dynamics.
problem State estimation of systems with non-linear dynamics and partial information.
method KalmanNet integrates a recurrent neural network with the Kalman filter to handle non-linearities and model mismatches.
result KalmanNet outperforms classic filtering methods in systems with both mismatched and accurate domain knowledge.
Paper investigates Lipschitz constants of self-attention modules in neural networks.
problem Lipschitz constants of self-attention modules in neural networks.
method Proved standard dot-product self-attention is not Lipschitz for unbounded input domain. Proposed L2 self-attention that is Lipschitz. Derived upper bound on L2 self-attention's Lipschitz constant.
result Proved standard self-attention is not Lipschitz for unbounded input domain and proposed an alternative L2 self-attention that is Lipschitz.
LQF linearizes deep models for better interpretability.
problem Lack of interpretability in deep neural networks.
method Simple modifications to architecture, loss function, and optimization.
result Comparable performance to non-linear fine-tuning, with interpretability.
Layer-wise relevance propagation (LRP) is a recently proposed technique for explaining predictions of complex non-linear classifiers in terms of input variables. In this paper, we apply LRP for the first time to natural language processing (NLP). More precisely, we use it to explain the predictions of a convolutional n…
CVAE improves stock volume forecasting with advanced input variables.
problem Improving accuracy of daily stock volume forecasts.
method Conditional Variational Auto-Encoder (CVAE) with advanced input variables.
result CVAE generates non-linear forecasts with better accuracy and correlation to actual data.
Multitask learning algorithms are typically designed assuming some fixed, a priori known latent structure shared by all the tasks. However, it is usually unclear what type of latent task structure is the most appropriate for a given multitask learning problem. Ideally, the "right" latent task structure should be learne…
AuxiLearn combines auxiliary tasks into a single loss function.
problem Improving neural network performance on a main task using auxiliary tasks.
method Implicit differentiation to learn a network that combines auxiliary tasks into a single coherent objective function.
result AuxiLearn consistently outperforms competing methods in various tasks and domains.
Analyzes generalization error in generalized linear models, explaining double descent phenomenon.
problem Understanding generalization of machine learning models in high dimensions.
method Develops a framework to characterize asymptotic generalization error for generalized linear models.
result Rigorously explains the double descent phenomenon in generalized linear models.
Reliable measures of statistical dependence could be useful tools for learning independent features and performing tasks like source separation using Independent Component Analysis (ICA). Unfortunately, many of such measures, like the mutual information, are hard to estimate and optimize directly. We propose to learn i…
For successful deployment of deep neural networks on highly--resource-constrained devices (hearing aids, earbuds, wearables), we must simplify the types of operations and the memory/power resources used during inference. Completely avoiding inference-time floating-point operations is one of the simplest ways to design …
Cross-Scale Residual Network for Multiple Tasks:Image Super-resolution, Denoising, and Deblockingeess.IV
In general, image restoration involves mapping from low quality images to their high-quality counterparts. Such optimal mapping is usually non-linear and learnable by machine learning. Recently, deep convolutional neural networks have proven promising for such learning processing. It is desirable for an image processin…
We consider the problem of personalization of online services from the viewpoint of ad targeting, where we seek to find the best ad categories to be shown to each user, resulting in improved user experience and increased advertisers' revenue. We propose to address this problem as a task of ranking the ad categories dep…
This paper investigates how transformers can learn to generalize to unseen examples in context.
problem Understanding how transformers can generalize to unseen examples in a prompt.
method Gradient descent analysis of one-layer multi-head transformers for in-context learning.
result The training loss for a one-layer multi-head transformer converges linearly to a global minimum, effectively learning ridge regression over basis functions.
Steerable E(3) Graph Neural Networks incorporate geometric and physical covariant information.
problem Incorporating covariant information like position, force, velocity, or spin in graph neural networks.
method Steerable E(3) Equivariant Graph Neural Networks (SEGNNs) that use steerable MLPs to incorporate geometric and physical covariant information.
result SEGNNs improve upon classic linear point convolutions and recent equivariant graph networks that send invariant messages.
KPCA improves OoD detection by separating InD and OoD data.
problem Insufficiency of PCA in detecting OoD data from InD data.
method Kernel PCA (KPCA) with task-specific kernels.
result KPCA achieves superior OoD detection performance.
New definitions of ESP for quantum reservoir computing handle non-stationary systems.
problem Traditional ESP does not apply to non-stationary systems.
method Introduce two new categories of ESP: non-stationary ESP and subset/subspace ESP.
result Demonstrates correspondence between non-stationary ESP and QRC with NARMA tasks.
This work develops methods to analyze data on curved spaces using deep learning.
problem Analyzing data in non-linear, curved spaces.
method Pullback Riemannian geometry through diffeomorphisms.
result Diffeomorphisms need to map data into geodesic subspaces to ensure proper data analysis.
This paper introduces an acceleration structure for hyperbolic embeddings.
problem Efficiently embedding and visualizing high-dimensional data in hyperbolic spaces.
method Building upon a polar quadtree, the paper introduces a new acceleration structure for hyperbolic embeddings.
result The new method computes embeddings in significantly less time compared to existing methods.
Two new algorithms reduce feature space while preserving non-linear relationships.
problem High-dimensional data and overfitting issues.
method Bias-variance analysis for non-linear transformations and generalized linear models.
result Competitive performance on regression and classification tasks.
MTL improves multi-dimensional regression in luminescence sensing.
problem Challenges in modeling multi-dimensional regression problems with classical methods.
method Multi-task learning (MTL) with feed-forward neural networks (FFNNs).
result MTL allows predicting multiple parameters from a single set of measurements.