Study on Privileged ERM showing limitations and providing capacity analysis.
problem Improving classification accuracy with privileged information.
method Theoretical analysis of Privileged ERM using VC dimension and generalization bounds.
result Worst-case guarantees for Privileged ERM cannot improve over standard ERM unless privileged information capacity is similar or smaller.
Privileged Information Dropout improves RL performance without distillation.
problem Improving sample efficiency and performance in reinforcement learning.
method Introducing Privileged Information Dropout to directly incorporate privileged information into RL agent inputs.
result Privileged Information Dropout outperforms distillation and auxiliary tasks in a partially-observed environment.
KOC+ uses privileged information to improve one-class classification performance.
problem Outlier detection and novelty detection using kernel methods.
method Kernel ridge regression with correction function for privileged information.
result KOC+ achieves better generalization performance compared to traditional methods.
A new method for Gaussian process classification using privileged information.
problem Efficiently incorporating privileged information into Gaussian process classification.
method Soft-to-Hard Label Transfer (SLT-GP) method.
result SLT-GP method achieves better performance than GPC and GPC+.
SPI detects anomalies using privileged information from training data.
problem Unsupervised anomaly detection in the absence of labeled test data.
method Constructs density estimates in privileged space and transfers them to anomaly scoring space.
result Significant improvement in anomaly detection performance with privileged information.
Study shows using time-series privileged information improves model efficiency.
problem Efficiently predicting future outcomes using supervised models with privileged information.
method Developed an algorithm for learning with privileged time-series data and proved its efficiency for non-stationary Gaussian-linear systems.
result Learning with privileged information is more efficient than without it for non-stationary Gaussian-linear systems.
Enhances reinforcement learning with partial state information.
problem Improving learning under partial observability with limited privileged signals.
method Introduced informed asymmetric actor-critic framework that uses arbitrary state-dependent privileged signals.
result Unbiased policy gradient estimates with arbitrary privileged signals.
Proposed SMO algorithm for OC-SVM+ significantly outperforms non-sequential algorithms.
problem One-class SVM with privileged information
method Sequential Minimal Optimization (SMO) algorithm
result Finite-time convergence established
Proposes a meta-loss function for regression and classification using privileged information.
problem Improving prediction performance in machine learning problems using privileged information.
method A simple yet effective custom loss function for regression and classification.
result The proposed model outperforms state-of-the-art predictors in protein binding affinity prediction.
Prior knowledge can be used to improve predictive performance of learning algorithms or reduce the amount of data required for training. The same goal is pursued within the learning using privileged information paradigm which was recently introduced by Vapnik et al. and is aimed at utilizing additional information avai…
We introduce a learning framework called learning using privileged information (LUPI) to the computer vision field. We focus on the prototypical computer vision problem of teaching computers to recognize objects in images. We want the computers to be able to learn faster at the expense of providing extra information du…
Paper improves detection accuracy using privileged training data.
problem Traditional detection systems rely on runtime features, limiting model accuracy.
method Adapting knowledge transfer, model influence, and distillation techniques to use training-time privileged information.
result Empirical evaluation shows up to 16.9% relative decrease in detection error.
PrML exploits label relationships using privileged information and low-rank constraints.
problem Improving multi-label learning performance by leveraging implicit and explicit label connections.
method Generates privileged label features and integrates them into low-rank based multi-label learning framework.
result PrML significantly improves multi-label learning performance compared to competing methods.
Paper uses SVM+ for valid prediction intervals in drug discovery datasets.
problem Valid prediction intervals in drug discovery datasets.
method LUPI paradigm, SVM+, inductive conformal predictor.
result Valid prediction intervals at specified significance levels.
Unsupervised domain adaptation improves with privileged information.
problem Domain adaptation under covariate shift and overlap limitations.
method Two-stage learning algorithm inspired by expected error analysis.
result Using privileged information reduces errors and increases sample efficiency.
Proposes a new method to selectively access privileged information in reinforcement learning.
problem Selective compression of privileged information in reinforcement learning.
method Formulates a variational bandwidth bottleneck to decide stochastically whether to access privileged information.
result Improves generalization and reduces access to costly information in reinforcement learning experiments.
The learning with privileged information setting has recently attracted a lot of attention within the machine learning community, as it allows the integration of additional knowledge into the training process of a classifier, even when this comes in the form of a data modality that is not available at test time. Here, …
New insights and algorithms improve prediction models with time-series privileged information.
problem Efficient learning of nonlinear prediction models with limited data.
method Generalization of LuPI to nonlinear tasks, using random features and representation learning.
result Theoretical and empirical evidence supports the use of privileged time-series information for nonlinear prediction.
New research suggests privileged information doesn't improve model performance.
problem Challenges in transferring knowledge using privileged information in machine learning.
method Critical examination of existing theoretical and empirical analyses of LUPI methods.
result LUPI methods often fail to effectively transfer knowledge from privileged information.
Enhances anomaly detection using privileged information in one-class SVM.
problem Anomaly detection in normal state description.
method Introduces a new one-class classification algorithm that incorporates privileged information.
result Improved performance in anomaly detection, validated on synthetic and real datasets.
Introduces privilege scores to measure and interpret protected attribute-related privilege in machine learning models.
problem Lack of explicit formulation of non-neutrality in fairness-aware machine learning methods.
method Privilege scores (PS) and privilege score contributions (PSCs) to measure and interpret protected attribute-related privilege.
result Demonstrates the broad applicability of PS and PSCs in gender and racial privilege in mortgage and college admissions applications.
New method uses dropout with varying variance to improve model learning efficiency.
problem Improving model learning efficiency with limited training data.
method Proposes heteroscedastic dropout for CNNs and RNNs.
result Significantly increases sample efficiency and accuracy with limited data.
New approach combines multi-view learning for improved convergence in knowledge transfer.
problem Improving convergence in knowledge transfer settings like learning with privileged information and distillation.
method Adopting a multi-view approach under reasonable assumptions about hypothesis spaces, encouraging agreement between teacher and student.
result Improved convergence rate achieved with regularized empirical risk minimization.
Improved meta-learning for dynamics using additional structured knowledge.
problem Meta-learning for dynamics with limited raw observations.
method Extended Neural ODE Process model to use privileged information.
result Improved accuracy and calibration on simulated dynamics tasks.
Distillation (Hinton et al., 2015) and privileged information (Vapnik & Izmailov, 2015) are two techniques that enable machines to learn from other machines. This paper unifies these two techniques into generalized distillation, a framework to learn from multiple machines and data representations. We provide theoretica…
Joint training improves model accuracy by selectively using privileged information.
problem Two-stage training can lead to model failure with noisy privileged information.
method Joint training of two models to use privileged information selectively.
result Joint training outperforms two-stage baselines on synthetic and real-world tasks.
Many machine learning algorithms assume that all input samples are independently and identically distributed from some common distribution on either the input space X, in the case of unsupervised learning, or the input and output space X x Y in the case of supervised and semi-supervised learning. In the last number of …
MPWTSVM improves multi-view learning by reducing redundancy and enhancing accuracy.
problem Improving multi-view learning models for better accuracy and efficiency.
method Proposes MPWTSVM, which combines WLTSVM's strengths with multi-view learning principles.
result Demonstrates better accuracy and efficiency compared to existing multi-view classification models.
The paper tackles feature selection for ordinal regression, considering feature redundancies and privileged information.
problem Discovering relevant factors in ranked data with potentially redundant features and privileged information.
method Develops feature relevance bounds for linear ordinal regression, considering feature redundancies and privileged information.
result Identifies all strongly and weakly relevant features and their type of relevance.
Proposes methods to correct bias and missing data in regression models.
problem Nonignorable selection bias and missing response in regression models.
method Imputation-based and importance weighted regression methods, including repeated regression and doubly robust combination.
result Repeated regression can effectively correct bias and outperforms weighted regression in extrapolation.
This paper clarifies privileged coordinates and nilpotent approximation for Carnot manifolds.
problem Understanding privileged coordinates and nilpotent approximation for Carnot manifolds.
method Systematic account on privileged coordinates and nilpotent approximation of Carnot manifolds.
result Description of all systems of privileged coordinates and algebraic characterization of nilpotent groups.
A new algorithm improves model-based reinforcement learning by guiding latent representations.
problem Improving model-based reinforcement learning through additional supervision.
method Proposed a novel asymmetric representation learning objective using latent guidance.
result Significantly improved performance over previous asymmetric approaches.
Study optimal portfolios for traders with asymmetric information and delay.
problem Optimizing portfolios for traders with delayed insider information.
method Anticipating stochastic calculus and white noise approach.
result Optimal portfolios maximize expected logarithmic utility under various financial models.
SAGE enhances reinforcement learning by injecting hints to prevent model stagnation.
problem Sparse rewards cause large language models to stall under relative policy optimization.
method SAGE injects privileged hints during training to increase within-group outcome diversity.
result SAGE consistently outperforms GRPO on 6 benchmarks with LLMs, achieving significant improvements.
The paper introduces Carnot coordinates for Carnot manifolds, simplifying nilpotent approximation.
problem Nilpotent approximation of Carnot manifolds.
method Identification of Carnot coordinates as privileged coordinates with specific properties.
result Carnot coordinates provide a precise and effective nilpotent approximation of Carnot manifolds.
ADVISOR dynamically balances imitation and reinforcement learning to overcome the imitation gap.
problem The gap between imitation learning and reinforcement learning when teaching agents have privileged information.
method Adaptive Insubordination (ADVISOR) dynamically weights imitation and reward-based reinforcement learning losses.
result On-the-fly switching with ADVISOR outperforms pure imitation, pure reinforcement learning, and their combinations.
Improves deep neural networks using soft labels through alternating minimization.
problem Improving deep neural networks training with soft labels.
method Co-Learns DNNs and soft labels via Alternating Minimization of two objectives.
result COLAM achieves improved performance on many tasks with better testing classification accuracy.
A new framework THPI for hashing with auxiliary data.
problem Data sparsity in hashing for real-world applications.
method Transfer learning approach to extend ITQ method.
result Extensive experiments show the effectiveness of THPI.
New method learns influential action sequences without privileged final states.
problem Learning meaningful action sequences in large action spaces.
method Model-free approach that considers the full trajectory.
result Successfully applied to large action spaces.
This survey reviews Heterogeneous Representation Learning (HRL) for diverse data types.
problem Challenges in learning from heterogeneous data types.
method Unified learning framework for modeling various learning settings.
result Unified mathematical framework for multi-view, transfer, privileged, and multi-task learning.
IReEn reveals functionality of black-box agents via iterative neural synthesis.
problem Revealing the functionality of a black-box agent without privileged information.
method Iterative refinement of candidate programs using neural program synthesis.
result The approach finds a functional equivalent program in 78% of cases, outperforming state-of-the-art.
Knowledge transfer speeds up neural classifier training.
problem Lack of theoretical analysis of knowledge transfer in neural networks.
method Regularization of fit between teacher and student networks using privileged information.
result Wide two-layer networks can interpolate between privileged information and data, improving generalization.
Supervised, semi-supervised, and unsupervised learning estimate a function given input/output samples. Generalization of the learned function to unseen data can be improved by incorporating side information into learning. Side information are data that are neither from the input space nor from the output space of the f…
We simplify motion control of a snake robot using nilpotent approximation.
problem Controlling motion of a trident snake robot.
method Constructing a privileged coordinate system and a nilpotent approximation.
result Motions corresponding to Lie brackets are well approximated by the nilpotent approximation.
The paper characterizes the efficiency of transferring knowledge from a teacher to a student classifier over finite domains.
problem Characterizing the statistical efficiency of knowledge transfer over finite domains.
method Three progressive levels of privileged information: hard labels, teacher probabilities, and soft labels. Novel empirical loss functions used to achieve the fundamental limits.
result Achieving the fundamental limits of knowledge transfer through specific levels of privileged information and novel loss functions.
CMRM improves robustness in noisy label settings without requiring privileged knowledge.
problem Learning with noisy labels without privileged knowledge.
method Conformal Margin Risk Minimization (CMRM) framework.
result CMRM consistently improves accuracy and reduces mislabeling under various noise conditions.
Study null hypersurfaces with privileged vector fields, extending surface gravity and defining new horizon types.
problem Characterize null hypersurfaces with privileged vector fields and extend surface gravity.
method Derive identities relating deformation tensor to intrinsic and extrinsic geometry, introduce generalized surface gravity, analyze Lie derivatives, and define new horizon types.
result Introduce three new horizon types that generalize existing concepts to arbitrary topologies and fixed points.
Representation learning systems typically rely on massive amounts of labeled data in order to be trained to high accuracy. Recently, high-dimensional parametric models like neural networks have succeeded in building rich representations using either compressive, reconstructive or supervised criteria. However, the seman…