Transformers capture combinatorial tasks with bounded error and logarithmic sample dependence.
problem Capturing complex combinatorial tasks with bounded error and sample efficiency.
method Formal definition of algorithmic capture, empirical analysis of infinite-width transformers, upper bounds on computational complexity.
result Transformers exhibit an inductive bias favoring simpler algorithmic procedures over higher complexity ones.
Bayesian nonparametrics are a class of probabilistic models in which the model size is inferred from data. A recently developed methodology in this field is small-variance asymptotic analysis, a mathematical technique for deriving learning algorithms that capture much of the flexibility of Bayesian nonparametric infere…
We utilize a recently developed genetic algorithm, in conjunction with discrete wavelets, for carrying out successful forecasts of the trend in financial time series, that includes the NASDAQ composite index. Discrete wavelets isolate the local, small scale variations in these non-stationary time series, after which th…
A novel hypergraph partitioning method using tensor eigenvalue decomposition captures super-dyadic interactions.
problem Capturing super-dyadic interactions in k-uniform hypergraphs.
method Tensor-based representation and tensor eigenvalue decomposition for capturing interactions.
result Improved min-cut solution on 2-uniform hypergraphs (graphs) compared to standard spectral partitioning.
New algorithm speeds up IRT model fitting for large datasets.
problem Inefficient algorithms for fitting Item Response Theory models to large datasets.
method Stochastic Variational Inference for IRT models.
result Higher log likelihoods and accuracy in imputing missing data.
New algorithm learns invariant representations for robust neural networks.
problem Learning robust neural network representations that are invariant to certain factors.
method Causal perspective and distribution matching approach.
result Empirically, the algorithm achieves state-of-the-art performance on domain generalization.
In this paper we explore the usage of deep reinforcement learning algorithms to automatically generate consistently profitable, robust, uncorrelated trading signals in any general financial market. In order to do this, we present a novel Markov decision process (MDP) model to capture the financial trading markets. We r…
We present network embedding algorithms that capture information about a node from the local distribution over node attributes around it, as observed over random walks following an approach similar to Skip-gram. Observations from neighborhoods of different sizes are either pooled (AE) or encoded distinctly in a multi-s…
Proposes a partially linear structure to capture nonlinear relationships in mixture of experts models.
problem Suboptimal estimates due to linearity assumption in mixture of experts models.
method Introduces a partially linear structure that incorporates unspecified functions to capture nonlinear relationships.
result Establishes the identifiability of the proposed model under mild conditions and introduces a practical estimation algorithm.
Diffusion Transformer captures spatial-temporal dependencies in sequential data.
problem Capturing rich spatial and temporal dependencies in sequential data.
method Established theoretical guarantees for diffusion transformers learning Gaussian process data.
result Spatial-temporal dependencies are captured within attention layers of diffusion transformers.
Interactions such as double negation in sentences and scene interactions in images are common forms of complex dependencies captured by state-of-the-art machine learning models. We propose Mahé, a novel approach to provide Model-agnostic hierarchical éxplanations of how powerful machine learning models, such as deep ne…
New algorithms recover sparse tensor principal components efficiently.
problem Recovering sparse tensor principal components from noisy data.
method Family of algorithms interpolating between polynomial-time and exhaustive search, tailored for sparse and highly sparse regimes.
result Our algorithms recover sparse vectors for signal-to-noise ratios beyond previous limits, with time complexity ildeO(np+t). This work develops secure distributed algorithms for machine learning to protect against data poisoning and network attacks.
problem Vulnerability of distributed machine learning algorithms to cyber threats.
method Game-theoretic framework to capture conflicting goals of a learner and an attacker, iterative distributed algorithm.
result Distributed SVM is prone to fail in different types of attacks, with impact depending on network structure and attack capabilities.
We present ShapeVis, a scalable visualization technique for point cloud data inspired from topological data analysis. Our method captures the underlying geometric and topological structure of the data in a compressed graphical representation. Much success has been reported by the data visualization technique Mapper, th…
Proposes an adversarial algorithm to learn unbiased representations via HGR coefficient.
problem Learning fair representations without sensitive attribute information.
method Adversarial algorithm using Hirschfeld-Gebelein-Renyi (HGR) maximal correlation coefficient.
result Significant improvements in bias mitigation compared to existing methods.
We present a solution to scale spectral algorithms for learning sequence functions. We are interested in the case where these functions are sparse (that is, for most sequences they return 0). Spectral algorithms reduce the learning problem to the task of computing an SVD decomposition over a special type of matrix call…
The Markov decision process (MDP) formulation used to model many real-world sequential decision making problems does not efficiently capture the setting where the set of available decisions (actions) at each time step is stochastic. Recently, the stochastic action set Markov decision process (SAS-MDP) formulation has b…
New research shows graph embeddings fail to capture key network properties.
problem Graph embeddings fail to capture salient properties of complex networks.
method Mathematical proof and empirical study of various embedding techniques.
result Any successful graph embedding must have a rank nearly linear in the number of vertices.
New algorithms optimize multiple tasks with shared similarities, reducing regret.
problem Optimizing multiple objectives with shared similarities in non-parametric Bayesian optimization.
method Developed two novel BO algorithms using multi-task kernels and random scalarizations.
result Derived worst-case regret bounds capturing inter-task similarities.
Captures data influence changes during training.
problem Traditional influence functions fail for modern training methods.
method Formalized trajectory-specific LOO influence, using data value embedding.
result Data influence varies by training stage, early and late stages have greater impact.
Multivariate boosted trees improve forecasting and control by capturing correlated predictions.
problem Capturing multivariate target cross-correlations and applying structured penalties to predictions.
method A computationally efficient algorithm for fitting multivariate boosted trees.
result Multivariate trees outperform univariate counterparts in correlated prediction scenarios.
Dynamic topic models (DTMs) are very effective in discovering topics and capturing their evolution trends in time series data. To do posterior inference of DTMs, existing methods are all batch algorithms that scan the full dataset before each update of the model and make inexact variational approximations with mean-fie…
New MMM captures hierarchical marketing effects and sign restrictions.
problem Measuring effectiveness of marketing activities with hierarchical structure and sign constraints.
method Proposes a constrained maximum likelihood approach using Hamiltonian Monte Carlo algorithm.
result Demonstrates superior performance on real datasets compared to multi-stage methods.
We introduce a notion of algorithmic stability of learning algorithms---that we term \emph{argument stability}---that captures stability of the hypothesis output by the learning algorithm in the normed space of functions from which hypotheses are selected. The main result of the paper bounds the generalization error of…
Proposes a new method to explain complex machine learning models.
problem Lack of joint feature effects in current explanation algorithms.
method Axiomatization of the Banzhaf index for capturing feature subsets.
result The generalized Banzhaf index optimally approximates black-box models.
This work presents a novel approach for robust PCA with total variation regularization for foreground-background separation and denoising on noisy, moving camera video. Our proposed algorithm registers the raw (possibly corrupted) frames of a video and then jointly processes the registered frames to produce a decomposi…
Paper develops models for better HFT and algorithmic trading.
problem Inaccurate LOB dynamics in financial markets.
method Semi-Markov and Hawkes jump-diffusion models for LOB dynamics.
result Improved trading strategies through precise model application.
Unified algorithm for any p-norm experimental design problems.
problem Experimental design problems for various p-norm objectives. method Randomized local search approach for all p. result First approximation algorithm for general p-norm objective. Statistical query (SQ) algorithms are algorithms that have access to an {\em SQ oracle} for the input distribution D instead of i.i.d.~ samples from D. Given a query function φ:X→[−1,1], the oracle returns an estimate of Ex∼D[φ(x)] within some tolerance τφ that roughly corresponds t…
DR-submodular continuous functions are important objectives with wide real-world applications spanning MAP inference in determinantal point processes (DPPs), and mean-field inference for probabilistic submodular models, amongst others. DR-submodularity captures a subclass of non-convex functions that enables both exact…
Future predictions on sequence data (e.g., videos or audios) require the algorithms to capture non-Markovian and compositional properties of high-level semantics. Context-free grammars are natural choices to capture such properties, but traditional grammar parsers (e.g., Earley parser) only take symbolic sentences as i…
The paper introduces a fast algorithm for learning and forecasting nonlinear dynamics from noisy time series data.
problem Challenges in capturing nonlinear dynamics from noisy time series data.
method A projected nonlinear state-space model with kernel functions applied to projected lines.
result The model effectively learns and forecasts complex nonlinear dynamics with computational efficiency.
Pursuit-evasion is a multi-agent sequential decision problem wherein a group of agents known as pursuers coordinate their traversal of a spatial domain to locate an agent trying to evade them. Pursuit evasion problems arise in a number of import application domains including defense and route planning. Learning to opti…
ExDBN learns dynamic Bayesian networks using mixed-integer programming.
problem Learning dynamic causal relationships from time series data.
method Score-based learning algorithm using mixed-integer quadratic programming with branch-and-cut method.
result The proposed method produces more accurate results than state-of-the-art approaches.
We consider the following multi-component sparse PCA problem: given a set of data points, we seek to extract a small number of sparse components with disjoint supports that jointly capture the maximum possible variance. These components can be computed one by one, repeatedly solving the single-component problem and def…
Spatial-temporal graph modeling is an important task to analyze the spatial relations and temporal trends of components in a system. Existing approaches mostly capture the spatial dependency on a fixed graph structure, assuming that the underlying relation between entities is pre-determined. However, the explicit graph…
Our everyday interactions with pervasive systems generate traces that capture various aspects of human behavior and enable machine learning algorithms to extract latent information about users. In this paper, we propose a machine learning interpretability framework that enables users to understand how these generated t…
Determinantal point processes (DPPs) have attracted significant attention as an elegant model that is able to capture the balance between quality and diversity within sets. DPPs are parameterized by a positive semi-definite kernel matrix. While DPPs have substantial expressive power, they are fundamentally limited by t…
Algorithm captures and refines features for efficient lifelong learning.
problem Efficiently learning multiple tasks sequentially with limited data.
method Proposes a lifelong learning algorithm that maintains and refines internal feature representation.
result Proves sample complexity improvement and matches lower bound for efficiency.
We present a novel algorithm, called Links, designed to perform online clustering on unit vectors in a high-dimensional Euclidean space. The algorithm is appropriate when it is necessary to cluster data efficiently as it streams in, and is to be contrasted with traditional batch clustering algorithms that have access t…
New copula-based models for binary outcomes capture complex interactions.
problem Complex interaction effects in binary outcome models.
method Additive models with copula-based components, no discretization required.
result Better or comparable predictive performance compared to other methods.
Learning image representations to capture fine-grained semantics has been a challenging and important task enabling many applications such as image search and clustering. In this paper, we present Graph-Regularized Image Semantic Embedding (Graph-RISE), a large-scale neural graph learning framework that allows us to tr…
Low-degree method fails to predict robust subspace recovery problem.
problem Predicting computational tractability of robust subspace recovery problem.
method Low-degree polynomial framework, anti-concentration properties.
result Low-degree method fails to predict computational tractability of robust subspace recovery problem even up to high degree.
Bayesian approach for handling incomplete clinical data.
problem Challenges in machine learning with multimodal, incomplete clinical data.
method Generative and discriminative learning, semi-supervised strategy, imputation of missing views.
result Automatic imputation of missing views and robust inference across different data sources.
TSVQR captures heterogeneous and asymmetric data using quantile regression.
problem Capturing heterogeneous and asymmetric information in modern data.
method Twin Support Vector Quantile Regression (TSVQR) with two nonparallel planes for quantile levels.
result TSVQR outperforms previous methods in capturing and learning from data.
Polynomial-time algorithm estimates mean with bounded covariance using differential privacy.
problem Estimating mean of a d-variate distribution with differential privacy constraints.
method Sum of Squares (SoS) exponential mechanism for polynomial-time differentially private estimation.
result First polynomial-time algorithm with O(d) samples for mean estimation under pure differential privacy. A new method warps inputs to learn nonstationary kernels efficiently.
problem Learning nonstationary patterns in data with varying smoothness.
method Sparse spectrum Gaussian processes with input warping as conditional Gaussian measures.
result Efficient learning of nonstationary patterns with fewer parameters.
HappyMap improves fairness and learning across domains by generalizing multi-calibration.
problem Improving fairness and learning across different domains for predictions.
method Proposes HappyMap, a generalized approach to multi-calibration.
result Unified understanding of fairness and learning across domains.