Chunking is a significant CL problem, accounting for half of performance drop, and current methods don't address it.
problem Chunking of data in continual learning.
method Analyzing and addressing the chunking sub-problem in continual learning.
result Current CL algorithms perform poorly on chunking, only as well as plain SGD training when there is no distribution shift.
This work simplifies IRL by using potential-based reward shaping.
problem Computational inefficiency in inverse reinforcement learning.
method Potential-based reward shaping to reduce RL sub-problems.
result Reduces computational burden of inverse reinforcement learning.
The paper analyzes inexact variants of iterative methods for solving optimization problems.
problem Solving optimization problems with inexact sub-problems.
method Inexact variants of stochastic gradient descent, Newton, proximal point, and subspace ascent methods.
result Iteration complexity results for inexact variants of various iterative methods.
A meta-learning approach improves the performance of alternating minimization for non-convex optimization problems.
problem Optimizing non-convex problems with multiple variables using alternating minimization.
method Meta-learning based alternating minimization (MLAM) to replace handcrafted updating rules.
result The proposed MLAM method outperforms traditional AM-based methods in various non-convex optimization problems.
Many problems in sequential decision making and stochastic control often have natural multiscale structure: sub-tasks are assembled together to accomplish complex goals. Systematically inferring and leveraging hierarchical structure, particularly beyond a single level of abstraction, has remained a longstanding challen…
Optimal control problem for firm cash flow with dividend and capital injection strategies.
problem Maximizing dividends while managing capital injections in a firm's cash flow.
method Proved two optimal strategies: mean-reverting dividends with capital injections or no injections until ruin.
result Optimal strategies are dichotomous: either mean-reverting dividends with injections or no injections.
R package `mvs` handles multi-view data for better model performance.
problem Classical machine learning ignores multi-view data structure, limiting interpretability and performance.
method Multi-view stacking (MVS) framework, training on each view separately, cross-validation for weights, ensemble learning.
result MVS improves model performance on multi-view data.
Meta-learning model divides tasks into sub-problems for efficient adaptation.
problem Training models to quickly adapt to new tasks.
method Hierarchical Expert Networks with information-theoretic partitioning and specialized experts.
result Specialized experts lead to efficient adaptation to new tasks.
Unsupervised speech recognition without labeled data using novel cost function and MAP refinement.
problem Training speech recognition systems without labeled data.
method Alternates between phoneme classifier learning and boundary refinement using Segmental Empirical Output Distribution Matching and MAP approach.
result Achieves phone error rate (PER) of 41.6% on TIMIT dataset.
We consider the problem of finding stationary Nash equilibria (NE) in a finite discounted general-sum stochastic game. We first generalize a non-linear optimization problem from Filar and Vrieze [2004] to a N-player setting and break down this problem into simpler sub-problems that ensure there is no Bellman error fo…
Study capacity constraints in continual learning with a simple model.
problem Understanding optimal resource allocation for agents with limited memory and compute resources.
method Analyzes a capacity-constrained linear-quadratic-Gaussian (LQG) sequential prediction problem and demonstrates optimal capacity allocation strategies.
result Derives a solution to the capacity-constrained LQG sequential prediction problem and shows how to optimally allocate capacity across sub-problems in the steady state.
We propose a novel class of Sequential Monte Carlo (SMC) algorithms, appropriate for inference in probabilistic graphical models. This class of algorithms adopts a divide-and-conquer approach based upon an auxiliary tree-structured decomposition of the model of interest, turning the overall inferential task into a coll…
In this paper, a new method is proposed for sparse PCA based on the recursive divide-and-conquer methodology. The main idea is to separate the original sparse PCA problem into a series of much simpler sub-problems, each having a closed-form solution. By recursively solving these sub-problems in an analytical way, an ef…
Simple framework decouples word alignment and multilingual embedding mapping.
problem Learning multilingual embeddings without supervision.
method Two-stage approach: 1) unsupervised word alignment, 2) mapping embeddings to shared space.
result Robust performance across various multilingual tasks, including distant languages.
Proposes TECU framework for efficient non-convex optimization.
problem Multivariate non-convex optimization problems with coupled objective functions.
method Embeds task-specific strategies into coordinate descent update schemes.
result Demonstrates improved efficiency and effectiveness in solving practical problems.
We reduce a broad class of machine learning problems, usually addressed by EM or sampling, to the problem of finding the k extremal rays spanning the conical hull of a data point set. These k "anchors" lead to a global solution and a more interpretable model that can even outperform EM and sampling on generalizatio…
Meta-learning improves CATE estimation from limited data.
problem Estimating heterogeneous treatment effects from scarce observational data.
method Meta-learning framework decomposes CATE estimation into sub-problems, using neural networks with shared and specific parameters, and optimizing task-specific parameters in closed form.
result Meta-learning method outperforms existing approaches in few-shot CATE estimation.
Proposes MOGFNs for generating diverse Pareto optimal solutions in multi-objective optimization.
problem Generating diverse candidates in multi-objective optimization with conflicting objectives.
method Introduces MOGFNs based on GFlowNets, with two variants: MOGFN-PC and MOGFN-AL.
result Improved candidate diversity compared to existing methods.
LineBO tackles high-dimensional Bayesian optimization by solving 1D subproblems.
problem Bayesian optimization struggles in high dimensions due to complex acquisition steps.
method LineBO restricts high-dimensional problems to 1D subproblems iteratively solved efficiently.
result LineBO converges globally and achieves a fast local rate for strongly convex functions.
Improved pricing method for American options in various models.
problem Efficient pricing of American options in jump-diffusion models and barrier options.
method Hybrid method combining perturbative arguments and quadratic approximation.
result Higher order approximations provide significantly more pricing accuracy.
EC^2-VAE generates music analogies by disentangling pitch and rhythm representations.
problem Disentangling music representations for generating creative analogies.
method Explicitly-constrained variational autoencoder (EC^2-VAE) for disentangling pitch and rhythm representations.
result EC^2-VAE enables the generation of music analogies by borrowing representations from different pieces.
Proposes a new method for medical image segmentation.
problem Medical image segmentation challenges.
method Decompose-and-Integrate Learning.
result Improves segmentation performance on multiple datasets.
In this paper, a new algorithm based on differential geometry viewpoint to solve the 3D rotating Navier-Stokes equations with complex Boundary is proposed, which is called Bi-parallel algorithm. For xample, it can be applied to passage flow between two blades in impeller and circulation flow through aircrafts with comp…
TASTE combines static and temporal data for phenotyping EHRs.
problem Phenotyping EHRs with both static and temporal data.
method Jointly models static and temporal tensors using PARAFAC2 and non-negative matrix factorization, alternatingly solving sub-problems.
result TASTE outperforms existing methods in speed and clinical meaningfulness of phenotypes.
Federated Learning over wireless networks tackles resource allocation challenges.
problem Heterogeneity in UE data and resources in Federated Learning.
method Proposed FL algorithm for heterogeneous data, convergence rate analysis, and resource allocation optimization.
result The proposed algorithm outperforms vanilla FedAvg in convergence rate and accuracy.
We consider multi-task learning, which simultaneously learns related prediction tasks, to improve generalization performance. We factorize a coefficient matrix as the product of two matrices based on a low-rank assumption. These matrices have sparsities to simultaneously perform variable selection and learn and overlap…
Proposes a method to learn graph structure and model parameters jointly in LRSM.
problem The sensitivity of graph weights in LRSM can be arbitrarily large under imbalanced scales and sample sizes.
method Jointly learns graph structure and model parameters by solving a single optimization problem, providing convergence guarantees.
result The proposed approach outperforms existing methods in various real-world numerical examples.
FSDH is a fast supervised hashing method that outperforms existing methods.
problem Efficiently hashing large datasets for fast retrieval.
method FSDH uses a simple regression of class labels to hash codes, avoiding iterative discrete optimization.
result FSDH is significantly faster (12-151x) than existing methods like SDH and FastHash.
In this paper we examine the claims reserving problem using Tweedie's compound Poisson model. We develop the maximum likelihood and Bayesian Markov chain Monte Carlo simulation approaches to fit the model and then compare the estimated models under different scenarios. The key point we demonstrate relates to the compar…
Graph neural networks struggle with proving unsatisfiability in complex logical formulas.
problem Proving unsatisfiability in complex logical formulas.
method Investigating the limitations of graph neural networks in logical reasoning tasks.
result Graph neural networks may fail in certifying unsatisfiability in Boolean formulae.
The ℓ1-norm fails to produce sparse solutions in Laplacian constrained graphical models, leading to a complete graph.
problem Learning a sparse graph under Laplacian constrained Gaussian graphical models.
method Introduced a nonconvex sparsity penalty and proposed a new estimator using a sequence of weighted ℓ1-norm penalized sub-problems. Developed a projected gradient descent algorithm with linear convergence rate. result The proposed estimator can recover the edges correctly with high probability and is effective on both synthetic and real-world data sets.
This paper speeds up OCSSVM training using SMO.
problem Training One-Class Slab SVMs is slow.
method Uses updated SMO to divide large problems into smaller, analytically solvable subproblems.
result Training OCSSVMs scales better with large datasets.
The paper tackles ranking experts based on their answers to questions, considering statistical and computational challenges.
problem Ranking experts based on their answers to questions, considering isotonic constraints.
method Investigates the existence of statistically optimal and computationally efficient procedures for ranking experts under isotonic constraints.
result Disproves the existence of computational-statistical gaps for the problem.
Solves portfolio optimization with cardinality constraints using column generation.
problem Portfolio optimization with cardinality constraints.
method Column generation method applied to a subset of assets in a master convex quadratic problem, using dual information to propose new assets.
result Solves portfolio optimization problems efficiently with cardinality constraints.
We consider variants of trust-region and cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under mild conditions on the inexact Hessian, and using approximate solution of the corresponding sub-problems, we provide iteration complexity to achieve ε-approximate seco…
Inspired by recent work on convex formulations of clustering (Lashkari & Golland, 2008; Nowozin & Bakir, 2008) we investigate a new formulation of the Sparse Coding Problem (Olshausen & Field, 1997). In sparse coding we attempt to simultaneously represent a sequence of data-vectors sparsely (i.e. sparse approximation (…
Revisits causal inference identifiability with positivity assumption.
problem General identifiability in causal inference without positivity assumption.
method Introduces new algorithm sound and complete under positivity assumption.
result New algorithm connects general identifiability to classical identifiability.
Consider a random vector with finite second moments. If its precision matrix is an M-matrix, then all partial correlations are non-negative. If that random vector is additionally Gaussian, the corresponding Markov random field (GMRF) is called attractive. We study estimation of M-matrices taking the role of inverse sec…
Deep learning predicts lightning strikes with high accuracy.
problem Predicting thunderstorms and lightning strikes accurately.
method Convolutional neural network architecture inspired by UNet++ and ResNet.
result Probability of detection of more than 94% for lightning strikes within 15 minutes.
Consider an unknown smooth function f:[0,1]→R, and say we are given n noisymod1 samples of f, i.e., yi=(f(xi)+ηi)mod1 for xi∈[0,1], where ηi denotes noise. Given the samples (xi,yi)i=1n our goal is to recover smooth, robust estimates of the clean samples $f…
Paper proposes a new method for sparse phase retrieval with fewer measurements.
problem Sparse phase retrieval in various fields.
method Stochastic alternating minimizing method (StormSpar) with HTP algorithm.
result The method recovers sparse signals from fewer measurements than existing methods.
We propose a variable decomposition algorithm -greedy block coordinate descent (GBCD)- in order to make dense Gaussian process regression practical for large scale problems. GBCD breaks a large scale optimization into a series of small sub-problems. The challenge in variable decomposition algorithms is the identificati…
We present a general Markovian framework for order book modeling. Through our approach, we aim at providing a tool enabling to get a better understanding of the price formation process and of the link between microscopic and macroscopic features of financial assets. To do so, we propose a new method of order book repre…
We study the problem of low-rank tensor factorization in the presence of missing data. We ask the following question: how many sampled entries do we need, to efficiently and exactly reconstruct a tensor with a low-rank orthogonal decomposition? We propose a novel alternating minimization based method which iteratively …
Solves optimal stopping problem with Poisson constraints using jumps.
problem Optimal stopping with Poisson constraints and jumps.
method Penalized backward stochastic differential equation (PBSDE) with jumps, decomposition method based on Jacod-Pham, comparison theorem of BSDEs with jumps.
result Solves American option pricing in nonlinear markets with Poisson constraints.
This paper addresses the problem of scalable optimization for L1-regularized conditional Gaussian graphical models. Conditional Gaussian graphical models generalize the well-known Gaussian graphical models to conditional distributions to model the output network influenced by conditioning input variables. While highly …
CNT leverages noisy targets to guide model learning.
problem Learning from noisy or incomplete labels.
method Conditioning model on noisy targets at inference time.
result Model focuses on simpler sub-problems and learns from easier examples first.
Tensors play a central role in many modern machine learning and signal processing applications. In such applications, the target tensor is usually of low rank, i.e., can be expressed as a sum of a small number of rank one tensors. This motivates us to consider the problem of low rank tensor recovery from a class of lin…