We consider the learning of algorithmic tasks by mere observation of input-output pairs. Rather than studying this as a black-box discrete regression problem with no assumption whatsoever on the input-output mapping, we concentrate on tasks that are amenable to the principle of divide and conquer, and study what are it…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “input-output pairs”
LESS combines local predictors for subsets to learn from heterogeneous input-output pairs.
problem Learning from heterogeneous input-output pairs in populations with varied behavior.
method LESS algorithm: generates subsets, trains local predictors, combines them.
result LESS is highly competitive compared to state-of-the-art methods.
Generates long programs from inputs, optimizing multiple tasks.
problem Creating long programs from input-output pairs.
method Trains a neural network to map state and outputs to next program statement, optimizing multiple tasks concurrently.
result Creates programs twice as long as existing solutions, improving success rate and runtime.
Tensor completion method identifies nonlinear systems from input-output data.
problem Identifying nonlinear functions from input-output data pairs.
method Formulated as tensor completion problem with smoothness regularization and solved using block coordinate descent.
result Provable correct nonlinear system identification under certain conditions.
Improved neural processes with attention for better predictions.
problem Underfitting in Neural Processes.
method Incorporating attention into Neural Processes to improve accuracy and range of functions.
result Significantly improved prediction accuracy and faster training.
A method for learning discontinuous functions using clustering, classification, and regression.
problem Supervised learning with highly nonlinear and discontinuous outputs.
method Three stages: clustering, classification, and separate regression for each class.
result Combining clustering, classification, and regression provides a robust and powerful approach.
Starting from a dataset with input/output time series generated by multiple deterministic linear dynamical systems, this paper tackles the problem of automatically clustering these time series. We propose an extension to the so-called Martin cepstral distance, that allows to efficiently cluster these time series, and a…
Entangled watermarks improve model defense against extraction attacks.
problem Model extraction attacks risk leaking valuable intellectual property.
method Entangled Watermarking Embeddings (EWE) leverages model capacity to overfit to outlier input-output pairs.
result The approach successfully claims model ownership with high confidence and minimal performance loss.
Recursive sketches summarize deep networks, aiding quick analysis and learning.
problem Understanding and analyzing complex deep learning models.
method Developed a recursive sketch mechanism to summarize inputs and outputs of modular deep networks.
result Sketches can identify key components and summarize essential information, even if partially erased.
Output-weighted optimal sampling for Bayesian regression and rare event statistics using few samplesstat.ML
New method selects inputs for Bayesian regression with few samples.
problem Optimal experimental design for high-dimensional inputs.
method Output-weighted optimal sampling using Bayesian regression.
result New criterion considers output values of existing samples.
Physics-informed DeepONets solve PDEs without paired data, predicting solutions quickly.
problem Lack of paired input-output data for solving PDEs.
method Physics-informed DeepONets use automatic differentiation to enforce physical laws as soft penalty constraints.
result Physics-informed DeepONets can solve PDEs without paired data, predicting solutions up to 3 orders of magnitude faster.
TCNs can approximate complex input-output maps with limited memory.
problem Approximating complex input-output maps with limited memory.
method Proved TCNs can approximate a wide class of input-output maps with arbitrary error tolerance.
result Deep ReLU TCNs can approximate input-output maps with finite memory to arbitrary error.
Safe neural networks for input-output specifications.
problem Ensuring machine learning models adhere to input-output constraints.
method Designing constrained predictors and combining them safely.
result Demonstrated on synthetic and real-world datasets.
New method uses sequential statistics for classifier parameter estimation without paired data.
problem Removing need for paired input-output data in classification problems.
method Introduces Caesar Cipher analogy and novel loss function for unsupervised learning.
result Estimates classifier parameters using sequential statistics without paired data.
Despite tremendous progress in outlier detection research in recent years, the majority of existing methods are designed only to detect unconditional outliers that correspond to unusual data patterns expressed in the joint space of all data attributes. Such methods are not applicable when we seek to detect conditional …
New method learns functions without paired data using mediating variables.
problem Learning functions without paired input-output data.
method Mediated Uncoupled Learning: Predicting to approximate .
result Statistical consistency and error bounds of the proposed method.
URNNs are as expressive as general RNNs with ReLU activations.
problem Expressiveness of URNNs compared to general RNNs.
method Input-output equivalence between URNNs and contractive RNNs with ReLU activations.
result URNNs are as expressive as general RNNs with ReLU activations.
Paper tackles active labeling for partial supervision.
problem Accessing stochastic gradients with partial supervision.
method Streaming technique to minimize generalization error.
result Proves minimization of generalization error ratio.
World Input-Output Networkphysics.soc-ph
Economic systems, traditionally analyzed as almost independent national systems, are increasingly connected on a global scale. Only recently becoming available, the World Input-Output Database (WIOD) is one of the first efforts to construct the multi-regional input-output (MRIO) tables at the global level. By viewing t…
Reduces transferability of adversarial examples between models.
problem Transferability of adversarial examples between different models trained on different datasets.
method Investigates the angle between input-output Jacobians of different models to demonstrate relevance and explains asymmetry in terms of gradient norms.
result Simple modification reduces adversarial example transferability between models.
Paper proposes a new ML approach to estimate g-vulnerability without estimating conditional probabilities.
problem Estimating information leakage in black-box systems with large output domains.
method Developed a novel ML-based approach to estimate g-vulnerability without estimating conditional probabilities.
result The proposed method outperforms frequentist approach when dealing with large output domains.
Sparse Stochastic Zeroth-Order Optimization with an Application to Bandit Structured Predictionstat.ML
Stochastic zeroth-order (SZO), or gradient-free, optimization allows to optimize arbitrary functions by relying only on function evaluations under parameter perturbations, however, the iteration complexity of SZO methods suffers a factor proportional to the dimensionality of the perturbed function. We show that in scen…
Convex optimization models predict outputs from inputs via optimization problems.
problem Predicting outputs from inputs using convex optimization models.
method Proposed a heuristic for learning parameters of convex optimization models from datasets.
result Demonstrated the effectiveness of the proposed method on three model classes.
New approach ties loss curvature to model performance in deep learning.
problem Understanding the relationship between loss curvature and model performance in deep learning.
method Empirical analysis of loss Hessians and theoretical results on input-output Jacobians.
result Novel generalization bound in terms of empirical Jacobian.
Unified Bayesian framework for LTV system identification using neural networks and Gaussian Processes.
problem Identifying Linear Time-Varying systems from input-output data.
method Bayesian modeling of impulse response as a stochastic process, using neural networks and Gaussian Processes for inference.
result Framework can infer LTI system properties from a single noisy input-output pair, achieving lower error than classical methods.
Quantifying similarity between data objects is an important part of modern data science. Deciding what similarity measure to use is very application dependent. In this paper, we combine insights from systems theory and machine learning, and investigate the weighted cepstral distance, which was previously defined for si…
This paper investigates how economic shocks propagate and amplify through the input-output network connecting industrial sectors in developed economies. We study alternative models of diffusion on networks and we calibrate them using input-output data on real-world inter-sectoral dependencies for several European count…
Wasserstein Neural Processes improve traditional NPs by using Wasserstein distance.
problem Traditional NPs fail to learn reasonable distributions for certain problem classes.
method Use approximations of Wasserstein distance to overcome limitations of KL divergence.
result Wasserstein Neural Processes maintain benefits of traditional NPs while approximating new function mappings.
We use Bayesian optimization to create efficient adversarial attacks with limited queries.
problem Creating adversarial examples with limited query access.
method Bayesian optimization for query-efficient adversarial attacks.
result Our method reduces query count by up to 80% compared to state-of-the-art methods.
JES optimizes expensive functions by considering joint entropy over input and output spaces.
problem Optimizing expensive functions with limited evaluations.
method Joint Entropy Search (JES) considers joint entropy over input and output spaces.
result JES outperforms other information-theoretic methods in Bayesian optimization.
Hierarchical Clustering and Matrix Completion for the Reconstruction of World Input-Output Tablesstat.ML
Method combines clustering and matrix completion for missing data in I/O tables.
problem Reconstructing missing entries in World Input-Output (I/O) matrices due to data collection issues.
method Hierarchical clustering and Matrix Completion with LASSO-like nuclear norm penalty.
result The method effectively predicts missing values from previous and similar countries' data.
Paper verifies safety of tree ensembles in safety-critical systems.
problem Ensuring safety of machine learning in safety-critical systems.
method Extract equivalence classes and formally verify input-output mappings of tree ensembles.
result Method is practical for tree ensembles trained on low-dimensional data.
Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systemsstat.ML
Deep learning for stochastic systems with multi-fidelity data.
problem Predicting stochastic, high-dimensional, and multi-fidelity systems with uncertainty.
method Probabilistic deep learning with variational inference for implicit distributions.
result Effective surrogate models for stochastic systems with quantified uncertainty.
Nonparametric, Nonasymptotic Confidence Bands with Paley-Wiener Kernels for Band-Limited Functionsstat.ML
The paper creates nonparametric confidence bands for band-limited functions.
problem Estimating confidence bands for band-limited functions with finite samples and unknown noise.
method Uses Paley-Wiener reproducing kernel Hilbert spaces and gradient-perturbation methods.
result Non-asymptotic guarantees for confidence regions without assuming a parametric model.
Theoretical insights into the optimization landscape of over-parameterized shallow neural networkscs.LG
In this paper we study the problem of learning a shallow artificial neural network that best fits a training data set. We study this problem in the over-parameterized regime where the number of observations are fewer than the number of parameters in the model. We show that with quadratic activations the optimization la…
Upstreamness and downstreamness in input-output analysis from local and aggregate informationphysics.soc-ph
New method ranks sectors and countries using local and aggregate I-O data.
problem Ranking sectors and countries in global value chains using incomplete I-O tables.
method Rank- approximation to I-O tables using local and aggregate information.
result Consistently good performance in reconstructing rankings of upstreamness and downstreamness.
Fine-tuning neural networks to guarantee performance on specific examples can also introduce incorrect inputs.
problem Ensuring reliable performance of neural networks on specific examples.
method Using SMT solvers to fine-tune ReLU neural networks to guarantee outcomes on a finite set of particular examples.
result Fine-tuning can introduce incorrect inputs that trigger unexpected performance.
Neural networks adapt to any input dimensionality.
problem Learning functions with inputs of varying dimensions.
method Equivariant neural networks using representation stability.
result Training on fixed dimensions allows extension to any input.
We consider the problem of learning a realization for a linear time-invariant (LTI) dynamical system from input/output data. Given a single input/output trajectory, we provide finite time analysis for learning the system's Markov parameters, from which a balanced realization is obtained using the classical Ho-Kalman al…
Develops neural network approximations for infinite-dimensional input-output maps.
problem Approximating input-output maps between infinite-dimensional spaces.
method Combines neural networks and model reduction techniques.
result Proves convergence of the proposed approximation methodology.
Graph-to-Tree Neural Networks improve structured input-output translation in tasks like semantic parsing and math word problems.
problem Improving performance on tasks like semantic parsing and math word problem solving.
method Graph-to-Tree Neural Networks, consisting of a graph encoder and a hierarchical tree decoder.
result Graph2Tree model outperforms or matches state-of-the-art models on neural semantic parsing and math word problem tasks.
Spanning attack improves black-box attacks with unlabeled data.
problem Query inefficiency in black-box attacks due to high input space dimensionality.
method Proposes spanning attack by constraining adversarial perturbations in a low-dimensional subspace via an auxiliary unlabeled dataset.
result Significantly improves query efficiency of black-box attacks.
We consider the problem of joint modelling of metabolic signals and gene expression in systems biology applications. We propose an approach based on input-output factorial hidden Markov models and propose a structured variational inference approach to infer the structure and states of the model. We start from the class…
New method learns transmission matrices from input-output data.
problem Recovering transmission matrices from input-output data in random media.
method Statistical mechanical formulation and pseudolikelihood maximization.
result Inference protocols can learn transmission couplings.
Within machine learning, the supervised learning field aims at modeling the input-output relationship of a system, from past observations of its behavior. Decision trees characterize the input-output relationship through a series of nested questions, the testing nodes, leading to a set of predictions, th…
Optimal intervention in economic networks modeled as influence maximization, with hard computational problems.
problem Optimal intervention in economic networks modeled as influence maximization.
method Transformed into influence maximization-like form, with theoretical and practical implications.
result Optimal intervention is NP-hard and cannot be approximated to a constant factor in polynomial time.
We consider the problem of learning a high-dimensional multi-task regression model, under sparsity constraints induced by presence of grouping structures on the input covariates and on the output predictors. This problem is primarily motivated by expression quantitative trait locus (eQTL) mapping, of which the goal is …
Sharp limits found for storing and retrieving input-output associations in linear associative memories.
problem Understanding the fundamental limits of storing and retrieving input-output associations in neural networks.
method Study of a minimal linear associative memory model, introducing a decoupled model and using statistical physics to characterize storage capacity.
result Linear associative memory can store up to 1/2 log(p) associations, providing a sharp statistical-physics characterization.