Introduces epistemic deep learning for better uncertainty estimation in neural networks.
problem Uncertainty quantification in deep neural networks.
method Random-set convolutional neural networks with belief function-based loss functions.
result Epistemic approach produces better performance in uncertainty estimation.
New framework models epistemic uncertainty in GNNs using random sets.
problem Uncertainty in graph neural network predictions.
method Introduces a belief function (random set) approach to model epistemic uncertainty in GNNs.
result Demonstrates superior uncertainty quantification on various graph datasets.
RS-NN predicts belief functions for classification, improving accuracy and uncertainty estimation.
problem Improving confidence in machine learning predictions for safety-critical applications.
method Random-Set Neural Network (RS-NN) using random set mathematics.
result RS-NN outperforms state-of-the-art methods in accuracy, uncertainty estimation, and OoD detection.
The paper outlines future work in random sets theory.
problem Developing a theory of statistical reasoning with random sets.
method Generalizing logistic regression, probability laws, and geometric uncertainty.
result A new geometric approach to uncertainty with general random sets.
Physics-informed neural networks (PINNs) [31] use automatic differentiation to solve partial differential equations (PDEs) by penalizing the PDE in the loss function at a random set of points in the domain of interest. Here, we develop a Petrov-Galerkin version of PINNs based on the nonlinear approximation of deep neur…
New framework improves worst-case generalization bounds for stochastic optimization.
problem Challenges in providing generalization guarantees for stochastic optimization algorithms.
method Introduces random set stability and empirically relevant complexity measures to avoid intractable mutual information terms.
result Bounded worst-case generalization error in terms of random set stability and empirically relevant complexity measures.
PAC-Bayesian theory applied to data-dependent hypothesis sets yields uniform generalization bounds.
problem Proving uniform generalization bounds for data-dependent hypothesis sets.
method Applying PAC-Bayesian framework on 'random sets' and considering data-dependent hypothesis sets.
result Data-dependent uniform generalization bounds are proven, providing tighter and unified results.
Deep learning models are vulnerable to external attacks. In this paper, we propose a Reinforcement Learning (RL) based approach to generate adversarial examples for the pre-trained (target) models. We assume a semi black-box setting where the only access an adversary has to the target model is the class probabilities o…
New gradient coding schemes reduce decoding error in both random and adversarial straggler settings.
problem Creating efficient approximate gradient coding schemes for distributed optimization.
method Introduced novel approximate gradient codes based on expander graphs, achieving optimal decoding coefficients.
result Achieved nearly optimal error in random setting and nearly half the error in adversarial setting compared to existing codes.
We present a first attempt to elucidate a theoretical and empirical approach to design the reward provided by a natural language environment to some structure learning agent. To this end, we revisit the Information Theory of unsupervised induction of phrase-structure grammars to characterize the behavior of simulated a…
We define two non-linear operations with random (not necessarily closed) sets in Banach space: the conditional core and the conditional convex hull. While the first is sublinear, the second one is superlinear (in the reverse set inclusion ordering). Furthermore, we introduce the generalised conditional expectation of r…
Injectivity of ReLU networks studied using statistical physics.
problem When can the input of a ReLU neural network be inferred from its output?
method Connection to spherical integral geometry and statistical physics.
result Replica symmetry-breaking theory and Gordon's min--max theorem provide insights into the injectivity threshold.
The paper addresses uncertainties in spectral clustering of corrupted data.
problem Uncertainties in spectral clustering due to measurement errors and missing data.
method Mathematical framework based on random set theory for Monte Carlo approximation of expected clusterings.
result Consistent quantities of interest for evaluating clusterings in corrupted data.
Several algorithms have been proposed to filter information on a complete graph of correlations across stocks to build a stock-correlation network. Among them the planar maximally filtered graph (PMFG) algorithm uses 3n−6 edges to build a graph whose features include a high frequency of small cliques and a good clust…
When assessing group solvency, an important question is to what extent intragroup transfers may be considered, as this determines to which extent diversification can be achieved. We suggest a framework to describe the families of admissible transfers that range from the free movement of capital to excluding any transac…
A determinantal point process (DPP) is a random process useful for modeling the combinatorial problem of subset selection. In particular, DPPs encourage a random subset Y to contain a diverse set of items selected from a base set Y. For example, we might use a DPP to display a set of news headlines that are relevant to…
New research shows imputation and regression together can predict better than separate steps.
problem Predicting with data missing values without strong assumptions.
method Proposes a joint imputation and regression approach using NeuMiss neural network.
result Joint imputation and regression outperforms separate imputation and regression methods.
Study relaxes identification assumptions for natural direct effects in non-randomized settings.
problem Identifying causal direct effects under unmeasured confounding.
method Developed relaxed conditions for identifying natural direct effects in non-randomized settings.
result Identified natural direct effect under unmeasured confounding conditions.
We provide tight upper and lower bounds on the complexity of minimizing the average of m convex functions using gradient and prox oracles of the component functions. We show a significant gap between the complexity of deterministic vs randomized optimization. For smooth functions, we show that accelerated gradient de…
We explore ways in which the covariance ellipsoid B={v∈Rd:E<X,v>2≤1} of a centred random vector X in Rd can be approximated by a simple set. The data one is given for constructing the approximating set consists of X1,...,XN that are independent and distribute…
Nearly all Gaussian points in high dimensions lie on a common ellipsoid.
problem Finding an ellipsoid that fits a large set of Gaussian points in high dimensions.
method Analyzing a random set of Gaussian points and proving a bound on their concentration.
result The bound nearly confirms a conjecture about fitting Gaussian points to ellipsoids.
We resolve the fundamental problem of online decoding with general nth order ergodic Markov chain models. Specifically, we provide deterministic and randomized algorithms whose performance is close to that of the optimal offline algorithm even when latency is small. Our algorithms admit efficient implementation vi…
Sublinear functionals of random variables are known as sublinear expectations; they are convex homogeneous functionals on infinite-dimensional linear spaces. We extend this concept for set-valued functionals defined on measurable set-valued functions (which form a nonlinear space), equivalently, on random closed sets. …
We consider two random group models: the hexagonal model and the square model, defined as the quotient of a free group by a random set of reduced words of length four and six respectively. Our first main result is that in this model there exists a sharp density threshold for Kazhdan's Property (T) and it equals 1/3. Ou…
We introduce a new random group model called the square model: we quotient a free group on n generators by a random set of relations, each of which is a reduced word of length four. We prove, as in the Gromov density model, that for densities >21 a random group in the square model is trivial with overwhel…
Solves super-hedging for financial models with uncertain prices.
problem Super-hedging European or Asian options in discrete-time models with uncertain prices.
method Numerical procedure under AIP condition to compute infimum price.
result Solves super-hedging problem under weak no-arbitrage condition.
The classical discrete time model of proportional transaction costs relies on the assumption that a feasible portfolio process has solvent increments at each step. We extend this setting in two directions, allowing for convex transaction costs and assuming that increments of the portfolio process belong to the sum of a…
Tensors play a central role in many modern machine learning and signal processing applications. In such applications, the target tensor is usually of low rank, i.e., can be expressed as a sum of a small number of rank one tensors. This motivates us to consider the problem of low rank tensor recovery from a class of lin…
Recently, researchers utilize Knowledge Graph (KG) as side information in recommendation system to address cold start and sparsity issue and improve the recommendation performance. Existing KG-aware recommendation model use the feature of neighboring entities and structural information to update the embedding of curren…
This paper improves signal reconstruction using determinantal sampling from random nodes.
problem Approximating square-integrable functions from random node evaluations.
method Combines determinantal point processes and mixtures thereof for RKHS-adapted approximations.
result Proves mean-square guarantees in L2 norm and shows faster convergence rates. AI-assisted interviews allow respondents to describe experiences naturally, but mapping those accounts into structured survey variables is fallible.
problem Mapping AI-assisted interview responses into structured survey variables is fallible.
method Adaptive Matrix Validation (AMV) is proposed, which involves mapping responses into tabular data and using a small set of structured questions for statistical adjustment.
result The estimator calibrates mapped values using validation answers from other respondents and corrects remaining error with validation answers observed for the target respondent.
Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.
problem Lack of equivariance in neural network representations of other neural networks.
method Represent neural networks as computational graphs and use graph neural networks to preserve permutation symmetry.
result Single model encodes diverse neural architectures, outperforming state-of-the-art methods.
Neural networks can approximate functions uniformly across various measures.
problem Universal approximation of functions across different probability measures.
method Proving neural networks are dense in Orlicz spaces, extending classical theorems.
result Neural networks uniformly approximate functions for weakly compact families of measures.
Optimal rates for shallow ReLU networks in nonparametric regression.
problem Approximating smooth and non-smooth functions with shallow ReLU networks.
method Analysis of shallow ReLUk neural networks, using variation norms and deep learning theory. result Optimal approximation rates for shallow ReLU networks in nonparametric regression.
Novel framework explains generalization in deep neural networks.
problem Understanding and improving generalization in deep neural networks.
method Topological Quantum Neural Networks as the semi-classical limit of Deep Neural Networks.
result Demonstrates that the perceptron, viewed as the semi-classical limit, achieves similar results to standard neural networks without training.
Investigates how neural network graph structure impacts predictive performance.
problem Lack of understanding between neural network graph structure and predictive performance.
method Developed relational graph representation to analyze neural networks, identifying a 'sweet spot' for improved performance.
result Identified a 'sweet spot' in relational graph structure that significantly improves neural network predictive performance.
Equivariant neural networks use symmetry to interpret complex data.
problem Interpreting and understanding the behavior of equivariant neural networks.
method Decompose layers into simple representations and analyze nonlinear activation functions.
result Equivariant neural networks can be interpreted using a filtration generalizing Fourier series.
Two new criteria help understand the advantage of deep neural networks.
problem Understanding the advantage of deepening neural networks.
method Proposed two new criteria to evaluate the expressivity of functions computable by deep neural networks.
result Increasing layers is more effective than increasing units in improving the expressivity of deep neural networks.
Paper benchmarks quantum neural networks against classical ones for binary classification tasks.
problem Comparing quantum neural networks with classical ones for binary classification.
method Evaluated with two toy examples, focusing on model complexity and training data size.
result EQNN and QNN outperform ENN and DNN for smaller parameter sets and training data samples.
The paper proves consistency of neural networks with regularization.
problem Overfitting in neural networks with large scale data.
method Theoretical framework of neural networks with regularization, sieves method, and minimal neural networks theory.
result The estimated neural network converges to the true underlying function as sample size increases.
New proof shows neural networks can represent all multivariate functions.
problem Representing all multivariate functions with neural networks.
method Proved that three-layer neural networks can represent both continuous and discontinuous functions.
result Three-layer neural networks can represent all multivariate functions, including discontinuous ones.
New learning rules for wide neural networks without backpropagation.
problem Training wide neural networks efficiently and without backpropagation.
method Input-weight alignment driven by gradient descent in the NTK regime.
result Biologically-motivated learning rules equivalent to backpropagation in wide networks.
DeepWeightFlow generates diverse neural network weights efficiently.
problem Generating complete neural network weights efficiently and accurately.
method Flow Matching in weight space with Git Re-Basin and TransFusion.
result DeepWeightFlow generates high-accuracy neural networks without fine-tuning.
Neural networks improve nonparametric regression with measurement errors.
problem Nonparametric regression with measurement errors.
method Proposes a neural network design using FNN, normalizing flow, and inference network.
result Neural network approach is more flexible and superior or comparable to classical methods.
Graph Metanetworks process diverse neural architectures efficiently.
problem Processing diverse neural architectures efficiently.
method Builds metanetworks using graph neural networks to process graphs representing input neural networks.
result Proves GMNs are expressive and equivariant to parameter permutation symmetries.
A powerful network teaches a weak one, improving its performance.
problem Improving the performance of a weak neural network using a more powerful one.
method During training, a weak network learns features from a strong network to minimize feature distance.
result A weak neural network can increase its performance by learning from a more powerful network.
Affine spiking neural networks learn efficiently and generalize well.
problem Learning with spiking neural networks, especially with positive weights.
method Affine encoders and decoders, continuous parameter dependence, gradient-based training.
result Affine spiking neural networks can approximate shallow ReLU networks and generalize well.
Unified platform SOCRATES for neural network analysis.
problem Analyzing neural networks for bugs and fairness.
method Standardized format, assertion language, and multiple analysis algorithms.
result Unified platform for neural network analysis.