Deep networks can approximate functions with fewer learnable parameters than previously thought.
problem High computational costs due to large number of parameters in deep neural networks.
method Theoretical design of ReLU networks with a few intrinsic parameters and numerical experiments.
result ReLU networks with a small number of intrinsic parameters can achieve good approximations of functions.
Improves generalization in learning problems with small parameter method.
problem Improving generalization in learning problems with high-dimensional nonlinear functions.
method Perturbation theory applied to a weakly-controlled gradient system.
result Approximate optimal solutions for improving generalization with small noise.
Optimal B-robust estimate is constructed for multidimensional parameter in drift coefficient of diffusion type process with small noise. Optimal mean-variance robust (optimal V -robust) trading strategy is find to hedge in mean-variance sense the contingent claim in incomplete financial market with arbitrary informatio…
Estimates matrix trace optimization with statistical learning theory.
problem Optimizing trace of parameter-dependent matrices.
method Monte Carlo estimator with bounds derived from epsilon nets and generic chaining.
result Predicts small sampling amount for matrices with small off-diagonal mass.
Small Nijenhuis tensor found on compact manifolds.
problem Finding compact manifolds with small Nijenhuis tensor.
method Provided explicit examples of manifolds with small Nijenhuis tensor.
result Examples of manifolds with small Nijenhuis tensor in various dimensions.
The study bounds the stability of Gaussian mixtures under small perturbations.
problem Stability of Gaussian mixtures under small changes in distribution.
method Deriving an explicit bound on parameter stability of spherical Gaussian Mixture Models (sGMM) in a pre-defined model class.
result Upper bound on parameter distance of close sGMMs to the original sGMM, dependent only on the original model.
We describe a simple fundamental domain for the holonomy group of the boundary unipotent spherical CR uniformization of the figure eight knot complement, and deduce that small deformations of that holonomy group (such that the boundary holonomy remains parabolic) also give a uniformization of the figure eight knot comp…
Study small eigenvalues of Riemann surfaces degenerating with Kähler metrics.
problem Determining small eigenvalues of the Laplacian on degenerating Riemann surfaces.
method Combining heat kernel estimates and Quillen metrics to compute asymptotic behavior of eigenvalues.
result Explicit calculation of small eigenvalues as a function of the parameter.
Improves model accuracy in medical imaging with small datasets using transfer learning.
problem Challenges in training neural networks on small medical imaging datasets.
method Comparison of current techniques, proposing one cycle training, discriminative learning rates, gradual freezing, and parameter modification.
result Transfer learning is crucial for small datasets, especially when images from the same part of the body are available.
Study on colored Jones polynomial of figure-eight knot for complex parameters.
problem Asymptotic behavior of colored Jones polynomial for figure-eight knot.
method Analyzing the asymptotic growth rate of the polynomial for complex parameters with small imaginary part.
result Growth rate of polynomial is related to the Chern-Simons invariant for large real part of the parameter and to the reciprocal of Alexander polynomial for small real part.
New framework assesses regularization norms in ill-posed problems, revealing L2 instability and proposing adaptive fractional RKHS solutions.
problem Comparative analysis of regularization norms in ill-posed problems.
method Small noise analysis framework for Tikhonov and RKHS regularizations.
result Optimal convergence rates achieved with adaptive fractional RKHS, but hyper-parameters decay too fast.
We prove uniqueness of solutions to complex Monge-Ampère equations for small temperature.
problem Proving uniqueness of solutions to complex Monge-Ampère equations.
method Local and global analysis of bounded hyperconvex domains and compact complex manifolds.
result Uniqueness of solutions confirmed for small temperature parameters.
Deep neural network solves portfolio optimization with MGARCH and small transaction costs.
problem Optimizing portfolios with MGARCH and small transaction costs.
method Fixed-point RL algorithm using neural networks.
result NN algorithm shows positive testing performance.
We compare communication efficiencies of two compelling distributed machine learning approaches of split learning and federated learning. We show useful settings under which each method outperforms the other in terms of communication efficiency. We consider various practical scenarios of distributed learning setup and …
A scalable method for Bayesian inference in large linear models.
problem High computational cost in Bayesian linear models for large networks.
method Sample-based inference and g-prior for hyperparameter selection.
result Linearised neural network inference on large datasets (ResNet-18, ResNet-50, U-Net).
Novel approach for SEM in small samples with p>n.
problem Small sample size and p>n issues in factor-based SEM. method Reformulates covariance structure into self-covariance and cross-covariance, defines a feasible set with relative error constraint.
result Improved stability and directional information in small-sample settings.
Study small eigenvalues of Toeplitz operators and their relation to Mabuchi geodesics.
problem Analyzing small eigenvalues of Toeplitz operators on complex projective manifolds.
method Proving the existence of exponentially decaying eigenvalues for Toeplitz operators with specific symbols, and establishing a connection to Mabuchi geodesics.
result Logarithmic distribution of small eigenvalues correlates with Mabuchi geodesics between polarizations.
Deep Neural Networks (DNNs) are usually over-parameterized, causing excessive memory and interconnection cost on the hardware platform. Existing pruning approaches remove secondary parameters at the end of training to reduce the model size; but without exploiting the intrinsic network property, they still require the f…
Simulation reveals relationships in stock market pyramid schemes.
problem Understanding pyramid scheme behavior in stock markets.
method Agent-based simulation with four investor types and parameters.
result Relationships between main fund's rate of return and trend investors' proportion.
This note provides an elementary proof of the folklore fact that draws from a Dirichlet distribution (with parameters less than 1) are typically sparse (most coordinates are small).
S2GPT-PINNs solve PDEs with sparse, small models.
problem Efficiently solving parametric PDEs with minimal resources.
method Sparse and small architecture, mathematically rigorous greedy algorithm, knowledge distillation, down-sampling.
result Achieves high efficiency with significantly fewer parameters.
Leave-one-out cross-validation (LOOCV) can be particularly accurate among cross-validation (CV) variants for machine learning assessment tasks -- e.g., assessing methods' error or variability. But it is expensive to re-fit a model N times for a dataset of size N. Previous work has shown that approximations to LOOCV…
SHADOWCAST generates graphs with user-specified attributes.
problem Controlling graph generation with understandable structures.
method Conditional generative adversarial network guided by Markov model.
result Competitive performance in generating desired graphs.
Training large and highly accurate deep learning (DL) models is computationally costly. This cost is in great part due to the excessive number of trained parameters, which are well-known to be redundant and compressible for the execution phase. This paper proposes a novel transformation which changes the topology of th…
Deep learning (DL) training-as-a-service (TaaS) is an important emerging industrial workload. The unique challenge of TaaS is that it must satisfy a wide range of customers who have no experience and resources to tune DL hyper-parameters, and meticulous tuning for each user's dataset is prohibitively expensive. Therefo…
A new method uses recurrent nets to efficiently estimate SEIR model parameters.
problem Estimating SEIR model parameters is slow and inaccurate with grid search.
method Transform non-differentiable problem to differentiable one using recurrent nets.
result Significantly better parameter estimations with fewer simulations.
We consider the problem of recovering material parameters in a transversely isotropic medium from the qP and qSV waves' travel times, given the axis of isotropy and the material parameters associated to the qSH wave speed. The operators obtained from the pseudolinearization argument are of parabolic type, and so we dis…
We compute a sharp small-time estimate for the price of a basket call under a bi-variate SABR model with both β parameters equal to 1 and three correlation parameters, which extends the work of Bayer,Friz&Laurence [BFL14] for the multivariate Black-Scholes flat vol model. The result follows from the heat kernel on …
Deep learning used for parameter estimation in hard-to-infer models.
problem Parameter estimation in intractable models like max-stable processes.
method Train deep neural networks on simulated data to estimate parameters.
result Deep learning provides accurate and faster parameter estimation.
Study small eigenvalues on Kähler manifolds degenerating with induced metrics.
problem Analyzing the asymptotic rates of small eigenvalues on degenerate Kähler manifolds.
method Combining Li's uniform Skoda inequality with Monge-Ampère equations.
result Established exact asymptotic rates for small eigenvalues.
We consider a market with fractional Brownian motion with stochastic integrals generated by the Riemann sums. We found that this market is arbitrage free if admissible strategies that are using observations with an arbitrarily small delay. Moreover, we found that this approach eliminates the discontinuity of the stocha…
PPI++ uses machine learning predictions to improve inference from small datasets.
problem Efficient inference from small labeled datasets with high-quality predictions.
method Adapts prediction-powered inference (PPI) to compute confidence sets for any parameter dimensionality.
result Improves classical intervals using only labeled data, always yielding better results.
We propose reinforcement learning on simple networks consisting of random connections of spiking neurons (both recurrent and feed-forward) that can learn complex tasks with very little trainable parameters. Such sparse and randomly interconnected recurrent spiking networks exhibit highly non-linear dynamics that transf…
In this paper we propose a Bayesian method for estimating architectural parameters of neural networks, namely layer size and network depth. We do this by learning concrete distributions over these parameters. Our results show that regular networks with a learnt structure can generalise better on small datasets, while f…
Simple private estimators for mean and covariance outperform existing methods.
problem Private estimation of mean and covariance at small sample sizes.
method Differentially private estimators for multivariate sub-Gaussian data.
result Asymptotic error rates match theoretical bounds and outperform previous methods.
Study develops a method to select penalty parameters for sparse neural networks without cross-validation.
problem Selecting optimal penalty parameters for sparse neural networks without cross-validation.
method Established theoretical foundation to bound the infinite norm of the gradient of the loss function at zero.
result Proposed method effectively selects penalty parameters for sparse neural networks.
An investor trades a safe and several risky assets with linear price impact to maximize expected utility from terminal wealth. In the limit for small impact costs, we explicitly determine the optimal policy and welfare, in a general Markovian setting allowing for stochastic market, cost, and preference parameters. Thes…
We improve robust parameter estimation in causal models from observational data.
problem Robustly estimating parameters in linear structural equation models from observational data.
method Extending Sankararaman et al. (2019) to a broader class of models, providing sufficient conditions for robust identifiability.
result For a large set of parameters, robust identifiability holds and existing algorithms achieve robust identifiability.
The effectiveness of utility-maximization techniques for portfolio management relies on our ability to estimate correctly the parameters of the dynamics of the underlying financial assets. In the setting of complete or incomplete financial markets, we investigate whether small perturbations of the market coefficient pr…
The cold posterior effect is explored through PAC-Bayes bounds for small sample sizes.
problem The cold posterior effect in approximate Bayesian inference for small datasets.
method Investigation through PAC-Bayes generalization bounds, focusing on temperature parameter λ.
result The temperature parameter λ in PAC-Bayes bounds captures the cold posterior effect.
We propose CAVIA for meta-learning, a simple extension to MAML that is less prone to meta-overfitting, easier to parallelise, and more interpretable. CAVIA partitions the model parameters into two parts: context parameters that serve as additional input to the model and are adapted on individual tasks, and shared param…
Paper optimizes neural network initialization using SMT solvers.
problem Improving neural network performance through better initialization.
method Reduces initialization to SMT problem solving.
result Proposed method achieves better performance than random initialization.
New method improves nonlinear filtering accuracy with reduced computation.
problem Complex nonlinear filtering with small system noise.
method Asymptotic expansion with ordinary differential equations and Edgeworth-type correction.
result Significantly lower computational cost with improved accuracy.
CDEFs reduce model complexity and uncover time correlations.
problem Model complexity and data efficiency in probabilistic modeling.
method Builds on deep exponential families, ties weights for reduced parameters.
result CDEFs uncover time correlations with fewer parameters.
Estimates for Schrödinger operators on manifolds with bounded Ricci curvature.
problem Quantifying unique continuation for Schrödinger operators on manifolds with specific curvature conditions.
method Proving quantitative unique continuation estimates for Schrödinger operators on manifolds with Ricci curvature bounded below.
result Upper bound for energy range and constant in terms of Ricci curvature and parameters of relatively dense set.
Two algorithms for linear contextual bandits with rare updates achieve optimal regret and efficiency.
problem Linear contextual bandits with infrequent parameter updates.
method Two practical algorithms with O(loglogT) updates, BLCE-G and BLCE. result Minimax-optimal regret with low computational complexity.
In this paper, we consider domain-invariant deep learning by explicitly modeling domain shifts with only a small amount of domain-specific parameters in a Convolutional Neural Network (CNN). By exploiting the observation that a convolutional filter can be well approximated as a linear combination of a small set of dict…
Proposes a deep learning method for modeling dynamic individual-level latent trajectories with changing parameters.
problem Modeling longitudinal data with changing individual-level dynamics parameters.
method Combines deep learning for dimensionality reduction and differential equations for dynamic modeling, allowing different parameters for sub-periods.
result Successfully identifies dynamic parameters and predictors of resilience.