Formula found for neural network error with fixed weights.
problem Understanding error in neural networks with fixed weights.
method Provided an explicit formula for approximation error.
result Explicit formula for neural network error with fixed weights.
Propose an XMSE-aware mixed estimator for EB that interpolates between ML and EB shrinkage.
problem Kernel-based EB estimation may be worse than ML when the kernel is poorly aligned with the true parameter.
method An XMSE-aware mixed estimator that interpolates between ML and EB shrinkage.
result Fixed-weight XMSE is a scalar quadratic, yielding a closed-form oracle mixing weight that is no worse than both ML and the base EB estimator at the XMSE scale.
Transformers can emulate various algorithms by prompting, proving universality.
problem How to emulate algorithms using fixed-weight Transformers.
method Two modes of in-context algorithm emulation: task-specific and prompt-programmable. Constructing prompts that encode algorithm parameters into token representations.
result Fixed-weight Transformers can emulate a broad class of algorithms via prompts.
We construct the first examples of families of bad Riemannian orbifolds which are isospectral with respect to the Laplacian but not isometric. In our case these are particular fixed weighted projective spaces equipped with isospectral metrics obtained by a generalization of Schüth's version of the torus method.
On a manifold with a projective connection we canonically assign a second order differential operator acting on the algebra of all densities to any tensor density Sij of fixed weight λ. In particular, this implies that on any projectively connected manifold, a `bracket' (symmetric biderivation) on the algebra of…
NTK-SAP improves neural network pruning by aligning training dynamics.
problem Improving neural network pruning to reduce training time and memory.
method Prune connections based on the spectrum of the Neural Tangent Kernel (NTK), using multiple random weight realizations and random inputs.
result Empirically, NTK-SAP achieves better performance than all baselines on multiple datasets.
Graph classification receives a great deal of attention from the non-Euclidean machine learning community. Recent advances in graph coarsening have enabled the training of deeper networks and produced new state-of-the-art results in many benchmark tasks. We examine how these architectures train and find that performanc…
Paper derives inequalities for eigenvalues of Witten-Laplacian under fixed volume constraint.
problem Eigenvalue inequalities of Witten-Laplacian on bounded domains.
method Rearrangement technique and trial functions under fixed weighted volume constraint.
result Several isoperimetric inequalities for eigenvalues of Witten-Laplacian.
Paper proves a new isoperimetric inequality for Steklov eigenvalues.
problem Finding a new isoperimetric inequality for Steklov eigenvalues.
method Proving a Brock-type inequality under specific conditions.
result Extension of Brock's classical result to Witten-Laplacian.
Compact character varieties of punctured spheres are proven.
problem Compact relative SO0(2,q)-character varieties of punctured spheres. method Non-abelian Hodge correspondence and Geometric Invariant Theory (GIT).
result Proves the existence of compact, totally non-hyperbolic character varieties.
This paper optimizes portfolio selection by penalizing tracking error, improving Sharpe ratio.
problem Optimizing portfolio allocation with a penalty for deviation from a reference portfolio.
method Formulated as a McKean-Vlasov control problem, provides explicit solutions and asymptotic expansions.
result The penalized portfolio strategy outperforms standard mean-variance and reference portfolios in most cases.
Reservoir Computing (RC) refers to a Recurrent Neural Networks (RNNs) framework, frequently used for sequence learning and time series prediction. The RC system consists of a random fixed-weight RNN (the input-hidden reservoir layer) and a classifier (the hidden-output readout layer). Here we focus on the sequence lear…
The study examines stable regions in weighted manifolds with boundary properties.
problem Studying stable regions in weighted manifolds with boundary properties.
method Using deformations constructed from parallel vector fields tangent to the boundary, the study deduces rigidity properties for stable sets.
result The classification of stable sets in some Riemannian cylinders and uniqueness results for minimizers.
Retrieving the most similar objects in a large-scale database for a given query is a fundamental building block in many application domains, ranging from web searches, visual, cross media, and document retrievals. State-of-the-art approaches have mainly focused on capturing the underlying geometry of the data manifolds…
EHVI outperforms scalarized EI in MOBO for molecule design.
problem Benchmarking MOBO strategies for molecule design.
method Compared EHVI against fixed-weight scalarized EI in MOBO.
result EHVI consistently outperforms scalarized EI in molecular optimization tasks.
The computational demands of computer vision tasks based on state-of-the-art Convolutional Neural Network (CNN) image classification far exceed the energy budgets of mobile devices. This paper proposes FixyNN, which consists of a fixed-weight feature extractor that generates ubiquitous CNN features, and a conventional …
We present a meta-learning approach for adaptive text-to-speech (TTS) with few data. During training, we learn a multi-speaker model using a shared conditional WaveNet core and independent learned embeddings for each speaker. The aim of training is not to produce a neural network with fixed weights, which is then deplo…
We propose to execute deep neural networks (DNNs) with dynamic and sparse graph (DSG) structure for compressive memory and accelerative execution during both training and inference. The great success of DNNs motivates the pursuing of lightweight models for the deployment onto embedded devices. However, most of the prev…
Data clustering has received a lot of attention and numerous methods, algorithms and software packages are available. Among these techniques, parametric finite-mixture models play a central role due to their interesting mathematical properties and to the existence of maximum-likelihood estimators based on expectation-m…
Proposes a new sampling policy for ranking and selection problems.
problem Improving ranking and selection in adaptive sampling policies.
method Annealed entropic allocation, using soft-min weights and saddlepoint corrections.
result Consistently competitive performance in various settings.
Traffic forecasting is of great importance to transportation management and public safety, and very challenging due to the complicated spatial-temporal dependency and essential uncertainty brought about by the road network and traffic conditions. Latest studies mainly focus on modeling the spatial dependency by utilizi…
On-device CNN inference for real-time computer vision applications can result in computational demands that far exceed the energy budgets of mobile devices. This paper proposes FixyNN, a co-designed hardware accelerator platform which splits a CNN model into two parts: a set of layers that are fixed in the hardware pla…
Annealed Entropic Allocation improves ranking and selection by mitigating hard switching and improving finite-budget discrimination.
problem Sequential budget allocation in ranking and selection
method Annealed weighted soft-min framework
result Surrogate converges uniformly to the hard minimum, soft-min weights concentrate on active challengers, and target allocation map is continuous.
Recently, reinforcement learning models have achieved great success, completing complex tasks such as mastering Go and other games with higher scores than human players. Many of these models collect considerable data on the tasks and improve accuracy by extracting visual and time-series features using convolutional neu…
New methods train neural networks without changing weights, achieving similar or higher performance.
problem Training neural networks efficiently with randomly initialized weights.
method Switching connections on and off, flipping weights' signs, minimizing changed connections.
result Achieves similar or higher performance with less computational cost than training all weights.
Protein function prediction is the important problem in modern biology. In this paper, the un-normalized, symmetric normalized, and random walk graph Laplacian based semi-supervised learning methods will be applied to the integrated network combined from multiple networks to predict the functions of all yeast proteins …
The paper introduces a portfolio construction method using Black-Litterman model and factors.
problem Developing an efficient portfolio construction method using Black-Litterman model and factors.
method The method involves selecting 20 factors based on global market, asset class, and stock characteristics, applying various weight allocation methods including Black-Litterman model, and incorporating deep learning for dynamic weight updates.
result The model using Black-Litterman and deep learning outperforms other weight allocation schemes.
We consider the problem of learning a one-hidden-layer neural network with non-overlapping convolutional layer and ReLU activation, i.e., f(Z,w,a)=∑jajσ(wTZj), in which both the convolutional weights w and the output weights a are paramete…
This paper shows neural networks can solve complex graph problems efficiently.
problem Solving exact maximum flow computation and minimum spanning tree problems.
method Introduces Max-Affine Arithmetic Programs and shows equivalence to neural networks.
result Two combinatorial optimization problems can be solved with polynomial-size neural networks.
New algorithm for training GNNs with learned weights.
problem Optimal sampling for GNNs with learned weights is intractable.
method Formulated as an adversary bandit problem, optimizing exploration and exploitation.
result Asymptotically approaches optimal variance within a factor of 3.
Interpolated-MLPs control inductive bias for better performance in low-compute tasks.
problem Low-compute performance gap between MLPs and CNNs.
method Introduced Interpolated MLP (I-MLP) approach to control inductive bias incrementally.
result Continuous logarithmic relationship between inductive bias and performance in low-compute tasks.
Scattering networks are a class of designed Convolutional Neural Networks (CNNs) with fixed weights. We argue they can serve as generic representations for modelling images. In particular, by working in scattering space, we achieve competitive results both for supervised and unsupervised learning tasks, while making pr…
AlphaForge mines and dynamically combines alpha factors for better investment performance.
problem Inconsistency and inflexibility of fixed factor weights in alpha factor mining.
method Generative-predictive neural network for factor generation and dynamic weight adjustment.
result Demonstrated superior performance in formulaic alpha factor mining and portfolio returns.
Optimal model averaging for conditional generative models improves performance across various data types.
problem Multiple plausible generators for conditional distributions can vary in performance.
method Sample-based maximum mean discrepancy, static model averaging, and mixture-of-experts model averaging.
result MoEMA improves over competing baselines across various data types.
Estimates TV distance between autoregressive models under different access models.
problem Estimating the total variation distance between two autoregressive distributions.
method Three access models: sample access, logit access, and noisy logit access; provides query complexity for each.
result Improved query complexity for estimating TV distance in autoregressive models.
The condition number predicts efficient information encoding in neural units, aiding model fine-tuning.
problem Efficient information encoding in neural units for various tasks and input modalities.
method Linking the condition number to the log-volume scaling factor and entropy of the output distribution.
result High condition number indicates efficient encoding, reducing overall information transfer.
Paper presents a modular RL framework for Forex trading, addressing limitations of prior studies.
problem Challenges in applying RL to Forex trading, including unrealistic environments, simplified rewards, and restricted action spaces.
method Integrates three components: a friction-aware execution engine, a decomposable reward architecture, and a discrete action interface.
result Empirical evaluation shows strong non-monotonic reward interactions and optimal Sharpe ratio with the full reward configuration.
AutoKE automates embedding physical knowledge into neural networks for complex engineering problems.
problem Complex physical equations in engineering problems.
method AutoKE framework using deep neural networks, equation parsing, automatic differentiation, adaptive weights, and NAS.
result Automatically embeds physical knowledge into neural networks for complex equations efficiently.
Study connects Gaussian processes and regularization for sequence-function mappings.
problem Understanding and interpreting sequence-function maps in biology.
method Relates Gaussian process priors, regularization, and gauge fixing in overparameterized weight space.
result Established the relationship between regularized regression and Gaussian processes in function space.
This paper introduces a new metric to improve the performance of AMMs over centralised exchanges.
problem Lack of a precise metric to compare AMM performance with centralised exchanges.
method Introduces Rebalancing-versus-Rebalancing (RVR) to measure AMM performance more accurately.
result AMMs can offer superior execution and rebalancing efficiency compared to centralised exchanges, even with low fees.
New research explores using exponential activation functions in neural networks, achieving convergence with over-parameterization.
problem Achieving neural network convergence with over-parameterization using exponential activation functions.
method Defined a neural function using an exponential activation function, initialized weights with random Gaussian distributions, and used gradient descent to find optimal weights.
result Gradient descent can find a weight matrix such that the neural function's output is within ε of the labels with high probability.
Introduces factor risk measures to assess risk relative to multiple factors.
problem Measuring risk relative to multiple factors.
method Introduces a double-argument mapping as a risk measure to assess risk relative to a vector of factors.
result Characterizes various types of factor risk measures including distortion, quantile, linear, and coherent measures.
Paper characterizes star-shaped risk measures and their properties.
problem Characterizing risk measures in the presence of liquidity risk and competitive delegation.
method Characterization of star-shaped risk measures, study of their properties.
result Star-shaped risk measures include all practically used risk measures.
Develops a new method for risk diversification using dynamic risk measures.
problem Dynamic risk diversification in investment portfolios.
method Introduces dynamic risk contributions and a recursive optimization approach for coherent dynamic distortion risk measures.
result Dynamic risk budgeting strategies can be solved using deep learning.
New risk measure considers horizon risk and interest rate uncertainty.
problem Dynamic risk evaluation considering horizon risk and interest rate uncertainty.
method Introduced a risk measure based on generalized Tsallis entropy.
result New q-entropic risk measure quantifies capital requirement.
Study examines risk premium convergence rates in risk sharing contracts.
problem Analyzing risk premium convergence rates in risk sharing contracts.
method Examines the limiting behavior of risk premium associated with Pareto optimal risk sharing contracts under general law-invariant risk measures.
result Risk premium convergence rate is typically n1/2, not n. Optimal risk sharing found for heterogeneous risk attitudes using distortion risk measures.
problem Risk sharing in economies with diverse risk attitudes.
method Modeling preferences with distortion risk measures, using comonotonic and counter-monotonic principles.
result Optimal risk sharing strategies identified based on risk attitudes, reducing the n-agent problem to a two-agent formulation. This paper extends risk parity to continuous-time, solving risk budgeting problems.
problem Achieving robust risk across different assets in continuous-time.
method Characterizing risk contributions and solving risk budgeting problems using continuous-time terminal variance.
result Risk contributions and risk budgets can be represented as predictable processes in continuous-time.