SRFRN accelerates image super-resolution using shallow residual units.
problem High computational complexity and time in deep learning image super-resolution.
method SRFRN uses a bicubic interpolated low-resolution image and residual representative units (RFR) for faster and more efficient high-resolution image reconstruction.
result SRFRN achieves superior performance and faster execution time compared to existing methods.
Algorithm learns two-layer residual units using ReLU activations from samples.
problem Learning two-layer residual units from samples.
method Design layer-wise objectives as functionals, formulate ERM as QP, solve using LP, prove statistical consistency.
result Strong statistical consistency and robustness of the algorithm.
We regard pre-trained residual networks (ResNets) as nonlinear systems and use linearization, a common method used in the qualitative analysis of nonlinear systems, to understand the behavior of the networks under small perturbations of the input images. We work with ResNet-56 and ResNet-110 trained on the CIFAR-10 dat…
This paper provides estimation and inference methods for a conditional average treatment effects (CATE) characterized by a high-dimensional parameter in both homogeneous cross-sectional and unit-heterogeneous dynamic panel data settings. In our leading example, we model CATE by interacting the base treatment variable w…
The covariance matrix is formulated in the framework of a linear multivariate ARCH process with long memory, where the natural cross product structure of the covariance is generalized by adding two linear terms with their respective parameter. The residuals of the linear ARCH process are computed using historical data …
Study on residual Monge-Ampère mass of complex functions with directional Lipschitz continuity.
problem Estimating the residual Monge-Ampère mass of plurisubharmonic functions.
method General decomposition formula under Sasakian structure, L1-apriori estimate, upper-bound estimate on residual mass. result Upper-bound estimate on residual mass for uniformly directional Lipschitz continuity confirmed.
Speeds up deep neural networks training by 10x using GPU concurrency.
problem Training deep residual neural networks efficiently.
method Layer-wise parallel training with GPU concurrency and Nonlinear Multigrid.
result 10.2x speedup over traditional techniques.
Better neural arithmetic logic units improve cell counting model generalization.
problem Neural networks struggle with high cell counts outside training data range.
method Introduced Neural Arithmetic Logic Units (NALU) for arithmetic operations in existing architectures.
result Improved cell counting accuracy for higher numeric ranges with better generalization.
RDL-Net improves speech enhancement with fewer parameters and better performance.
problem Improving speech enhancement with fewer parameters and better performance.
method Proposes RDL-Net, a CNN combining residual and dense aggregations without over-allocating parameters.
result RDL-Net achieves higher speech enhancement performance with fewer parameters and lower computational requirements.
In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be represented by a maximum of a number of functions that can be realized by a shallow network with a …
We present a self-consistent model for explosive financial bubbles, which combines a mean-reverting volatility process and a stochastic conditional return which reflects nonlinear positive feedbacks and continuous updates of the investors' beliefs and sentiments. The conditional expected returns exhibit faster-than-exp…
New proof for weak mixing in polygonal billiards.
problem Proving weak mixing in polygonal billiards.
method Using Baire category and eigenvalue analysis.
result Billiard flow is weakly mixing for non-rational polygons.
Causalfe estimates treatment effects in panel data with fixed effects.
problem Spurious heterogeneity in treatment effect estimates due to fixed effects in panel data.
method CFFE approach with node-level residualization during tree construction.
result Validates the estimator's performance through simulation studies.
New framework analyzes deep neural networks using feature probabilities.
problem Degenerate situation in over-parameterized DNNs.
method Mean-field framework representing DNNs by feature probabilities and functions.
result Global convergence proof for over-parameterized Res-Net training.
In this paper, we study the problem of node representation learning with graph neural networks. We present a graph neural network class named recurrent graph neural network (RGNN), that address the shortcomings of prior methods. By using recurrent units to capture the long-term dependency across layers, our methods can…
Training deep recurrent neural network (RNN) architectures is complicated due to the increased network complexity. This disrupts the learning of higher order abstracts using deep RNN. In case of feed-forward networks training deep structures is simple and faster while learning long-term temporal information is not poss…
A major contributing factor to the recent advances in deep neural networks is structural units that let sensory information and gradients to propagate easily. Gating is one such structure that acts as a flow control. Gates are employed in many recent state-of-the-art recurrent models such as LSTM and GRU, and feedforwa…
We revisit the initialization of deep residual networks (ResNets) by introducing a novel analytical tool in free probability to the community of deep learning. This tool deals with non-Hermitian random matrices, rather than their conventional Hermitian counterparts in the literature. As a consequence, this new tool ena…
A residual network (or ResNet) is a standard deep neural net architecture, with state-of-the-art performance across numerous applications. The main premise of ResNets is that they allow the training of each layer to focus on fitting just the residual of the previous layer's output and the target output. Thus, we should…
Multivariate time-series modeling and forecasting is an important problem with numerous applications. Traditional approaches such as VAR (vector auto-regressive) models and more recent approaches such as RNNs (recurrent neural networks) are indispensable tools in modeling time-series data. In many multivariate time ser…
Itô processes are the most common form of continuous semimartingales, and include diffusion processes. This paper is concerned with the nonparametric regression relationship between two such Itô processes. We are interested in the quadratic variation (integrated volatility) of the residual in this regression, over a un…
AGCA approximates angular variation on the unit sphere, reducing extremal dependence problems to eigenanalysis.
problem Approximating angular variation in multivariate extremes.
method Anchored geodesic component analysis (AGCA) approximates angular variation by great subspheres constrained to pass through a chosen reference direction.
result AGCA finds concentrated tail directions in daily equity-portfolio losses, explaining about 91% of anchored variation.
Celiac Disease (CD) is a chronic autoimmune disease that affects the small intestine in genetically predisposed children and adults. Gluten exposure triggers an inflammatory cascade which leads to compromised intestinal barrier function. If this enteropathy is unrecognized, this can lead to anemia, decreased bone densi…
Study on deep multi-head self-attention dynamics, proving homogenized limits under specific scalings.
problem Understanding the behavior of deep multi-head self-attention models as depth increases.
method Random model of deep multi-head self-attention, viewing depth as time, and analyzing the residual stream as a particle system.
result Homogenized limit of the dynamics, leading to deterministic or stochastic behavior depending on scaling, with implications for representation collapse.
Earthquake signal detection is at the core of observational seismology. A good detection algorithm should be sensitive to small and weak events with a variety of waveform shapes, robust to background noise and non-earthquake signals, and efficient for processing large data volumes. Here, we introduce the Cnn-Rnn Earthq…
It is a known fact that training recurrent neural networks for tasks that have long term dependencies is challenging. One of the main reasons is the vanishing or exploding gradient problem, which prevents gradient information from propagating to early layers. In this paper we propose a simple recurrent architecture, th…
The paper investigates how target normalization and momentum affect dying ReLUs in neural networks.
problem Understanding and mitigating the dying ReLU problem in neural networks.
method Empirical analysis and theoretical modeling of a discrete-time linear autonomous system.
result Target variance plays a crucial role in the dying ReLU phenomenon, and momentum exacerbates this issue.
A convolutional sequence to sequence non-intrusive load monitoring model is proposed in this paper. Gated linear unit convolutional layers are used to extract information from the sequences of aggregate electricity consumption. Residual blocks are also introduced to refine the output of the neural network. The partiall…
Bayesian MS-VAR model for pricing equity-linked life insurance products.
problem Pricing and hedging equity-linked life insurance products on maximum of several assets.
method Introduces Bayesian Markov-Switching Vector Autoregressive (MS-VAR) process to model economic variables and insured's lifetime.
result Obtains net single premiums and hedging formulas for equity-linked life insurance products.
AReLU uses attention-based rectification to improve neural network performance.
problem Improving neural network performance through better activation functions.
method Integrates attention mechanism with rectified linear unit (ReLU) to learn and scale feature maps.
result AReLU significantly boosts performance of most network architectures with minimal changes.
Abstract: Non-residually finite hyperbolic groups imply non-residually finite rigid hyperbolic groups.
problem Existence of non-residually finite hyperbolic groups
method Direct implication
result Existence of non-residually finite rigid hyperbolic groups
We have modeled the employment/population ratio in the largest developed countries. Our results show that the evolution of the employment rate since 1970 can be predicted with a high accuracy by a linear dependence on the logarithm of real GDP per capita. All empirical relationships estimated in this study need a struc…
Okun's law for the biggest developed countries is re-estimated using the most recent data on real GDP per capita and the rate of unemployment. Our results show that the change in unemployment rate can be predicted with a high accuracy. The link needs the introduction of a structural break which might be caused by the c…
Residual finiteness is known to be an important property of groups appearing in combinatorial group theory and low dimensional topology. In a recent work [2] residual finiteness of quandles was introduced, and it was proved that free quandles and knot quandles are residually finite. In this paper, we extend these resul…
In this note, residual finiteness of quandles is defined and investigated. It is proved that free quandles and knot quandles of tame knots are residually finite and Hopfian. Residual finiteness of quandles arising from residually finite groups (conjugation, core and Alexander quandles) is established. Further, residual…
Develops a local Fokker--Planck geometric framework for more accurate score estimation.
problem Inaccurate estimation of score function in non-linear, state-dependent drifts.
method Local Fokker--Planck geometric framework, time change to cumulative-variance coordinate, heat-ball mean-value representations, exact high-dimensional sampling.
result Exact local mean-value representations for the score and density, improved accuracy in low-density regions.
Every non-trivial knot group is fully residually perfect.
problem Understanding the residual properties of knot groups.
method Analyzing the residual properties of knot groups using group theory.
result Every non-trivial knot group is fully residually perfect.
Enhanced TSFMs improve time series forecasting accuracy and reliability.
problem Variance, bias, and uncertainty in TSFMs' predictions on real data.
method Statistical and ensemble techniques including bagging, stacking, residual modeling, and prediction intervals.
result Hybrid models consistently outperform standalone TSFMs across multiple horizons.
We seek to improve the data efficiency of neural networks and present novel implementations of parameterized piece-wise polynomial activation functions. The parameters are the y-coordinates of n+1 Chebyshev nodes per hidden unit and Lagrangian interpolation between the nodes produces the polynomial on [-1, 1]. We show …
Residual flows are shown to approximate MMD well.
problem Lack of theoretical understanding of normalizing flows' expressiveness.
method Proved residual flows are universal approximators in MMD.
result Residual flows can approximate MMD with a bounded number of blocks.
Let p be a prime. In this paper, we classify the geometric 3-manifolds whose fundamental groups are virtually residually p. Let M=M3 be a virtually fibered 3-manifold. It is well-known that G=π1(M) is residually solvable and even residually finite solvable. We prove that G is always virtually residually p…
Researchers identify critical protein residues using advanced graph theory.
problem Identifying essential residues in proteins for function.
method Learning Random Geometric Graphs (RGG) with Cramer's V correlation and organic thresholding.
result Advanced RGG methods accurately identify critical residues compared to existing techniques.
Defines Wodzicki residue using groupoids and fibered distributions.
problem Defining and understanding the Wodzicki residue in noncommutative geometry.
method Using groupoid language and filtered manifolds, defining the residue and showing its properties.
result The groupoidal residue is a trace on pseudodifferential operators and matches the usual residue in certain cases.
Study of measured laminations on surfaces using Newton polytopes and Poisson brackets.
problem Understanding the space of measured laminations on surfaces from a valuative perspective.
method Introducing Newton polytopes for character variety functions, defining tangent spaces, and identifying symplectic structures.
result Trace functions have unit coefficients at the extremal points of their Newton polytopes.
In this work we prove a Baum-Bott type residue theorem for flags of holomorphic foliations. We prove some relations between the residues of the flag and the residues of their correspondent foliations. We define the Nash residue for flags and we give a partial answer to the Baum-Bott type rationality conjecture in this …
Method estimates heterogeneous causal effects on networks using orthogonal learning.
problem Challenges in estimating causal effects on networks due to treatment effects on both treated and neighbors, and network homophily.
method Two-stage orthogonal learning framework: first stage uses graph neural networks for nuisance components, second stage residualizes and interpretable attention-based model for causal effects.
result Improves heterogeneous effect estimation and supports interpretable analyses.
We revisit residual algorithms in both model-free and model-based reinforcement learning settings. We propose the bidirectional target network technique to stabilize residual algorithms, yielding a residual version of DDPG that significantly outperforms vanilla DDPG in the DeepMind Control Suite benchmark. Moreover, we…
Wide residual networks generalize well with uniform convergence to RNTK as width increases.
problem Understanding the generalization ability of wide residual networks.
method Uniform convergence of residual network kernel to residual neural tangent kernel (RNTK).
result Generalization error converges to kernel regression error with respect to RNTK.