arXiv research

This paper improves disentanglement in VAEs by progressively learning hierarchical representations.

We describe a new training methodology for generative adversarial networks. The key idea is to grow both the generator and discriminator progressively: starting from a low resolution, we add new layers that model increasingly fine details as training progresses. This both speeds the training up and greatly stabilizes i…

Develops a new method to create object models from medical images.

PaGoDA reduces diffusion model training costs by 64x.

In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent …

We introduce a novel generative autoencoder network model that learns to encode and reconstruct images with high quality and resolution, and supports smooth random sampling from the latent space of the encoder. Generative adversarial networks (GANs) are known for their ability to simulate random high-quality images, bu…

The extension of image generation to video generation turns out to be a very difficult task, since the temporal dimension of videos introduces an extra challenge during the generation process. Besides, due to the limitation of memory and training stability, the generation becomes increasingly challenging with the incre…

Deep Learning has enabled remarkable progress over the last years on a variety of tasks, such as image recognition, speech recognition, and machine translation. One crucial aspect for this progress are novel neural architectures. Currently employed architectures have mostly been developed manually by human experts, whi…

The contradiction between physical and economical sciences concerning the growth of the production/consumption mechanism is analyzed. It is then shown that if one wishes to keep the security level stable or to enhance it in a growing economy the cost of security grows faster than the gross wealth. The result is a typic…

An algorithm solves optimization problems with large sample sets, improving worst-case complexity.

Paper proposes TC3 algorithm to predict crowdfunding funding progress.

We develop a progressive training approach for neural networks which adaptively grows the network structure by splitting existing neurons to multiple off-springs. By leveraging a functional steepest descent idea, we derive a simple criterion for deciding the best subset of neurons to split and a splitting gradient for …

Improved NODEs for long-term time series forecasting.

We first pose the Unsupervised Progressive Learning (UPL) problem: an online representation learning problem in which the learner observes a non-stationary and unlabeled data stream, learning a growing number of features that persist over time even though the data is not stored or replayed. To solve the UPL problem we …

Deep learning accelerators efficiently train over vast and growing amounts of data, placing a newfound burden on commodity networks and storage devices. A common approach to conserve bandwidth involves resizing or compressing data prior to training. We introduce Progressive Compressed Records (PCRs), a data format that…

Although deep neural networks (NNs) have achievedstate-of-the-art accuracy in many visual recognition tasks,the growing computational complexity and energy con-sumption of networks remains an issue, especially for ap-plications on platforms with limited resources and requir-ing real-time processing. Filter pruning tech…

The theory of complex hyperbolic discrete groups is still in its childhood but promises to grow into a rich subfield of geometry. In this paper I will discuss some recent progress that has been made on complex hyperbolic deformations of the modular group and, more generally, triangle groups. These are some of the simpl…

EPC curriculum improves MARL performance as agent population grows.

We present a generative autoencoder that provides fast encoding, faithful reconstructions (eg. retaining the identity of a face), sharp generated/reconstructed samples in high resolutions, and a well-structured latent space that supports semantic manipulation of the inputs. There are no current autoencoder or GAN model…

Satellite imagery helps assess sustainable development with machine learning.

Improves spatio-temporal forecasting by reducing errors between training and inference.

ProAGAN stabilizes GANs for learning SOMs from noisy medical imaging data.

We present a framework for translating unlabeled images from one domain into analog images in another domain. We employ a progressively growing skip-connected encoder-generator structure and train it with a GAN loss for realistic output, a cycle consistency loss for maintaining same-domain translation identity, and a s…

This paper explains a mechanism called phase collapse that improves image classification accuracy.

StreamBP optimally detects communities in growing networks.

Submanifolds of finite type were introduced by the author during the late 1970s. The first results on this subject were collected in author's books [26,29]. In 1991, a list of twelve open problems and three conjectures on finite type submanifolds was published in [40]. A detailed survey of the results, up to 1996, on t…

Agent learns diverse hierarchical structures in unknown environments.

Classical stochastic gradient methods for optimization rely on noisy gradient approximations that become progressively less accurate as iterates approach a solution. The large noise and small signal in the resulting gradients makes it difficult to use them for adaptive stepsize selection and automatic stopping. We prop…

A submanifold $M$ of a Euclidean $m$-space is said to be biharmonic if $Δ\overrightarrow H=0$ holds identically, where $\overrightarrow H$ is the mean curvature vector field and $Δ$ is the Laplacian on $M$. In 1991, the author conjectured that every biharmonic submanifold of a Euclidean space is minimal. The study of b…

Model shows how relaxed leverage can lead to asset price bubbles.

Forecasting technological progress is of great interest to engineers, policy makers, and private investors. Several models have been proposed for predicting technological improvement, but how well do these models perform? An early hypothesis made by Theodore Wright in 1936 is that cost decreases as a power law of cumul…

This research formalizes inductive generalization and proposes a new learning paradigm called Inductive Learning.

Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, the memory and compute requirements for training these models also increases. We introduce a technique to train deep neural networks using hal…

Current XAI research lacks solid foundations and clear goals.

Establishing unambiguously the existence of speculative bubbles is an on-going controversy complicated by the need of defining a model of fundamental prices. Here, we present a novel empirical method which bypasses all the difficulties of the previous approaches by monitoring external indicators of an anomalously growi…

BOSH optimizes functions with stochastic evaluations more efficiently and precisely.

Distributionally robust optimization (DRO) problems are increasingly seen as a viable method to train machine learning models for improved model generalization. These min-max formulations, however, are more difficult to solve. We therefore provide a new stochastic gradient descent algorithm to efficiently solve this DR…

Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-p…

In this article, we investigate when the set of primitive geodesic lengths on a Riemannian manifold have arbitrarily long arithmetic progressions. We prove that in the space of negatively curved metrics, a metric having such arithmetic progressions is quite rare. We introduce almost arithmetic progressions, a coarsific…

We analyze dropout in deep networks with rectified linear units and the quadratic loss. Our results expose surprising differences between the behavior of dropout and more traditional regularizers like weight decay. For example, on some simple data sets dropout training produces negative weights even though the output i…

With inspiration from Random Forests (RF) in the context of classification, a new clustering ensemble method---Cluster Forests (CF) is proposed. Geometrically, CF randomly probes a high-dimensional data cloud to obtain "good local clusterings" and then aggregates via spectral clustering to obtain cluster assignments fo…

Tokenized RWAs face liquidity issues despite promising markets.

Deep RL algorithms can overfit to early experiences, leading to poor performance.

The style-based GAN architecture (StyleGAN) yields state-of-the-art results in data-driven unconditional generative image modeling. We expose and analyze several of its characteristic artifacts, and propose changes in both model architecture and training methods to address them. In particular, we redesign the generator…

Sharp representation theorems show depth benefits for ReLU networks.

Industry evolution caused by various reasons, among which technology progress driving industry development has been approved, but with the new trend of industry convergence, inter-industry convergence also plays an increasing important role. This paper plans to probe the industry synergetic evolution mechanism based on…

Convex clustering is a recent stable alternative to hierarchical clustering. It formulates the recovery of progressively coalescing clusters as a regularized convex problem. While convex clustering was originally designed for handling Euclidean distances between data points, in a growing number of applications, the dat…

Paper introduces a new curriculum generation method for reinforcement learning.