A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
R package huge simplifies graph estimation for high-dimensional data.
With the advent of Big Data era, data reduction methods are highly demanded given its ability to simplify huge data, and ease complex learning processes. Concretely, algorithms that are able to filter relevant dimensions from a set of millions are of huge importance. Although effective, these techniques suffer from the…
Develops a fast method to learn graph structures from large datasets.
Preconditioned SGD accelerates convergence for ill-conditioned huge-scale matrix completion.
The paper develops fast Bayesian methods for estimating huge PVARs with competitive forecasts.
Recent advances in pre-training huge models on large amounts of text through self supervision have obtained state-of-the-art results in various natural language processing tasks. However, these huge and expensive models are difficult to use in practise for downstream tasks. Some recent efforts use knowledge distillatio…
New method selects features for big data efficiently.
A framework for data-driven decision-making in infectious disease control.
Canonical Correlation Analysis (CCA) is a widely used spectral technique for finding correlation structures in multi-view datasets. In this paper, we tackle the problem of large scale CCA, where classical algorithms, usually requiring computing the product of two huge matrices and huge matrix decomposition, are computa…
Canonical Correlation Analysis (CCA) is a widely used statistical tool with both well established theory and favorable performance for a wide range of machine learning problems. However, computing CCA for huge datasets can be very slow since it involves implementing QR decomposition or singular value decomposition of h…
The time complexity of support vector machines (SVMs) prohibits training on huge data sets with millions of data points. Recently, multilevel approaches to train SVMs have been developed to allow for time-efficient training on huge data sets. While regular SVMs perform the entire training in one -- time consuming -- op…
The task of classifying X-ray data is a problem of both theoretical and clinical interest. Whilst supervised deep learning methods rely upon huge amounts of labelled data, the critical problem of achieving a good classification accuracy when an extremely small amount of labelled data is available has yet to be tackled.…
A new algorithm for optimizing huge-scale black-box problems with reduced memory usage.
Survey on using large models to train smaller datasets in NLP.
Proposes resilience metrics for large blackout costs with logarithmic resilience.
We propose an sorting algorithm by Machine Learning method, which shows a huge potential sorting big data. This sorting algorithm can be applied to parallel sorting and is suitable for GPU or TPU acceleration. Furthermore, we discuss the application of this algorithm to sparse hash table.
Gradient boosted decision trees (GBDT) is the leading algorithm for many commercial and academic data applications. We give a deep analysis of this algorithm, especially the histogram technique, which is a basis for the regulized distribution with compact support. We present three new modifications. 1) Share memory tec…
We present a matrix-factorization algorithm that scales to input matrices with both huge number of rows and columns. Learned factors may be sparse or dense and/or non-negative, which makes our algorithm suitable for dictionary learning, sparse component analysis, and non-negative matrix factorization. Our algorithm str…
A general theory of innovation and progress in human society is outlined, based on the combat between two opposite forces (conservatism/inertia and speculative herding "bubble" behavior). We contend that human affairs are characterized by ubiquitous ``bubbles'', which involve huge risks which would not otherwise be tak…
Simpler model outperforms deep learning for disease prediction.
The S&P500 daily values and log-returns fail to conform to Benford's laws, revealing underlying trends.
Derives formulae linking SABR model parameters to ATM and option prices.
Widely-used public benchmarks are of huge importance to computer vision and machine learning research, especially with the computational resources required to reproduce state of the art results quickly becoming untenable. In medical image computing, the wide variety of image modalities and problem formulations yields a…
Fast algorithm for analyzing huge social networks.
Paper proposes a new sparse Bayesian neural network for simpler, more efficient DNNs.
With the rapid development of high-throughput technologies, parallel acquisition of large-scale drug-informatics data provides huge opportunities to improve pharmaceutical research and development. One significant application is the purpose prediction of small molecule compounds, aiming to specify therapeutic propertie…
New methods distill data for deep networks efficiently.
We study safe screening for metric learning. Distance metric learning can optimize a metric over a set of triplets, each one of which is defined by a pair of same class instances and an instance in a different class. However, the number of possible triplets is quite huge even for a small dataset. Our safe triplet scree…
Modern computing and communication technologies can make data collection procedures very efficient. However, our ability to analyze large data sets and/or to extract information out from them is hard-pressed to keep up with our capacities for data collection. Among these huge data sets, some of them are not collected f…
We introduce a mean-field type approximation for description of company's income statistics. Utilizing huge company data we show that a discrete version of Langevin equation with additive and multiplicative noises can appropriately describe the time evolution of a company's income fluctuation in statistical sense. The …
Optimal smooth subspaces approximate large data sets efficiently.
Semi-supervised learning deals with the problem of how, if possible, to take advantage of a huge amount of not classified data, to perform classification, in situations when, typically, the labelled data are few. Even though this is not always possible (it depends on how useful is to know the distribution of the unlabe…
Unified model trained on images and videos using masked autoencoding.
Data augmentation is essential for medical research to increase the size of training datasets and achieve better results. In this work, we experiment three GAN architectures with different loss functions to generate new brain MRIs. The results show the importance of hyperparameter tuning and the use of mini-batch simil…
Mining graph data has become a popular research topic in computer science and has been widely studied in both academia and industry given the increasing amount of network data in the recent years. However, the huge amount of network data has posed great challenges for efficient analysis. This motivates the advent of gr…
The negative externalities from an individual bank failure to the whole system can be huge. One of the key purposes of bank regulation is to internalize the social costs of potential bank failures via capital charges. This study proposes a method to evaluate and allocate the systemic risk to different countries/regions…
Structured prediction provides a general framework to deal with supervised problems where the outputs have semantically rich structure. While classical approaches consider finite, albeit potentially huge, output spaces, in this paper we discuss how structured prediction can be extended to a continuous scenario. Specifi…
Recently, there has been an increasing interest in designing distributed convex optimization algorithms under the setting where the data matrix is partitioned on features. Algorithms under this setting sometimes have many advantages over those under the setting where data is partitioned on samples, especially when the …
We are losing biodiversity at an unprecedented scale and in many cases, we do not even know the basic data for the species. Traditional methods for wildlife monitoring are inadequate. Development of new computer vision tools enables the use of images as the source of information about wildlife. Social media is the rich…
In this paper, we propose a data collaboration analysis method for distributed datasets. The proposed method is a centralized machine learning while training datasets and models remain distributed over some institutions. Recently, data became large and distributed with decreasing costs of data collection. If we can cen…
Kurdistan Region is a tourist hub. This research analyzes other Non-Oil Sectors that have huge attractions of Foreign Direct Investments into the Kurdistan Region from 2005 to 2013. Comparative analysis was carried out between Iraq and the Region, and among influential Sectors of the Economy. T-test and ANOVA are stati…
TSML tackles anomaly detection and pattern discovery in industrial time series data.
Fast and accurate methods for low-rank learning problems.
This paper proposes and evaluates the k-greedy equivalence search algorithm (KES) for learning Bayesian networks (BNs) from complete data. The main characteristic of KES is that it allows a trade-off between greediness and randomness, thus exploring different good local optima. When greediness is set at maximum, KES co…
In many data mining applications collection of sufficiently large datasets is the most time consuming and expensive. On the other hand, industrial methods of data collection create huge databases, and make difficult direct applications of the advanced machine learning algorithms. To address the above problems, we consi…
In recent years, a specific machine learning method called deep learning has gained huge attraction, as it has obtained astonishing results in broad applications such as pattern recognition, speech recognition, computer vision, and natural language processing. Recent research has also been shown that deep learning tech…
In this paper we present a very simple way to price a class of barrier options when the underlying process is driven by a huge class of Lévy processes. To achieve our goal we assume that our market satisfies a symmetry property. In case of not satisfying that property some approximations can be obtained.
Deep learning applied to SAR data is explored in this paper.