With this work we try to analyse the agglomeration process in the Portuguese regions, using the New Economic Geography models. In these models the base idea is that where has increasing returns to scale in the manufactured industry and low transport costs, there is agglomeration. Of referring, as summary conclusion, th…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “Data Agglomeration”
Newly available data on the spatial distribution of retail activities in cities makes it possible to build models formalized at the level of the single retailer. Current models tackle consumer location choices at an aggregate level and the opportunity new data offers for modeling at the retail unit level lacks a theore…
The aim of this paper is to analyze the processes of polarization and agglomeration, to explain the mechanisms and causes of these phenomena in order to identify similarities and differences. As the main implication of this study should be noted that both process pretend to explain the concentration of economic activit…
DEFRAG accelerates extreme classification by reducing feature dimensions.
problem High precision and scalability in assigning labels from a vast label space.
method Adaptive feature agglomeration to reduce feature dimensions.
result Significant reduction in training and prediction times (up to 40%) for extreme classification algorithms.
ANN clusters multi-view data by agglomerating subviews and avoiding postprocessing.
problem Challenges in multi-view clustering, especially subview agglomeration.
method Agglomerative Neural Network (ANN) using Constrained Laplacian Rank.
result ANN outperforms state-of-the-art methods in multi-view clustering.
The aim of this paper is to analyze the relationship between inter-industry, intra-industry and inter-regional clustering and demand for labor by companies in Portugal. Is expected at the outset that there is more demand for work where the agglomeration is greater. It should be noted, as a summary conclusion, the resul…
Efficiently clusters data with weak assumptions, robust to contamination.
problem General-shaped clustering under weak parametric assumptions with data contamination.
method Two-step hybrid robust clustering algorithm combining trimmed k-means and hierarchical agglomeration.
result Outperforms state-of-the-art methods in various applications.
This work aims to study the Portuguese regional agglomeration process, using the linear form the New Economic Geography models that emphasize the importance of spatial factors (distance, costs of transport and communication) in explaining of the concentration of economic activity in certain locations. In a theoretical …
A new text clustering method using NMF and LSA improves stability and performance.
problem Text data's large, sparse term-document matrix makes clustering difficult.
method Proposes a new feature agglomeration method based on NMF and deterministic K-Means initialization.
result Significantly improves clustering performance and stability.
Recursive nearest agglomeration (ReNA): fast clustering for approximation of structured signalsstat.ML
In this work, we revisit fast dimension reduction approaches, as with random projections and random sampling. Our goal is to summarize the data to decrease computational costs and memory footprint of subsequent analysis. Such dimension reduction can be very efficient when the signals of interest have a strong structure…
The paper explores various forms of calibration scores and their implications for fairness.
problem The evaluation of probabilistic predictions through calibration.
method The authors organize three grouping choices and one agglomeration of group errors, providing a framework for comparing and creating new calibration scores.
result The study demonstrates that appropriate choices of grouping can provide notions of (sub-)group or individual fairness.
New bounds on the cohesion of complete-link and other linkage methods for agglomeration clusteringcs.LG
New bounds improve linkage methods for clustering, distinguishing complete-link from single-link.
problem Improving bounds on linkage methods for clustering quality.
method Developed new bounds for complete-link and average-link methods in agglomeration clustering.
result Separated complete-link from single-link in terms of approximation for diameter.
Using open source data, we observe the fascinating dynamics of nighttime light. Following a global economic regime shift, the planetary center of light can be seen moving eastwards at a pace of about 60 km per year. Introducing spatial light Gini coefficients, we find a universal pattern of human settlements across dif…
Unified method for simultaneous denoising and clustering.
problem Clustering noisy signals.
method Sparse convex wavelet clustering with fusion and group-sparse penalties.
result Unified approach that denoises and clusters simultaneously.
Infinite mixture models are commonly used for clustering. One can sample from the posterior of mixture assignments by Monte Carlo methods or find its maximum a posteriori solution by optimization. However, in some problems the posterior is diffuse and it is hard to interpret the sampled partitionings. In this paper, we…
Study a simplified model of multiverse structure with synchronized timelines.
problem Understanding the complex structure of the local multiverse with multiple universes.
method Time-amalgamated globally hyperbolic model of multiverse as a collection of parallel universes.
result Elementary particles are transcosmic strings with multiple endpoints on parallel universes.
Sum-of-norms clustering is a method for assigning points in to clusters, , using convex optimization. Recently, Panahi et al.\ proved that sum-of-norms clustering is guaranteed to recover a mixture of Gaussians under the restriction that the number of samples is not too large. The pu…
Spatial orderness metric improves CNN performance for non-spatial data.
problem Improving CNN performance for data lacking spatial order.
method Proposed spatial orderness metric to quantify spatial ordering.
result Adding convolutional layers is counterproductive for non-spatial data.
Inverse inference, or "brain reading", is a recent paradigm for analyzing functional magnetic resonance imaging (fMRI) data, based on pattern recognition and statistical learning. By predicting some cognitive variables related to brain activation maps, this approach aims at decoding brain activity. Inverse inference ta…
It has always been a great challenge for clustering algorithms to automatically determine the cluster numbers according to the distribution of datasets. Several approaches have been proposed to address this issue, including the recent promising work which incorporate Bayesian Nonparametrics into the -means clusterin…
The scaling of income inequality in citiesphysics.soc-ph
Developing a scientific understanding of cities in a fast urbanizing world is essential for planning sustainable urban systems. Recently, it was shown that income and wealth creation follow increasing returns, scaling superlinearly with city size. We study scaling of per capita incomes for separate census defined incom…
With the large-scale penetration of the internet, for the first time, humanity has become linked by a single, open, communications platform. Harnessing this fact, we report insights arising from a unified internet activity and location dataset of an unparalleled scope and accuracy drawn from over a trillion (1.5$\times…
With this study we want to test the validity of the well known "Verdoorn's Law" which considers the relationship between the growth of productivity and output in the case of the Portuguese economy at a regional and sectoral levels (NUTs II) for the period 1995-1999. The importance of some additional variables in the or…
A Time Series Analysis-Based Stock Price Prediction Using Machine Learning and Deep Learning Modelsq-fin.ST
This study predicts stock prices using various machine and deep learning models.
problem Predicting stock price movements is challenging but possible.
method Agglomerative approach combining statistical, machine learning, and deep learning models.
result Deep learning models outperform traditional methods in stock price prediction.
CAF-HFCM automatically forms a cluster hierarchy and optimizes the number of clusters without trial-and-validation.
problem Challenges in determining the optimal number of clusters in fuzzy c-means.
method CAF-HFCM, an auto-fused hierarchical fuzzy c-means method.
result Automatic agglomeration and optimal number of clusters without validity indices.
Clarifies contributions of model architectures and resources in SOTA language models.
problem Difficulty in disentangling contributions of model architectures and resources.
method Overview of large pre-trained language models, focusing on their use of new architectures and resources.
result Identifies potential starting points for benchmark comparisons and areas for improvement.
Data preprocessing improves data quality for robust data mining.
problem Noisy and incomplete data hinders data mining models.
method Overview of data cleaning, transformation, and preprocessing methods.
result Preprocessing significantly affects data mining model performance.
Big data sets must be carefully partitioned into statistically similar data subsets that can be used as representative samples for big data analysis tasks. In this paper, we propose the random sample partition (RSP) data model to represent a big data set as a set of non-overlapping data subsets, called RSP data blocks,…
A new method for handling imbalanced big data using ensembles and smart data.
problem Imbalanced data distribution in big data scenarios.
method Smart Data driven Decision Trees Ensemble (SD_DeTE) methodology.
result SD_DeTE outperforms Random Forest in handling imbalanced binary classification problems in big data.
Prevents sensitive data generation in diffusion models using labeled and unlabeled data.
problem Generating sensitive data in diffusion models using unlabeled data.
method Positive-Unlabeled Diffusion Models, approximating ELBO with labeled and unlabeled data.
result Prevents the generation of sensitive data without compromising image quality.
Study reveals Data Shapley's inconsistent performance in data selection tasks.
problem Inconsistency of Data Shapley's performance in data selection across different settings.
method Hypothesis testing framework and identification of utility functions.
result Data Shapley's performance is no better than random selection without specific constraints.
PRRO generates synthetic tabular data that improves SL performance and class distribution.
problem Low SL utility of synthetic data due to class imbalance and overlooked data relationships.
method Data pruning and column reordering to optimize SL utility.
result Synthetic data generated with PRRO enhances predictive performance and class distribution.
Defines data science as a natural ecosystem with challenges and missions.
problem Challenges and missions in data science due to 5D complexities and data life cycle phases.
method Systemic and data-centric view of data science as a fusion of data universe and its challenges, formalizing a general-purpose architecture.
result Essential data science as a natural ecosystem integrating specific disciplines and high-impact applications.
Synthetic data enhances analytics but requires careful volume management.
problem Accuracy of statistical methods on synthetic data vs. raw data.
method Synthetic Data Generation for Analytics framework using tabular diffusion models.
result Error rate decreases with more synthetic data but may stabilize or increase.
This paper introduces C-DSL to improve data mining outcomes by considering context.
problem Data collection ambiguities, data imbalance, hidden biases, lack of domain info, and data incompleteness.
method Developed Context-Driven Data Science Lifecycle (C-DSL) to address data quality issues.
result Tangible improvements to data mining outcomes were achieved through C-DSL.
Proposes using probabilistic models for privacy-preserving synthetic data.
problem Designing high-quality synthetic data for privacy preservation.
method Formulate the problem through probabilistic modelling, choosing a model for the data.
result Statistical discoveries can be reliably reproduced from synthetic data.
New test ensures quality of shared data in machine learning.
problem Ensuring quality of external data in machine learning tasks.
method Distribution-free two-sample testing procedures grounded in conformal outlier detection.
result Identifies valuable external data agents for model personalization.
Paper creates fair synthetic data ensuring equal predictions across sensitive attributes.
problem Ensuring fair predictions across sensitive attributes in synthetic data.
method Equalizing target probability distributions across sensitive attributes in synthetic data generation.
result Synthetic data provides strong fair predictions, equal across all thresholds.
A new method classifies multiple correlated data streams simultaneously.
problem Classifying multiple correlated data streams in practical scenarios.
method Double-Coupling Support Vector Machines (DC-SVM) considers both internal and external correlations.
result The proposed method outperforms traditional methods on artificial and real-world data streams.
DPA preserves data distribution in reduced dimensions.
problem Loss of data distribution in dimension reduction.
method DPA combines encoder and decoder to match data distribution.
result DPA successfully reconstructs data distribution.
Efficient synthetic data generation improves model performance on tabular data.
problem Improving model robustness and performance with scarce or low-quality data.
method Hardness characterization to identify high-value training points, generating synthetic data only from these points.
result Synthetic data generated from hardest points outperforms non-targeted methods on tabular datasets.
For most problems in science and engineering we can obtain data sets that describe the observed system from various perspectives and record the behavior of its individual components. Heterogeneous data sets can be collectively mined by data fusion. Fusion can focus on a specific target relation and exploit directly ass…
DAERNN models censored data using neural networks with data augmentation.
problem Handling censored data in expectile regression.
method Data augmentation based Expectile Regression Neural Networks (ERNNs).
result DAERNN outperforms existing censored ERNNs methods and achieves comparable predictive performance to fully observed data.
Data preprocessing techniques are devoted to correct or alleviate errors in data. Discretization and feature selection are two of the most extended data preprocessing techniques. Although we can find many proposals for static Big Data preprocessing, there is little research devoted to the continuous Big Data problem. A…
Data collection is a major bottleneck in machine learning and an active research topic in multiple communities. There are largely two reasons data collection has recently become a critical issue. First, as machine learning is becoming more widely-used, we are seeing new applications that do not necessarily have enough …
This paper quantifies uncertainty in Data Shapley using statistical inference.
problem Uncertainty in data valuation due to dynamic data distribution.
method Established relationship with U-statistics and quantified uncertainty using statistical inference.
result Confidence intervals for Data Shapley estimations are provided.
DCoM uses deep neural networks to detect semantic data types from raw column values.
problem Detecting semantic data types from dirty and unseen data.
method DCoM employs multi-input NLP-based deep neural networks trained on 686,765 data columns.
result DCoM outperforms existing methods significantly on 78 different semantic data types.
Causal inference from observational data is the goal of many data analyses in the health and social sciences. However, academic statistics has often frowned upon data analyses with a causal objective. The introduction of the term "data science" provides a historic opportunity to redefine data analysis in such a way tha…