Extracts high-quality monolingual datasets from web crawl data.
problem Improving text representation quality through larger corpora.
method Automated pipeline using deduplication and language identification, augmented with filtering for high-quality documents.
result Extracted massive high-quality monolingual datasets from Common Crawl.
The paper tackles uniform sampling from databases with duplicates.
problem Sampling uniformly from entities with duplicate records.
method Two-stage process: frequency estimation followed by rejection sampling.
result Efficient sampling algorithms under various data properties.
Entity resolution (ER) is the task of identifying records belonging to the same entity (e.g. individual, group) across one or multiple databases. Ironically, it has multiple names: deduplication and record linkage, among others. In this paper we survey metrics used to evaluate ER results in order to iteratively improve…
Framework evaluates privacy cost of non-private pre-processing in DP pipelines.
problem Privacy cost of non-private data-dependent pre-processing in DP machine learning pipelines.
method Establishes upper bounds on overall privacy guarantees using Smooth DP and bounded sensitivity.
result Explicit overall privacy guarantees for various pre-processing algorithms.
New validation method prevents privacy breaches and biases in federated learning.
problem Privacy breaches and data leakage in federated learning.
method Stratified cross-validation for unbiased and privacy-preserving federated learning.
result Stratified cross-validation prevents data leakage without demanding deduplication algorithms.
NIFTY dataset for financial forecasting models.
problem Improving financial market forecasting with LLMs.
method Two datasets for supervised and reinforcement learning.
result Demonstrates applications in stock price prediction.
Machine learning improves invoice matching speed and accuracy.
problem Efficiently matching similar invoices from various businesses and organizations.
method Combining machine learning with string similarity functions on invoice datasets.
result Neural network and boosted decision tree outperformed existing solutions.
For statistical learning, categorical variables in a table are usually considered as discrete entities and encoded separately to feature vectors, e.g., with one-hot encoding. "Dirty" non-curated data gives rise to categorical variables with a very high cardinality but redundancy: several categories reflect the same ent…
IPO Finance Agent evaluates LLMs on SpaceX IPO due diligence, surpassing Finance Agent v2.
problem Evaluating language models on financial tasks like IPO due diligence.
method Introducing IPO Finance Agent and an evaluator-optimizer pipeline.
result The best-performing model reaches 79.4% accuracy at 0.30 USD per query.
IPO Finance Agent extends Finance Agent v2 for SpaceX S-1 filings, improving accuracy and cost-efficiency.
problem Evaluating IPO due diligence tasks with long-form documents.
method Extended task domain, improved agentic harness with contextual retrieval, automated rubric generation.
result Best-performing model reaches 79.8% accuracy, cost-efficient model at 77.2% with 0.05 USD per query.