Chameleon learns model initializations across tasks with different schemas.
problem Meta-learning parameter initialization across tasks with varying predictor schemas.
method Chameleon aligns different predictor schemas to a common representation.
result Chameleon successfully learns parameter initializations across tasks with different schemas.
Automatically extracts hyperparameter schemas from AI library documentation.
problem Lack of machine-readable hyperparameter schemas for AI automation tools.
method Automatically mines Python docstrings in AI libraries to extract JSON Schemas.
result Effective at extracting machine-readable schemas from 119 AI models.
Traditional relation extraction predicts relations within some fixed and finite target schema. Machine learning approaches to this task require either manual annotation or, in the case of distant supervision, existing structured sources of the same schema. The need for existing datasets can be avoided by using a univer…
When translating natural language questions into SQL queries to answer questions from a database, we would like our methods to generalize to domains and database schemas outside of the training set. To handle complex questions and database schemas with a neural encoder-decoder paradigm, it is critical to properly encod…
In this paper, we consider advancing web-scale knowledge extraction and alignment by integrating OpenIE extractions in the form of (subject, predicate, object) triples with Knowledge Bases (KB). Traditional techniques from universal schema and from schema mapping fall in two extremes: either they perform instance-level…
The ML-Schema, proposed by the W3C Machine Learning Schema Community Group, is a top-level ontology that provides a set of classes, properties, and restrictions for representing and interchanging information on machine learning algorithms, datasets, and experiments. It can be easily extended and specialized and it is a…
A framework for federated learning with heterogeneous data.
problem Federated learning with data from clients using different data representations.
method FLIC framework that maps client data into a common feature space via local embedding functions, learned federally using Wasserstein barycenters and trained locally via distribution alignment.
result FLIC outperforms FL benchmarks with heterogeneous input feature spaces.
Paper uses Gaussian mixture models and Wasserstein distance for schema matching.
problem Schema matching between different datasets.
method Gaussian mixture models and Wasserstein distance for comparison.
result Derives an approximation for Wasserstein distance between Gaussian mixture models.
While label fusion from multiple noisy annotations is a well understood concept in data wrangling (tackled for example by the Dawid-Skene (DS) model), we consider the extended problem of carrying out learning when the labels themselves are not consistently annotated with the same schema. We show that even if annotators…
Paper improves text-to-SQL models with schema-aware denoising.
problem Text-to-SQL models struggle with schema linking and grammar correctness.
method Adapts transformer-based seq-to-seq model with SeaD denoising objectives and clause-sensitive decoding.
result Improves seq-to-seq model performance on WikiSQL benchmark.
FastSGT improves accuracy in BERT-based DST for SGD datasets.
problem Dialog State Tracking in goal-oriented dialogue systems.
method BERT-based model with two carry-over procedures and multi-head attention.
result Significantly improved accuracy compared to baseline model.
GOLOMB improves dialogue state tracking for unseen services.
problem Improving dialogue state tracking for multiple services and APIs.
method GOLOMB uses a BERT-based model that queries dialogue history with slot descriptions and values.
result GOLOMB achieves a joint goal accuracy of 53.97% on the SGD dataset.
Recent studies have significantly improved the state-of-the-art on common-sense reasoning (CSR) benchmarks like the Winograd Schema Challenge (WSC) and SWAG. The question we ask in this paper is whether improved performance on these benchmarks represents genuine progress towards common-sense-enabled systems. We make ca…
The Hawkes process (HP) has been widely applied to modeling self-exciting events including neuron spikes, earthquakes and tweets. To avoid designing parametric triggering kernel and to be able to quantify the prediction confidence, the non-parametric Bayesian HP has been proposed. However, the inference of such models …
Paper proposes a method to predict optimal data partitioning based on query execution costs.
problem Finding optimal data partitioning for improved system performance and scalability.
method Formal model abstraction of workload queries, genetic algorithm for optimization, evaluation using PostgreSQL's query optimizer.
result The approach effectively reduces workload execution cost and improves system performance.
New framework tackles deep financial reporting bottleneck by improving hallucination and coherence.
problem Statistical smoothing trap in LLMs limits deep financial reporting quality.
method DeepNews Framework integrates information foraging, schema-guided planning, and adversarial prompting.
result DeepNews system achieves 25% acceptance rate in blind test, significantly outperforming SOTA.
The notion of a pseudoknot is defined as an equivalence class of knot diagrams that may be missing some crossing information. We provide here a topological invariant schema for pseudoknots and their relatives, 4-valent rigid vertex spatial graphs and singular knots, that is obtained by replacing unknown crossings or ve…
CTSyn generates high-quality synthetic tabular data.
problem Challenges in generating high-quality synthetic tabular data.
method Diffusion-based generative foundation model with autoencoder and conditional latent diffusion.
result CTSyn outperforms existing table synthesizers on standard benchmarks.
DCoM uses deep neural networks to detect semantic data types from raw column values.
problem Detecting semantic data types from dirty and unseen data.
method DCoM employs multi-input NLP-based deep neural networks trained on 686,765 data columns.
result DCoM outperforms existing methods significantly on 78 different semantic data types.
New protocol evaluates synthetic data for temporal consistency.
problem Synthetic data generators can produce invalid timestamps and trajectories.
method Characterize datasets by four properties, then measure timestamp validity and dynamics.
result Temporal fidelity must be measured, not inferred from static data.
The task of dialog management is commonly decomposed into two sequential subtasks: dialog state tracking and dialog policy learning. In an end-to-end dialog system, the aim of dialog state tracking is to accurately estimate the true dialog state from noisy observations produced by the speech recognition and the natural…
Meta-learning, or learning to learn, is a machine learning approach that utilizes prior learning experiences to expedite the learning process on unseen tasks. As a data-driven approach, meta-learning requires meta-features that represent the primary learning tasks or datasets, and are estimated traditonally as engineer…
Logarithmic regret strategies for safe multi-armed bandits with safety risk constraints.
problem Maximizing reward while avoiding unsafe arms under safety risk constraints.
method Doubly optimistic strategies with pseudo-regret formulation.
result Logarithmic regret bounds for safe multi-armed bandits.
GenIE extracts structured text with fewer errors and more entities.
problem Closed information extraction with errors and limited entities.
method End-to-end autoregressive generation with bi-level constraints.
result State-of-the-art performance on closed information extraction.
Instead of requiring a domain expert to specify the probabilistic dependencies of the data, in this work we present an approach that uses the relational DB schema to automatically construct a Bayesian graphical model for a database. This resulting model contains customized distributions for columns, latent variables th…
Improves text-to-SQL models by selecting the best SQL query from beam output.
problem Simplifying database query writing for natural language questions.
method Discriminative re-ranker using BERT fine-tuned classifier.
result Achieved top 4 score on Spider leaderboard.
KEEL improves causal discovery with fuzzy knowledge and complex data.
problem Challenges in causal discovery due to prior knowledge, domain inconsistencies, and small sample sizes.
method Weakly-supervised fuzzy knowledge and data co-driven causal discovery method (KEEL).
result KEEL outperforms state-of-the-art methods in accuracy, robustness, and computational efficiency.
This paper presents a general iterative bias correction procedure for regression smoothers. This bias reduction schema is shown to correspond operationally to the L2 Boosting algorithm and provides a new statistical interpretation for L2 Boosting. We analyze the behavior of the Boosting algorithm applied to commo…
This paper proposes the Mesh Neural Network (MNN), a novel architecture which allows neurons to be connected in any topology, to efficiently route information. In MNNs, information is propagated between neurons throughout a state transition function. State and error gradients are then directly computed from state updat…
Dirichlet process mixture models (DPMM) are a cornerstone of Bayesian non-parametrics. While these models free from choosing the number of components a-priori, computationally attractive variational inference often reintroduces the need to do so, via a truncation on the variational distribution. In this paper we presen…
'Sharing of statistical strength' is a phrase often employed in machine learning and signal processing. In sensor networks, for example, missing signals from certain sensors may be predicted by exploiting their correlation with observed signals acquired from other sensors. For humans, our hands move synchronously with …
In this paper we address a classification problem where two sources of labels with different levels of fidelity are available. Our approach is to combine data from both sources by applying a co-kriging schema on latent functions, which allows the model to account item-dependent labeling discrepancy. We provide an exten…
Reweighting improves risk bounds in certain data regions.
problem Improving risk bounds in classification and heteroscedastic regression.
method Weighted empirical risk minimization with a data-dependent weight function.
result A weighted ERM estimator can achieve superior performance in specific sub-regions.
This paper surveys scalable automated alignment methods for LLMs.
problem Scalability issues in traditional human-annotated alignment methods for LLMs.
method Categorizes and discusses various automated alignment methods.
result Emerging automated alignment methods are effective and scalable.
Inference-aware meta-alignment of LLMs reduces computational cost.
problem Aligning LLMs to diverse human preferences is challenging due to conflicting criteria.
method IAMA trains a base model to be aligned to multiple tasks via different inference-time alignment algorithms, using non-linear GRPO for optimization.
result IAMA enables effective alignment of LLMs to multiple criteria with limited computational budget.
Conformal Alignment ensures trustworthy outputs from foundation models.
problem Ensuring outputs from foundation models align with human values in high-stakes tasks.
method A framework that trains an alignment predictor using reference data to select trustworthy outputs.
result Conformal Alignment accurately identifies trustworthy outputs via lightweight training over moderate reference data.
LPL optimizes embeddings to align local neighborhoods, improving cross-lingual word alignment.
problem Aligning embeddings across different datasets and languages.
method Locality Preserving Loss (LPL) optimizes model to project embeddings while maintaining local neighborhoods and aligning them.
result LPL-based alignment leads to better and consistent accuracy, especially in small training set settings.
Extends reinforcement learning alignment to scalar rewards, improving math reasoning.
problem Designing reinforcement learning algorithms for general LLM alignment.
method Introduces f-GRPO and f-HAL, estimating f-divergences between reward-aligned and unaligned distributions.
result Improves math reasoning RLVR tasks and mitigates reward hacking.
The paper proposes a learning-theoretic perspective on representation alignment.
problem Understanding how AI models' representations become aligned as they scale.
method Reviewing and connecting different notions of alignment, focusing on stitching.
result Relating properties of stitching to kernel alignment of representations.
Survey of alignment techniques for large language models.
problem Ensuring large language models align with human values.
method Analysis of diverse alignment methods and training paradigms.
result Preference-based methods offer more flexibility for nuanced alignment.
INCAD clusters and detects anomalies in streaming data without thresholds.
problem Clustering and anomaly detection for streaming data with unknown clusters and thresholds.
method Probabilistic clustering and anomaly detection in a streaming model.
result More reliable definition of normal vs abnormal behavior in streaming data.
The paper addresses rigid alignment of noisy patches, providing a polynomial time algorithm and convergence conditions.
problem Finding a rigid alignment of overlapping local views (patches) that minimizes alignment error in a noisy setting.
method Characterizes non-degeneracy based on kernel and positivity of a matrix, provides polynomial time algorithm for testing non-degeneracy, and uses Riemannian gradient descent for alignment.
result The algorithm converges locally linearly to a non-degenerate perfect alignment under certain conditions.
Adaptive personalized federated learning improves local model personalization.
problem Maximizing global model performance limits local model personalization.
method APFL algorithm trains local models while contributing to global model, with optimal mixing parameter and communication-efficient optimization.
result Demonstrates effectiveness of personalization schema and correctness of generalization theories.
Study connects database alignment and planted matching using Gaussian features.
problem Identify matching between correlated user features in anonymized databases.
method Derived results for database alignment and planted matching, showing connections and thresholds.
result Performance thresholds for database alignment converge to planted matching when feature dimensionality is sufficiently high.
Bayesian framework proves thresholds for multi-graph alignment feasibility.
problem Determining when multi-graph alignment is statistically possible.
method Developed a Bayesian estimation framework over metric spaces.
result Identified thresholds for Gaussian and sparse Erdős-Rényi models.
SMAI framework tests and integrates single-cell data alignability.
problem Lack of a rigorous statistical test for alignability and distortion during alignment.
method Spectral manifold alignment and inference (SMAI) framework.
result SMAI outperforms existing methods in alignability testing and integration.
New method aligns brain data across individuals for better brain decoding.
problem Inter-individual variability in brain response patterns limits decoder generalization.
method SpectralOT method that embeds cortical geometry into Laplace-Beltrami eigenmodes.
result SpectralOT strikes balance between aligning functional features and preserving anatomical structure.
Method aligns multilingual news for better stock return prediction.
problem Losing nuances in multilingual news translations.
method Optimal transport for sentence alignment.
result Aligned sentences correlate better with stock returns.