Linear properties are either universal or absent across language models.
problem Explaining the prevalence of linear properties in language models.
method Proved identifiability of distribution-equivalent next-token predictors and analyzed various notions of linearity.
result Linear properties either hold in all or none distribution-equivalent next-token predictors.
We consider the problem of structure learning for bow-free acyclic path diagrams (BAPs). BAPs can be viewed as a generalization of linear Gaussian DAG models that allow for certain hidden variables. We present a first method for this problem using a greedy score-based search algorithm. We also prove some necessary and …
We characterize distributional equivalence in latent-variable models with cycles.
problem Lack of an equivalence characterization for latent-variable causal models with cycles.
method Established graphical criterion for distributional equivalence and developed edge rank constraints.
result First equivalence characterization without structural assumptions for latent-variable models with cycles.
We present Distributed Equivalent Substitution (DES) training, a novel distributed training framework for large-scale recommender systems with dynamic sparse features. DES introduces fully synchronous training to large-scale recommendation system for the first time by reducing communication, thus making the training of…
Bayesian method identifies causal DAG structure from non-Gaussian errors.
problem Learning causal structure from non-Gaussian errors in Bayesian networks.
method Bayesian hierarchical model with DAG prior for non-Gaussian errors.
result Posterior DAG selection consistency achieved under mild assumptions.
New TTP framework fuses control arms while controlling Type-I error.
problem Bias in borrowing control data from previous trials.
method Kernel two-sample testing via MMD and equivalence testing.
result Higher power than standard TTP methods while maintaining error control.
We provide a distribution-free test that can be used to determine whether any two joint distributions p and q are statistically different by inspection of a large enough set of samples. Following recent efforts from Long et al. [1], we rely on joint kernel distribution embedding to extend the kernel two-sample test…
SGD-trained neural networks generalize well even with adversarial label noise.
problem Generalization of neural networks trained on adversarial label noise.
method Training a one-hidden-layer neural network with SGD on arbitrary width networks.
result SGD-trained networks achieve classification accuracy competitive with the best halfspace over adversarial label noise.
We solve structure learning for cyclic linear causal models using observational data.
problem Learning the structure of cyclic linear causal models from observational data.
method Assuming simple graphs, we use a criterion for distributional equivalence and implement a greedy search method.
result We show that simple cyclic models are of expected dimension and justify score-based methods for structure learning.
The main approach to defining equivalence among acyclic directed causal graphical models is based on the conditional independence relationships in the distributions that the causal models can generate, in terms of the Markov equivalence. However, it is known that when cycles are allowed in the causal structure, conditi…
EPIC quantifies reward differences without policy optimization.
problem Distinguishing reward function quality from policy optimization issues.
method EPIC distance to compare reward functions directly.
result EPIC bounds policy training success and regret.
We consider the problem of approximating the set of eigenvalues of the covariance matrix of a multivariate distribution (equivalently, the problem of approximating the "population spectrum"), given access to samples drawn from the distribution. The eigenvalues of the covariance of a distribution contain basic informati…
Study reveals an equivalence principle for the spectrum of random inner-product kernel matrices in polynomial scaling.
problem Understanding the spectrum of random kernel matrices in polynomial scaling regimes.
method Investigates random matrices with nonlinear kernel functions applied to inner products of uniformly distributed vectors.
result The spectrum of the random kernel matrix is asymptotically equivalent to a simpler matrix model through free additive convolution.
FAST selects coresets more efficiently by matching distributions in the frequency domain.
problem Efficiently selecting representative subsets of large datasets for deep learning.
method FAST uses spectral graph theory and CFD to match distributions, addressing limitations of existing methods.
result FAST significantly outperforms state-of-the-art coreset selection methods in accuracy and energy efficiency.
New method identifies causal structure in count data using cumulants and path analysis.
problem Challenges in discovering causal structure from count data, especially due to non-identifiability.
method Poisson Branching Structural Causal Model (PB-SCM) with path analysis using high-order cumulants.
result Causal order is identifiable under specific conditions in PB-SCM using cumulant information.
Develops new Markov processes with switching rates and past dependence.
problem Modeling processes with dynamic switching rates and path dependence.
method Introduces a new class of Markov jump processes with regime switching and path dependence. Derives distributional properties and maximum likelihood estimates.
result Maximum likelihood estimates of the process parameters are derived in closed form and have asymptotic normality.