GNN-FiLM uses feature-wise linear modulation to improve graph neural networks.
problem Improving graph neural networks for better performance.
method Feature-wise linear modulation applied to target node representations in GNNs.
result GNN-FiLM outperforms baseline methods on a regression task for molecular graphs.
TFiLM expands convolutional models' receptive field with minimal overhead.
problem Capturing long-range dependencies in sequential data.
method A novel architectural component using a recurrent neural network to modulate convolutional model activations.
result TFiLM significantly improves learning speed and accuracy on various tasks.
We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation. FiLM layers influence neural network computation via a simple, feature-wise affine transformation based on conditioning information. We show that FiLM layers are highly effective for visual reasoning - an…
Improved multi-user VoiceFilter-Lite model for speech recognition.
problem Limited performance of multi-user VoiceFilter-Lite models.
method Dual learning rate schedule and FiLM for feature conditioning.
result Closed the performance gap between multi-user and single-user VoiceFilter-Lite models.
Recent breakthroughs in computer vision and natural language processing have spurred interest in challenging multi-modal tasks such as visual question-answering and visual dialogue. For such tasks, one successful approach is to condition image-based convolutional network computation on language via Feature-wise Linear …
FLANs process each feature separately for better interpretability.
problem Need for interpretable machine learning models in critical scenarios.
method Feature-wise latent representations summed for prediction.
result FLANs enhance interpretability without sacrificing performance.
A family of recent successful approaches to few-shot learning relies on learning an embedding space in which predictions are made by computing similarities between examples. This corresponds to combining information between support and query examples at a very late stage of the prediction pipeline. Inspired by this obs…
The goal of supervised feature selection is to find a subset of input features that are responsible for predicting output values. The least absolute shrinkage and selection operator (Lasso) allows computationally efficient feature selection based on linear dependency between input features and output values. In this pa…
We study the phenomenon of bias amplification in classifiers, wherein a machine learning model learns to predict classes with a greater disparity than the underlying ground truth. We demonstrate that bias amplification can arise via an inductive bias in gradient descent methods that results in the overestimation of the…
AdaTrans adapts to feature and sample transfer in high-dimensional regression.
problem High-dimensional linear regression with more features than samples.
method F-AdaTrans and S-AdaTrans methods using fused-penalties and adaptive weights.
result AdaTrans achieves convergence rates close to oracle estimators and near-minimax optimal rates.
This paper improves image super-resolution by integrating cross-scale non-local attention.
problem Improving image super-resolution by leveraging long-range and cross-scale feature correlations.
method Proposes a Cross-Scale Non-Local (CS-NL) attention module integrated into a recurrent neural network.
result Significantly improved performance on SISR benchmarks.
Paper tackles robust decision-making from multiple sites with shared structure.
problem Learning robust sequential decisions from heterogeneous multi-site datasets.
method Group-Robust MDPs with d-rectangular uncertainty sets, feature-wise worst-case aggregation, and cluster-level pooling.
result Proves suboptimality bound for robust planning policy under robust partial coverage assumption.
New method learns PDE solutions from low-fidelity data.
problem Challenges in learning PDE surrogates with scarce data.
method Flow matching in infinite-dimensional space with conditional neural operators.
result Accurately learns PDE solutions across different resolutions and fidelities.
VJE learns latent representations without contrastive learning, providing probabilistic semantics.
problem Learning latent representations without contrastive signals.
method VJE maximizes a symmetric conditional evidence lower bound (ELBO) on paired encoder embeddings, using a Student-t distribution on a polar representation.
result VJE outperforms standard non-contrastive baselines in ImageNet-1K, CIFAR-10/100, and STL-10.
CDSSL improves representation quality by integrating linear and nonlinear dependencies.
problem Scarcity of labeled data and neglect of nonlinear dependencies in SSL.
method CDSSL combines linear correlations and nonlinear dependencies using HSIC in RKHS.
result CDSSL enhances representation quality on diverse benchmarks.
Paper proposes a feature-wise change detection method for improving indoor positioning accuracy.
problem Improving the quality of reference fingerprint maps in indoor positioning systems.
method Inspired by RANSAC, the paper uses resampling of features to estimate intermediate locations and identifies candidate locations using MJI.
result The approach improves positioning accuracy by 20% and achieves 90% change detection accuracy.
Unified normative modeling for neuroimaging phenotypes using denoising diffusion models.
problem Discarding multivariate dependence in neuroimaging pipelines.
method Denoising diffusion probabilistic models (DDPMs) with FiLM and SAINT backbones.
result Unified multivariate normative modeling with better calibration and dependence preservation.
Kernel testing compares cell states in single-cell data.
problem Comparing non-linear cell states in single-cell data.
method Kernel-based testing framework for non-linear distribution comparison.
result Identifies subtle population variations in cell states.
Introduces modular q-holonomic modules to solve q-difference equations.
problem Solving q-difference equations in quantum invariants and Chern-Simons theory. method Defines modular q-holonomic modules with improved analyticity properties. result Modular q-holonomic modules explain structural properties of quantum invariants and Chern-Simons theory. Studies modules over a category of Jacobi diagrams in handlebodies.
problem Understanding modules over a specific category of Jacobi diagrams.
method Generalizes adjunctions and studies subquotient modules.
result Generalizes adjunctions between modules and Casimir Lie algebra modules.
Deviance-style normalization for sparse, jointly overdispersed count matrices
problem Jointly overdispersed count matrices
method Dirichlet-multinomial deviance residualization
result Preserves exact sparsity, evaluates in constant time, recovers multinomial residual
DAG-FM discovers causal relationships from heterogeneous data.
problem Challenges in causal discovery from heterogeneous causal mechanisms.
method DAG-FM uses two specialized Transformer-based sub-modules and a robust tabular interaction block to model complex row-column interactions.
result DAG-FM achieves state-of-the-art performance on synthetic and real-world datasets.
Generalized Steinberg module presentation for Gaussian and Eisenstein integers.
problem Presenting Steinberg modules for specific number rings.
method Generalization of Bykovskii's presentation to Gaussian and Eisenstein integers.
result Generalization does not yield a presentation for all Euclidean number rings.
We prove that the Steinberg module of the special linear group of a quadratic imaginary number ring which is not Euclidean is not generated by integral apartments. Assuming the generalized Riemann hypothesis, this shows that the Steinberg module of a number ring is generated by integral apartments if and only if the ri…
Study of cubic skein modules in 3-sphere and arbitrary 3-manifolds.
problem Lack of systematic study of higher degree skein modules.
method Investigation of cubic skein module structure and properties in 3-sphere and arbitrary 3-manifolds.
result Establishment of a foundational framework for higher skein modules.
Proposes a novel network-based neighborhood regression for biological systems.
problem Lack of comprehensive analysis on biological modules using both global and local network data.
method Develops a community-wise least square optimization approach to analyze gene modules and their regulatory strength.
result Achieves exact minimax optimality and linear consistency in identifying gene module associations.
SympNets identify Hamiltonian systems from data using linear, activation, and gradient modules.
problem Identifying Hamiltonian systems from data.
method Composition of linear, activation, and gradient modules; universal approximation theorems.
result SympNets can approximate arbitrary symplectic maps and generalize well to various Hamiltonian systems.
This thesis generalizes structures on Q-manifolds and Lie n-algebroids.
problem Representation theory and linear structures of Q-manifolds and Lie n-algebroids. method Introduces differential graded modules and representations up to homotopy, defines Weil algebra, and studies VB-Lie n-algebroids. result Establishes an equivalence between VB-Lie n-algebroids and (n+1)-term representations up to homotopy of Lie n-algebroids. We categorify the notion of an infinitesimal braiding in a linear strict symmetric monoidal category, leading to the notion of a (strict) infinitesimal 2-braiding in a linear symmetric strict monoidal 2-category. We describe the associated categorification of the 4-term relation, leading to six categorified relations. …
Study the module structure of homology of Artin kernels.
problem Characterize the module structure of homology of Artin kernels.
method Use flag complex and double covers of toric complexes to analyze properties of torsion part.
result Determine dimensions and sizes of Jordan forms of the torsion part.
The spaces of linear differential operators on Rn acting on tensor densities of degree λ and the space of functions on T∗Rn which are polynomial on the fibers are not isomorphic as modules over the Lie algebra $\Vect({\mathbb{R}}^n)$ of vector fields on Rn. However, these mo…
The paper characterizes vector bundles and differential operators using Lie algebras and their symbols.
problem Characterizing vector bundles and differential operators using algebraic methods.
method Lie-algebraic characterization of vector bundles and differential operators.
result The Lie algebras P(E,M) and S(P(E,M)) characterize vector bundles and their smooth sections. In the last chapter of his book "The Algebraic Theory of Modular Systems " published in 1916, F. S. Macaulay developped specific techniques for dealing with " unmixed polynomial ideals " by introducing what he called " inverse systems ". The purpose of this paper is to extend such a point of view to differential module…
A commuting n-tuple (T1,…,Tn) of bounded linear operators on a Hilbert space $\clh$ associate a Hilbert module H over C[z1,…,zn] in the following sense: \[\mathbb{C}[z_1, \ldots, z_n] \times \mathcal{H} \rightarrow \mathcal{H}, \quad \quad (p, h) \mapsto p(T_1, \ldots, T_n)h…
Paper investigates Lipschitz constants of self-attention modules in neural networks.
problem Lipschitz constants of self-attention modules in neural networks.
method Proved standard dot-product self-attention is not Lipschitz for unbounded input domain. Proposed L2 self-attention that is Lipschitz. Derived upper bound on L2 self-attention's Lipschitz constant.
result Proved standard self-attention is not Lipschitz for unbounded input domain and proposed an alternative L2 self-attention that is Lipschitz.
In this paper, a generalized multivariate Student-t mixture model is developed for classification and clustering of Low Probability of Intercept radar waveforms. A Low Probability of Intercept radar signal is characterized by a pulse compression waveform which is either frequency-modulated or phase-modulated. The propo…
Let F_λ(S1) be the space of tensor densities of degree (or weight) λ on the circle S1. The space Dk_λ,μ(S1) of k-th order linear differential operators from F_λ(S1) to F_μ(S1) is a natural module over Diff(S1), the diffeomorphism group of S1. We deter…
Projective resolves symplectic Steinberg module for number rings.
problem Constructing a projective resolution for symplectic Steinberg module.
method Similar to special linear group, but more complex construction.
result Computed top degree cohomology of congruence subgroups.
Let {T1,…,Tn} be a set of n commuting bounded linear operators on a Hilbert space H. Then the n-tuple (T1,…,Tn) turns H into a module over C[z1,…,zn] in the following sense: \[\mathbb{C}[z_1, \ldots, z_n] \times \mathcal{H} \raro \clh, \quad \quad …
Let Dk be the space of k-th order linear differential operators on R: A=ak(x)dxkdk+⋯+a0(x). We study a natural 1-parameter family of $\Diff(\bf R)$- (and $\Vect(\bf R)$)-modules on Dk. (To define this family, one considers arguments of differential operators as tensor-d…
MoNODEs improve neural ODEs by separating dynamic states from static factors.
problem Learning non-linear dynamics with variations across trajectories.
method Introduces time-invariant modulator variables to separate dynamic states from static factors.
result Consistently improves model generalization and far-horizon forecasting.
The moduli space of jets of certain G-structures (basically those which admit a canonical linear connection) is shown to be isomorphic to the quotient of a natural G-module by G.
New statistical methods improve explainability of boosting models.
problem Uncertainty quantification for boosting models is computationally intensive and hard to interpret.
method Derive methods for statistical inference using gradient boosting and Boulevard regularization.
result Achieve asymptotically normal predictions with theoretical guarantees and runtime independent of data size.
Let M be a smooth manifold, S the space of polynomial on fibers functions on T∗M (i.e., of symmetric contravariant tensor fields). We compute the first cohomology space of the Lie algebra, Vect(M), of vector fields on M with coefficients in the space of linear differential operators on S. This co…
Paper analyzes why deeper layers of ViTs perform worse on out-of-distribution tasks.
problem Performance degradation of intermediate layers in ViTs under distribution shift.
method Extensive linear probing experiments across various benchmarks and fine-grained analysis of transformer modules.
result Probing feedforward network activations yields best performance under significant distribution shift.
In this paper we introduce a new algebraic device, which enables us to treat the quaternions as though they were a commutative field. This is of interest both for its own sake, and because it can be applied to develop an "algebraic geometry" of noncompact hypercomplex manifolds. The basic building blocks of the theory …
Three definitions of graded vector bundles are shown to be equivalent.
problem Defining graded vector bundles in three different ways.
method Equivalence of categories among sheaves, graded modules, and locally trivial graded manifolds.
result All three approaches to graded vector bundles are equivalent.
We find a presentation of symplectic Steinberg modules and show vanishing cohomology for certain groups.
problem Cohomology vanishing for specific groups and modules.
method Presented a symplectic Steinberg module and used it to prove cohomology vanishing.
result Cohomology of Sp2n(Z) vanishes in a specific degree for n≥2.