Paper proposes a method to estimate scientific parameters in hybrid models without relying on model architecture.
problem Estimating unknown parameters in hybrid models combining machine learning and scientific models.
method Sharpness-aware minimization adapted for hybrid modeling, focusing on model simplicity.
result Demonstrates effectiveness of SAM-based hybrid model learning for scientific parameter estimation.
Proposes DMOC for more nuanced neural network robustness.
problem Lipschitz continuity is too coarse for nuanced data-dependent behavior.
method Data-driven, architecture-agnostic framework based on DMOC.
result DMOC provides a finer notion of robustness relative to data distribution.
Study Gaussian-process limits of neural networks using tensor programs.
problem Understanding the behavior of neural networks as they approach infinite width.
method Quantitative analysis through tensor programs and Wasserstein distance.
result Explicit finite-width error bounds, showing convergence to Gaussian-process limits.
MetaPerturb learns to improve generalization across different tasks and architectures.
problem Improving generalization on unseen data for diverse tasks and architectures.
method MetaPerturb is a meta-learned set-based perturbation function that improves generalization performance across heterogeneous tasks and architectures.
result MetaPerturb significantly outperforms baselines on most tasks and architectures with minimal increase in parameter size and no hyperparameters to tune.
ProbRes calibrates probabilistic forecasts by learning volatility dynamics.
problem Quantifying risk and uncertainty in time series forecasting.
method ProbRes learns conditional mean and volatility separately, generating well-calibrated prediction intervals.
result ProbRes accurately captures predictive distributions and produces well-calibrated prediction intervals.
Paper explores anchoring for vision models, improving generalization and safety.
problem Anchoring can lead to undesirable shortcuts, limiting generalization.
method Introduced a new anchored training protocol with a regularizer to mitigate undesirable shortcuts.
result Significant performance gains in generalization and safety metrics.
In many real-world prediction tasks, class labels include information about the relative ordering between labels, which is not captured by commonly-used loss functions such as multi-category cross-entropy. Recently, the deep learning community adopted ordinal regression frameworks to take such ordering information into…
NeuralSurv models survival analysis with Bayesian uncertainty.
problem Capturing time-varying risk relationships in survival analysis.
method Two-stage data-augmentation scheme, mean-field variational algorithm, coordinate-ascent updates, locally linearized Bayesian neural network.
result Delivers superior calibration compared to state-of-the-art models.
The increasingly common use of neural network classifiers in industrial and social applications of image analysis has allowed impressive progress these last years. Such methods are however sensitive to algorithmic bias, i.e. to an under- or an over-representation of positive predictions or to higher prediction errors i…
With access to large datasets, deep neural networks (DNN) have achieved human-level accuracy in image and speech recognition tasks. However, in chemistry, data is inherently small and fragmented. In this work, we develop an approach of using rule-based knowledge for training ChemNet, a transferable and generalizable de…
Proposes integrating global and local entropy for more reliable LLMs.
problem Uncertainty in large language models (LLMs) leads to unreliable predictions.
method Measures global uncertainty from hidden-state matrices and local uncertainty from tokens, combining them via a multiplicative gate.
result Global-Local Uncertainty (GLU) outperforms unsupervised baselines across multiple models and benchmarks.
Efficiently learns from partial labels using variational inference.
problem Learning from noisy and ambiguous partial labels in crowdsourcing.
method Amortized variational inference for probabilistic posterior approximation.
result Achieves state-of-the-art performance in accuracy and efficiency.
Degree-Quant improves GNN efficiency by quantizing them without losing accuracy.
problem Efficiency of graph neural networks at inference time.
method Architecturally-agnostic method, Degree-Quant, for quantizing GNNs.
result Degree-Quant trained models perform as well as full-precision models and achieve up to 26% gains.
New method controls posterior collapse in VAEs without network architecture constraints.
problem Posterior collapse in VAEs reduces diversity of generated samples.
method Introduces Latent Reconstruction (LR) loss to control posterior collapse.
result Controls posterior collapse on various datasets without architectural constraints.
AMF-VI uses adaptive mixtures of flows for robust VI across diverse distributions.
problem Inconsistent behavior of single-flow models across different distributions.
method Sequential expert training of individual flows and adaptive global weight estimation via likelihood-driven updates.
result AMF-VI achieves lower negative log-likelihood and stable gains in transport metrics across various posterior families.
APD method decomposes neural network parameters into simple, faithful components.
problem Understanding the internal mechanisms learned by neural networks.
method Attribution-based Parameter Decomposition (APD) method.
result Demonstrated effectiveness in recovering features, separating computations, and identifying representations.
GIBLy adds geometric priors to 3D segmentation models, improving performance with minimal overhead.
problem Lack of explicit geometric information in 3D semantic segmentation models.
method Introduces GIBLy, a lightweight geometric inductive bias layer that integrates learnable geometric priors into existing 3D segmentation pipelines.
result Consistent performance gains across multiple benchmarks, including up to +11.5% mIoU on TS40K with PTV3.
CRUMB: Efficient Prior Fitted Network Inference via Distributionally Matched Context Batching
problem Inference of tabular foundation models with large training datasets
method CRUMB (Clustered Retrieval Using Minimised-MMD Batching)
result CRUMB outperforms state-of-the-art context selection strategies on the TabArena benchmark
Enhances neural operators with physics knowledge for more accurate simulations.
problem Improving accuracy and generalization of neural operators for physical systems.
method Jointly learns from original PDEs and simplified forms, incorporating fundamental physics.
result Significant improvement in nRMSE across various PDE problems.
A new optimisation framework for neural networks without hyperparameters.
problem Lack of explicit architectural information in optimisation frameworks.
method Extends mirror descent to account for neural architecture, transforming a Bregman divergence.
result Automatic gradient descent: a first-order optimiser without hyperparameters.
Photonic co-processor speeds up training of large neural networks.
problem Training large neural networks with backpropagation is inefficient and communication is a bottleneck.
method Direct Feedback Alignment (DFA) with a photonic accelerator.
result Photonic accelerator can compute random projections with trillions of parameters.
Unified study of stateful replay for streaming learning, reducing forgetting by 2-3x.
problem Catastrophic forgetting in streaming generative and predictive learning.
method Unified analysis of stateful replay for autoencoding, forecasting, and classification tasks.
result Stateful replay reduces average forgetting by a factor of 2-3 on heterogeneous multi-task streams.
Random Fourier features improve tabular deep learning convergence.
problem Tabular deep learning convergence issues.
method Random Fourier projections as a pre-processing step, projecting inputs into a fixed feature space.
result Random Fourier pre-processing accelerates tabular deep learning convergence.
New robustness measure accounts for task-specific symmetries.
problem Traditional robustness measures fail for tasks with inherent symmetries.
method Sound notion of adversarial robustness for equivariant tasks, using randomized smoothing and graph edit distance certificates.
result Provable robustness can be achieved for various tasks with inherent symmetries.
Improved deep learning models with less labelled data and better label quality.
problem High costs and effort in training deep neural networks with label errors.
method Iterative label improvement using confidence-based filtering and dataset partitioning.
result Significant improvement in label quality and model accuracy.
Enhances deep neural networks with fixed-mean Gaussian processes for uncertainty estimation.
problem Post-hoc uncertainty estimation of pre-trained deep neural networks.
method Fixed-mean Gaussian processes with variational inference for efficient stochastic optimization.
result FMGP improves uncertainty estimation and computational efficiency compared to state-of-the-art methods.
Symile learns joint representations across multiple modalities, outperforming pairwise CLIP.
problem Pairwise contrastive learning fails to capture joint information between multiple modalities.
method Symile uses a flexible, architecture-agnostic objective to learn modality-specific representations by deriving a lower bound on total correlation.
result Symile outperforms pairwise CLIP on cross-modal classification and retrieval across various datasets.
Unified PAC-Bayesian framework for deep learning generalization.
problem Limitations of existing PAC-Bayesian norm-based bounds for deep neural networks.
method Unified framework using anisotropic Gaussian posteriors and sensitivity matrix.
result Comparable or tighter generalization bounds compared to state-of-the-art approaches.
Real-time pruning during training reduces network size and training time.
problem Efficiently reducing neural network size without sacrificing accuracy.
method Activation density-based pruning during training.
result Up to 200x reduction in parameters and 60x reduction in inference compute operations.
CoDeQ simplifies joint model compression by integrating pruning and quantization.
problem Joint pruning and quantization methods are complex and require additional procedures.
method CoDeQ uses a dead-zone quantizer to directly induce sparsity and learn quantization parameters.
result CoDeQ achieves high sparsity and low-precision accuracy with minimal bit operations.
GABI learns geometry from diverse systems to improve Bayesian inference.
problem Bayesian inversion of physical systems with varying geometries.
method Geometric Autoencoders for Bayesian Inversion (GABI) learns geometry-aware priors from large datasets.
result GABI yields comparable predictive accuracy to deterministic methods and well-calibrated uncertainty quantification.
Privacy-preserving GNNs for graph data with sensitive node data.
problem Privacy concerns in learning node representations for graphs with sensitive data.
method Developed a privacy-preserving GNN learning algorithm based on Local Differential Privacy (LDP). Proposed an LDP encoder, an unbiased rectifier, and a denoising mechanism (KProp).
result Our method maintains a satisfying level of accuracy with low privacy loss.
A simple strategy prevents negative transfer in transfer learning.
problem Negative transfer in transfer learning where source representations harm target performance.
method Residual feature integration with a trainable target-side encoder.
result The method provably prevents negative transfer with theoretical guarantees.
The paper uses geometry to assess how hard examples are for NLP models.
problem Challenges in NLP datasets and classifiers, especially with shallow features.
method Information geometry to quantify example difficulty, exploring BERT, CNN, and fasttext.
result Deep learning models are vulnerable to word substitutions in difficult examples.
A new concordance loss improves model performance and reliability in survival prediction.
problem Inconsistent evaluation of deep survival models using likelihood losses.
method Proposed a value-monotone concordance loss (SCL) to improve reliability and optimization.
result SCL achieves comparable discrimination and is the best or within one standard deviation of the best C-index across multiple datasets.