A heuristic method for determining input ranges for complex processes.
problem Determining input variable ranges for non-numeric, high-dimensional processes.
method Create synthetic training data and use a decision tree classifier.
result Validated on a real use case in a lamination factory.
A new algorithm estimates output ranges for deep neural networks efficiently.
problem Estimating output ranges in deep neural networks with complex non-linearities.
method Integrates Simulated Annealing tailored for constrained domains and global optima.
result Guaranteed convergence and robust performance across various DNN architectures.
Paper analyzes output range of deep neural networks for verification.
problem Verifying deep neural networks for high-assurance applications.
method Combines local search and linear programming to find output ranges.
result Efficient algorithm for guaranteed range estimation of NN outputs.
Ranger improves DNNs' fault resilience without re-computation.
problem Transient faults in DNNs cause errors, reducing reliability.
method Range restriction to transform critical faults to benign faults.
result Significant improvement in error resilience (3x to 50x) with no accuracy loss.
OC-BNNs enforce output constraints in BNNs, improving model robustness.
problem Hard to encode prior knowledge in function space for BNNs.
method Formulate a prior that incorporates functional constraints on output.
result OC-BNNs improve model robustness and prevent infeasible predictions.
We investigate how simultaneously recorded long-range power-law correlated multi-variate signals cross-correlate. To this end we introduce a two-component ARFIMA stochastic process and a two-component FIARCH process to generate coupled fractal signals with long-range power-law correlations which are at the same time lo…
Optimistic bounds for multi-output learning using self-bounding Lipschitz condition.
problem Learning vector-valued functions from supervised data.
method Introducing self-bounding Lipschitz condition and proving optimistic bounds using local Rademacher complexity and Srebro's inequality.
result Minimax optimal generalization bounds for multi-output learning, up to logarithmic factors.
Conditional Restricted Boltzmann Machines (CRBMs) are rich probabilistic models that have recently been applied to a wide range of problems, including collaborative filtering, classification, and modeling motion capture data. While much progress has been made in training non-conditional RBMs, these algorithms are not a…
New kernel models multi-output Gaussian processes accurately.
problem Challenges in modelling cross-covariances for multiple-output Gaussian processes.
method Replaced Gaussian components with block components of finite bandwidth in spectral mixture kernel.
result First multi-output generalization of spectral mixture kernel that can approximate any stationary multi-output kernel to arbitrary precision.
A new algorithm computes Fourier coefficients for a specified range efficiently.
problem Inefficiency in FFT due to fixed output size for all applications.
method Fast Partial Fourier Transform (PFT) that allows specifying the range of Fourier coefficients to compute.
result PFT achieves significant speedup over state-of-the-art FFT algorithms for small output sizes.
We consider a framework for structured prediction based on search in the space of complete structured outputs. Given a structured input, an output is produced by running a time-bounded search procedure guided by a learned cost function, and then returning the least cost output uncovered during the search. This framewor…
New algorithm verifies deep neural networks with guaranteed bounds.
problem Verifying deep neural networks' correctness is hard.
method Adaptive nested optimisation for reachability analysis.
result Efficiently verifies a broader class of DNNs than previous methods.
A new method for forming learning objectives using the sum of ranked range.
problem Forming learning objectives from aggregated values.
method Sum of ranked range (SoRR) minimization with DCA.
result The proposed method effectively forms learning objectives and is applicable to binary and multi-label/multi-class classification.
A new method for analyzing multifractal cross correlations in complex systems.
problem Characterizing long-range cross-correlations in complex systems.
method Multifractal Cross Wavelet Analysis (MFXWT)
result MFXWT accurately captures joint multifractality in binomial multifractal measures but may produce spurious results for bivariate fractional Brownian motions.
MinimalRNN simplifies RNNs for better interpretability and efficiency.
problem Improving interpretability and efficiency of RNNs.
method MinimalRNN uses a simplified structure with minimal updates, leading to efficient learning and testing.
result MinimalRNN learns disentangled RNN states and captures longer range dependencies.
We develop a method for quantile-based sensitivity analysis in models with discontinuities.
problem Uncertainty in interpreting discontinuous models using traditional derivatives.
method Quantile-based derivatives for discontinuous models with discrete inputs.
result Derivatives of quantile-based outputs are well-defined and provide meaningful insights.
Paper studies t-SNE convergence with generalized kernels.
problem Understanding convergence of t-SNE with generalized kernels.
method Concrete formulation of generalized kernels, proving convergence to an equilibrium distribution.
result t-SNE converges to an equilibrium distribution under certain conditions for generalized kernels.
Several machine learning problems arising in natural language processing can be modeled as a sequence labeling problem. We provide Gaussian process models based on pseudo-likelihood approximation to perform sequence labeling. Gaussian processes (GPs) provide a Bayesian approach to learning in a kernel based framework. …
Paper develops a duality approach for robust loss functions in infinite-dimensional RKHSs.
problem Robustness issues in infinite-dimensional RKHSs with operator-valued kernels.
method Develops a duality approach to solve OVK machines for various loss functions.
result Empirical improvements and theoretical stability analysis for robust structured data applications.
Kernel methods summarize and integrate posterior similarity matrices from Bayesian clustering.
problem Summarizing and integrating posterior similarity matrices from Bayesian clustering.
method Positive semi-definite PSMs, kernel matrices, kernel methods, combining kernels.
result Kernel methods effectively summarize and integrate posterior similarity matrices.
New method speeds up MOGP inference to linear in m.
problem High computational cost of MOGP inference.
method Use of sufficient statistic with orthogonal bases.
result Linear scaling in m, allowing large m without sacrificing expressivity.
Gradient-free learning uses kernel and range space for solving linear equations.
problem Solving linear equations and least squares problems.
method Manipulating kernel and range space to solve linear matrix equations, adapting for neural networks.
result Gradient-free learning framework for neural networks, showing good performance on real-world data.
Bayesian Optimization improved for high-dimensional outputs.
problem Optimizing many correlated outcomes efficiently.
method Efficient multi-task Gaussian Process sampling.
result Substantial improvements in sample efficiency.
Improved neural processes with attention for better predictions.
problem Underfitting in Neural Processes.
method Incorporating attention into Neural Processes to improve accuracy and range of functions.
result Significantly improved prediction accuracy and faster training.
Paper extends Bayes Theorem for interval probability estimates.
problem Real-world input probabilities are often interval estimates, not precise.
method Developed IT2 version of Bayes Theorem and a novel algorithm for encoding intervals.
result Conservative method avoids invalid output results from inconsistent input.
A new UNet variant reduces spectral artifacts in image transformations.
problem Spectral artifacts caused by traditional UNet upsampling layers.
method Introduced a Guided UNet (GUNet) architecture using a novel upsampling module.
result GUNet produces higher fidelity outputs in image transformations.
Shared classical randomness improves quantum generative models' output distributions.
problem Improving generative performance of shallow unitary quantum models.
method Introducing stochasticity into unitary quantum models via shared classical randomness.
result Shared classical randomness allows shallow unitary quantum models to represent a strictly larger family of distributions.
SnareNet adds repair layers to neural networks to ensure outputs meet physical constraints.
problem Unconstrained neural network predictions violate physical or safety requirements.
method SnareNet appends a differentiable repair layer that navigates constraints to produce feasible outputs.
result SnareNet consistently improves objective quality while satisfying constraints more reliably.
A new method detects long-range cross correlations in complex systems.
problem Detecting long-range cross correlations in complex systems.
method Joint multifractal analysis based on wavelet leaders (MF-X-WL).
result MF-X-WL detects cross correlations in synthetic and real-world data.
New method interprets neural networks at multiple scales.
problem Interpreting neural networks at various scales and identifying important subsets of inputs.
method Rank Projection Trees framework using any scoring function.
result Successfully identifies biologically important genes and gene sets.
Study analyzes low-energy behavior of Schrödinger operators with Coulomb potentials.
problem Analyzing the limiting resolvent of Schrödinger operators at low energies.
method Using Vasy's second microlocal approach (Lagrangian approach), uniformly analyzing the resolvent from E=0. result Obtained oscillatory asymptotics for the resolvent output at low energy, differing from short-range cases.
The paper tackles stable maxima optimization for expensive functions.
problem Finding stable maxima of expensive functions with input variations.
method Uses multiple gradient Gaussian Process models to estimate stability and guide optimization.
result Demonstrates effective finding of stable maxima on synthetic and real-world problems.
Algorithm samples fair rankings to ensure individual fairness while maintaining group fairness.
problem Fair ranking tasks with group fairness constraints and uncertainty in item utilities.
method Efficient algorithm that samples rankings from an individually-fair distribution ensuring group fairness.
result Expected utility of output ranking is at least α times optimal fair solution, where α depends on utilities and constraints.
Wave-U-Net improves audio source separation by modeling phase information.
problem Fixed spectral transformations and high sampling rates limit audio source separation performance.
method Wave-U-Net adapts U-Net to time-domain, using repeated resampling to capture different time scales.
result Wave-U-Net achieves comparable performance to spectrogram-based U-Net on singing voice separation.
Paper presents a new method for inference in deep networks, achieving precise performance guarantees in high dimensions.
problem Inference in deep networks with known parameters, especially in high dimensions.
method Multi-Layer Vector Approximate Message Passing (ML-VAMP) derived from expectation propagation.
result Exact prediction of mean-squared error (MSE) in a large system limit (LSL) for deep networks.
SGPA calibrates transformer uncertainty for safety-critical tasks.
problem Uncertainty estimation in transformer models for safety-critical domains.
method Bayesian inference in transformer's output space using sparse Gaussian processes.
result SGPA-based Transformers improve in-distribution calibration and out-of-distribution robustness.
In a multi-class classification problem, it is standard to model the output of a neural network as a categorical distribution conditioned on the inputs. The output must therefore be positive and sum to one, which is traditionally enforced by a softmax. This probabilistic mapping allows to use the maximum likelihood pri…
Introduces SoRR for aggregating losses in supervised learning.
problem Aggregating individual losses into a single output for machine learning models.
method Sum of ranked range (SoRR) minimization using DCA.
result Demonstrates effectiveness of AoRR and TKML in improving robustness of multi-label learning.
New neural process models produce correlated predictions for better estimation tasks.
problem Need for models that can handle correlated predictions for tasks like weather forecasting.
method Developed new Neural Process models that can produce correlated predictions and support exact maximum likelihood training.
result Improved predictive performance on various experiments with synthetic and real data.
A new kernel improves Volterra series model selection and prediction.
problem Hard identification of Volterra series from limited data.
method Proposes a novel regularization network using a multiplicative polynomial kernel.
result Better selection of influential monomials improves model prediction.
Cohesion uses deep Koopman operators to generate long-range forecasts of chaotic dynamics.
problem Challenges in data-driven emulation of chaotic dynamics, especially long-range skill decay.
method Generative modeling with coherent priors estimated using reduced-order models.
result Superior long-range forecasting skill on chaotic systems, including climate dynamics.
We adapt Shapley values to explain model uncertainty, connecting it to information theory.
problem Explaining uncertainty in model predictions.
method Adapted Shapley value framework to quantify feature contributions to predictive uncertainty.
result Deep connections between Shapley values and information theory quantities.
Generative models emulate climate model outputs for impact assessment.
problem Outdated climate model projections hinder adaptation and mitigation planning.
method Score-based diffusion on a spherical mesh, trained on monthly ESM fields.
result Generative models produce distributions closely matching ESM outputs.
SLED improves factuality in LLMs without external knowledge.
problem Unreliable or factually incorrect outputs from large language models.
method Contrasts final layer logits with early layers' logits, uses approximate gradient to refine outputs.
result Consistently improves factual accuracy over existing methods.
Novel framework for Bayesian neural networks incorporating task-specific constraints.
problem Task-specific constraints in supervised model deployment.
method Introduces Output-Constrained BNN (OC-BNN) framework.
result OC-BNNs effectively incorporate prior expert knowledge and desiderata like safety and fairness.
Novel kernel-based PSI algorithm handles non-linearity and structured data.
problem Non-linearity and structured data in independence measures.
method Develops a PSI algorithm using HSIC, capable of handling non-linearity and structured data.
result Successfully identifies important features in real-world data.
Multi-output inference tasks, such as multi-label classification, have become increasingly important in recent years. A popular method for multi-label classification is classifier chains, in which the predictions of individual classifiers are cascaded along a chain, thus taking into account inter-label dependencies and…
New method improves accuracy of quantized neural networks.
problem Accuracy drop in quantized neural networks, especially MobileNet family.
method Weight equalizing shift scaler, binary shifting to recover output range.
result Top-1 accuracy improved from 0.1% to 69.78% ~ 70.96% in MobileNets.