arXiv research

Adversarial MoE learns category-specific models for product search.

We discuss an autoencoder model in which the encoding and decoding functions are implemented by decision trees. We use the soft decision tree where internal nodes realize soft multivariate splits given by a gating function and the overall output is the average of all leaves weighted by the gating values on their path. …

Dropout is a very effective method in preventing overfitting and has become the go-to regularizer for multi-layer neural networks in recent years. Hierarchical mixture of experts is a hierarchically gated model that defines a soft decision tree where leaves correspond to experts and decision nodes correspond to gating …

Long Short-Term Memory (LSTM) is one of the most widely used recurrent structures in sequence modeling. It aims to use gates to control information flow (e.g., whether to skip some information or not) in the recurrent computations, although its practical implementation based on soft gates only partially achieves this g…

This study analyzes communication constraints in MoE architectures using information theory.

APINNs improve physics-informed neural networks through flexible domain decomposition.

Meta-gradient D4PG optimizes performance and constraint adherence in RL.

In this article, we propose a novel ensemble technique with a multi-scheme weighting based on a technique called coopetitive soft gating. This technique combines both, ensemble member competition and cooperation, in order to maximize the overall forecasting accuracy of the ensemble. The proposed algorithm combines the …

In this article, we propose the Coopetititve Soft Gating Ensemble or CSGE for general machine learning tasks and interwoven systems. The goal of machine learning is to create models that generalize well for unknown datasets. Often, however, the problems are too complex to be solved with a single model, so several model…

HyperFair integrates fairness in recommender systems using probabilistic soft logic.

Federated Learning with L0 constraint improves sparsity and performance.

ConViT combines CNN and ViT strengths, improving image classification.

LightGCNet simplifies AI for soft sensors, reducing complexity and training time.

RPPs improve deep learning models with soft equivariance constraints.

Framework learns stochastic dynamics from endpoint and intermediate distributions using soft energy constraints.

New model optimizes portfolios with realistic transaction costs.

Generative AI connects to Schrödinger bridge problems with soft constraints for stability.

Soft-Radial Projection solves gradient saturation in constrained deep learning.

Model accurately gates ocean microbes from high-frequency flow cytometry data.

New method combines gradient optimization with constraint-based techniques for causal discovery.

We propose a novel adaptive approximation approach for test-time resource-constrained prediction. Given an input instance at test-time, a gating function identifies a prediction model for the input among a collection of models. Our objective is to minimize overall average cost without sacrificing accuracy. We learn gat…

PolyLUT uses polynomials to reduce FPGA latency.

We present a dynamic model selection approach for resource-constrained prediction. Given an input instance at test-time, a gating function identifies a prediction model for the input among a collection of models. Our objective is to minimize overall average cost without sacrificing accuracy. We learn gating and predict…

Automatically learns flexible symmetry constraints in neural networks using gradients.

In industrial systems, certain process variables that need to be monitored for detecting faults are often difficult or impossible to measure. Soft sensor techniques are widely used to estimate such difficult-to-measure process variables from easy-to-measure ones. Soft sensor modeling requires training datasets includin…

Study reveals biases in gradient descent for GLNs, improving neural network performance.

ValueBlindBench tests LLM-generated investment rationales for validity before returns are known.

Study stability of selective SSMs with discontinuous gating.

Hybrid quantum-classical method optimizes financial index tracking.

Paper proposes a new sparse group k-max regularization for sparsity constraints.

Extended CSGE improves power and cyclist movement forecasting.

Iterative thresholding algorithms seek to optimize a differentiable objective function over a sparsity or rank constraint by alternating between gradient steps that reduce the objective, and thresholding steps that enforce the constraint. This work examines the choice of the thresholding operator, and asks whether it i…

Optimal control problems on Riemannian manifolds are solved by penalizing constraint violations.

FedSGM tackles constrained federated learning with unified framework.

GIB improves neural network generalization by dynamically selecting task-relevant features across different sequential environments.

Entropy regularization improves sparse model discovery in federated learning.

Soft-constrained PINN solves ODEs with minimal data, improving efficiency and robustness.

2D-PT improves sampling in constrained optimization problems.

New algorithm improves on EM for streaming data, outperforming existing methods.

Study improves fraud detection in e-commerce with a stacked model combining CNNs, GNNs, and confidence gating.

Hybrid SAC improves RL for video games with discrete, continuous actions.

We propose to jointly learn multi-view geometry and warping between views of the same object instances for robust cross-view object detection. What makes multi-view object instance detection difficult are strong changes in viewpoint, lighting conditions, high similarity of neighbouring objects, and strong variability i…

An important form of prior information in clustering comes in form of cannot-link and must-link constraints. We present a generalization of the popular spectral clustering technique which integrates such constraints. Motivated by the recently proposed $1$-spectral clustering for the unconstrained problem, our method is…

The present work extends the randomized shortest-paths framework (RSP), interpolating between shortest-path and random-walk routing in a network, in three directions. First, it shows how to deal with equality constraints on a subset of transition probabilities and develops a generic algorithm for solving this constrain…

CriticSMC improves planning efficiency in constrained environments.

HS-MoE selects sparse experts using adaptive priors and data-adaptive gating.

One primary focus in multimodal feature extraction is to find the representations of individual modalities that are maximally correlated. As a well-known measure of dependence, the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation becomes an appealing objective because of its operational meaning and desirable propert…

Localized Multidirectional Correction improves non-refusal target-response behavior in foundation models.