Paper introduces a new deep-learning method for quantum mechanics.
problem Simulating time-evolving Schrödinger equations efficiently.
method Generative diffusion models and stochastic mechanics.
result Significantly lower computational complexity compared to existing methods.
Study reveals attention mechanism's similarity computation parallels traditional machine learning.
problem Understanding the essence and principles of attention mechanism in deep learning.
method Examined classic metrics and vector space properties in manifold learning, clustering, and supervised learning to identify key characteristics of similarity computation and information propagation.
result Self-attention mechanism in deep learning adheres to the same principles but operates more flexibly and adaptively.
Paper introduces a Gaussian Process for operator learning in computational mechanics.
problem Efficient and accurate solutions for large datasets with reliable uncertainty quantification.
method Gaussian Process (GP) embedded in a neural operator framework with stochastic dual descent (SDD) algorithm.
result Improves GP resolution independence and scalability for high-dimensional and non-linear systems.
New DP mechanisms improve ML privacy-utility-computational tradeoffs.
problem Improving privacy in machine learning with multiple passes over data.
method Formalized DP for adaptive streams, extended matrix factorization techniques, Fourier-transform-based mechanism.
result Substantial improvements in privacy-utility-computational tradeoffs over previous methods.
Expands differential privacy mechanisms to include the Generalized Gaussian mechanism for improved private machine learning.
problem Improving privacy in machine learning algorithms while maintaining utility.
method Introduces and analyzes the Generalized Gaussian (GG) mechanism for differential privacy.
result The GG mechanism provides better performance than the Laplace and Gaussian mechanisms across various values of β.
Investigates the fundamental components of attention mechanisms.
problem Understanding the building blocks of attention in deep learning.
method Classified and studied three key mechanisms: additive, multiplicative output, and synaptic attention.
result Additive activation attention is central in proofs of lower bounds.
New insights into privacy guarantees for subsampled mechanisms under composition.
problem Tight privacy guarantees for the composition of subsampled differentially private mechanisms.
method Addressed confusion points in privacy accounting for subsampled mechanisms, providing examples and counterexamples.
result Privacy guarantees for subsampled mechanisms differ significantly between Poisson subsampling and sampling without replacement.
We propose a new input perturbation mechanism for publishing a covariance matrix to achieve (ε,0)-differential privacy. Our mechanism uses a Wishart distribution to generate matrix noise. In particular, We apply this mechanism to principal component analysis. Our mechanism is able to keep the positive semi-definitene…
Mechanism design for large item sets using topic models.
problem Designing optimal mechanisms for large item sets when exact priors are unknown or intractable.
method Proposes a framework that disentangles statistical estimation and mechanism design, leveraging topic models to reduce complexity.
result Reduces the complexity of mechanism design for large item sets, making it feasible with topic models.
Privacy improves robustness in statistical estimation.
problem Sparse mean estimation under privacy constraints.
method Sum-of-Squares method and exponential-time mechanisms.
result Private algorithms matching optimal tradeoffs are not known, but achieved via Sum-of-Squares.
The softmax content-based attention mechanism has proven to be very beneficial in many applications of recurrent neural networks. Nevertheless it suffers from two major computational limitations. First, its computations for an attention lookup scale linearly in the size of the attended sequence. Second, it does not enc…
A novel incentive mechanism improves fairness and participation in federated learning.
problem Low-quality clients and lack of fairness in federated learning.
method Client selection process and money transfer mechanism to ensure fairness and participation.
result The proposed incentive mechanism improves the duration and fairness of federated learning.
FEA-Net uses physics knowledge to predict material responses efficiently.
problem Predicting material mechanical responses accurately and efficiently.
method Physics-guided deep learning with FEA integration.
result FEA-Net accurately predicts mechanical responses under external loading.
Although deep neural networks generally have fixed network structures, the concept of dynamic mechanism has drawn more and more attention in recent years. Attention mechanisms compute input-dependent dynamic attention weights for aggregating a sequence of hidden states. Dynamic network configuration in convolutional ne…
Optimal Gaussian noise mechanisms achieve nearly optimal error in unbiased mean estimation.
problem Efficiently estimating the mean of high-dimensional data while preserving privacy.
method Differential privacy mechanisms with Gaussian noise, focusing on optimal covariance.
result Gaussian noise mechanisms achieve nearly optimal error among all private unbiased mean estimation mechanisms.
New mechanisms improve differential privacy for scalar queries.
problem Improving differential privacy for scalar, real-valued query functions.
method Mixing multiple Gaussian distributions to satisfy differential privacy.
result Mechanisms yield lower noise amplitudes and variances compared to the analytic Gaussian mechanism.
Efficient trainable front-end for neural speech enhancement.
problem Inefficient STFT front-ends in neural speech enhancement models.
method Butterfly mechanism for Fast Fourier Transform, trainable STFT window.
result Accuracy and efficiency improvements for low-compute systems.
Chemical transport models (CTMs), which simulate air pollution transport, transformation, and removal, are computationally expensive, largely because of the computational intensity of the chemical mechanisms: systems of coupled differential equations representing atmospheric chemistry. Here we investigate the potential…
Paper tackles dynamic behavior of variable topology mechanisms, presenting new transition conditions.
problem Dynamic behavior of mechanisms with changing kinematic topology.
method Presented new transition conditions for variable topology mechanisms using projected motion equations and Voronets equations.
result Results show the dynamic behavior of joint locking in 3R and 6DOF mechanisms.
Attention mechanisms in deep neural networks have achieved excellent performance on sequence-prediction tasks. Here, we show that these recently-proposed attention-based mechanisms---in particular, the Transformer with its parallelizable self-attention layers, and the Memory Fusion Network with attention across modalit…
A simple method to create new 4-manifolds by altering fundamental groups.
problem Creating new 4-manifolds with specific fundamental groups.
method Cut-and-paste mechanism to alter fundamental groups.
result Constructs new 4-manifolds with fixed-point free involutions.
The paper improves privacy accounting for discrete-valued mechanisms and the subsampled Gaussian mechanism.
problem Improving the accuracy and efficiency of differential privacy accounting for discrete outputs.
method Uses fast Fourier transform (FFT) for rigorous error analysis and accounting of privacy loss.
result Provides strict lower and upper bounds for (ε,δ)-values, demonstrating up to 75% reduction in noise variance. Identifying computational mechanisms for memorization and retrieval of data is a long-standing problem at the intersection of machine learning and neuroscience. Our main finding is that standard overparameterized deep neural networks trained using standard optimization methods implement such a mechanism for real-valued…
This paper demonstrates the use of genetic algorithms for evolving: 1) a grandmaster-level evaluation function, and 2) a search mechanism for a chess program, the parameter values of which are initialized randomly. The evaluation function of the program is evolved by learning from databases of (human) grandmaster games…
Study asymptotics of unitary matrix elements in quantum mechanics.
problem Asymptotic behavior of unitary matrix elements in quantum mechanics.
method Uses Berezin-Toeplitz quantization and symplectic geometry.
result Recover asymptotics of Wigner's d-matrix elements for spin representations.
Econophysics provides a strategy for understanding the potential mechanisms underlying the anomalous distribution of wealth found in real societies. We present a computational nonlinear stochastic model for the distribution of wealth that depends upon three parameters and two mechanisms: trade and investment. To avoid …
We introduce preferential behavior into the study on statistical mechanics of money circulation. The computer simulation results show that the preferential behavior can lead to power laws on distributions over both holding time and amount of money held by agents. However, some constraints are needed in generation mecha…
We introduce a theory-driven mechanism for learning a neural network model that performs generative topology design in one shot given a problem setting, circumventing the conventional iterative process that computational design tasks usually entail. The proposed mechanism can lead to machines that quickly response to n…
Paper proposes an attention sampler for reducing attention mechanism computation.
problem Computational challenges in large-scale attention-based models.
method Importance sampling in streaming setting, attention sampler.
result Significantly reduces the computational burden of attention mechanisms.
New mechanisms from primate vision improve neural network robustness.
problem Demonstrating robust neural networks to small adversarial perturbations.
method Investigated two biologically plausible mechanisms: non-uniform retina sampling and receptive field diversity.
result Non-uniform retina sampling and receptive field diversity improve adversarial robustness.
Unified Bayesian framework for uncertainty quantification in mechanics.
problem Propagation of input uncertainties and inference of unknown parameters.
method Bayesian probability theory for forward and inverse problems.
result Unified theoretical framework for both forward and inverse UQ.
Modern neural networks are often augmented with an attention mechanism, which tells the network where to focus within the input. We propose in this paper a new framework for sparse and structured attention, building upon a smoothed max operator. We show that the gradient of this operator defines a mapping from real val…
Analyzes unsupervised neural networks using statistical mechanics and Monte Carlo simulations.
problem Understanding computational capabilities of unsupervised neural networks.
method Statistical mechanics approach and Monte Carlo simulations.
result Obtained a phase diagram summarizing network performance.
We consider the problem of fitting a linear model to data held by individuals who are concerned about their privacy. Incentivizing most players to truthfully report their data to the analyst constrains our design to mechanisms that provide a privacy guarantee to the participants; we use differential privacy to model in…
Formulates mechanics for probability distributions on statistical manifold.
problem Formulating mechanics for probability distributions on statistical manifold.
method Information-geometric formulation of Classical Mechanics on statistical manifold, using dually-flat connection and Hilbert bundle structure.
result Provides coherent formalism for Lagrangian and Hamiltonian mechanics on statistical bundle.
New method for efficient matrix completion with nonignorable missing data.
problem Nonignorable missing data in matrix completion.
method Nuclear norm regularized U-statistic loss function and accelerated proximal gradient algorithm.
result Near minimax optimal statistical convergence rate for nonignorable missing data.
This study presents a rapid multiple incremental and decremental mechanism based on Weight-Error Curves (WECs) for support-vector analysis. Recursion-free computation is proposed for predicting the Lagrangian multipliers of new samples. This study examines Ridge Support Vector Models, subsequently devising a recursion-…
We develop a geometric version of the inverse problem of the calculus of variations for discrete mechanics and constrained discrete mechanics. The geometric approach consists of using suitable Lagrangian and isotropic submanifolds. We also provide a transition between the discrete and the continuous problems and propos…
GOAT improves attention mechanisms by learning better priors.
problem Standard attention mechanisms use a naive uniform prior, limiting flexibility and generalization.
method GOAT introduces a trainable, continuous prior that replaces the uniform assumption, maintaining compatibility with optimized kernels.
result GOAT avoids representational trade-offs and learns an extrapolatable prior that combines positional flexibility with length generalization.
New method uses quantum annealing and VAN for better statistical mechanics calculations.
problem Difficulty in computing partition function in statistical mechanics.
method Combines quantum annealing samples with variational autoregressive networks.
result Enhanced accuracy in finite-size Sherrington-Kirkpatrick model.
A new mechanism for differentially private Fréchet mean on SPD matrices.
problem Privacy-preserving statistical summaries for SPD matrices.
method Tangent Gaussian mechanism for log-Euclidean metric.
result Significantly better utility and computational efficiency.
Computing equilibrium states in condensed-matter many-body systems, such as solvated proteins, is a long-standing challenge. Lacking methods for generating statistically independent equilibrium samples in "one shot", vast computational effort is invested for simulating these system in small steps, e.g., using Molecular…
A computational model for the distribution of wealth among the members of an ideal society is presented. It is determined that a realistic distribution of wealth depends upon two mechanisms: an asymmetric flux of wealth in trading transactions that advantages the poorer of the two traders and a non-stationary creation …
This paper presents a framework for estimating the remaining useful life (RUL) of mechanical systems. The framework consists of a multi-layer perceptron and an evolutionary algorithm for optimizing the data-related parameters. The framework makes use of a strided time window to estimate the RUL for mechanical component…
Local laGPR speeds up multiscale mechanics simulations without neural networks.
problem High computational costs in multiscale mechanics simulations.
method Local approximate Gaussian process regression (laGPR) combined with FE schemes.
result laGPR offers better accuracy than neural networks for stress predictions.
This paper analyzes MFVBI for GMM using statistical mechanics.
problem Approximate fast computation of Gaussian Mixture Model.
method Statistical mechanics and MFVBI applied to GMM.
result Rigorous analysis and mathematical foundation for MFVBI applied to GMM.
Paper develops a framework to discover bioprocessing regulatory mechanisms using symbolic and statistical learning.
problem Challenges in modeling complex intracellular regulation, stochastic system behavior, and limited experimental data.
method Symbolic and statistical learning framework based on stochastic differential equations and Bayesian learning.
result Improved sample efficiency and robust model selection compared to state-of-the-art approaches.
Algorithm learns optimal dynamic mechanisms from data.
problem Designing optimal mechanisms for dynamic settings with unknown reward functions.
method Offline reinforcement learning with pessimism principle.
result Learned mechanisms are efficient, individually rational, and truthful.