Paper accelerates L-BFGS with second-order info and stochastic batches.
problem Optimizing finite-sum minimization problems.
method Proposes L-BFGS with smooth gradient difference estimates and well-scaling initial Hessians.
result Achieves acceleration in both convex and nonconvex cases.
AdaSub optimizes with second-order info in low-dims subspace.
problem Efficiently use second-order optimization methods with low computational cost.
method Adaptive subspace selection for second-order optimization.
result AdaSub outperforms other stochastic optimizers in time and iterations.
Enhances SMC² with Hessian info for more efficient posterior approximation.
problem Improving accuracy and efficiency in Bayesian inference.
method Integrates second-order information (Hessian) into SMC²'s proposal distribution.
result Second-order proposals lead to more accurate posterior approximations and better step-size selection.
Introduces info intervention to handle causal questions and check counterfactual variables.
problem Controversial interpretation of causal questions for non-manipulable variables and lack of power to check counterfactual variables.
method Intervenes input/output information of causal mechanisms, providing causal diagrams for communication and theoretical focus.
result Causal diagrams based on info intervention provide a new perspective on information transfer as causality.
We present an information-theoretic framework for sequential adaptive compressed sensing, Info-Greedy Sensing, where measurements are chosen to maximize the extracted information conditioned on the previous measurements. We show that the widely used bisection approach is Info-Greedy for a family of k-sparse signals b…
BackPACK extends PyTorch to compute additional gradient info.
problem Lack of efficient tools for computing mini-batch variance and Hessian approximations.
method BackPACK builds on PyTorch to automatically compute additional derivatives.
result BackPACK enables efficient computation of various derivative quantities.
We characterize the performance of sequential information guided sensing, Info-Greedy Sensing, when there is a mismatch between the true signal model and the assumed model, which may be a sample estimate. In particular, we consider a setup where the signal is low-rank Gaussian and the measurements are taken in the dire…
Deep reinforcement learning has shown its success in game playing. However, 2.5D fighting games would be a challenging task to handle due to ambiguity in visual appearances like height or depth of the characters. Moreover, actions in such games typically involve particular sequential action orders, which also makes the…
Study multiplayer bandits without collision info, achieving regret bounds.
problem Multiplayer bandits without collision info.
method Two feedback models considered; algorithms for both models.
result First theoretical guarantees for second model with square-root regret.
Proper balance between exploitation and exploration is what makes good decisions, which achieve high rewards like payoff or evolutionary fitness. The Infomax principle postulates that maximization of information directs the function of diverse systems, from living systems to artificial neural networks. While specific a…
Rating Prediction is a basic problem in Recommender System, and one of the most widely used method is Factorization Machines(FM). However, traditional matrix factorization methods fail to utilize the benefit of implicit feedback, which has been proved to be important in Rating Prediction problem. In this work, we consi…
Estimates non-parametric logistic model using case-control data and external summary info.
problem Imbalanced binary data in case-control studies.
method Two-step estimation procedure with deep neural network for functional approximation.
result Proposed estimator achieves optimal convergence rate in non-parametric regression.
VTrackIt creates a synthetic dataset with infrastructure and vehicle info for AVs.
problem Lack of infrastructure and pooled vehicle info in existing AV datasets.
method Developed VTrackIt, a synthetic dataset with intelligent infrastructure and pooled vehicle info, and introduced InfraGAN for trajectory predictions.
result VTrackIt reduces high-risk edge cases in AV trajectory predictions.
Study achieves fairness without demographic info, improving regression tasks.
problem Achieve fairness in models without prior demographic info.
method VFair method to minimize training loss variance, dynamic update approach.
result Regression tasks can achieve significant fairness improvement without prior demographics.
ViterbiNet uses deep learning to detect symbols without channel info.
problem Symbol detection in digital receivers without channel state info.
method Integrates DNNs into Viterbi algorithm, leaving structure intact.
result Performance of ViterbiNet approaches CSI-based Viterbi, tracks dynamic channels.
β-VAEs retain label info even at high compression.
problem Retaining label information in compressed representations.
method Investigation of β-VAE architectures on Binary Static MNIST and Omniglot. result β-VAEs can retain label information even at high compression.
MFAI uses gradient boosted trees to leverage auxiliary info for scalable Bayesian matrix factorization.
problem Matrix factorization struggles with poor data quality, especially high sparsity and low SNR.
method Integrates gradient boosted trees into probabilistic matrix factorization framework.
result MFAI effectively leverages auxiliary information, improving model performance.
Paper tackles circularity issues in machine learning predictions.
problem Circularity problems in machine learning predictions.
method Not specified in the abstract.
result Not specified in the abstract.
Improved semantic segmentation accuracy by addressing class imbalance.
problem Class imbalance in training data leads to misclassification of rare classes.
method Localized weighting of posterior class probabilities with pixel-wise priors.
result Significant improvement in recall and reduction of non-detection rate for rare classes.
Bayesian optimization with binary auxiliary info for faster target function optimization.
problem Optimizing target functions with expensive binary auxiliary information.
method Mixed-type Gaussian process (MOGP) and information-based acquisition functions (MT-ES, MT-PES).
result Efficient approximation of mixed-type predictive ES via random features.
Second-order economic theory considers new variables to improve price volatility predictions.
problem Current economic models focus on first-order variables, missing second-order variables that affect price volatility.
method Introduces second-order economic theory with new variables composed of sums of squares of agents' transactions.
result Second-order economic theory complements first-order variables and introduces new macroeconomic variables.
Second-order optimizers retain residual information after data deletion, affecting machine unlearning.
problem Residual information in second-order optimizers after data deletion.
method Comparison of first-order and second-order learners, eigendecomposition analysis.
result Second-order optimizers retain residual information, not detectable by first-order analysis.
Exact second-order optimization for deep learning reduces computational cost and improves performance.
problem Inadequate use of second-order optimization methods in deep learning due to high computational cost and non-convexity.
method Developed an exact stochastic second-order Newton method that addresses the non-convexity issue and provides an expression for the stochastic Hessian.
result Exact second-order Newton direction formula and its application in deep learning datasets.
SOLBP extends efficient inference to uncertain Bayesian networks.
problem Inference in uncertain Bayesian networks with second-order probabilities.
method Extends Loopy Belief Propagation to second-order Bayesian networks.
result Generates inferences consistent with sum-product networks, more efficient and scalable.
Developed a theory of local convexity for second order differential equations on Lie algebroids.
problem Analyzing convexity in differential equations on Lie algebroids.
method Theory development for local convexity of SODEs on Lie algebroids.
result Extensive discussion of homogeneous quadratic SODEs on Lie algebroids.
Paper examines risk measure expansions under FGM dependence, improving accuracy at extreme levels.
problem Capturing higher-order tail behavior and dependence effects in risk measures.
method Second-order asymptotic expansions using extreme value theory and regular variation theory.
result Second-order approximations reduce approximation errors, especially at extreme confidence levels.
Negative step sizes improve second-order methods for neural networks.
problem Second-order methods discard negative curvature, limiting their effectiveness.
method Introduce negative step sizes in second-order methods combined with Wolfe line search.
result Negative step sizes lead to global convergence and improved performance.
Simplified argument for second order estimate in quaternionic Calabi-Yau problem.
problem Second order estimates for quaternionic Calabi-Yau problem on hyperkähler manifolds.
method Simplified argument to derive the estimate.
result Simplified derivation of second order estimate.
Paper studies second order tail probabilities in risk models.
problem Analyzing tail probabilities in risk models with constant interest force.
method Asymptotic expansion and weighted Kesten-type inequality for second order subexponential random variables.
result Second order asymptotic formulae for continuous-time renewal risk models are derived.
A dynamical system on the total space of the fibre bundle of second order accelerations, T2M, is defined as a third order vector field S on T2M, called semispray, which is mapped by the second order tangent structure into one of the Liouville vector field. For a regular Lagrangian of second order we prove that …
Paper studies second order symmetric parallel tensors in generalized f.pk-space forms.
problem Exploring properties of second order symmetric parallel tensors in generalized f.pk-space forms.
method Analyzes the properties of second order symmetric parallel tensors and deduces the existence or non-existence of certain tensors and hypersurfaces.
result There does not exist second order skew-symmetric parallel tensor in f.pk-space form. There is no parallel hypersurface in a generalized f.pk-space form but there is semi-parallel hypersurface.
New method controls linear systems with partial info and disturbances.
problem Controlling linear dynamical systems under partial observation and adversarial disturbances.
method Double Spectral Control (DSC) using two-level spectral approximation strategy.
result Matches best known regret guarantees with exponential runtime improvement.
Paper generalizes connections between Lie groups and affine connections.
problem Exploring properties of infinitesimal groups and affine connections.
method Introducing second-order infinitesimal groups and using them to define Lie brackets and connections.
result Generalized correspondence between symmetric and non-symmetric affine connections.
In this present paper, we study geometric structures of rank two prolongations of implicit second-order partial differential equations (PDEs) for two independent and one dependent variables and characterize the type of these PDEs by the topology of fibers of the rank two prolongations. Moreover, by using properties of …
Modified Newton step for online learning reduces matrix size for large datasets.
problem Handling large multi-class datasets efficiently in online learning.
method Element-wise multiplication to reduce matrix size of second order matrices.
result Proposed method achieves similar mistake rates to popular methods but with faster computations.
In this paper, we study stochastic non-convex optimization with non-convex random functions. Recent studies on non-convex optimization revolve around establishing second-order convergence, i.e., converging to a nearly second-order optimal stationary points. However, existing results on stochastic non-convex optimizatio…
AdamQLR optimizes Adam with K-FAC heuristics, achieving comparable performance to tuned benchmarks.
problem Improving the performance of Adam optimizers with stabilizing heuristics.
method Combining Adam's update directions with K-FAC's heuristics (damping and learning rate selection).
result Untuned AdamQLR can achieve comparable performance to tuned benchmarks.
The quantification of diversification benefits due to risk aggregation plays a prominent role in the (regulatory) capital management of large firms within the financial industry. However, the complexity of today's risk landscape makes a quantifiable reduction of risk concentration a challenging task. In the present pap…
We show that, for mechanical system with external forces, the equations of deviations of solution curves of the corresponding Lagrange equations,determine a nonlinear connection on the second order osculator (second order tangent) bundle. In particular, Jacobi equations in Finsler and Riemann spaces determine such a no…
A first-order model for a stock market assigns to each stock a return parameter and a variance parameter that depend only on the rank of the stock. A second-order model assigns these parameters based on both the rank and the name of the stock. First- and second-order models exhibit stability properties that make them a…
Second-order guarantees for federated learning algorithms.
problem Non-convex optimization in federated learning with saddle-points as bottlenecks.
method Drawing on recent results on second-order optimality in centralized and decentralized settings, establish second-order guarantees for federated learning algorithms.
result Established second-order guarantees for federated learning algorithms.
New findings show second-order scoring rules can't accurately represent epistemic uncertainty.
problem Lack of epistemic uncertainty representation in second-order learners.
method Generalised second-order scoring rules introduced to prove theoretical limitations.
result No loss function incentivizes second-order learners to accurately represent epistemic uncertainty.
The paper explores parabolic regularity in geometric variational analysis.
problem Developing calculus rules and computation formulas for second-order generalized differential constructions.
method Introducing and applying the concept of parabolic regularity to geometric aspects of second-order variational analysis.
result Established new calculus rules and computation formulas for second-order generalized differential constructions.
New method uses mutual info and network science to explain deep learning models.
problem Interpreting deep neural networks for understanding their decision-making process.
method Coupling mutual information with network science to quantify information flow in deep learning models.
result Proposed NIF technique for codifying information flow in deep learning models.
Paper learns robot's end-effector position without prior info.
problem Lack of prior robot structure or sensor info.
method Generates internal end-effector config from raw data.
result Can control robot without prior kinematic info.
New method explains predictive uncertainty by focusing on second-order effects.
problem Explaining predictive uncertainty in machine learning models.
method CovLRP, CovGI, etc., based on second-order effects.
result Predictive uncertainty is dominated by second-order effects.
AMM models complex data with minimal labeled info.
problem Clustering complex data with little labeled info.
method Adversarial optimization of mixture models.
result Achieves 2.86% unsupervised clustering error on MNIST.
We apply the Cartan equivalence method to the study of real analytic second order ODEs under the local real analytic diffeomorphism of $\C^2$ which are area-preserving. This enables us to give a characterization of the second order ODEs which are equivalent to y′′=0 under such transformations. Moreover w…