arXiv research

New method uses robust estimators for Newton's method in empirical risk minimization.

New matrix reveals cluster info in sparse directed graphs.

New approach to counterfactual reasoning in AI and psychology.

This paper presents VEC-NBT, a variation on the unsupervised graph clustering technique VEC, which improves upon the performance of the original algorithm significantly for sparse graphs. VEC employs a novel application of the state-of-the-art word2vec model to embed a graph in Euclidean space via random walks on the n…

DeepBC method computes backtracking counterfactuals in deep causal models.

A new graph neural network (NBA-GNN) avoids revisiting nodes to improve accuracy.

A major drawback of backpropagation through time (BPTT) is the difficulty of learning long-term dependencies, coming from having to propagate credit information backwards through every single step of the forward computation. This makes BPTT both computationally impractical and biologically implausible. For this reason,…

New Q-Newton's method avoids saddle points and converges quadratically.

Spectral method detects communities in sparse hypergraphs, achieving detection threshold.

In many environments only a tiny subset of all states yield high reward. In these cases, few of the interactions with the environment provide a relevant learning signal. Hence, we may want to preferentially train on those high-reward states and the probable trajectories leading to them. To this end, we advocate for the…

Develops active intervals for geodesics in Teichmüller space.

Recent studies incorporate Nesterov's accelerated gradient method for the acceleration of gradient based training. The Nesterov's Accelerated Quasi-Newton (NAQ) method has shown to drastically improve the convergence speed compared to the conventional quasi-Newton method. This paper implements NAQ for non-convex optimi…

We propose an L-BFGS optimization algorithm on Riemannian manifolds using minibatched stochastic variance reduction techniques for fast convergence with constant step sizes, without resorting to linesearch methods designed to satisfy Wolfe conditions. We provide a new convergence proof for strongly convex functions wit…

While Standard gradient descent is one very popular optimisation method, its convergence cannot be proven beyond the class of functions whose gradient is globally Lipschitz continuous. As such, it is not actually applicable to realistic applications such as Deep Neural Networks. In this paper, we prove that its backtra…

BRACE generates efficient counterfactual explanations by integrating causal reasoning.

Spectral clustering is a standard approach to label nodes on a graph by studying the (largest or lowest) eigenvalues of a symmetric real matrix such as e.g. the adjacency or the Laplacian. Recently, it has been argued that using instead a more complicated, non-symmetric and higher dimensional operator, related to the n…

We study the problem of learning a good search policy for combinatorial search spaces. We propose retrospective imitation learning, which, after initial training by an expert, improves itself by learning from \textit{retrospective inspections} of its own roll-outs. That is, when the policy eventually reaches a feasible…

Boosting Variational Inference improves posterior approximations with adaptive step-sizes.

Unified spectral clustering for sparse networks with heterogeneous degrees.

Spectral algorithms are classic approaches to clustering and community detection in networks. However, for sparse networks the standard versions of these algorithms are suboptimal, in some cases completely failing to detect communities even when other algorithms such as belief propagation can do so. Here we introduce a…

The study proves unique path lifting properties and their implications on quotient spaces and covering maps.

Motivated by community detection, we characterise the spectrum of the non-backtracking matrix $B$ in the Degree-Corrected Stochastic Block Model. Specifically, we consider a random graph on $n$ vertices partitioned into two equal-sized clusters. The vertices have i.i.d. weights $\{ φ_u \}_{u=1}^n$ with second moment $Φ…

Local search improves GFlowNets' ability to generate high-reward samples.

Variance-reduced algorithms, although achieve great theoretical performance, can run slowly in practice due to the periodic gradient estimation with a large batch of data. Batch-size adaptation thus arises as a promising approach to accelerate such algorithms. However, existing schemes either apply prescribed batch-siz…

A distinguishing property of communities in networks is that cycles are more prevalent within communities than across communities. Thus, the detection of these communities may be aided through the incorporation of measures of the local "richness" of the cyclic structure. In this paper, we introduce renewal non-backtrac…

New method detects communities in complex hypergraphs, matching theoretical limits.

In unconstrained optimisation on an Euclidean space, to prove convergence in Gradient Descent processes (GD) $x_{n+1}=x_n-δ_n \nabla f(x_n)$ it usually is required that the learning rates $δ_n$'s are bounded: $δ_n\leq δ$ for some positive $δ$. Under this assumption, if the sequence $x_n$ converges to a critical point $…

Researchers compute large quantum invariants for 3-manifolds.

There have been several spectral bounds for the percolation transition in networks, using spectrum of matrices associated with the network such as the adjacency matrix and the non-backtracking matrix. However they are far from being tight when the network is sparse and displays clustering or transitivity, which is repr…

New findings support a new community recovery threshold for Stochastic Block Model with many communities.

The paper develops methods for unconstrained optimization on Riemannian manifolds.

Improves generative models by optimizing rewards and sample editing.

New methods solve MI problems with locally Lipschitz operators, improving solution efficiency.

HollowFlow speeds up likelihood evaluation for large-scale models.

We show that the driving force behind the regularizing effect of Laplacian smoothing on surface elements is the popular mean ratio quality measure. We use these insights to provide natural generalizations to polygons and polyhedra. The corresponding functions measuring the quality of meshes are easily seen to be convex…

Stochastic approximation is one of the effective approach to deal with the large-scale machine learning problems and the recent research has focused on reduction of variance, caused by the noisy approximations of the gradients. In this paper, we have proposed novel variants of SAAG-I and II (Stochastic Average Adjusted…

Paper introduces a privacy-preserving line search method for optimization.

Transformer learns to search through reinforcement learning, mimicking DFS.

Paper tackles sample-efficient RL for linearly realizable MDPs with limited revisiting.

We consider the problem of clustering partially labeled data from a minimal number of randomly chosen pairwise comparisons between the items. We introduce an efficient local algorithm based on a power iteration of the non-backtracking operator and study its performance on a simple model. For the case of two clusters, w…

We consider the problem of minimizing a difference-of-convex (DC) function, which can be written as the sum of a smooth convex function with Lipschitz gradient, a proper closed convex function and a continuous possibly nonsmooth concave function. We refine the convergence analysis in [38] for the proximal DC algorithm …

Paper uses RL and diffusion models to solve Bayesian inverse problems.

New formula refutes random CSPs with fewer constraints.

Our main result concerns the following condition: {\bf Condition C.} Let $X$ be a Banach space. A $C^1$ function $f:X\rightarrow \mathbb{R}$ satisfies Condition C if whenever $\{x_n\}$ weakly converges to $x$ and $\lim _{n\rightarrow\infty}||\nabla f(x_n)||=0$, then $\nabla f(x)=0$. We assume that there is given a cano…

New findings on community recovery in SBM with many communities.

Community detection is a fundamental problem in network analysis with many methods available to estimate communities. Most of these methods assume that the number of communities is known, which is often not the case in practice. We study a simple and very fast method for estimating the number of communities based on th…

We say that a subset $S\subseteq F_N$ is \emph{spectrally rigid} if whenever $T_1, T_2\in cv_N$ are points of the (unprojectivized) Outer space such that $||g||_{T_1}=||g||_{T_2}$ for every $g\in S$ then $T_1=T_2$ in $\cvn$. It is well-known that $F_N$ itself is spectrally rigid; it also follows from the result of Smil…

Efficient algorithms solve joint graphical lasso problems.