Improved math problem solvers using Transformer networks and diverse notations.
problem Challenges in constructing accurate and automatic solvers for math word problems.
method Transformer networks trained to translate math word problems to arithmetic expressions in infix, prefix, and postfix notations. Pre-training on general text corpus to improve performance.
result Significant improvements in accuracy, up to 10% over previous state of the art.
A new robust prefix-tuning framework improves model robustness against adversarial attacks.
problem Lack of robustness in prefix-tuning for adversarial attacks.
method Leveraging layerwise activations of pretrained models for additional prefix finetuning during the test phase.
result Framework substantially improves robustness over strong baselines while maintaining comparable accuracy on clean texts.
CROP verifies clean prefixes in reasoning traces, improving downstream repair accuracy.
problem Uncertainty in reasoning traces prevents full certification of entire responses.
method CROP selects a calibrated threshold to certify the longest prefix with low risk proxies.
result CROP improves downstream repair accuracy by preserving valid reasoning and discarding misleading suffixes.
Prefix consistency improves model reliability by weighting answers based on their reproducibility.
problem Improving the reliability of large language models' reasoning traces.
method Use prefix consistency to weight candidate answers based on their reproducibility during regeneration.
result Prefix consistency is the best correctness predictor, reaching Standard MV plateau accuracy with up to 21x fewer tokens.
Improved text generation with discourse-aware soft prompts.
problem Efficient fine-tuning methods don't generalize across all generation tasks.
method Hierarchical blocking and attention sparsity on prefix parameters.
result Structured design of prefix parameters yields more coherent generations.
ReOPD uses pre-collected teacher trajectories to distill knowledge from multi-turn interactions.
problem The cost of fully online on-policy distillation for multi-turn interactions.
method ReOPD, an off-environment alternative that reuses pre-collected teacher trajectories as replayed prefixes, addressing the prefix trap and distribution shift.
result ReOPD preserves or improves OPD-level accuracy, uses zero tool calls, and is at least 4imes faster per training step. Proposes a new notation for biracks to simplify rack structure analysis.
problem Simplifying birack notation for easier rack structure analysis.
method Introduces a new notation that includes rack structure knowledge from the start.
result Generalizes results from involutive to non-involutive biracks and clarifies rack structure relations.
Graphical notation simplifies tensor operations and decompositions.
problem Complex tensor operations are difficult to understand and represent.
method Introduces graphical notation to represent tensor operations.
result Simplified representation of tensor operations and decompositions.
Unified notation simplifies information-theoretic concepts in machine learning.
problem Opaque notation for information-theoretic quantities in machine learning.
method Proposed a practical and unified notation for information-theoretic quantities.
result Unified notation facilitates new intuitions and rederivations in machine learning.
Graphical notation simplifies complex polynomial constraints in linear models.
problem Complex polynomial constraints in linear structural equation models are impractical.
method Developed a graphical notation to represent these constraints.
result The graphical notation simplifies the representation of many polynomial constraints.
Formula found for a specific knot's A-polynomial.
problem Computing the A-polynomial of a specific knot.
method Explicit formula derived for the knot with Conway's notation C(2n, 4).
result The A-polynomial contains exactly the same irreducible factors as the one defined in~\cite{CCGLS1}.
Exposure bias has been regarded as a central problem for auto-regressive language models (LM). It claims that teacher forcing would cause the test-time generation to be incrementally distorted due to the training-generation discrepancy. Although a lot of algorithms have been proposed to avoid teacher forcing and theref…
Paper simplifies concentration inequalities for easier probabilistic analysis.
problem Complexity in probabilistic analysis of random variables.
method Compact notations for concentration inequalities.
result Simplified expressions for typical sizes and tails of random variables.
KVCOMM optimizes multi-agent LLM systems by reusing KV-caches, reducing redundant processing.
problem Substantial overhead from reprocessing overlapping contexts across multi-agent systems.
method KVCOMM reuses KV-caches and adjusts offsets for shared content using a pool of cached examples (anchors).
result Achieves over 70% reuse rate across diverse multi-agent tasks, up to 7.8x speedup.
We construct an integer polynomial whose coefficients enumerate the Kauffman states of the two-bridge knot with Conway's notation C(n,r).
An explicit formula for the A-polynomial of the knot with Conway's notation C(2n,3) is obtained from the explicit Riley-Mednykh polynomial of it.
We describe a method of encoding various types of link diagrams, including those with classical, flat, rigid, welded, and virtual crossings. We show that this method may be used to encode link diagrams, up to equivalence, in a notation whose length is a cubic function of the number of 'riser marks'. For classical knots…
Autostackability for finitely generated groups is defined via a topological property of the associated Cayley graph which can be encoded in a finite state automaton. Autostackable groups have solvable word problem and an effective inductive procedure for constructing van Kampen diagrams with respect to a canonical fini…
Study risk-constrained Kelly optimization for mutually exclusive outcomes, proving support invariance and developing a structured algorithm.
problem Risk-constrained Kelly optimization for mutually exclusive outcomes with explicit state prices.
method Analyzes the finite mutually exclusive outcome version of risk-constrained Kelly optimization with explicit state prices, proving support invariance and developing a structured algorithm.
result Support is invariant across CRRA parameter and drawdown-surrogate parameter in the overround regime.
We provide a proof of backpropagation algorithm in matrix notation.
problem The lack of a full induction proof of backpropagation algorithm in matrix notation.
method We provide a full induction proof of the BP algorithm in matrix notation, situating it in the framework of matrix differential calculus.
result We prove the validity of the backpropagation algorithm in inductive form.
This paper explains how to optimize prompts for model adaptation.
problem Understanding and optimizing prompt tuning for model adaptation.
method Bayesian view and meta-learning to explain prompt optimization.
result Optimal prompting can be studied formally as conditioning Bayesian predictors.
New algorithm reduces regret in private online learning with optimal gap-dependent rate.
problem Optimal gap-dependent regret rate for private stochastic decision-theoretic online learning.
method Horizon-free pure-DP algorithm with exponential block partitioning and softmax selection.
result Explicit regret bound of 1000⋅(ΔminlogK+εlogK). We review the actions of the supergravity theory in eleven dimensions as well as the type IIA and IIB supergravities in ten dimensions and derive the bosonic equations of motion in a coordinate-free notation. We also consider the existence of supersymmetries and the associated generalized Killing spinor equations. The …
This article contains general formulas for Tutte and Jones polynomial for families of knots and links given in Conway notation.
We calculate the Chern-Simons invariants of the hyperbolic orbifolds of the knot with Conway's notation C(2n,3) using the Schläfli formula for the generalized Chern-Simons function on the family of C(2n,3) cone-manifold structures. We present the concrete and explicit formula of them. We apply the general instruct…
Improved bounds on acylindricity for right-angled Artin groups.
problem Bounding the acylindrical action of right-angled Artin groups on their extension graphs.
method Exploring lattice properties, studying prefixes of powers, and extending quasi-root uniqueness.
result Cardinality of r-quasi-stabilizer is bounded by a linear function of r. A new method improves language model extrapolation without changing training.
problem Improving language model performance on sequences not seen during training.
method Perturbation-based training procedure for language models.
result The method consistently improves out-of-support prediction.
As deep learning techniques advance more than ever, hyper-parameter optimization is the new major workload in deep learning clusters. Although hyper-parameter optimization is crucial in training deep learning models for high model performance, effectively executing such a computation-heavy workload still remains a chal…
We consider the problem of path inference: given a path prefix, i.e., a partially observed sequence of nodes in a graph, we want to predict which nodes are in the missing suffix. In particular, we focus on natural paths occurring as a by-product of the interaction of an agent with a network---a driver on the transporta…
We improve private training accuracy with learning rate schedules and matrix factorizations.
problem Private training with learning rate schedules and correlated noise.
method General upper and lower bounds for learning rate schedules, memory-efficient constructions, and schedule-aware factorizations.
result Schedule-aware factorizations improve accuracy in private training.
We introduce a matrix representation of a chord on a tangle which leads us to representing tangle chord diagrams as stacks of matrices that we call books. We show that band sum moves, Reidemeister moves as well as orientation changes are implemented on \widetilde{Z}_f - a framed link invariant constructed from the Kont…
For families of knots and links given in Conway notation we compute lower maximal and upper minimal bound of hyperbolic volume by using source links and augmented links.
A new fairness metric for decision-making algorithms, conditioning on known fair variables.
problem Fairness issues in decision-making systems.
method Conditional fairness metric, Derivable Conditional Fairness Regularizer (DCFR), adversarial representation.
result Traditional fairness notations are special cases of the new conditional fairness notation.
The paper identifies patterns in language model weights used for memorizing paragraphs.
problem Locating the specific mechanisms and weights used by language models to memorize paragraphs.
method Examined gradients and attention patterns in language models to identify memorized paragraphs.
result Gradients of memorized paragraphs have a distinguishable spatial pattern, and localized attention heads are involved in paragraph memorization.
This article contains general formulas for Tutte and Jones polynomials for families of knots and links given in Conway notation and "portraits of families"-- plots of zeroes of their corresponding Jones polynomials.
After defining reduced minimum braid word and criteria for a braid family representative, different braid family representatives are derived, and a correspondence between them and families of knots and links given in Conway notation is established.
The following discourse is inspired by the works on hyperbolic groups of Epstein, and Neumann/Reeves. Epstein showed that geometrically finite hyperbolic groups are biautomatic. Neumann/Reeves showed that virtually central extensions of word hyperbolic groups are biautomatic. We prove the following generalisation: Theo…
In this paper, we present a new comparative study on automatic essay scoring (AES). The current state-of-the-art natural language processing (NLP) neural network architectures are used in this work to achieve above human-level accuracy on the publicly available Kaggle AES dataset. We compare two powerful language model…
Probabilistic models can handle causal inference without special tools.
problem Confusion over necessary tools for causal inference.
method Demonstrated through concrete examples that causal questions can be answered using standard probabilistic models.
result Causal questions can be addressed using standard probabilistic modelling and inference.
This paper contains general formulae for the reduced relative Tutte, Kauffman bracket and Jones polynomials of families of virtual knots and links given in Conway notation and discussion of a counterexample to the Z-move conjecture of Fenn, Kauffman and Manturov.
Improved financial VA intent classification accuracy.
problem Determining user intents for unseen open intents.
method Supervised pre-training of intent representations using prefix-tuning and fine-tuning.
result 1.63% - 2.07% higher accuracy on banking77 benchmark.
For the numerical solution of the American option valuation problem, we provide a script written in MATLAB implementing an explicit finite difference scheme. Our main contribute is the definition of a posteriori error estimator for the American options pricing which is based on Richardson's extrapolation theory. This e…
Braids can be represented geometrically as laminations of punctured disks. The geometric complexity of a braid is the minimal complexity of a lamination that represents it, and tight laminations are representatives of minimal complexity. These laminations give rise to a normal form of braids, via a relaxation algorithm…
Penrose's two-spinor notation for 4-dimensional Lorentzian manifolds can be extended to two-component notation for quaternionic manifolds, which is a very useful tool for calculation. We construct a family of quaternionic complexes over unimodular quaternionic manifolds by elementary calculation. On complex quaternio…
PPT optimizes transformer behavior by steering its latent posterior using prior samples.
problem Eliciting desired behavior from transformers without backpropagation.
method Posterior Prefix Tuning (PPT) uses predictive Monte Carlo (PMC) samples and importance sampling to optimize the latent posterior.
result PPT optimizes transformer behavior without backpropagation, achieving high utility across different utility functions.
In this paper, we define a new special curve in Euclidean 3-space which we call {\it k−slant helix} and introduce some characterizations for this curve. This notation is generalization of a general helix and slant helix. Furthermore, we have given some necessary and sufficient conditions for the k−slant helix.
We give explicit formulae for the volumes of hyperbolic cone-manifolds of double twist knots, a class of two-bridge knots which includes twist knots and two-bridge knots with Conway notation C(2n,3). We also study the Riley polynomial of a class of one-relator groups which includes two-bridge knot groups.
Power-SMC reduces inference latency for training-free LLM reasoning.
problem Training-free LLM reasoning with low latency.
method Power-SMC, a training-free Sequential Monte Carlo scheme targeting sequence-level power distribution.
result Power-SMC reduces inference latency from 16-28× to 1.4-3.3× over baseline decoding.