Model forgets examples; this research predicts which ones to replay.
problem Language models forget examples during updates, leading to errors.
method Train forecasting models to predict which examples will be forgotten.
result Forecasting models can reduce forgetting of upstream pretraining examples.
The paper analyzes how forgetting in LLMs is linked to simple task-upstream example associations.
problem Forgetting of upstream knowledge in fine-tuned LLMs.
method Empirical analysis of forgotten examples in N upstream examples after M new tasks, using low-rank matrix approximation. result Forgetting can be predicted efficiently using matrix completion over empirical associations.
Rediscovered by a systematic search, a forgotten class of integrable surfaces is shown to disprove the Finkel-Wu conjecture. The associated integrable nonlinear partial differential equation zyy+(1/z)xx+2=0 possesses a zero curvature representation, a third-order symmetry, and a nonlocal transformatio…
Inspired by the phenomenon of catastrophic forgetting, we investigate the learning dynamics of neural networks as they train on single classification tasks. Our goal is to understand whether a related phenomenon occurs when data does not undergo a clear distributional shift. We define a `forgetting event' to have occur…
We prove three theorems giving fixed points for orientation preserving homeomorphisms of the plane following forgotten results of Brouwer.
Proposes a new method to unlearn from specific data points in conformal predictors.
problem Challenges of existing unlearning methods in conformal predictors.
method Formalizes conformal unlearning, introduces practical metrics, and presents an optimization algorithm.
result Demonstrates effective removal of targeted information while preserving utility.
AI models forget statistics' lesson: correlation doesn't imply causation.
problem AI models often produce flawed causal models due to ignoring correlation vs causation.
method Demonstrates examples of flawed AI models and proposes rethinking core models.
result Current efforts to make AI models ethical are insufficient.
MaxRR efficiently unlearns models by splitting and selecting core samples.
problem Efficiently unlearning models while verifying unlearning guarantees.
method Model splitting and core sample selection with a generalized unlearning metric.
result MaxRR achieves efficient unlearning with properties matching full retraining.
This work bridges continual learning, active learning, and open set recognition in deep neural networks.
problem Protecting previously acquired representations from catastrophic forgetting in deep neural networks.
method Surveying the literature and proposing a consolidated view to integrate open set recognition and active learning principles.
result Joint improvement in alleviating catastrophic forgetting, querying data, selecting task orders, and robust open world application.
This research debiases machine unlearning by using counterfactual examples.
problem Machine unlearning processes can be biased, leading to inaccurate results.
method Intervention-based approach using counterfactual examples to mitigate biases.
result The method outperforms existing baselines on evaluation metrics.
We explore the problem of selectively forgetting a particular subset of the data used for training a deep neural network. While the effects of the data to be forgotten can be hidden from the output of the network, insights may still be gleaned by probing deep into its weights. We propose a method for "scrubbing'" the w…
This is an review on the point classification of second order ODE's by Ruslan Sharipov. His works were published in 1997-1998 at the Electronic Archive at LANL and undeservedly forgotten. Last chapter is an application of this classification to the investigation of Painleve equations.
Cycloids, hipocycloids and epicycloids have an often forgotten common property: they are homothetic to their evolutes. But what if use convex symmetric polygons as unit balls, can we define evolutes and cycloids which are genuinely discrete? Indeed, we can! We define discrete cycloids as eigenvectors of a discrete doub…
Bayesian inference forgetting framework removes influence of single data points.
problem Enforcement of the right to be forgotten in machine learning causes high costs for companies.
method Develops forgetting algorithms for variational and Markov chain Monte Carlo in Bayesian inference.
result Proves removal of influence of single datums on learned models with guaranteed generalizability.
Modern treatment of space curve evolutes and involutes.
problem Understanding space curve evolutes and involutes.
method Unified treatment with novel observations and computer graphics.
result Illustration of space curve evolutes and involutes with computer graphics.
The paper proposes selective forgetting for deep neural networks at a finer level than samples.
problem Selective forgetting of deep neural networks to handle outliers, poisoned data, or sensitive information.
method Formulated selective forgetting at a finer level than samples, introduced as an optimization problem on three criteria.
result Experimental results show the model can forget specific information for classification, improving accuracy in specific cases.
Intense recent discussions have focused on how to provide individuals with control over when their data can and cannot be used --- the EU's Right To Be Forgotten regulation is an example of this effort. In this paper we initiate a framework studying what to do when it is no longer permissible to deploy models derivativ…
Abstract sketches historical development of Lie brackets, crossed modules, and Lie-Rinehart algebras.
problem Characterizing and understanding the relationships between Lie brackets, crossed modules, and Lie-Rinehart algebras.
method Historical review and combinatorial group theory considerations.
result The mutual relationship between Lie-Rinehart algebras and Lie brackets, and the historical development of these concepts.
The paper calculates R and Racah matrices for SO(5) and finds Kauffman polynomials.
problem Generalizing Reshetikhin-Turaev approach to SO(2n+1) case.
method Provided R and Racah matrices for SO(5) symmetric representation.
result Found Kauffman polynomials for SO(5) symmetric representation.
Loewner's theorem connects two curve properties via simple functions.
problem Understanding the rotation number of curves defined by smooth periodic functions.
method Elementary proof following Bol's work.
result Two curve properties have non-negative rotation number.
Machine unlearning can compromise privacy, study shows.
problem Machine unlearning may leave data imprints in ML models, risking privacy.
method Proposed a membership inference attack to detect leakage.
result Machine unlearning can lead to unintended privacy risks.
Order patterns and permutation entropy have become useful tools for studying biomedical, geophysical or climate time series. Here we study day-to-day market data, and Brownian motion which is a good model for their order patterns. A crucial point is that for small lags (1 up to 6 days), pattern frequencies in financial…
Is the present economic and financial crisis similar to some previous one? It would be so nice to prove that universality laws exist for predicting such rare events under a minimum set of realistic hypotheses. First, I briefly recall whether patterns, like business cycles, are indeed found, and can be modeled within a …
Learning and memory are intertwined in our brain and their relationship is at the core of several recent neural network models. In particular, the Attention-Gated MEmory Tagging model (AuGMEnT) is a reinforcement learning network with an emphasis on biological plausibility of memory dynamics and learning. We find that …
New method quantifies uncertainty in fine-tuned LLMs using LoRA ensembles.
problem Uncertainty in fine-tuned LLMs and how to trust their predictions.
method Posterior approximations using low-rank adaptation ensembles.
result Unexpected retention of acquired knowledge during fine-tuning in overfitting regime.
This paper presents the contemporary Fundamental Theorem of Asset Pricing as being equivalent to approaches to pricing that emerged before 1700 in the context of Virtue Ethics. This is done by considering the history of science and mathematics in the thirteenth and seventeenth century. An explanation as to why these ap…
This paper develops efficient federated learning and unlearning methods in Bayesian models.
problem Managing epistemic uncertainty and legal right to be forgotten in decentralized networks.
method Develops federated variational inference solutions based on decentralized local free energy minimization.
result Demonstrates efficient unlearning mechanisms in federated learning and unlearning.
Paper uses second-order differential geometry to study stochastic mechanics.
problem Stochastic differential equations and their symmetries.
method Develops second-order differential geometry to study symmetries of SDEs and constructs stochastic mechanics.
result Establishes stochastic Lagrangian and Hamiltonian mechanics and their relations with HJB equations.
DVWU framework improves model performance by considering data value heterogeneity.
problem Existing machine unlearning algorithms ignore data value heterogeneity, potentially degrading model performance.
method Data Value-Weighted Unlearning (DVWU) framework that integrates data values into the unlearning process.
result DVWU achieves superior predictive performance and robustness compared to conventional unlearning approaches.
A novel method for parallel transport and geodesics on submanifolds.
problem Understanding parallel transport and geodesics on submanifolds.
method Rolling tangent space to visualize and analyze parallel transport and geodesics.
result Conditions for parallel transport and geodesics are simplified and visualized in the tangent space.
Linear filtration helps delete training data from models.
problem Deleting training data from models when individuals request it.
method Linear filtration as a computationally efficient sanitization method.
result Demonstrates benefits in an adversarial setting over naive deletion schemes.
Consider a set represented by an inequality. An interesting phenomenon which occurs in various settings in mathematics is that the interior of this set is the subset where strict inequality holds, the boundary is the subset where equality holds, and the closure of the set is the closure of its interior. This paper disc…
Conditional GANs are at the forefront of natural image synthesis. The main drawback of such models is the necessity for labeled data. In this work we exploit two popular unsupervised learning techniques, adversarial training and self-supervision, and take a step towards bridging the gap between conditional and uncondit…
Study on deleting user data in linear regression models to maintain limited memory.
problem Deleting user data in a limited time frame for statistical models.
method Proposed FIFD-OLS and FIFD-Adaptive Ridge algorithms for low-dimensional and online settings.
result Demonstrated effectiveness of FIFD-Adaptive Ridge in maintaining statistical efficiency.
Procedure removes training data dependency from deep networks, improving generalization.
problem Removing dependency on training data in deep networks for better generalization.
method Deterministic and stochastic parts to ensure forgetting, leveraging activation and weight dynamics.
result New bound on information extraction from black-box networks, ensuring forgetting in activations.
New method removes specific training data influence from neural networks.
problem Removing specific training data influence from neural networks for privacy and regulatory reasons.
method Noisy fine-tuning on retain data to ensure provable unlearning guarantees without restrictive assumptions.
result Achieves formal unlearning guarantees and performs effectively in practice.
This paper considers an often forgotten relationship, the time delay between a cause and its effect in economies and finance. We treat the case of Foreign Direct Investment (FDI) and economic growth, - measured through a country Gross Domestic Product (GDP). The pertinent data refers to 43 countries, over 1970-2015, - …
Paper proposes first unlearning algorithm for MCMC models.
problem Enforcing right to be forgotten in AI causes high costs for data deletion.
method Converts MCMC unlearning to explicit optimization problem, designs MCMC influence function.
result MCMC unlearning does not compromise generalizability of models.
New copula models learn to forget dependencies, improving data representation.
problem Restrictive assumptions and poor scaling in existing copula models.
method Diffusion and flow-based copulas that progressively forget dependencies.
result Provable valid copulas at all times, superior performance in complex dependencies.
New framework uses symmetry-based matrices for efficient, flexible NNs.
problem Designing neural networks with relaxed equivariance.
method Symmetry-based structured matrices, Group Matrices (GMs).
result GMs enable competitive performance with fewer parameters.
Algorithm removes specific training data from models efficiently in high-dimensional settings.
problem Efficiently removing specific training data from high-dimensional models without full retraining.
method Starts from original model parameters, performs Newton steps, adds isotropic Laplacian noise.
result Two Newton steps are sufficient for effective unlearning in high-dimensional problems.
Paper proposes machine unlearning method to forget user data from neural networks.
problem Memorization of user data in neural networks violates GDPR's right to be forgotten.
method Proposes Forsaken method to measure and achieve high forgetting rates without significant accuracy loss.
result Forsaken method achieves over 90% forgetting rate with less than 5% accuracy loss.
A framework for certified unlearning in decentralized federated learning.
problem Privacy-preserving machine learning in decentralized federated learning.
method Newton-style updates to quantify and correct data influence, using Fisher information matrices for scalability.
result The proposed framework ensures that the unlearned model is difficult to distinguish from a retrained model without the deleted data.
Modified PCA algorithm with continual learning preserves features of previous modes for multimode process monitoring.
problem Catastrophic forgetting of previous modes in monitoring models for successive modes.
method Modified PCA algorithm with elastic weight consolidation (EWC) to preserve features of previous modes.
result PCA-EWC algorithm effectively monitors multimode processes without performance decrease.
We consider parametric exponential families of dimension K on the real line. We study a variant of \textit{boundary crossing probabilities} coming from the multi-armed bandit literature, in the case when the real-valued distributions form an exponential family of dimension K. Formally, our result is a concentration…
Paper shows incorrectness of approximate unlearning definitions and challenges exact unlearning verification.
problem Incorrectness of approximate unlearning definitions and challenges in verifying exact unlearning.
method Analysis of machine unlearning approaches, including exact and approximate methods.
result Unlearning is only well-defined at the algorithmic level, and auditable claims are limited.
A new method for forgetting data from trained models using information theory.
problem Efficiently forgetting private or copyrighted data from trained machine learning models.
method An information-theoretic approach to zero-shot unlearning, minimizing gradient smoothing.
result Our method successfully unlearns data while maintaining model performance.
Paper proposes ManiF-SMC for effective approximate machine unlearning.
problem Limited unlearning effectiveness and potential to undermine original learning objectives.
method Reformulates approximate unlearning as pushing erased samples towards semantic neighbors in retained data, using a margin-based triplet loss.
result Achieves unlearning effectiveness comparable to state-of-the-art methods while operating purely in representation space.