LIT compresses deep networks by training intermediate representations, outperforming traditional methods.
problem Reducing the computational overhead of deep network inference.
method LIT trains a student model with the same width but shallower depth, using the intermediate representations from the teacher model.
result LIT achieves substantial network depth reductions without accuracy loss, outperforming traditional methods.
Self-supervised and supervised methods learn similar intermediate visual representations but diverge in final layers.
problem Comparing self-supervised and supervised methods for visual learning.
method Comparison of contrastive self-supervised and supervised methods on simple image data.
result Contrastive and supervised methods learn similar intermediate representations but diverge in final layers.
New method compresses Green's function data efficiently.
problem Efficiently representing complex correlation functions.
method Intermediate representation (IR) of analytical continuation.
result IR yields significantly compact form of correlation functions.
Neural networks benefit from intermediate representations, reducing sample complexity.
problem Understanding how neural networks leverage intermediate representations for hierarchical learning.
method Fixed, randomly initialized neural network as a representation function, compared with raw inputs and other trainable networks.
result Neural representations can achieve improved sample complexities compared to raw inputs, especially for low-rank polynomials.
Proposes KTAN for better training of student networks with both intermediate representations and probability distributions.
problem Reduces large computation and storage cost of deep networks by transferring generalization ability.
method Holistically considers intermediate representations and probability distributions; uses a Teacher-to-Student layer and adversarial learning.
result Significantly improves performance of student networks on image classification and object detection tasks.
This paper introduces a novel approach to measuring privacy risks in deep computer vision models based on intermediate outputs.
problem The exposure of intermediate results in hidden layers of deep computer vision models poses significant privacy concerns.
method The approach leverages Degrees of Freedom (DoF) to evaluate the amount of information retained in each layer and combines this with the rank of the Jacobian matrix to assess sensitivity to input variations.
result The proposed framework provides deeper insights into privacy risks associated with intermediate representations without requiring adversarial attack simulations.
End-to-end learnable network for safer self-driving with interpretable intermediate representations.
problem Safe motion planning for self-driving vehicles.
method Differentiable semantic occupancy representation for cost calculation in motion planning.
result Significantly outperforms state-of-the-art planners in imitating human behaviors and producing safer trajectories.
New analysis of annealing paths in sampling and estimation.
problem Sampling from complex distributions and estimating normalization constants.
method Extending known results on Bregman divergence to quasi-arithmetic means under monotonic embedding.
result Analogous result for quasi-arithmetic means, highlighting the interplay between means, parametric families, and divergence functionals.
Workshop reviews techniques to understand neural NLP models.
problem Understanding the inner workings of neural networks in natural language processing.
method Systematic manipulation of inputs, decoding intermediate representations, modifying architectures, and testing on simplified languages.
result Various techniques can improve explainability of neural network models.
TIPRDC anonymizes data features to protect privacy while retaining useful information.
problem Privacy concerns from crowdsourced data hinder deep learning applications.
method Hybrid training method combining adversarial and mutual information estimation.
result Feature extractor hides private information while preserving original data features.
GANs can improve image reconstruction by using intermediate layers.
problem Improving the quality of image reconstruction using GANs.
method Exploiting the representation in intermediate layers of the generator.
result Intermediate layers in GANs can represent natural images with high visual fidelity.
Paper proposes a new method to learn code semantics using an Intermediate Representation (IR) and embeddings.
problem Lack of robust methods to comprehend program semantics robustly.
method Defines an embedding space (inst2vec) based on IR of code, leveraging both data- and control-flow.
result A single RNN architecture and fixed inst2vec embeddings outperform specialized approaches on various tasks.
Method uses autoregressive models to interpret neural network representations.
problem Understanding and quantifying information preserved in neural network layers.
method Trains autoregressive models to invert model representations and estimate mutual information.
result Mutual information between inputs and network layers decreases over training.
New approach decouples skill learning and language grounding for autonomous agents.
problem Autonomous acquisition of skills without external instructions and feedback.
method Language-Goal-Behavior (LGB) architecture with semantic representation.
result Decouples skill learning and language grounding, enabling diversity and strategy switching.
Study of Anosov representations with Lipschitz limit set and applications to rigidity.
problem Characterizing Anosov representations with specific limit set properties.
method Introducing an unstable Jacobian and analyzing its orbit growth rate.
result Many higher rank representations belong to the studied class.
Divide and conquer quantizes neural networks, improving accuracy.
problem Quantizing neural networks to reduce memory and compute.
method Divide a pretrained network into sections, train each section independently, then stitch them.
result Improves quantized training accuracy by 21.6% on average.
Researchers analyze the geometric and statistical properties of transformer model representations.
problem Understanding the semantic structure of large transformer models across various data types.
method Characterization of geometric and statistical properties through analysis of intrinsic dimension and neighbor composition.
result The semantic information of the dataset is better expressed at the end of the first peak in transformer models.
The paper analyzes different neural network architectures for 3D point cloud processing.
problem Understanding the effects of specific intermediate-layer network architectures on 3D point cloud processing.
method Designing five metrics to diagnose various types of DNNs and conducting comparative studies.
result The hypotheses on the effects of specific intermediate-layer network architectures on the representation capacity of DNNs are verified.
Enhances adversarial example transferability by fine-tuning existing examples.
problem Adversarial examples are often overfit to a source model, limiting black-box transferability.
method Intermediate Level Attack (ILA) fine-tunes adversarial examples on a pre-specified layer of the source model.
result ILAs achieve high transferability to target models without knowledge of their architecture.
LaPool improves molecular graph representation learning by capturing interaction importance.
problem Lack of efficient intermediate pooling steps in GNNs leads to poor molecular substructure representation.
method LaPool is a novel, data-driven, and interpretable hierarchical graph pooling method that considers node features and graph structure.
result LaPool outperforms recent GNNs on molecular graph prediction and understanding tasks.
Myia compiler optimizes ML models with efficient AD for array programming.
problem Efficient automatic differentiation for array programming in ML.
method Introduces a new graph-based IR that supports function calls, higher-order functions, and recursion.
result Myia compiler enables efficient AD using source transformation without a tape, supporting higher-order derivatives.
Proposes a method to analyze distributed datasets without sharing original data.
problem Difficulty in centralizing large, distributed datasets due to size and privacy concerns.
method Centralizes intermediate representations instead of original datasets.
result Achieves higher prediction performance compared to individual analyses.
Study finds more flood risk strategies can improve outcomes in NYC.
problem Managing future flood risks with complex models.
method Used an intermediate complexity model to analyze flood risk strategies.
result More combinations of risk mitigation strategies expand the solution set and improve outcomes.
AP-Calculus offers a new framework for causal inference in Bayesian networks.
problem Causal inference in Bayesian networks with complex architectures.
method Introduces Attribution Projection Calculus (AP-Calculus) to determine causal relationships.
result Proves that for each label, exactly one intermediate node acts as a deconfounder.
Paper analyzes why deeper layers of ViTs perform worse on out-of-distribution tasks.
problem Performance degradation of intermediate layers in ViTs under distribution shift.
method Extensive linear probing experiments across various benchmarks and fine-grained analysis of transformer modules.
result Probing feedforward network activations yields best performance under significant distribution shift.
Quantized-TinyLLaVA reduces communication costs in split learning for multimodal models.
problem High communication costs in split learning for multimodal models.
method Integrates a compression module that quantizes intermediate features into discrete representations before transmission.
result Achieves an approximate 87.5% reduction in communication overhead with 2-bit quantization.
RAE improves image representation learning with simplified design choices.
problem Improving image representation learning using pretrained vision encoders.
method Generalized RAE formulation, complementary working mechanisms of RAE and REPA, and free CFG guidance.
result RAEv2 achieves state-of-the-art results with 10x faster convergence and less training time.
Framework learns stochastic dynamics from endpoint and intermediate distributions using soft energy constraints.
problem Learning stochastic dynamics from endpoint and intermediate distributional observations.
method Formulates generation as a McKean-Vlasov control problem with soft energy constraints, solving it through FBSDE.
result Model learns coherent stochastic trajectories matching prescribed marginal laws.
A new method aggregates generative classifiers to resist adversarial attacks.
problem Adversarial attacks on deep neural networks.
method Rank-aggregating ensemble of generative classifiers trained on intermediate layer responses.
result The ensemble of generative classifiers shows robustness to adversarial attacks.
NKI integrates obfuscated datasets using nonlinear kernels for improved data collaboration.
problem Privacy-preserving data collaboration with reduced reconstruction risk.
method Formulates linear kernel integration, kernelizes it, and introduces graph regularization and centering constraints.
result NKI improves classification accuracy over existing linear integration methods under nonlinear dimensionality reduction.
This paper generalizes the envelope of mid-lines to intermediate lines for a plane curve.
problem Understanding the envelope of intermediate lines for a plane curve.
method Using singularity theory techniques to analyze the local behavior of the envelope of intermediate lines.
result The envelope of intermediate lines (EIL) is formed by three disconnected sets: AEIL, the curve itself, and IPTL. Study on rigidity with non-negative intermediate curvature on low-dimensional manifolds.
problem Extending non-existence theorem of positive scalar curvature to product manifolds.
method Introduced intermediate curvature and studied rigidity conditions.
result Rigidity when intermediate curvature is non-negative in low dimensions.
Preserves positive intermediate curvature on manifolds.
problem Obstructs positive intermediate curvature on partial tori.
method Shows smooth interpolation of metrics with positive intermediate curvature.
result Proves non-existence of certain manifolds with positive intermediate curvature.
The study connects manifold topology to metrics with positive intermediate curvature.
problem Understanding the relationship between manifold topology and metrics with positive intermediate curvature.
method Formulated a conjecture and proved it for specific dimensions and conditions.
result Closed, aspherical 6-manifolds cannot admit metrics with positive 4-intermediate curvature.
ie-HGCN addresses HIN challenges by efficiently learning node representations.
problem Lack of flexibility in exploring meta-paths and high computational complexity in HIN GCN methods.
method Hierarchical aggregation architecture that automatically extracts useful meta-paths and reduces computational cost.
result ie-HGCN outperforms state-of-the-art methods on real network datasets.
New characterisation of Stokes decompositions for vector bundles.
problem Understanding connections on vector bundles on algebraic curves.
method Formalizing Stokes decompositions as an intermediate category.
result Simple characterisation of Stokes decompositions established.
Enhances adversarial examples for better transferability across models.
problem Adversarial examples often have limited black-box transferability.
method Fine-tunes existing adversarial examples by increasing perturbation on a pre-specified layer of the source model.
result Our method achieves greater black-box transferability of adversarial examples.
New rigidity results for manifolds with maximal symmetry rank and positive intermediate Ricci curvature.
problem Understanding the structure of manifolds with maximal symmetry rank and positive intermediate Ricci curvature.
method Recovering stronger topological rigidity results using higher intermediate Ricci curvatures and nontrivial fundamental groups.
result Stronger topological rigidity results for manifolds with maximal symmetry rank and positive intermediate Ricci curvature.
Study on spaces of metrics with intermediate curvature bounds.
problem Understanding spaces of metrics with lower bounds on intermediate curvatures.
method Analyzing spaces of Riemannian metrics with specific curvature bounds on high-dimensional Spin-manifolds.
result Spaces of metrics with positive p-curvature and k-positive Ricci curvature have non-trivial homotopy groups.
A new trajectory representation method for AI problems.
problem Trajectory prediction and optimization in AI problems.
method Sub-goal trees, recursively partitioning trajectories into sub-segments.
result Sub-goal trees predict trajectories faster and more accurately.
Neural network feature optimization for causal inference.
problem Estimating heterogeneous treatment effects from data.
method Genetic algorithm optimization of intermediate neural network layers for feature representations.
result Retains useful features for outcome prediction even if related to treatment assignment.
Proves metrics with positive intermediate Ricci curvature on complex manifolds.
problem Establishing metrics with positive intermediate Ricci curvature on complex manifolds.
method Canonical variation and surgery techniques.
result Existence of metrics with positive intermediate Ricci curvature on various examples.
Reduces data leakage in distributed deep learning models.
problem Prevents reconstruction of sensitive raw data patterns during client communications.
method Reduces distance correlation between raw data and learned representations.
result Resilient to reconstruction attacks while maintaining model accuracy.
Extends Perelman's theorem to positive intermediate curvature conditions.
problem Positive intermediate curvature conditions and their implications.
method Generalization of Perelman's gluing theorem to positive intermediate curvature conditions.
result Observer moduli space can have non-trivial higher homotopy groups.
The study proves that certain manifolds with boundary cannot have metrics with positive intermediate curvatures.
problem Proving the nonexistence of metrics with positive intermediate curvatures on manifolds with boundary.
method Curvature obstruction theorems for manifolds with boundary.
result Topologically nontrivial compact manifolds with boundary cannot have metrics of positive m-intermediate curvature if the boundary is m-convex. New model learns graph neural networks equivariant to various transformations.
problem Learning equivariant graph neural networks for complex transformations.
method E(n)-Equivariant Graph Neural Networks (EGNNs) that are computationally efficient and scalable.
result Achieves competitive or better performance without higher-order representations.
Paper develops a neural network method to anonymize data without losing important information.
problem Protecting sensitive information while preserving useful data for analysis.
method Adversarial neural networks training with three sub-networks to prevent private labels from being predictive.
result Demonstrated success in anonymizing handwritten digits and sentiment analysis data.
Study finds metrics with positive intermediate Ricci curvature on specific low-dimensional manifolds.
problem Existence of invariant metrics with positive intermediate Ricci curvature on low-dimensional cohomogeneity one manifolds.
method Construction of invariant metrics with positive intermediate Ricci curvature on specific manifolds.
result Invariant metrics with positive 4th-intermediate Ricci curvature exist but not for 3rd-intermediate Ricci curvature on certain manifolds.