arXiv cs.LG - 2026-07-13

187 items collected.

1. A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions

Author: Qingzhuo Wang, Ruiyang Qin, Zhenxin Qin, Wen Shen, Zhihua Wei
Published: 7/13/2026, 4:00:00 AM
Categories: cs.LG, cs.AI, cs.CL, cs.GT

arXiv:2607.08776v1 Announce Type: new Abstract: Despite the success of knowledge distillation (KD) in Large Language Models (LLMs), the underlying mechanism behind its efficacy remains unclear. In this paper, we propose a unified approach to explore the common mechanism of various KD methods using i...

arXiv cs.LG - 2026-07-13 ​

1. A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions ​

2. iLENS: Interpretable LLM-Guided Mixture-of-Experts for Neuroimaging Survival Analysis ​

3. Signed Symmetric Quantization for Few-Bit Integers ​

4. Sticky Routing: Training MoE Models for Memory-Efficient Inference ​

5. Reward Transport: Property Control in Flow Matching via Noise-Space Alignment ​

6. Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement ​

7. LieBN: Batch Normalization over Lie Groups ​

8. HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning ​

9. DaDaDa: A Dataset for Data Pricing in Data Marketplaces ​

10. Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices ​

11. Adaptive Bayes exactly tracks information over intrinsic time ​

12. Prompt-Driven Exploration ​

13. How are linear representations learned? Exact solutions to the dynamics of abstraction ​

14. Optimizing Against Safety Representations: Activation-Guided Adversarial Suffixes and the Geometry of Refusal ​

15. Pattern-Aware Graph Neural Networks for Handling Missing Data ​

16. A Machine Learning Surrogate for Component Criticality Ranking in Interdependent Power-Communication Networks ​

17. SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions ​

18. BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving ​

19. TSRouter: Dynamic Modality-Model Selection for Time Series Reasoning ​

20. Training, Reading, and Editing Legible Transformers ​

21. FairSelect: A Systematic Evaluation of Multi-Level and Intersectional Algorithmic Fairness ​

22. Eluna: An Agentic LLM System for Automating Warehouse Operations with Reasoning and Task Execution ​

23. NL-PAC: Specification Ambiguity and Certified Minimax Risk Floors in LLM-Mediated Supervision ​

24. Stochastic Linear Bandits with Partially Observed Actions ​

25. Federated Low-Rank Koopman Learning for Multivariate Time-Series Anomaly Detection in IoT Systems ​

26. Optimal Top-$k$ Identification from Pairwise Comparisons ​

27. AlphaZero in Sparsely Rewarded Games: Limits and Auxiliary Supervision ​

28. Group Invariant Spectral Embedding ​

29. Sensitivity-Aware Thresholding and Token Routing for Activation Sparsification in Large Language Models ​

30. Model Agnostic Graph Prompt Learning for Crystal Property Prediction ​

31. Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing ​

32. Variable-Length Generative Protein Design via Generalized Poisson Flow ​

33. Learning More from Less: Reinforcement Learning from Hindsight ​

34. COBS: Cumulant Order Block Sparse Attention ​

35. EvoLP: Self-Evolving Latency Predictor for Model Compression in Real-Time Edge Systems ​

36. Pitfalls and Remedies for Multi-Task Bayesian Optimization ​

37. A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Design ​

38. EXHOLD: Experience-Aware Real-Time Hold Control for Large-Scale Ride-Hailing Matching at DiDi ​

arXiv cs.LG - 2026-07-13

1. A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions

2. iLENS: Interpretable LLM-Guided Mixture-of-Experts for Neuroimaging Survival Analysis

3. Signed Symmetric Quantization for Few-Bit Integers

4. Sticky Routing: Training MoE Models for Memory-Efficient Inference

5. Reward Transport: Property Control in Flow Matching via Noise-Space Alignment

6. Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement

7. LieBN: Batch Normalization over Lie Groups

8. HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learning

9. DaDaDa: A Dataset for Data Pricing in Data Marketplaces

10. Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices

11. Adaptive Bayes exactly tracks information over intrinsic time

12. Prompt-Driven Exploration

13. How are linear representations learned? Exact solutions to the dynamics of abstraction

14. Optimizing Against Safety Representations: Activation-Guided Adversarial Suffixes and the Geometry of Refusal

15. Pattern-Aware Graph Neural Networks for Handling Missing Data

16. A Machine Learning Surrogate for Component Criticality Ranking in Interdependent Power-Communication Networks

17. SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions

18. BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving

19. TSRouter: Dynamic Modality-Model Selection for Time Series Reasoning

20. Training, Reading, and Editing Legible Transformers

21. FairSelect: A Systematic Evaluation of Multi-Level and Intersectional Algorithmic Fairness

22. Eluna: An Agentic LLM System for Automating Warehouse Operations with Reasoning and Task Execution

23. NL-PAC: Specification Ambiguity and Certified Minimax Risk Floors in LLM-Mediated Supervision

24. Stochastic Linear Bandits with Partially Observed Actions

25. Federated Low-Rank Koopman Learning for Multivariate Time-Series Anomaly Detection in IoT Systems

26. Optimal Top-$k$ Identification from Pairwise Comparisons

27. AlphaZero in Sparsely Rewarded Games: Limits and Auxiliary Supervision

28. Group Invariant Spectral Embedding

29. Sensitivity-Aware Thresholding and Token Routing for Activation Sparsification in Large Language Models

30. Model Agnostic Graph Prompt Learning for Crystal Property Prediction

31. Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing

32. Variable-Length Generative Protein Design via Generalized Poisson Flow

33. Learning More from Less: Reinforcement Learning from Hindsight

34. COBS: Cumulant Order Block Sparse Attention

35. EvoLP: Self-Evolving Latency Predictor for Model Compression in Real-Time Edge Systems

36. Pitfalls and Remedies for Multi-Task Bayesian Optimization

37. A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Design

38. EXHOLD: Experience-Aware Real-Time Hold Control for Large-Scale Ride-Hailing Matching at DiDi