Loading
Life sciences · Preprint
arXiv · September 10, 2026
Posted before peer review. The findings may change or fail to hold.
LILA is a proposed calibration-free structured pruning method for large language models that uses Kolmogorov–Smirnov distance on singular value distributions to score neuron importance. The preprint reports superior zero-shot accuracy compared to calibrated and policy-based baselines on LLaMA-2-7B and Phi-2 across multiple sparsity levels, with theoretical support from Neural Tangent Kernel analysis, but results are unrefereed and lack statistical significance testing or multiple independent runs.
Empirical method comparison on standard benchmarks. LLaMA-2-7B (7 billion parameters) and Phi-2 language models; evaluation on zero-shot accuracy benchmarks and generative tasks.. Intervention: LILA structured pruning method using spectral importance via KS-distance; applied at 25% and other sparsity levels. Compared with: PruneNet (45M-parameter RL policy), WikiText-2-calibrated SliceGPT, random pruning.
Without fine-tuning, LILA surpasses PruneNet by 1.57 percentage points in zero-shot accuracy on LLaMA-2-7B at 25% sparsity LILA outperforms WikiText-2-calibrated SliceGPT by up to 6.0 percentage points across all sparsity levels without any calibration data After one epoch of LoRA recovery fine-tuning, LILA matches SliceGPT to within 0.48 percentage points on LLaMA-2-7B and Phi-2
Safety was not reported in the material analysed. Check the source before drawing any conclusion about harm.
The source did not state who this applies to in practice.
This is an unrefereed arXiv preprint describing a novel computational method for neural network compression; it reports empirical results and theoretical analysis but has not undergone peer review.
As stated by the source record.
Quoted from the source exactly as published.
Graded across the dimensions that decide whether you should act, each from what the source actually supports. There is no single score, and where a dimension was not assessed it says so.
Structured pruning of large language models (LLMs) offers hardware-efficient compression, yet existing methods require calibration data, gradient computation, or large auxiliary policy networks at pruning time. LILA (\emph{Latent-Informed Layer Analysis}) scores neuron importance via the Kolmogorov--Smirnov (KS) distance between empirical singular value distributions of the full and neuron-ablated feed-forward network (FFN) weight matrix, providing a closed-form spectral rule requiring no training, calibration data, or auxiliary network. Without any fine-tuning, LILA surpasses PruneNet (45M-parameter RL policy) by 1.57~pp in zero-shot accuracy on LLaMA-2-7B at 25\% sparsity, and outperforms WikiText-2-calibrated SliceGPT by up to 6.0~pp across all sparsity levels, while preserving the original architecture. After one epoch of LoRA recovery fine-tuning, LILA achieves highly competitive performance, matching the heavily calibrated SliceGPT baseline to within a 0.48~pp margin across LLaMA-2-7B and Phi-2, despite using zero calibration data. A Neural Tangent Kernel analysis confirms a 22$\times$ reduction in functional distortion versus random pruning, providing theoretical grounding for the spectral importance criterion. Finally, extending LILA to dynamically allocate sparsity budgets via KS-scores yields state-of-the-art generative preservation at moderate compression, while uncovering fundamental single-layer architectural bottlenecks at higher compression regimes.
Taken from the source record, never inferred. Follow any of these and new work involving them reaches your briefing.