generative-latent-prior/glp-llama8b-d6

Llama8B Activation Diffusion Model

This repository contains model weights accompanying the paper Learning a Generative Meta-Model of LLM Activations.

• Project page: https://generative-latent-prior.github.io
• Code: https://github.com/g-luo/generative_latent_prior

Quick Start

This model is trained on Llama-3.1-8B activations from Layer 15, using FineWeb data. GLPs are activation diffusion models useful for applications like on-manifold steering and sparse probing.

from glp.denoiser import load_glp

model = load_glp("generative-latent-prior/glp-llama8b-d6", device="cuda:0", checkpoint="final")

Citation

@article{luo2026glp,
  title={Learning a Generative Meta-Model of LLM Activations},
  author={Grace Luo and Jiahai Feng and Trevor Darrell and Alec Radford and Jacob Steinhardt},
  journal={arXiv preprint arXiv:2602.06964},
  year={2026}
}