[★circuit tracing tools★] what's going on inside LMs?

tools to understand model behaviors

https://github.com/decoderesearch/circuit-tracer

GitHub - decoderesearch/circuit-tracer

 

Our approach is to generate attribution graphs, which (partially) reveal the steps a model took internally to decide on a particular output. The open-source library we’re releasing supports the generation of attribution graphs on popular open-weights models—and a frontend hosted by Neuronpedia lets you explore the graphs interactively.

 

An overview of the interactive graph explorer UI on Neuronpedia. To get started, you can visit the Neuronpedia interface t

 

https://www.neuronpedia.org/gemma-2-2b/graph

gemma-2-2b Graph | Neuronpedia

 

To get started, you can visit the Neuronpedia interface to generate and view your own attribution graphs for prompts of your choosing. For more sophisticated usage and research, you can view the code repository. This release enables researchers to:

1. Trace circuits on supported models, by generating their own attribution graphs;
2. Visualize, annotate, and share graphs in an interactive frontend;
3. Test hypotheses by modifying feature values and observing how model outputs change.

---