BERT attention visualizer
Type a sentence and watch a Transformer read it - every attention head, every layer, and why it decides what it decides.
Attention, made visible
A Transformer lets every word look at every other word. NeuroScope shows which words each head attends to, which heads matter most for the answer, and how the prediction builds up layer by layer.
It also hides one word at a time to show which words the answer really depends on, and works out one attention weight from query and key, so the formula becomes concrete.
Small BERTs that run in your browser
BERT-Tiny and BERT-Mini (including small sentiment and emotion models) run right in the page; DistilBERT runs on the NeuroScope server.
Fine-tune them on your own text data, or on built-in datasets like movie reviews and SMS spam.
Try it in a minute
- Press Try it now - a small BERT trained for sentiment loads (about 18 MB the first time).
- Type your own sentence.
- Open the self-attention panel and click any head.
Questions
Which models are included?
BERT-Tiny, BERT-Mini and small sentiment / emotion models in the browser; DistilBERT on the server.
Can I fine-tune BERT on my data?
Yes - upload a CSV with a text column and a label column.
Is attention the same as explanation?
Not always - that is why NeuroScope also shows which hidden words change the answer.