Geek Out Time: Where LLMs “Keep” Their Facts? (VSCode with Colab)

A few weeks ago, we tried “ Reducing Hallucination with DoLa in Google Colab (Microsoft Phi-2)”( https:// nedvedyang.medium.com/geek-out-time-reducing-hallucination-with-dola-in-google-colab-microsoft-phi-2-ec878793df50 ) , a technique that improves factual accuracy by mixing signals from different transformer layers. It sparked some very interesting questions from the folks, one of which kept coming back:

“So… does this mean the lower layers of an LLM are actually more factual?”

We’re going to see the layers in action, using the new Google Colab integration inside VS Code released a few days ago . ( https://developers.googleblog.com/en/google-colab-is-coming-to-vs-code/ )

We run some deeper experiments, and clarify what’s really happening inside a modern 7B model.

This is an excerpt — the full article continues on Medium.