we can inference Meta's Llama 2 7B, yay

README.md

@@ -43,9 +43,32 @@ This still runs at interactive rates and samples more coherent and diverse stori
 
 *Once upon a time, there was a little girl named Lily. She loved playing with her toys on top of her bed. One day, she decided to have a tea party with her stuffed animals. She poured some tea into a tiny teapot and put it on top of the teapot. Suddenly, her little brother Max came into the room and wanted to join the tea party too. Lily didn't want to share her tea and she told Max to go away. Max started to cry and Lily felt bad. She decided to yield her tea party to Max and they both shared the teapot. But then, something unexpected happened. The teapot started to shake and wiggle. Lily and Max were scared and didn't know what to do. Suddenly, the teapot started to fly towards the ceiling and landed on the top of the bed. Lily and Max were amazed and they hugged each other. They realized that sharing was much more fun than being selfish. From that day on, they always shared their tea parties and toys.*
 
+## Meta's Llama 2 models
+
+As the neural net architecture is identical, we can also inference the Llama 2 models released by Meta. First you'll have to export these weights in the llama2.c format. Git clone the main repo from Meta, and cp the `export_meta_llama_bin.py` file (in the root directory of this project) over, and run it:
+
+```bash
+git clone https://github.com/facebookresearch/llama.git
+cd llama
+cp /path/to/llama2.c/export_meta_llama_bin.py .
+torchrun --nproc_per_node 1 export_meta_llama_bin.py
+```
+
+The export will take ~20 minutes or so and generate a 26GB file (the weights of the 7B model in float32) called `llama2_7b.bin` in the current directory. Go back to the root directory of llama2.c and run!
+
+```bash
+./run path/to/llama2_7b.bin
+```
+
+This ran at about 4 tokens/s compiled with OpenMP on 96 threads on my CPU Linux box in the cloud. Example output:
+
+*<s>The purpose of this document is to highlight the state-of-the-art of CoO generation technologies, both recent developments and those in commercial use. The focus is on the technologies with the highest merit to become the dominating processes of the future and therefore to be technologies of interest to S&amp;T ... R&amp;D. As such, CoO generation technologies developed in Russia, Japan and Europe are described in some depth. The document starts with an introduction to cobalt oxides as complex products and a short view on cobalt as an essential material. The document continues with the discussion of the available CoO generation processes with respect to energy and capital consumption as well as to environmental damage.*
+
+base models... ¯\_(ツ)_/¯. 
+
 ## models
 
-It looks like I will have multiple models that I will train on TinyStories, I will catalogue them here.
+For the sake of examples of smaller, from-scratch models, I trained multiple models on TinyStories and catalogue them here:
 
 | model | dim | n_layers | n_heads | max context length | parameters | download
 | --- | --- | --- | --- | --- | --- | --- |