schihei/llama2.c
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #10 from luigifcruz/patch-1

Browse Source 
Bigger number better with -funsafe-math-optimizations flag.
This commit is contained in:
Andrej
2023-07-23 18:52:36 -07:00
committed by GitHub
parent 1b63a9510e 1eb111adc7
commit 15e7c92fad
1 changed files with 7 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+7 -1
View File
@@ -19,7 +19,7 @@ Let's just run a baby Llama 2 model in C. You need a model checkpoint. Download
wget https://karpathy.ai/llama2c/model.bin -P out
```
(if that doesn't work try [google drive](https://drive.google.com/file/d/1aTimLdx3JktDXxcHySNrZJOOk8Vb1qBR/view?usp=share_link)). Compile and run the C code:
(if that doesn't work try [google drive](https://drive.google.com/file/d/1aTimLdx3JktDXxcHySNrZJOOk8Vb1qBR/view?usp=share_link)). Compile and run the C code (check [howto](#howto) for faster optimization flags):
```bash
gcc -O3 -o run run.c -lm
@@ -68,6 +68,12 @@ Once we have the model.bin file, we can inference in C. Compile the C code first
gcc -O3 -o run run.c -lm
```
Alternatively, if you want to increase the inference performance and are confident in using unsafe math optimizations, which are probably fine for this application, you can compile the code with the `-funsafe-math-optimizations` flag as shown below:
```bash
gcc -O3 -funsafe-math-optimizations -o run run.c -lm
```
You can now run it simply as
```bash