schihei/llama2.c
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

todo changes

Browse Source
This commit is contained in:
Andrej Karpathy
2023-08-13 03:24:31 +00:00
parent 1d14cb8dd8
commit 9ff459b925
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+2 -2
View File
@@ -292,12 +292,12 @@ If your candidate PRs have elements of these it doesn't mean they won't get merg
## unsorted todos ## unsorted todos
- revive tests; train a tiny Llama test model (committed to repo) and use it as reference in unit tests
- make it easier to add a new dataset with not too much pain
- add multiquery support into run.c - add multiquery support into run.c
- add custom bpe training code and the ability to train a smaller vocabulary (32K is to much)
- should calculate freq_cis online in the script run.c instead of loading them - should calculate freq_cis online in the script run.c instead of loading them
- int4/8 quantization - int4/8 quantization
- export the model in a more sensible output format with a proper header, etc. - export the model in a more sensible output format with a proper header, etc.
- train a tiny Llama test model (committed to repo) and use it as reference in unit tests
- support Llama 2 7B Chat models and tune run.c to Chat UI/UX - support Llama 2 7B Chat models and tune run.c to Chat UI/UX
- llama2.cu investigate and merge - llama2.cu investigate and merge
- (LoRA) finetuning and export of Llama 2 models - (LoRA) finetuning and export of Llama 2 models