Repository navigation
No useful output #33
Description
After installing the code and models successfully I ran per the directions in the README. However the output is useless from a code development perspective and does not at all match the README files reported output. Specifically:
./main -m models/bigcode/gpt_bigcode-santacoder-ggml.bin -p "def fibonnaci(" --top_k 0 --top_p 0.95 --temp 0.2
main: seed = 1693067463
starcoder_model_load: loading model from 'models/bigcode/gpt_bigcode-santacoder-ggml.bin'
starcoder_model_load: n_vocab = 49280
starcoder_model_load: n_ctx = 2048
starcoder_model_load: n_embd = 2048
starcoder_model_load: n_head = 16
starcoder_model_load: n_layer = 24
starcoder_model_load: ftype = 1
starcoder_model_load: qntvr = 0
starcoder_model_load: ggml ctx size = 3475.60 MB
starcoder_model_load: memory size = 768.00 MB, n_mem = 49152
starcoder_model_load: model size = 2707.45 MB
main: prompt: 'def fibonnaci('
main: number of tokens in prompt = 7, first 8 tokens: 563 24240 78 2658 64 2819 7
def fibonnaci(!
main: mem per token = 320504 bytes
main: load time = 777.92 ms
main: sample time = 0.18 ms
main: predict time = 84.90 ms / 12.13 ms per token
main: total time = 924.55 ms
Activity
mvording commentedon Feb 22, 2024
I was able to get results for this as well the starcoder, starcoderbase models by tweaking the parameter values from the ones provided in the readme.
% ./main --help .... gives some information on parameter flags
below param values returned source code for the function
% ./main -m models/bigcode/gpt_bigcode-santacoder-ggml-q4_1.bin -p "def fibonnaci(" --top_k 10 --top_p 0.90 --temp 0.4
% ./main -m models/bigcode/starcoderbase-ggml-q4_1.bin -p "def fibonnaci(" --top_k 10 --top_p 0.90 --temp 0.4