Skip to content

Commit b391579

Browse files
authored
Update README and comments for standalone perplexity tool (#525)
1 parent 7a87d31 commit b391579

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -248,7 +248,7 @@ cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach.
248248

249249
### Perplexity (Measuring model quality)
250250

251-
You can pass `--perplexity` as a command line option to measure perplexity over the given prompt. For more background,
251+
You can use the `perplexity` example to measure perplexity over the given prompt. For more background,
252252
see https://huggingface.co/docs/transformers/perplexity. However, in general, lower perplexity is better for LLMs.
253253

254254
#### Latest measurements
@@ -271,10 +271,10 @@ Perplexity - model options
271271
#### How to run
272272

273273
1. Download/extract: https://s3.amazonaws.com/research.metamind.io/wikitext/wikitext-2-raw-v1.zip?ref=salesforce-research
274-
2. Run `./main --perplexity -m models/7B/ggml-model-q4_0.bin -f wiki.test.raw`
274+
2. Run `./perplexity -m models/7B/ggml-model-q4_0.bin -f wiki.test.raw`
275275
3. Output:
276276
```
277-
Calculating perplexity over 655 chunks
277+
perplexity : calculating perplexity over 655 chunks
278278
24.43 seconds per pass - ETA 4.45 hours
279279
[1]4.5970,[2]5.1807,[3]6.0382,...
280280
```

examples/perplexity/perplexity.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ std::vector<double> softmax(const std::vector<float>& logits) {
1919

2020
void perplexity(llama_context * ctx, const gpt_params & params) {
2121
// Download: https://s3.amazonaws.com/research.metamind.io/wikitext/wikitext-2-raw-v1.zip?ref=salesforce-research
22-
// Run `./main --perplexity -m models/7B/ggml-model-q4_0.bin -f wiki.test.raw`
22+
// Run `./perplexity -m models/7B/ggml-model-q4_0.bin -f wiki.test.raw`
2323
// Output: `perplexity: 13.5106 [114/114]`
2424
auto tokens = ::llama_tokenize(ctx, params.prompt, true);
2525

0 commit comments

Comments
 (0)