Closed
Description
Question
Hello - having an issue getting this code to run in the browser. Using Xenova/TinyLlama-1.1B-Chat-v1.0 on "@xenova/transformers": "^2.13.2"
It runs perfectly in node.
import { pipeline } from '@xenova/transformers';
console.log('Loading model...');
const generator = await pipeline('text-generation', 'Xenova/TinyLlama-1.1B-Chat-v1.0');
console.log('Model loaded!');
const messages = [
{ role: 'system', content: 'You are a friendly Assistant' },
{ role: 'user', content: 'Explain JavaScript Scopes in simple terms' },
];
const prompt = generator.tokenizer.apply_chat_template(messages, {
tokenize: false,
add_generation_prompt: true,
});
console.log('Generating...');
const result = await generator(prompt, {
max_new_tokens: 256,
temperature: 0.5,
do_sample: true,
top_k: 50,
});
console.dir(result);In Node it runs:
But in the browser I see this:
Same issue in Firefox.
This issue seems to say it's memory: #8
Is this one too large to run in the browser?