You are right in that 1.5B sized models can’t be very intelligent. But I do expect it to not write what has already been written. It’s been finetued on this completion task, with tokens designed specifically for it. So I assumed it’d be okay at it.

Sometimes it does actually generate what I want it to. When I provide a simple if statement with a for loop under it, it will autocomplete the else branch mostly correctly almost every time.

Example:

I wrote this code

	var start_with_code := markdown.begins_with("```") or markdown.begins_with("\n```\n")
	var result:String
	var mark_i := 0
	var code_i := 0
	if start_with_code:
		for i in markdown_texts.size() + code_blocks.size():
			if i % 2 == 0.0:
				result += code_blocks[code_i]
				code_i += 1
			else:
				result += markdown_texts[mark_i]
				mark_i += 1
	else:

And the model added this

	else:
		for i in markdown_texts.size() + code_blocks.size():
			if i % 2 == 0.0:
				result += markdown_texts[mark_i]
				mark_i += 1
			else:
				result += code_blocks[code_i]
				code_i += 1

(I’m writing a markdown parser here)