Skip to content

Encoder buffer grows exponentially #177

Closed
@pnemtsev

Description

@pnemtsev
package main

import (
	"fmt"

	jsoniter "github.com/json-iterator/go"
)

type NopWriter struct {
	total int
}

func (w *NopWriter) Write(p []byte) (n int, err error) {
	fmt.Println("len", len(p))
	fmt.Println("cap", cap(p))
	w.total += len(p)
	fmt.Println("total", w.total)
	fmt.Println()
	return len(p), nil
}

func main() {
	jsoniter.NewEncoder(new(NopWriter)).Encode(make([]int, 10000000))
}

This code produces following output:

len 493
cap 512
total 493

len 1006
cap 1024
total 1499

len 2030
cap 2048
total 3529

len 4078
cap 4096
total 7607

len 8174
cap 8192
total 15781

len 16366
cap 16384
total 32147

len 32750
cap 32768
total 64897

len 65518
cap 65536
total 130415

len 131054
cap 131072
total 261469

len 262126
cap 262144
total 523595

len 524270
cap 524288
total 1047865

len 1048558
cap 1048576
total 2096423

len 2097134
cap 2097152
total 4193557

len 4194286
cap 4194304
total 8387843

len 8388590
cap 8388608
total 16776433

len 3223568
cap 16777216
total 20000001

As you can see, the inner Encoder buffer grows exponentially on each Write. I want to set a fixed capacity of the inner buffer to limit the memory allocated by Encoder.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions