Skip to content

badger v1 eventually segfaults when open file limit is reached #1395

Closed
@RubenKelevra

Description

@RubenKelevra

What version of Go are you using (go version)?

1.14.x

What operating system are you using?

Linux

What version of Badger are you using?

1.6.1

Does this issue reproduce with the latest master?

unknown

What Badger options were set?

"sync_writes": true,
"table_loading_mode": 2,
"value_log_loading_mode": 0,
"num_versions_to_keep": 1,
"max_table_size": 16777216,
"level_size_multiplier": 10,
"max_levels": 7,
"value_threshold": 32,
"num_memtables": 5,
"num_level_zero_tables": 1,
"num_level_zero_tables_stall": 2,
"level_one_size": 268435456,
"value_log_file_size": 10485760,
"value_log_max_entries": 1000000,
"num_compactors": 2,
"compact_l_0_on_close": true,
"read_only": false,
"truncate": true

Logfile with stacktrace is attached here: ipfs-cluster/ipfs-cluster#1180

Example:

Jun 25 18:28:50 loki.pacman.store ipfs-cluster-service[1682307]: 2020-06-25T18:28:50.075+0200        ERROR        providers        providers/providers_manager.go:138        error adding new providers: Create value log file. Path=/var/lib/ipfs/.ipfs-cluster/badger/000987.vlog. Error=open /var/lib/ipfs/.ipfs-cluster/badger/000987.vlog: too many open files
Jun 25 18:28:50 loki.pacman.store ipfs-cluster-service[1682307]: 2020-06-25T18:28:50.075+0200        ERROR        badger        [email protected]/logger.go:38        writeRequests: Create value log file. Path=/var/lib/ipfs/.ipfs-cluster/badger/000987.vlog. Error=open /var/lib/ipfs/.ipfs-cluster/badger/000987.vlog: too many open files
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]: panic: runtime error: invalid memory address or nil pointer dereference
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x1029522]
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]: goroutine 447157279 [running]:
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]: github.com/dgraph-io/badger.(*valueLog).write(0xc00013e870, 0xc008acaa00, 0x1, 0xa, 0x0, 0x0)
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]:         pkg/mod/github.com/dgraph-io/[email protected]/value.go:1149 +0x272
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]: github.com/dgraph-io/badger.(*DB).writeRequests(0xc00013e700, 0xc008acaa00, 0x1, 0xa, 0x43fbe6, 0x15de9a0)
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]:         pkg/mod/github.com/dgraph-io/[email protected]/db.go:627 +0x106
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]: github.com/dgraph-io/badger.(*DB).doWrites.func1(0xc008acaa00, 0x1, 0xa)
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]:         pkg/mod/github.com/dgraph-io/[email protected]/db.go:699 +0x5a
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]: created by github.com/dgraph-io/badger.(*DB).doWrites
Jun 25 18:29:07 loki.pacman.store ipfs-cluster-service[1682307]:         pkg/mod/github.com/dgraph-io/[email protected]/db.go:752 +0x30e

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions