[elastic] Cache the results of 'invokeGo()' call #76
Conversation
|
❌ Build go-langserver 1.0.178 failed (commit 0e831b861f by @movie-travel-code) |
movie-travel-code
force-pushed
the
OptimizeGoLangserver
branch
from
c173120 to
ee43752
Compare
go/packages/golist.go
Outdated
| var ( | ||
| goListLRUCache *lru.Cache | ||
| createGoListLRUCache sync.Once | ||
| goListLRUEntries = 32 |
There was a problem hiding this comment.
isn't 32 too smal?
I would imaging a cache at least 4MB of size in total
There was a problem hiding this comment.
It's difficult to give a perfect cache size. IIUC kibana will send the full request in the order of folders, that means we won't touch the folder again in a short period of time. goListDriver() has the same result for the same folder, like A\a.go, A\b.go, A\c.go, A\d.go, once we cache the go list result of A\a.go, we can reuse the result for three remains go files. And we won't handle the same folder again in the same indexing process. So holding the go list results in the memory for a long time is not the best choice.
I use TiDB which has 820 .go file and 96 subfolders to test the which number is the best candidate. Setting 32 has the shortest time and memory usage is also within a reasonable range. I will improve the entry numbers in case we index many repos simultaneously.
There was a problem hiding this comment.
Test more repos to get the best cache size.
movie-travel-code
force-pushed
the
OptimizeGoLangserver
branch
from
ee43752 to
c43556d
Compare
'invokeGo' is used to get the import outline of the specified folder(package), so there are duplications that will slow down the go langserver. Since go langserver serves several repos at the same time, use LRU cache in case of blow up the memory. 'determineRootDirsCached' serves single view, i.e. single workspace folder, 'golistDriverLRUCached' serves single subfolder, i.e. package. Given that we will skip 'vendor' folder, for now, the number of the repos are simultaneously indexing is very low, that's why set the 'determineRootDirsCached' entry number to 16. Future plan: skip the 'vendor' file download the deps during the initialize request eliminate the 'invokeGo' call find a better approach to handle the 'vendor' folder
|
I choose 7 repos to test the cache size, these repos contain 592 I index these repo simultaneously, the rough results are as follows.
80 |
movie-travel-code
force-pushed
the
OptimizeGoLangserver
branch
from
c43556d to
6d40969
Compare
'invokeGo' is used to get the import outline of the specified folder(package), so there are duplications that will slow down the go langserver. Since go langserver serves several repos at the same time, use LRU cache in case of blow up the memory. 'determineRootDirsCached' serves single view, i.e. single workspace folder, 'golistDriverLRUCached' serves single subfolder, i.e. package. Given that we will skip 'vendor' folder, for now, the number of the repos are simultaneously indexing is very low, that's why set the 'determineRootDirsCached' entry number to 16. Future plan: skip the 'vendor' file download the deps during the initialize request eliminate the 'invokeGo' call find a better approach to handle the 'vendor' folder
'invokeGo' is used to get the import outline of the specified folder(package),
so there are duplications that will slow down the go langserver.
Since go langserver serves several repos at the same time, use LRU cache
in case of blow up the memory. 'determineRootDirsCached' serves single view,
i.e. single workspace folder, 'golistDriverLRUCached' serves single
subfolder, i.e. package. Given that we will skip 'vendor' folder, for now,
the number of the repos are simultaneously indexing is very low, that's
why set the 'determineRootDirsCached' entry number to 16.
Future plan: