[Summary] Go to file (Find files in repo) optimization

[wxiaoguang]

Follow:

• Feature: Find files in repo #15028

TODO:

• Optimize the performance. should have been improved by Refactor parseTreeEntries, speed up tree list #21368
• Use a config option to disable this feature if some users do not like it. not needed anymore.
• Use weighted algorithm for string matching. Use weighted algorithm for string matching when finding files in repo #21370
  • When a user searches word, they may want to see /{word}.txt before /{w}e-g{o}t-{r}esult.{d}at

[lunny]

For the find files, we have two ways to optimize. One is cache all the file paths, another is to use a backend search rather than frontend search or both.

[lunny]

Both are not easy, so for 1.17, a configuration to allow users disable should be a quick patch.

[wxiaoguang]

This feature is not the only slow one, and not the most urgent one to be optimized.

There are already a lot of slow handlers in Gitea.

For example, for the same repo:

$ time curl https://gitea.com/.../.../commits/branch/master/..../.... > /dev/null
curl 0.02s user 0.03s system 0% cpu 56.408 total

$ time curl https://gitea.com/.../.../tree-list/branch/master > /dev/null
curl  0.05s user 0.02s system 0% cpu 19.313 total

---

Update: now the 56s handler has been optimized by 20765. That repo has got commit-graph write --changed-paths (19812 comment).

A new test shows that the gitea.com/.../tree-list/branch/master still needs 19 seconds for the first time, and 6-9 seconds for later. On my side (1G + 512M VPS), the ls-tree is pretty fast:

$ time git ls-tree -r HEAD | wc -l
76950
git ls-tree -r HEAD  0.12s user 0.04s system 87% cpu 0.185 total
wc -l  0.01s user 0.01s system 5% cpu 0.184 total

Before ListEntriesRecursive uses -l (show sizes), it seems that this flag makes the ls-tree much slower. So it could be optimized with a dedicated function to call ls-tree without -l

[wxiaoguang]

--> Refactor parseTreeEntries, speed up tree list #21368

[wxiaoguang]

--> Use weighted algorithm for string matching when finding files in repo #21370

Then this issue could be closed as completed.

[wxiaoguang]

All done