Improve the performance of tree listing #570

typeless · 2017-01-03T11:22:01Z

Add two query strings start and count for the home page of repository. If the values are omitted or invalid they are set to 0 and the total number of rows respectively.

For #502

lunny · 2017-01-03T11:25:13Z

routers/repo/view.go

@@ -40,12 +41,33 @@ func renderDirectory(ctx *context.Context, treeLink string) {
 		return
 	}

+	query := ctx.Req.URL.Query()
+
+	var start, count int = 0, 100


I prefer the count could be configured and If we ignored some files we have to give a hint on the UI.

That would be better. Let me figure out how to do it properly.

lunny · 2017-01-03T11:26:53Z

routers/repo/view.go

+
+	var start, count int = 0, 100
+
+	if _start, ok := query["start"]; ok && len(_start) == 1 {


ctx. QueryInt ("start") is more convenient.

lunny · 2017-01-03T11:27:09Z

routers/repo/view.go

+		}
+	}
+
+	if _count, ok := query["count"]; ok && len(_count) == 1 {


ctx.QueryInt ("count")

lunny · 2017-01-03T11:27:26Z

routers/repo/view.go

 	entries, err := tree.ListEntries()
 	if err != nil {
 		ctx.Handle(500, "ListEntries", err)
 		return
 	}
-	entries.Sort()
+
+	entries = entries[start : start+count]


According to ftp://www.kernel.org/pub/software/scm/git/docs/git-mktree.html, that has been done by git itself when creating the tree object. And my trivial testing seems to confirm it.

Bwko · 2017-01-05T21:34:03Z

When count is not set it shows no commits. When coun/startt > commits Gitea panics

typeless · 2017-01-06T00:49:47Z

Please mark this as work-in-progress, I will finish this PR later (adding front-end controls as @lunny suggested).

@Bwko Thanks for pointing it out.

Edit: Or perhaps I can set the default count to the total number of rows, so it can still work as before and have follow-up PRs of UI supports in the future.

Edit2: The latest commit should fix the bug found by @Bwko

They are pre-sorted by git-mktree.

Bwko · 2017-01-06T19:28:16Z

@typeless Good work 👍 Only the sorting is alphabetically and not folder first.

The sorting done by git-mktree is alphabetical but it doesn't take directories into consideration.

typeless · 2017-01-07T02:29:24Z

@Bwko My bad 😆 , didn't notice the nuances.

appleboy · 2017-01-07T02:50:50Z

routers/repo/view.go

@@ -12,6 +12,9 @@ import (
 	"path"
 	"strings"



Remove redundant empty line.

ethantkoenig · 2017-01-13T01:45:35Z

routers/repo/view.go

+	count := ctx.QueryInt("count")
+
+	// Check invalid values
+	if (start == 0 && count == 0) || start+count < start || start+count > len(entries) {


Could we replace start+count < start with count < 0?

lunny · 2017-01-23T11:24:15Z

@typeless Is this PR ready to review?

typeless · 2017-01-24T01:41:23Z

@lunny The UI part is still missing if that is desired to be included in this PR.

bkcsoft · 2017-02-12T07:44:01Z

@typeless UI would be required yes, otherwise there may be issue in the future :( Would be nice to have it fetch and render the next set of files via API, like GitHub has load more entries... when there's to many files 🙂

bkcsoft · 2017-02-12T07:46:11Z

You could also just include the base/paginate.tmpl, but then you need to use ?page=0 and a static count 🙁

lunny · 2017-02-14T06:25:15Z

@typeless Is this completed?

typeless · 2017-02-14T09:49:04Z

@lunny I don't have a good idea about how the UI design should be done, before starting to get familiar with Semantic UI. Please bear with me as my spare timeslices for OSS projects are very limited recently.

If someone is willing to help implement the web UI (I am really not good at it), that would be appreciated.

lunny · 2017-02-14T09:54:29Z

OK. Please give the permissions to let maintainers to push to your branch, it's in this PR right sidebar. And I will move this to v1.2.

typeless · 2017-02-14T09:58:50Z

@lunny Thanks. I have the "allow edits from maintainers" checked.

bkcsoft · 2017-02-14T22:46:37Z

@typeless @lunny I can look at implementing lazy-loading during the weekend 😄

lunny · 2017-02-23T13:50:09Z

@bkcsoft any update?

bkcsoft · 2017-04-07T02:19:57Z

@lunny I've been swamped with work unfortunately 🙁

lunny added this to the 1.1.0 milestone Jan 3, 2017

lunny added the type/enhancement An improvement of existing functionality label Jan 3, 2017

lunny requested changes Jan 3, 2017

View reviewed changes

typeless changed the title ~~Improve the performance of tree listing~~ Improve the performance of tree listing Jan 3, 2017

typeless changed the title ~~Improve the performance of tree listing~~ Improve the performance of tree listing (WIP) Jan 3, 2017

typeless changed the title ~~Improve the performance of tree listing (WIP)~~ [WIP] Improve the performance of tree listing Jan 3, 2017

tboerger added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jan 5, 2017

lunny added the pr/wip This PR is not ready for review label Jan 6, 2017

typeless added 4 commits January 6, 2017 09:48

Preliminary implementation

79cfbe8

Do not redundantly sort the tree entries

0088b59

They are pre-sorted by git-mktree.

Use ctx.QueryInt("start") and ctx.QueryInt("count")

791d3e4

Check invalid query values

1fe0d48

typeless force-pushed the limit-commit-log branch from d3a80bd to 1fe0d48 Compare January 6, 2017 01:49

Should check if the values are not provided (start=0 count=0)

702565e

Revert the change of removing entries sorting

0c6d9ce

The sorting done by git-mktree is alphabetical but it doesn't take directories into consideration.

appleboy suggested changes Jan 7, 2017

View reviewed changes

routers/repo/view.go

@@ -12,6 +12,9 @@ import (

"path"

"strings"

Copy link

Member

appleboy Jan 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove redundant empty line.

Remove redundant empty lines

f155fc1

ethantkoenig suggested changes Jan 13, 2017

View reviewed changes

Reduce the condition formula of input validation

7594d24

clandmeter mentioned this pull request Jan 23, 2017

Slow page loads with a large repo #491

Closed

lunny approved these changes Feb 14, 2017

View reviewed changes

lunny modified the milestones: 1.2.0, 1.1.0 Feb 14, 2017

bkcsoft self-assigned this Feb 14, 2017

bkcsoft removed their assignment Apr 7, 2017

lunny modified the milestones: 1.3.0, 1.2.0 Apr 30, 2017

lunny modified the milestones: 1.3.0, 1.x.x Oct 10, 2017

typeless changed the title ~~[WIP] Improve the performance of tree listing~~ Improve the performance of tree listing Dec 22, 2018

typeless closed this Dec 22, 2018

lunny removed this from the 1.x.x milestone Dec 23, 2018

typeless deleted the limit-commit-log branch April 3, 2019 04:59

go-gitea locked and limited conversation to collaborators Nov 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve the performance of tree listing #570

Improve the performance of tree listing #570

typeless commented Jan 3, 2017 •

edited

Loading

lunny Jan 3, 2017

typeless Jan 3, 2017

lunny Jan 3, 2017

lunny Jan 3, 2017

lunny Jan 3, 2017

typeless Jan 3, 2017

Bwko commented Jan 5, 2017

typeless commented Jan 6, 2017 •

edited

Loading

Bwko commented Jan 6, 2017

typeless commented Jan 7, 2017

appleboy Jan 7, 2017

ethantkoenig Jan 13, 2017

lunny commented Jan 23, 2017

typeless commented Jan 24, 2017

bkcsoft commented Feb 12, 2017

bkcsoft commented Feb 12, 2017

lunny commented Feb 14, 2017

typeless commented Feb 14, 2017

lunny commented Feb 14, 2017

typeless commented Feb 14, 2017

bkcsoft commented Feb 14, 2017

lunny commented Feb 23, 2017

bkcsoft commented Apr 7, 2017


		var start, count int = 0, 100

		if _start, ok := query["start"]; ok && len(_start) == 1 {

Improve the performance of tree listing #570

Improve the performance of tree listing #570

Conversation

typeless commented Jan 3, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Bwko commented Jan 5, 2017

typeless commented Jan 6, 2017 • edited Loading

Bwko commented Jan 6, 2017

typeless commented Jan 7, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lunny commented Jan 23, 2017

typeless commented Jan 24, 2017

bkcsoft commented Feb 12, 2017

bkcsoft commented Feb 12, 2017

lunny commented Feb 14, 2017

typeless commented Feb 14, 2017

lunny commented Feb 14, 2017

typeless commented Feb 14, 2017

bkcsoft commented Feb 14, 2017

lunny commented Feb 23, 2017

bkcsoft commented Apr 7, 2017

typeless commented Jan 3, 2017 •

edited

Loading

typeless commented Jan 6, 2017 •

edited

Loading