Skip to content

Normalise keywords to lowercase when searching for crates. #140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Normalise keywords to lowercase when searching for crates. #140

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/bin/migrate.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -440,6 +440,14 @@ fn migrations() -> Vec<Migration> {
ON crates (lower(name))", &[]));
Ok(())
}),
Migration::new(20150320174400, |tx| {
try!(tx.execute("CREATE INDEX index_keywords_lower_keyword ON keywords (lower(keyword))",
&[]));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this should be ALTER TABLE keywords (although I also thought it would be UPDATE keywords ...?).

I think that this may also not land well in production as there's currently a uniqueness constraint on the keyword column, but this will generate duplicates (e.g. FFT => fft).

I think that this may want to iterate through all keywords, deleting those with a downcase equivalent and migrating all current members of that keyword to the downcase equivalent. This should also probably delete the old uniqueness constraint and add a new one on lower(keyword) to speed up queries and provide a db-level gurantee that we don't have duplicates.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I totally mangled this.

Ok(())
}, |tx| {
try!(tx.execute("DROP INDEX index_keywords_lower_keyword", &[]));
Ok(())
}),
];
// NOTE: Generate a new id via `date +"%Y%m%d%H%M%S"`

Expand Down
2 changes: 1 addition & 1 deletion src/krate.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -463,7 +463,7 @@ pub fn index(req: &mut Request) -> CargoResult<Response> {
ON crates.id = crates_keywords.crate_id
INNER JOIN keywords
ON crates_keywords.keyword_id = keywords.id
WHERE keywords.keyword = $1";
WHERE lower(keywords.keyword) = lower($1)";
(format!("SELECT crates.* {} {} LIMIT $2 OFFSET $3", base, sort_sql),
format!("SELECT COUNT(crates.*) {}", base))
})
Expand Down
10 changes: 7 additions & 3 deletions src/tests/krate.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,9 @@ fn index_queries() {
krate.readme = Some("readme".to_string());
krate.description = Some("description".to_string());
::mock_crate(&mut req, krate);
::mock_crate(&mut req, ::krate("BAR"));
let mut krate2 = ::krate("BAR");
krate2.keywords.push("KW1".to_string());
::mock_crate(&mut req, krate2);

let mut response = ok_resp!(middle.call(req.with_query("q=baz")));
assert_eq!(::json::<CrateList>(&mut response).meta.total, 0);
Expand All @@ -77,7 +79,7 @@ fn index_queries() {
let mut response = ok_resp!(middle.call(req.with_query("q=foo")));
assert_eq!(::json::<CrateList>(&mut response).meta.total, 1);
let mut response = ok_resp!(middle.call(req.with_query("q=kw1")));
assert_eq!(::json::<CrateList>(&mut response).meta.total, 1);
assert_eq!(::json::<CrateList>(&mut response).meta.total, 2);
let mut response = ok_resp!(middle.call(req.with_query("q=readme")));
assert_eq!(::json::<CrateList>(&mut response).meta.total, 1);
let mut response = ok_resp!(middle.call(req.with_query("q=description")));
Expand All @@ -99,7 +101,9 @@ fn index_queries() {
assert_eq!(::json::<CrateList>(&mut response).crates.len(), 0);

let mut response = ok_resp!(middle.call(req.with_query("keyword=kw1")));
assert_eq!(::json::<CrateList>(&mut response).crates.len(), 1);
assert_eq!(::json::<CrateList>(&mut response).crates.len(), 2);
let mut response = ok_resp!(middle.call(req.with_query("keyword=KW1")));
assert_eq!(::json::<CrateList>(&mut response).crates.len(), 2);
let mut response = ok_resp!(middle.call(req.with_query("keyword=kw2")));
assert_eq!(::json::<CrateList>(&mut response).crates.len(), 0);
}
Expand Down