New addon: Fuzzy Search with PostgreSQL pg_trgm extension

[christophlsa]

This addon provides the ability to create GIN or GiST indexes of char and text fields and also to use the search operator % in search domains. This is a first basic implementation which should be extended later.

Requirements

First you need to have the pg_trgm extension available. In debian based distribution you have to install the postgresql-contrib module. Then you have to either install the pg_trgmextension to your database or you have to give your postgresql user the superadmin right (then the extension will be installed by this module).

Explanation

If you installed the odoo module you can define in Settings -> Database Structure -> Trigam Index the indexes. There the index name will automatically created.

For example you can create an index for the name field of res.partner. Then in a search you can use self.env['res.partner'].search([('name', '%', 'Jon Miller)]). In this Example it can find existing names like John Miller or John Mill.

Which strings can be found depends on the limit which is set (default: 0.3). Currently you have to set the limit by executing SQL: self.env.cr.execute("SELECT set_limit(0.2);").

Also you can use the similarity(column, 'text') function in the order parameter to order by the similarity. This is just a basic implementation which doesn't contains validations and has to start with this function. For example you can define this like "similarity(%s.name, 'John Mil') DESC" % self.env['res.partner']._table.

For further questions read the Documentation of the pg_trgm module.

Tasks/Goals

For this PR:

• Set permissions/groups
• Add possibility to order by similarity
• Add Readme etc
• German Translations
• Add unittests

Later:

• Modify the general search parts (e.g. in tree view or many2one fields)

[lmignon]

remove pprint lines

[christophlsa]

Thanks. I think I committed to fast ^^

[dreispt]

The initial approach looks good.
Take care with the technique used to patch the core, it can be tricky, so unit tests will be your friend.

[christophlsa]

• Renamed module base_trgm_search to base_search_fuzzy,
• added Unit tests
• added german translations
• added access permissions
  moved the monkey patching to method _register_hook of ir.model
• fixed _auto_init (wrong parameter)
• added README
• cleaned up some parts

[dreispt]

Is it ready for review?

[christophlsa]

for me yes :)

[elicoidal]

s/something/anything

[elicoidal]

Thanks for the contribution!

[moylop260]

Please, Could you review our beta checks https://travis-ci.org/OCA/server-tools/jobs/139978373#L424-L430?

************* Module base_search_fuzzy.models.trgm_index
base_search_fuzzy/models/trgm_index.py:122: [C8107(translation-required), TrgmIndex.create_index] String parameter of raise "UserError" requires translation. Use _('The pg_trgm extension does not exists or cannot be installed.')

Ignore

************* Module base_search_fuzzy.models.ir_model
base_search_fuzzy/models/ir_model.py:24: [W0402(deprecated-module), ] Uses of a deprecated module 'openerp.osv'
************* Module base_search_fuzzy.tests.test_query_generation
base_search_fuzzy/tests/test_query_generation.py:3: [W0402(deprecated-module), ] Uses of a deprecated module 'openerp.osv'

[StefanRijnhart]

@moylop260 FYI two of those are the dreaded false positive for imports from openerp.osv again. OCA/maintainer-quality-tools#306. What is the recommended solution for that?

[moylop260]

@StefanRijnhart
You are right I'll review this check

[moylop260]

👍
(Just add _translate method in raises)
Good job! thanks

[christophlsa]

Thanks for the comments. I updated all. :)

[christophlsa]

I added a fix to check if the functions which I overwrite are already decorated by the new decorator. We had problems that these functions got this decorator much times (in particular in the cron worker) and a maximum recursion depth error was thrown.

I also rebased the branch.

[gurneyalex]

travis red build should be fixed by #507

[gurneyalex]

s/SUEPRUSER/SUPERUSER/

[gurneyalex]

one small typo, but 👍

[dreispt]

PR https://github.com/blooparksystems/server-tools/pull/1

[dreispt]

TravisCI is red 🚧

[christophlsa]

After rerunning the checks (and merging the Readme fix) all are green.