Speed up LeaveOneOutEncoder with vectorization.

[jkleint]

Over 400X faster while using less memory.

Store category mappings as DataFrames, then do vectorized apply
with .map(). fit() just computes the sum and count of y for each
level of each column of X. These mappings are stored as a dict
mapping the column name to a DataFrame with sum and count columns.
transform() then applies the map to each column of X, plus a little
vectorized math.

There is a speed/space tradeoff, whether to store the mean of y
for each level as well as the sum and count. This was resolved
in favor of space, recomputing the mean in transform(), so that
e.g. pickled Transformers will take less space on disk.

rows, cats = 1000000, 1000
X = pd.DataFrame({'x': np.random.randint(0, cats, rows).astype(str)})
y = pd.Series(np.random.rand(rows))

# old
%time LeaveOneOutEncoder().fit_transform(X, y)
CPU times: user 2min 47s, sys: 241 ms, total: 2min 48s
Wall time: 2min 48s

# new
CPU times: user 390 ms, sys: 17.3 ms, total: 408 ms
Wall time: 407 ms

[jkleint]

Minor API suggestion for LeaveOneOutEncoder: the randomized and sigma parameters are redundant. You could get rid of randomized and just have sigma default to None. If you want randomization, just set sigma > 0.

[janmotl]

Thank you. It always makes me happy when the size of the code decreases.

Note however, that the behaviour changed when we encounter a unique value during the training time:

    def test_leave_one_out_unique(self):
        X = pd.DataFrame(data=['1', '2', '2', '2', '3'], columns=['col'])
        y = [1, 0, 1, 0, 1]

        encoder = encoders.LeaveOneOutEncoder(impute_missing=False)
        result = encoder.fit(X, y).transform(X, y)

        self.assertFalse(any(result.isnull()), 'There should not be any missing value')

I am not sure we should treat a unique value as missing.

The API change makes sense. Go ahead.

[jkleint]

You're welcome! Glad to help.

Yeah, I didn't fully understand the algorithm when I was translating it and I think I have a mistake. I believe the intent is that if there's only one sample for a level, we should use the overall target mean, yes? And that should happen both when y is provided and when it's not? That's a little cleaner. I can do that.

[janmotl]

That is correct.

[jkleint]

Fixed the issue with missing values, and added some documentation and comments.

[jkleint]

Also added patch to remove randomize arg in favor of sigma.

[janmotl]

What if there is a missing value in the input:

    def test_leave_one_out_missing(self):
        X = pd.DataFrame(data=['1', '2', '2', '2', '3', None], columns=['col'])
        y = [1, 0, 1, 0, 1, 0]

        encoder = encoders.LeaveOneOutEncoder(impute_missing=False)
        result_fit = encoder.fit(X, y).transform(X, y)

        self.assertTrue(pd.isna(result_fit['col'][5]), 'we expect NaN or None because impute_missing=False')

? Shouldn't we preserve the missing value?

[janmotl]

Note: The suggested change depends on the outcome of #92.