strutils.multiReplace() crashes if search string is ""

[genotrance]

import strutils
var
  search = ""
  replacement = "abc"
echo "string".multiReplace(search, replacement)
# EDIT(@timotheecour ) I think this should be: `echo "string".multiReplace(("", "abc"))`

Crash output

strutils.nim             multiReplace
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

This works on 0.18.0 but fails on 0.19.0 and #head.

[krux02]

I am not sure if "" should be supported. If the search engine is non utf-8 multibyte characters aware, it can destroy characters. I would prefer if this would just raise an error for illegal argument.

[Araq]

I am not sure if "" should be supported. If the search engine is non utf-8 multibyte characters aware, it can destroy characters. I would prefer if this would just raise an error for illegal argument.

Excellent point that I never considered before. However, IMO an empty string simply doesn't "match" and so the original string is returned unmodified.

[genotrance]

Search string could be "" at runtime, I would prefer the proc would handle it rather than having to check myself. It's not too much work but everyone else using the proc will also benefit.

That being said, even the replace() proc is broken from Araq's definition.

import strutils

echo "abc".replace("", "a")

# Prints aaabaca

[timotheecour]

However, IMO an empty string simply doesn't "match" and so the original string is returned unmodified.

import nre
doAssert replace("foo", re"", "-") == "-f-o-o-"

• likewise with nim-regex (pending Unexpected replacements for "^", "$", "" nitely/nim-regex#29 ; currently returns "-f-o-o")

• "likewise" with re pending re.replace("foo", re"", "-") returns -foo, inconsistent with nre.replace #9437 ; currently returns "-foo"

• see also relevant discussion here:
  Proposed solution for issue #8919 #9280 (comment)

This was discussed and we choose this implemementation because Python/JS/whatever also does it this way.

[dom96]

This should be consistent with replace I guess.

[Araq]

This was discussed and we choose this implemementation because Python/JS/whatever also does it this way.

That is what I said before knowing it breaks UTF-8 and can only be fixed by even more special logic for empty matches which is bad.

[timotheecour]

@Araq
knowing that, would it be possible to enforce the same behavior across these for dealing with empty match string:

• re.replace
• nre.replace
• count
• strutils.replace
• strutils.multiReplace
• nim-regex (external)

and could that behavior be to throw an error (either doAssert or ValueError)
seems like a saner behavior, that would catch some bugs

[timotheecour]

@krux02 there were a few options discussed in this issue; which option did you pick in your commit?

[krux02]

@timotheecour well you can look in the commit and see. But sure. In this PR strutils won't match empty strings on anything anymore. I didn't touch re or nre because those are wrappers for PCRE and I can't control what PCRE matches and what it doesn't match.

[timotheecour]

for clarity, here's new behavior:

echo multiReplace("abcd", ("", "123"))
abcd

also, @genotrance I think your original issue was wrong, since replacement has to be varargs[(string, string)] ; I'll edit it in top-post

[dom96]

Really? We've changed the way replace works again?!