Porting guide from pcre1

[MatthewVernon]

Hi,

I don't suppose there's any chance of a porting guide from pcre1 to pcre2, is there, please?

I know you want to be shot of pcre1; I've recently filed bugs against the outstanding packages in Debian which still Build against pcre1, and there are a lot of responses of the form "is there any guidance on porting to pcre2?" I don't feel I have deep enough knowledge of the two libraries (especially the older one) to do so myself, but I think having something to point folk at might help in getting more of the remaining ~200(!) packages that still need old-pcre ported, which in turn will make it plausible for me to drop old-pcre from Debian...

Thanks :)

[PhilipHazel]

I'm afraid you won't get one from me. After 7 years I have effectively forgotten PCRE1. I would have to go and read both sets of documentation - which anyone else could do just as well. Off the top of my head, the executive summary for is: (1) There is no separate study() function; (2) Use pcre2_match() instead of pcre_exec(); (3) Call pcre2_match_data_create() before calling pcre2_match(); (4) Adjust the number and type of various arguments and options bits; (5) If using JIT, call pcre2jit_compile() before pcre2_match(). For guidance, take a look at pcre2demo.c.

[stapelberg]

As one of the upstream software authors who has received a debian bug to update my software (https://i3wm.org/) from pcre to pcre2, I would also appreciate a porting guide.

In fact, I would have expected a porting guide and examples to be provided in the bug to begin with. Now a lot of people need to look separately at what’s necessary, when we could have avoided much of that duplicate effort with a little more preparation…

One initial point of confusion to me is the PCRE2_CODE_UNIT_WIDTH macro, which seems to be a new concept in pcre2?

[PhilipHazel]

Ah, yes, I had forgotten about that, which just goes to demonstrate the state of my memory. I'm sorry we didn't provide a porting guide at the time. I can't remember what happened in PCRE1, but in PCRE2, if you are just working with a single code unit width, you set PCRE2_CODE_UNIT_WIDTH and can then use generic function names such as pcre2_match() instead of using the width-specific names such as pcre2_match_16(). The pcre2demo program demonstrates and comments on this.

[stapelberg]

Looks like one pcre→pcre2 porting example is i3/i3#4682 (comment) (unless we find any issues with it). Maybe this helps anyone else.

[kamahen]

I'm in the process of porting SWI-Prolog's support from PCRE1 to PCRE2. If you're interested in what I do, watch SWI-Prolog/packages-pcre#2
But don't expect fast progress on this. ;)

[inkydragon]

Another example: PHP

• PHP: rfc:pcre2-migration
• PCRE2 support by weltling · Pull Request #2857 · php/php-src

Backward Incompatible Changes
Internal library API has changed

• The 'S' modifier has no effect, patterns are studied automatically. No real impact.
• The 'X' modifier is the default behavior in PCRE2. The current patch reverts the behavior to the meaning of 'X' how it was in PCRE, but it might be better to go with the new behavior and have 'X' turned on by default. So currently no impact, too.
• Some behavior change due to the newer Unicode engine was sighted. It's Unicode 10 in PCRE2 vs Unicode 7 in PCRE.
• Some behavior change can be sighted with invalid patterns.

[smoretti]

Hi
I have inherited some C code using PCRE. I think I managed most of the changes to PCRE2 with the help of your examples. But pcre_exec to pcre2_match is the trickiest for me, notably with pcre2_match_data compared to what was in PCRE.
Don't know if this is the right place for asking help, but here it the code I have:

      const pcre * const restrict rg = regex->regexCompiled[iPatternPrf];
      int ovector[2*8];
      unsigned int offset = 0;
      const unsigned int len = PFSeq->Length;
      int rc;
      size_t count = 0;
      while (offset < len && (rc = pcre_exec(rg, 0, CleanSeq, len, offset, 0, ovector, 8)) >= 0)
      {
        for(int k = 0; k < rc; ++k)
        {
          if (count < nmatch) {
            Matches[2*count] = ovector[2*k];
            Matches[2*count+1] = ovector[2*k+1];
            ++count;
          }
          else {
            fprintf(stderr, "Warning: maximum number of matches reached for %s with %s\n",
              prfs[iPatternPrf]->Identification, prfs[iPatternPrf]->Pattern);
          }
        }
        offset = ovector[1];
      }

I have replaced it by

      const pcre2_code * const restrict rg = regex->regexCompiled[iPatternPrf];
      CLEANUP_PCRE2_MATCH_DATA_FREE pcre2_match_data *match_data = pcre2_match_data_create_from_pattern (rg, NULL);
      int ovector[2*8];
      unsigned int offset = 0;
      const unsigned int len = PFSeq->Length;
      int rc;
      size_t count = 0;
      while (offset < len && (rc = pcre2_match(rg, (PCRE2_SPTR)CleanSeq, PCRE2_ZERO_TERMINATED, offset, 0, match_data, NULL)) >= 0)
      {
        for(int k = 0; k < rc; ++k)
        {
          if (count < nmatch) {
            Matches[2*count] = ovector[2*k];
            Matches[2*count+1] = ovector[2*k+1];
            ++count;
          }
          else {
            fprintf(stderr, "Warning: maximum number of matches reached for %s with %s\n",
              prfs[iPatternPrf]->Identification, prfs[iPatternPrf]->Pattern);
          }
        }
        offset = ovector[1];
      }

but I don't know really how to replace ovector regarding match_data.

[kamahen]

Maybe pcre2_get_ovector_pointer(match_data)?
Anyway, that's what pcre2demo.c uses.

(I don't have my conversion fully working, so can't be sure that I've got it right either)

[smoretti]

Thanks @kamahen, I will look at pcre2demo.c.

[smoretti]

I think I did it (https://github.com/sib-swiss/pftools3/tree/master/src/C/prg)

The code contains both PCRE and PCRE2, users can switch between them with cmake

[kamahen]

My code is almost ready for review (maybe in a day or so).
If you want to look at it sooner: https://github.com/kamahen/packages-pcre/tree/pcre2_wip and diff with commit 8876f2a

[kamahen]

My migration code is now here, awaiting code inspection:
SWI-Prolog/packages-pcre#11

It does not use pcre2_substitute(), but instead uses existing code for doing split, replace, and fold operations using the regexps.