grok pattern IPTABLES does not always match

[jdelker]

This applies to logstash-patterns-core 4.3.4:

The provided grok pattern for IPTABLES in patterns/ecs-v1/firewalls is incomplete and does not match all variants.
Obviously that applies for logged ICMP packages, which show less attributes (missing SPT, DPT, ...).

The following message does not match `%{IPTABLES}':

IN=eth6 OUT=eth1.13 MAC=00:1a:8c:17:da:4e:30:e4:db:34:88:31:08:00 SRC=1.2.3.4 DST=5.6.7.8 LEN=578 TOS=0x00 PREC=0x00 TTL=49 ID=29312 DF PROTO=47

Reason: The IPTABLES pattern expects SPT and DPT fields, which are not present for ICMP packets.

Suggestion for corrected pattern:

IPTABLES IN=(?:%{NOTSPACE:[observer][ingress][interface][name]})?\s+OUT=(?:%{NOTSPACE:[observer][egress][interface][name]})?\s+(?:MAC=(?:%{COMMONMAC:[destination][mac]})?(?::%{COMMONMAC:[source][mac]})?(?::[A-Fa-f0-9]{2}:[A-Fa-f0-9]{2})?\s+)?(:?%{IPTABLES4_PART}|%{IPTABLES6_PART}).*?PROTO=(?:%{WORD:[network][transport]})?(?:\s+SPT=(?:%{INT:[source][port]:int})?\s+DPT=(?:%{INT:[destination][port]:int})?\s+(?:%{IPTABLES_TCP_PART})?)?

This basically makes everything after PROTO optional.

[jsvd]

Hi @jdelker, any chance you could submit a PR, adding the fix to the regex and using your example line as a test?
You can see a couple of example PRs that do a similar task (fix + add or update tests) here: https://github.com/logstash-plugins/logstash-patterns-core/pull/313/files and https://github.com/logstash-plugins/logstash-patterns-core/pull/311/files

[jdelker]

I'm sorry, @jsvd. While I respect your request - and generally like to contribute - I'm not into Ruby at all.
Providing a diff for the ecs-v1 pattern is no problem (basically it's exchanging the provided line above), but coding the particular test is beyond what I can provide.

[jsvd]

@jdelker that's ok, can I ask you to just create a PR with the change to the grok pattern so that we can attribute this change to you? I'll carry it forward from there.