27 Aug 10:44

lbudai

d5900a2

syslog-ng-3.29.1

3.29.1

Highlights

panos-parser(): parse Palo Alto PAN-OS logs

Example:

@include "scl.conf"

log {
  source { network(transport("udp")); };

  parser { panos-parser(); };

  destination {
   elasticsearch-http(
     index("syslog-ng-${YEAR}-${MONTH}-${DAY}")
     type("")
     url("http://localhost:9200/_bulk")
     template("$(format-json
       --scope rfc5424
       --scope dot-nv-pairs --rekey .* --shift 1 --exclude *future_* --exclude *dg_hier_level_*
       --scope nv-pairs --exclude DATE --key ISODATE @timestamp=${ISODATE})")
   );
  };
};

(#3234)

Features

snmptrap: improve error message when missing dependency
(#3363)
disk queue: reduce memory usage during load
(#3352)
Allow dupnames flag to be used in PCRE expressions, allowing duplicate names for named subpatterns
as explained here: https://www.pcre.org/original/doc/html/pcrepattern.html#SEC16 .

Example:
```
filter f_filter1 {
  match("(?<FOOBAR>bar)|(?<FOOBAR>foo)" value(MSG) flags(store-matches, dupnames));
};
```
(#3381)

Bugfixes

filter/regex: if there was a named match (?..)? that is optional to match, the previose or the next named matches might not be saved as named match.
(#3393)
tls: Fixed a bug, where ecdh-curve-list() were not applied at client side.
(#3356)
scratch-buffers: fix global.scratch_buffers_bytes.queued counter bug
This bug only affected the stats_counter value, not the actual memory usage (i.e. memory usage was fine before)
(#3355)
wsl: fix infinite loop during startup
(#3340)
openbsd: showing grammar debug info for openbsd too, when -y command line option is used
(#3339)
stats-query: speedup syslog-ng-ctl query get "*" command.

An algorithmic error view made syslog-ng-ctl query get "*" very slow with large number of counters.
(#3376)
syslogformat: fixing crashing with small invalid formatted logs see example in #3328
(#3364)
cfg: fix config reload crash via introducing on_config_inited in LogPipe
(#3176)
config: fix error reporting
- Error reporting was fixed for lines longer than 1024 characters.
- The location of the error was incorrectly reported in some cases.
  (#3383)
disk queue: fix possible crash during load, and possible false positive corruption detection
(#3342)
db-parser, pdbtool, graphite-output: fix glib assertion error

The assertion happened in these cases
- dbparser database load
- argument parsing in graphite-output
- pdbtool merge commad
Syslog-ng emitted a glib assertion warning in the cases above, even in successful executions.

If G_DEBUG=fatal-warnings environment variable was used, the warning turned into a crash.
(#3344)
stats: fix stats-ctl query crash when trying to reset all the counters
syslog-ng-ctl query get '*' --reset
(#3361)

Packaging

RHEL 7 packaging: fix logrotate file conflict with rsyslog
(#3324)
Debian packaging: python3-nose was removed from package dependencies.
Pytest will run Python related unittests (for modules/python/pylib/syslogng/debuggercli/tests/)
instead of nose.
(#3343)

Notes to developers

light: test for assertion errors in glib for each testcases
(#3344)
Fix signal handling when an external library/plugin sets SIG_IGN

Previously, setting SIG_IGN in a plugin/library (for example, in a Python module) resulted in a crash.
(#3338)
func-test: removed logstore_reader check, which was never reached
(#3236)
plugin_skeleton_creator: fixing a compiler switch

Wrong compiler switch used in plugin_skeleton_creator. This caused a compiler warning. The grammar debug info did not appear for that module, when -y command line option was used.
(#3339)
Light test framework: get_stats and get_query functions to DestinationDriver class

Two new functions added to DestinationDriver class which can be used for getting the stats
and query output of syslog-ng-ctl.
(#3211)

Other changes

internal(): limit the size of internal()'s temporary queue

The internal() source uses a temporary queue to buffer messages.
From now on, the queue has a maximum capacity, the log-fifo-size() option
can be used to change the default limit (10000).

This change prevents consuming all the available memory in special rare cases.
(#3229)
network plugins: better timer defaults for TCP keepalive

From now on, syslog-ng uses the following defaults for TCP keepalive:
- tcp-keepalive-time(): 60
- tcp-keepalive-intvl(): 10
- tcp-keepalive-probes(): 6
Note: so-keepalive() is enabled by default.
(#3357)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:
Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Christian Tramnitz, chunmeng, Gabor Nagy, Laszlo Budai, Laszlo Szemere, László Várady, MileK, Norbert Takacs, Peter Czanik, Péter Kókai, Terez Nemes.

Assets 3

27 Aug 10:44

lbudai

syslog-ng-3.28.1

d4523a1

syslog-ng-3.28.1

3.28.1

Highlights

http: add support for proxy option

Example:

log {
   source { system(); };
   destination { http( url("SYSLOG_SERVER_IP:PORT") proxy("PROXY_IP:PORT") method("POST") ); };
};

(#3253)

Features

map: template function

This template function applies a function to all elements of a list. For example: $(map $(+ 1 $_) 0,1,2) => 1,2,3.
(#3301)
use-syslogng-pid(): new option to all sources

If set to yes, syslog-ng overwrites the message's ${PID} macro to its own PID.
(#3323)

Bugfixes

affile: eliminate infinite loop in case of a spurious file path

If the template evaluation of a log message will result to a spurious
path in the file destination, syslog-ng refuses to create that file.
However the problematic log message was left in the msg queue, so
syslog-ng was trying to create that file again in time-reopen periods.
From now on syslog-ng will handle "permanent" file errors, and drop
the relevant msg.
(#3230)
Fix minor memory leaks in error scenarios
(#3265)
crypto: fix hang on boot due to lack of entropy
(#3271)
Fix IPv4 UDP destinations on FreeBSD

UDP-based destinations crashed when receiving the first message on FreeBSD due
to a bug in destination IP extraction logic.
(#3278)
network sources: fix TLS connection closure

RFC 5425 specifies that once the transport receiver gets close_notify from the
transport sender, it MUST reply with a close_notify.

The close_notify alert is now sent back correctly in case of TLS network sources.
(#2811)
disk-buffer: fixes possible crash, or fetching wrong value for logmsg nvpair
(#3281)
packaging/debian: fix mod-rdkafka Debian packaging
(#3282)
kafka destination: destination halts if consumer is down, and kafka's queue is filled
(#3305)
file-source: Throw error, when follow-freq() is set with a negative float number.
(#3306)
stats-freq: with high stats-freq syslog-ng emits stats immediately causing high memory and CPU usage
(#3320)
secure-logging: bug fixes (#3284)
- template arguments are now consistently checked
- fixed errors when mac file not provided
- fixed abort when derived key not provided
- fixed crash with slogkey missing parameters
- fixed secure-logging on 32-bit architectures
- fixed CMake build

Other changes

dbld: Fedora 32 support (#3315)
dbld: Removed Ubuntu Eoan (#3313)
secure-logging: improvements (#3284)
- removed 1500 message length limitation
- slogimport has been renamed to slogencrypt
- $(slog) will not start anymore when key is not found
- internal messaging (warning, debug) improvements
- improved memory handling and error information display
- CMake build improvements
- switched to GLib command line argument parsing
- the output of slogkey -s is now parsable
- manpage improvements

Notes to developers

dbld: devshell is now upgraded to Ubuntu Focal
(#3277)
dbld/devshell: Multiple changes:
- Added snmptrapd package.
- Added support for both python2 and python3.
  (#3222)
threaded-source: fully support default-priority() and default-facility()
(#3304)
CMake: fix libcap detection
(#3294)
Fix atomic_gssize_set() warning with new glib versions
(#3286)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Airbus Commercial Aircraft, Andras Mitzki, Antal Nemes, Attila Szakacs,
Balazs Scheidler, Gabor Nagy, Laszlo Budai, Laszlo Szemere, László Várady,
Péter Kókai, Vatsal Sisodiya, Vivin Peris.

Assets 3

27 Aug 10:45

lbudai

syslog-ng-3.27.1

0db9435

syslog-ng-3.27.1

3.27.1

Highlights

DESTIP/DESTPORT/PROTO: new macros. (#2899)
set-facility(): add new rewrite operation to change the syslog facility associated with the message. (#3136)
network tls: Added ca-file() option. With this option the user can set a bundled CA-file to verify the peer. (#3145)
Forward integrity and confidentiality of logs (#3121): It is an experimental feature currently, we are still working on the final form with the authors.

Features

DESTIP/DESTPORT/PROTO: new macros

These new macros express the destination ip, destination port and used protocol on a source.

The use-case behind the PR is as follows:
- someone has an appliance which sends out log messages via both UDP and TCP
- the format of the two are different, and he wants to capture either with the simplest possible filter
- netmask() doesn't work because the IP addresses are the same
- host() doesn't work because the hostnames are the same
Example:
```
log {
  source { network(localip(10.12.15.215) port(5555) transport(udp)); };
  destination { file("/dev/stdout" template("destip=$DESTIP destport=$DESTPORT proto=$PROTO\n")); };
};
```
Output:
```
destip=10.12.15.215 destport=5555 proto=17
```
(#2899)

set-facility(): add new rewrite operation to change the syslog facility
associated with the message.

log {
    source { system(); };
    if (program("postfix")) {
      rewrite { set-facility("mail"); };
    };
    destination { file("/var/log/mail.log"); };
    flags(flow-control);
};

(#3136)

network tls: Added ca-file() option. With this option the user can set a bundled CA-file to verify the peer.
(#3145)

http: When a HTTP response is received, emit a signal with the HTTP response code.
(Later it can be extended to read the response and parse it in a slot...).

This PR also extends the Python HTTP header module with the possibility of writing custom HTTP response code handlers. When someone implements an auth header plugin in Python, it could be useful (for example invalidating a cache).

Example config, click to expand!


@version: 3.25

python {
from syslogng import Logger

logger = Logger()

class TestCounter():
    def __init__(self, options):
        self.header = options["header"]
        self.counter = int(options["counter"])
        logger.debug(f"TestCounter class instantiated; options={options}")

    def get_headers(self, body, headers):
        logger.debug(f"get_headers() called, received body={body}, headers={headers}")

        response = ["{}: {}".format(self.header, self.counter)]
        self.counter += 1
        return response

    def on_http_response_received(self, http_code):
        self.counter += http_code
        logger.debug("HTTP response code received: {}".format(http_code))

    def __del__(self):
        logger.debug("Deleting TestCounter class instance")
};

source s_network {
  network(port(5555));
};

destination d_http {
    http(
        python_http_header(
            class("TestCounter")
            options("header", "X-Test-Python-Counter")
            options("counter", 11)
            # this means that syslog-ng will trying to send the http request even when this module fails
            mark-errors-as-critical(no)
        )
        url("http://127.0.0.1:8888")
    );
};

log {
    source(s_network);
    destination(d_http);
    flags(flow-control);
};

(#3159)

java/python: add support for the "arrow" syntax.
```
options("key" => "value")
```
(#3161) (#3247)
python: persist support for python

This feature enables users to persist data between reloads or restarts. The intended usage is to support bookmarking and acknowledgement in the future. It is not suitable for local database use cases.
(#3171)
rewrite: Added conditional set-tag() option. With this option the user can put condition statement inside set-tag option.
```
rewrite { set-tag("tag" condition(match("test" value("MSG")))); };
```
(#3190)
scl: add sumologic destinations: sumologic-syslog() and sumologic-http()
(#3194)
iterate: new template function

The iterate template function generates a series from an initial number and a next function.

For example you can generate a sequence of nonnegative numbers with
```
source {
  example-msg-generator(
    num(3)
    template("$(iterate $(+ 1 $_) 0)")
  );
};
```
(#3205)
telegram: new max-size option

Telegram message will be truncated for max-size size. Telegram does not accept message larger than 4096 utf8 characters. The default value is 4096.
(#3206)

example-message-generator : add support for values(name1 => value1, name2 => value2,..) syntax.

Example

@version: 3.27
log {
  source { example-msg-generator(template("message parameter")
                                 num(10)
                                 values("PROGRAM" => "program-name"
                                        "current-second" => "$C_SEC"
                                ));
         };
  destination { file(/dev/stdout template("$(format-json --scope all-nv-pairs)\n")); };
};

(#3237)

example-msg-generator: support freq(0) for fast message generation

log {
   source { example-msg-generator(freq(0) num(100)); };
   destination { file("/dev/stdout"); };
};

(#3245)

Bugfixes

file: changed time-reap() timer's schedule to respect the documentation (expires after last message)
(#3133)
dbld: fix building problems
- fix rpm package build on centos-7
- fix devshell image build
- fix ubuntu-trusty image build
- fix deb package build on ubuntu-trusty
- fix rpm package build on fedora-30
  (#3143)
tls (network): Properly log an error message, when key-file() or cert-file() is missing.
(#3145)
loggen: fix crash with invalid parameterization
(#3146)
format-json: fix printing of embedded zeros

Prior to 2.64.1, g_utf8_get_char_validated() in glib falsely identified embedded zeros as valid utf8 characters. As a result, format json printed the embedded zeroes as \u0000 instead of \x00. This change fixes this problem.
(#3175)
configure: fix --with-net-snmp configure option
(#3180)
python: fix Py_None reference counting in logger methods (trace, debug, info, warning, error)
(#3187)
afmongodb: do not build module when ENABLE_MONGODB=OFF
(#3188)
telegram: automatically truncate messages larger than 4096 utf8 characters to avoid telegram destination to get stuck
(#3206)
compat/glib: fix recursive call issue on CentOS-6/RHEL-6/platforms
(#3212)
timeutils: fix crash in %f conversion when non-numeric character is in usec field (e.g. ".asd123")
(#3270)

Packaging

macOS: add example startup configuration.
(#3172)
rpm: fix --without maxminddb option

If maxminddb development package was installed on the build system: rpmbuild fails if --without maxminddb was used.
(#3208)

Notes to developers

light: Support to relocate reports dir other than current base dir

For example

python -m pytest -lvs functional_tests/source_drivers/file_source/test_acceptance.py --installdir=/install --reports /tmp/

(#3157)

CONTRIBUTING.md: contribution guide updated
(#3174)
libtest: Now we install config_parse_lib.h, fake-time.h, mock-cfg-parser.h and queue_utils_lib.h
which help unit testing outside of core.
(#3179)
tests: Wait until snmptrapd process able to write traps into output file
(#3185)
mongodb: Replaced th...

Assets 3

27 Aug 10:38

lbudai

syslog-ng-3.26.1

f9a3495

syslog-ng-3.26.1

3.26.1

Highlights

file source: Added a new option to multi-line file sources: multi-line-timeout()
After waiting multi-line-timeout() seconds without reading new data from the file, the last (potentially partial)
message will be flushed and sent through the pipeline as a LogMessage.
Since the multi-line file source detects the end of a message after finding the beginning of the subsequent message
(indented or no-garbage/suffix mode), this option can be used to flush the last multi-line message
in the file after a multi-line-timeout()-second timeout.
There is no default value, i.e. this timeout needs to be explicitly configured.
Example config:
```
file("/some/folder/events"
    multi-line-mode("prefix-garbage")
    multi-line-prefix('^EVENT: ')
    multi-line-timeout(10)
    flags("no-parse")
);
```
(#2963)

python-http-header: Added this new plugin, which makes it possible for users to implement HTTP header plugins in Python language.
It is built on top of signal-slot mechanism: currently HTTP module defines only one signal, that is signal_http_header_request and python-http-header plugin implements a python binding for this signal. This means that when the signal_http_header_request signal is emitted then the connected slot executes the Python code.
The Python interface is:

def get_headers(self, body, headers):

It should return string List. The headers that will be appended to the request's header.
When the plugin fails, http module won't try to send the http request without the header items by default.
If you want http module to trying to send the request without these headers, just disable mark-errors-as-critical() function.
Original code was written by Ferenc Sipos.

Example config, click to expand!

@version: 3.26
python {
from syslogng import Logger
logger = Logger()
class TestCounter():
    def __init__(self, options):
        self.header = options["header"]
        self.counter = int(options["counter"])
        logger.debug(f"TestCounter class instantiated; options={options}")
    def get_headers(self, body, headers):
        logger.debug(f"get_headers() called, received body={body}, headers={headers}")
       
        response = ["{}: {}".format(self.header, self.counter)]
        self.counter += 1
        return response
    def __del__(self):
        logger.debug("Deleting TestCounter class instance")
};
source s_network {
  network(port(5555));
};
destination d_http {
    http(
        python_http_header(
            class("TestCounter")
            options("header", "X-Test-Python-Counter")
            options("counter", 11)
            # this means that syslog-ng will trying to send the http request even when this module fails
            mark-errors-as-critical(no)
        )
        url("http://127.0.0.1:8888")
    );
};
log {
    source(s_network);
    destination(d_http);
    flags(flow-control);
};

(#3123)

azure-auth-header: Added this new plugin, which generates authorization header for applications connecting to Azure.
It can be used as a building block in higher level SCLs.
Implemented as a signal-slot plugin.

Example config, click to expand!

@version: 3.26
@include "scl.conf"
destination d_http {
  http(
    url("http://127.0.0.1:8888")
    method("PUT")
    user_agent("syslog-ng User Agent")
    body("${ISODATE} ${MESSAGE}")
    azure-auth-header(
      workspace-id("workspace-id")
      secret("aa1a")
      method("POST")
      path("/api/logs")
      content-type("application/json")
    )
  );
};
source s_gen {
  example-msg-generator(num(1) template("Test message\n"));
};
log {
  source(s_gen);
  destination(d_http);
};

(#3078)

python: From now on users can specify a persist name template from python code.
```
@staticmethod
def generate_persist_name(options):
    return options["file_name"]
```
- Usage of this function is necessary, when one python destination is used multiple times in one config.
- Persist name from config takes precedence over generate_persist_name.
- Persist name is exposed through self.persist_name. (#3016)

Features

set-severity(): Added this new rewrite rule for changing message severity.
It receives a template and sets message severity by evaluating the template.
Numerical and textual severity levels are both supported.
Examples:
```
rewrite {
  set-severity("info");
  set-severity("6");
  set-severity("${.json.severity}");
};
```
(#3115)
$(list-search): Added a new template function, which returns the first index of a pattern in a list.
Starts the search at start_index. 0 based. If not found, returns empty string.
Usage: $(list-search --mode MODE <pattern> ${list})
Where mode can be: literal (default), prefix, substring, glob, pcre.
Add --start-index <index> to change the start index. (#2955)
config version: Made the config version check of the configuration more liberal by accepting version numbers
that had no changes relative to the current version. This means that if you are running 3.26 and the last
semantic change in the configuration was 3.22, then anything between 3.22 and 3.26 (inclusive) is accepted
by syslog-ng without a warning at startup. (#3074)
$SEVERITY instead of $LEVEL: syslog-ng now follows the RFC3164 trend of using severity instead of level
to refer to the severity of the message that is used in the template language ($SEVERITY),
filter expressions (severity()) and so on. (#3128)
http: Added ssl_version("tlsv1_3") and ssl_version("no-tlsv13") options to respectively force and disable TLSv1.3. (#3063)
scl: Improved error message at init, when an scl is missing a dependency. (#3015)
geoip2: Added template() option as an alias for the positional argument string, to match the grammar convention. (#3051)
loggly: Added transport() option, so users can now use it with tls (or any network() supported transport). (#3149)
config-option-database: Added support for parser, diskq and hook-commands blocks. (#3029)

Bugfixes

configure.ac: Fixed gethostbyname() function location detection (#3135)
http: Fixed a crash, when workers() was set to 0. We do not allow nonnegative values anymore. (#3116)
snmp-dest: engine-id() option now handles 5 to 32 characters, instead of the strict 10 before. (#3058)
http: Fixed handling of ssl-version() option, which was ignored before.
Prior this fix, these values of ssl-version in http destination were ignored by syslog-ng:
tlsv1_0, tlsv1_1, tlsv1_2, tlsv1_3. (#3083)
network sources: Added workaround for a TLS 1.3 bug to prevent data loss.
Due to a bug in the OpenSSL TLS 1.3 implementation (openssl/openssl#10880),
it is possible to lose messages when one-way communication protocols are used, -
such as the syslog protocol over TLS (RFC 5425,
RFC 6587) - and the connection is closed by the client right after sending data.
The bug is in the TLS 1.3 session ticket handling logic of OpenSSL.

To prevent such data loss, we've disabled TLS 1.3 session tickets in all syslog-ng network sources.
Tickets are used for session resumption, which is currently not supported by syslog-ng.

The loggen testing tool also received some bugfixes (#3064), which reduce the
likelihood of data loss if the target of loggen has not turned off session tickets.

If you're sending logs to third-party OpenSSL-based TLS 1.3 collectors, we recommend turning session
tickets off in those applications as well until the OpenSSL bug is fixed. (#3082)
cmake: Now we install loggen headers, as we do with autotools. (#3067)
graylog2, format-gelf: Fixed sending empty message, when ${PID} is not set.
Also added a default value "-" to empty short_message and host as they are mandatory fields. (#3112)
loggen: fix dependency error with cmake + openssl from nonstandard location (#3062)
config-option-database: Fixed reading 'grammar' and 'parser' files on 'POSIX' environment (#3125)
file source: Fixed file source not able to process new message after log-msg-size() increase. (#3075)
checkpoint parser: Fixed parsing ISO timestamp. (#3056)
secret-storage: Fixed some cases, where diagnostical logs were truncated. (#3141)
loggen, dqtool: Fixed a crash, when writing error/debug message or relocating qfile. (#3069)
build: Fixed a compatibility related build error on Solaris 11. (#3070)
loggen: Fixed address resolution when only loopback interface was configured. (#3048)

Packaging

scl: Moved scl files to the core package. (#2979)
RHEL: Now we include the packaging/rhel/ folder in our release tarball. (#3071)
RHEL 8 / CentOS 8: Added RHEL 8 / CentOS 8 support to syslog-ng.spec (#3034)

Notes to developers

signal-slot-connector: Introduced a generic event handler interface for syslog-ng modules.
- The concept is simple:
  - There is a SignalSlotConnector which stores Signal - Slot connections
  - Signal : Slot = 1 : N, so multiple slots can be assigned to the same Signal.
  - When a Signal is emitted, the connected Slots are executed.
  - Signals are string lite...

Assets 3

27 Aug 10:38

lbudai

syslog-ng-3.25.1

b880087

syslog-ng-3.25.1

3.25.1

Highlights

http-destination: Users now can specify the action for any HTTP result code.
Use with response-action(response_code => action) in your http block.
Available actions are: success, retry, drop and disconnect. (#3007)
syslog-ng-cfg-db: Added a new script, which can provide the options of
sources and destinations queried by the user. This tool can make the configuration
of syslog-ng a lot easier. Use with ./syslog-ng-cfg-db.py from the
contrib/config_database dir.(#2997)
redis-destination: Improved the performance by 2 orders of magnitude.
In our labor environment, now it operates at 25k EPS. (#2972)

Features

create-dirs(): Added to pipe() source/destination, and standardize the behavior.
(#3018, #2635)
default-network-drivers: Added max-connections() option, to change the limit
from 10. (#2961)
checkpoint: Added support for timezone value at the end of timestamps. (#3033)
filter/rewrite: Added disable-jit flag to disable JIT PCRE compilation. (#2992, #2986)
syslog-ng-ctl: Added export-config-graph option to visualize config graph. (#2990)
build/travis: Added ARM64 arch support. (#2967)
build/dbld: Readded CentOS 6 support. (#2860, #2971, #3028)
python: Added Python 3.8 support. (#3017)

Bugfixes

tls: Fixed an infinite loop which occured, when a TLS connection broke. (#3026, #3009)
log-block: Fixed an issue, where inline network destinations disjointed
the rest of the config. (#2989, #2820)
kafka/network-load-balancer: Fixed a crash when an argument was set to empty. (#3002)
python-source: Fixed a memory corruption during reload. (#3014)
python-destination: Actually use return value of open method. (#2998, #2513)
python-fetcher: Fixed FETCH_NO_DATA and FETCH_TRY_AGAIN constants. (#3012)
python: Fixed python Exception reporting when no Exception happened. (#2995)
telegram: Fixed the syntax error of the use-system-cert-store() option. (#2977)
config: Throw error to single dots, which were ignored before. (#3000)
file-destination: Delay ACKs until messages are written to disk. This fixes message
drop on I/O error and message lost in the LogProtoFileWriter in case of a crash, by
retrying to send the message. (#2985)
http-destination: Handle global template options values. (#3020)
timeutils: Fixed month and day name parsing, when only the first 2 characters
matched. (#3035)
logmsg: Added default PRI value (LOG_USER | LOG_NOTICE) to log messages
created without initial parsing. (#2974)
packaging: Added ordering dependencies network.target and network-online.target
to the service files. (#2994, #2667)
amqp: Support older (0.7.1) version (#2999)
loggen: Set plugin path in installation time. (#3019)
timeutils/patterndb: Fixed some undefined behaviours. (#2969)
stomp: Fixed a buffer over-read on connection. (#2988)
pseudofile: Fixed a crash, when template() option is not set. (#2988)
wildcard-source: Fixed a crash, when max-files() was set to 0. (#2988)

Other changes

syslog-ng-debun: Various maintenance updates and small fixes. (#2993)
scl: Avoid @requires loading the plugins themselves. (#2887)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Clément Besnier,
Gabor Nagy, jadhavsumit98, Janos Szigetvari, Laszlo Budai, Laszlo Szemere,
László Várady, MikeLim, Nikita Uvarov, Norbert Takacs, pabloli, Péter Kókai,
Zoltan Pallagi.

Assets 3

27 Aug 10:38

lbudai

syslog-ng-3.24.1

f219fbb

syslog-ng-3.24.1

3.24.1

Highlights

Add a new template function called $(format-flat-json), which generates
flattened json output. This is useful for destinations, where the json
parsing does not handle nested json format. (#2890)
Add ISO 8601 compliant week numbering. Use it with the ${ISOWEEK} macro
and and all its variants: S_ISOWEEK, R_ISOWEEK and C_ISOWEEK. (#2878)
Add add-contextual-data() glob selector. It matches the message with shell
style globbing. Enable it by setting selector(glob("$my_template") in the
add-contextual-data() block. (#2936)
Add new rewrite operations to manipulate the timezone portion of timestamps have
been added. set-timezone() to set the timezone value to a specific value,
fix-timezone() to fix up an incorrectly recognized timezone and guess-timezone()
to automatically deduce the timezone value on the assumption that the message
is received in near real time. (#2818)
Send Server Name Identification (SNI) information with transport(tls).
Enable it by setting the sni(yes) option in the tls block in your
destination. (#2930)

Features

templates: change the $LOGHOST macro to honour use-fqdn() (#2894)
Define syslog-ng-sysconfdir (#2932)
dqtool: add assign dqfile to persist file feature (#2872)

Bugfixes

Fix backtick subsitution of defines/environment variables in the main configuration file. (#2906, #2909)
Fix SCL block parameter substitution of quoted escaped newline (#2901)
python, diskq, random-generator source: crash after failed reload (#2907)
Fix crash at shutdown on 32bit systems (#2893, #2895)
Invalidate the value of the LEGACY_MSGHDR macro in case either the PID or the PROGRAM
macros are unset() using a rewrite rule. Previously LEGACY_MSGHDR would retain the old values. (#2896)
on 32bit platform diskq ftruncate could fail due to size 32/64 interface (#2892)
Support new tzdata format, starting from version 2009.XXX, in tzinfo parser. (#2898)
udp, udp6, tcp, tcp6, syslog, network destination: Correctly detect and set IP_MULTICAST_TTL
in case of multicast ip address (#2905)
Fix hostname resolve on systems with only the loopback network interface configured (#2933)
wildcard-file(): Add multi-line(), pad_size() and multi-line-mode() option validation. (#2922)
kafka-c: Fix multiple memleaks (#2944)

Other changes

geoip: remove deprecated module, geoip2 database location detection (#2780)
various refactor, build issue fixes (#2902)

Notes to the developers

LightRunWithStrace: Run syslog-ng behind strace (#2921)
LightVerboseLogOnError: Increase default pytest verbosity on error (#2919)
Dbld image caching (#2858)
Dbld gradle caching (#2857)
logreader,logsource: move scratch-buffer mark and reclaim into LogSource (#2903)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Bertrand Jacquin,
Gabor Nagy, Henrik Grindal Bakken, Kerin Millar, kjhee43, Laszlo Budai,
Laszlo Szemere, László Várady, Péter Kókai, Raghunath Adhyapak, Zoltan Pallagi.

Assets 3

27 Aug 10:37

lbudai

syslog-ng-3.23.1

26c0fe2

syslog-ng-3.23.1

3.23.1

Features

The redis() destination now handles any number of command parameters.
(#2816)
The format() option of date-parser() supports lists. From now on, a
single date-parser() instance is able to process different date formats,
making it easy to catch on when some programs change the way they log
information. (#2779)
Add relocation support for disk-buffer(). The relocate subcommand of
dqtool can be used to move a single or multiple queue files. (#2855)
file(), pipe(): The time-reap() option now can be set or disabled for
each destination separately.
Use time-reap(0) to disable closing idle destination files. (#2798)
syslog-ng-ctl: Add list-files subcommand to print files present in the
current configuration. (#2797)

Bugfixes

Fix minor memory leaks (#2868)
Add global context to standalone parsers (#2876)
Fix heap usage after free in cfg_run_parser_with_main_context (#2884)
Fix g_thread_init call order issue with older glibs (#2853)
SNMP destination: fixing statistics format (#2854)
eventlog: flush escaped_buffer when full (#2837)
dbparser: remove unnecessary lock (#2838)
dbparser: fix crash when context times out in the middle of another rule (#2832)
radix: fix grouping in PCRE (#2808)
add-contextual-data: make filters config plugin aware (#2886)
Undefined warning regression (#2829)

Other changes

Disable time-reap() on non-templated filenames by default (#2798)
The --preprocess-into command line flag accepts -, and writes the
preprocessed configuration to stdout. (#2869)
Add information on environment variables passed to the confgen script (#2888)
Light: minor fixes (#2867, #2844)
python: use malloc_debug for python unit tests (#2866)
Travis: verbose unit test output (#2851)
Travis cleanup (#2809)
filter: add unit tests to filter-op (#2835)
Fix clang/gcc diagnostic differences (#2810)
Fix leak in stats test (#2874)
cmake, autotools: -Wundef to enable-extra-warning (#2806)

Notes to the developers

Check whether commit messages are properly formatted (#2803, #2807)
gitignore: tags file and build directory (#2794)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Gabor Nagy,
Laszlo Budai, Laszlo Szemere, László Várady, Mark Bonsack, Mehul Prajapati,
Péter Kókai, Romain Tartière, Zoltan Pallagi.

Assets 3

27 Aug 10:37

lbudai

syslog-ng-3.22.1

64eeea4

syslog-ng-3.22.1

3.22.1

Highlights

Sending SNMP traps: Using the new snmp() destination, incoming log messages
can be converted to SNMP traps, as the fields of the SNMP messages can be
customized with macros. (#2693)
$(template) dynamic binding: Extends the $(template) template function to
allow dynamic binding. For example, the name of the template to be invoked
can come from the message (name-value pairs). (#2716)
syslog(), network(): Add dynamic-window-size() option to enable dynamic
flow control that distributes the specified amount of window between active
connections at runtime. This can be used in low-memory environments, where
only a small subset of the active clients sends messages at high rate.
(#2772)

Features

match(): Add support for the template() option (#2715)
add-contextual-data(): Allow using templates in name-value pairs (#2711)
Add support for floating point operations in template functions (#2742)
Add support for usec precision when parsing time (#2709)

Bugfixes

Fix null pointer access when destinations are suspended (#2778)
Fix grouping-by() deadlock (#2758)
Fix a general source-related crash and enhance wildcard-file()'s bookmark
handling (#2589)
Fix infinite loop (reload/reopen) (#2739)
Fix python() package/module name collision (#2438)
Fix escaped quote in block argument (#2781)
Reintroduce test on SYSLOG_NG_HAVE_TIMEZONE (#2774)
snmp(): Fix template leak (#2746)

Other changes

Never drop flow-controlled messages: The meaning of log-fifo-size() has
changed to avoid dropping flow-controlled messages when log-fifo-size() is
misconfigured. From now on, log-fifo-size() only affects messages that are
not flow-controlled. (#2753)
The -d/--debug syslog-ng command line flag no longer implies
-e/--stderr. If you want to redirect internal() source to stderr,
use the -e/--stderr option explicitly. (#2731)
dbld, RPM and DEB packaging improvements (#2724)
Checkpoint parser improvements (#2740)
Reset the timezone on config reload event (#2691)
geoip2(): Include IP into the error message (#2743)
Improve regexp error messages (#2796)
http(): Warn if less workers used than urls (#2757)
http(): Allow URLs to be specified by a space/comma separated string
(#2699)
loggen: Change message rate at runtime using signals (#2756)
debun: add acquire_running_syslog_config function (#2752)
FreeBSD fixes for the test suite (#2783)

Notes to the developers

ivykis: update to 0.42.4 (#2736)
Support generator plugins in global options (#2747)
logthrfetcher: new constants (#2766)
logthrsourcedrv: support position tracking (#2750)
Light: Support pre-commit and tox (#2725)
Enable Bison error flags: conflicts-sr/rr (#2762)
Dynamic stats constant registration (#2761)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:
Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler,
Christian Michallek, Fabien Wernli, Gabor Nagy, Kyeong Yoo, Laszlo Budai,
Laszlo Szemere, László Várady, Mehul Prajapati, Norbert Takacs, Oleksii Hamov,
Péter Kókai, Romain Tartière, Zoltan Pallagi.

Assets 3

27 Aug 10:39

lbudai

syslog-ng-3.21.1

42e54fe

syslog-ng 3.21.1

3.21.1

Highlights

Add an alternative, native, librdkafka based kafka-c() destination in
parallel of the existing Java implementation, that provides the same
configuration interface. Eventually, we expect this to replace the Java
one (#2496)
Add a native, http() based destination based driver for elasticsearch
called elasticsearch-http(), as an alternative of the Java one.
Eventually, we expect this to replace the Java implementation. (#2509)
Add the ability to automatically determine the timezone value for an
incoming log entry as long as the incoming stream is close to real time
and the timezone information is missing from the timestamp. Enable this
function by using flags(guess-timezone) for sources and the
date-parser(). (#2517, #2673)

Features

syslog(): Add the ability to work with messages larger than log-msg-size()
in the source driver by using the trim-large-messages(yes) option.
The characters over the limit will be truncated. Previously messages
longer than the limit caused the connection to be closed abruptly.
(#2644)
amqp(): add support for heartbeats and the "external" authentication
mechanism. (#2676, #2626)
graylog2(): add support for TLS and UDP. (#2657)
udp(): Add spoof-source-max-msglen() option to allow setting the
maximum spoofed datagram size, which was hard-wired to 1024 previously.
(#2535)
db-parser(): add an option program-template() that customizes the
value used for matching the PROGRAM field. (#2651)
pdbtool: Add sort option to pdbtool merge (#2664)
$(implode) and $(explode): add template functions to split and join
strings based on a simple separator. The exploded array is represented as
a syslog-ng list that can be manipulated with the $(list-*) template
functions. (#2700)
Add an --omit-empty-values option for value-pairs based destinations &
template functions. (#2519)
grouping-by() parser: add sort-key() option (#2701)

Support for non-syslog or non-standard formats in SCL

apache-accesslog-parser(): support for vhost:port as the first field in
common/combined log formats (#2688)
Add application adapter for Junos classification (#2684)
Add parser and adapter for CheckPoint LogExporter output (#2665)

Bugfixes

Fix race condition of idle timer and scheduled I/O job (#2650)
Few leaks find via sanitizer (#2696)
syslogformat: set $MSG even if the incoming message is empty (#2672)
Fix double-free error in logproto unit tests (#2662)
groupingby: identical persist name (#2659)
stats: deindex pruned counters/clusters (#2648)
Type hinting should not accept empty values (#2639)
app-parser, pseudofile: fix crash with grammar error (#2640)
python: set_timestamp normalization (#2643)
db-parser: fix memory leak (#2652)
grouping-by: use after free, memory leak, missing init calls of filters (#2655)
amqp: fixing double connect (#2660)
old style definition warning fixes (#2680)
Fix "!=" filter (#2683)
dbparser: fix memleak (#2706)
nondumpable-allocator: fixing mmap error handling (#2666)
Fix timeutils warning (#2604)
Fix old style include statement compatibility (#2600)
Fix config revert (threaded destinations) (#2596)
Add warning on old style include statement (#2592)

Other changes

cfg-parser: add aliases for yesno (#2671)
Include json-c in the dist tarball (#2590)
cmake: disable_all_modules support (#2647)
Cmake clang sanitizer (#2562)
timeutils refactor (#2483)
Expedite threaded flush at reload (#2656)
elasticsearch2: Added deprecation warning (#2628)
Astyle fixes (#2624)
Force C99 with GNU (#2623)
Make rewording and other small edits to README (#2608)
Port tests to Criterion (#2607, #2661, #2621, #2620, #2619, #2618, #2617,
#2616, #2615, #2599, #2594, #2593, #2591, #2586, #2584, #2583)
test_reliable_backlog: fix random failure (#2668)
Fix unit test with function pointer dereference in case of ASLR, Criterion (#2669)
test-stats-query: fix unit test (#2603)

Notes to the developers

Version from git describe (#2627)
light: example-msg-generator support (#2571)
light: test app parser applications (#2686)
light: Switch to native logger (#2546)
light: Remove SetupTestcase() dependency (#2587)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:
Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Chris Spencer,
David Liew, Fabien Wernli, Gabor Nagy, Laszlo Budai, Laszlo Szemere, Layne,
László Várady, Mehul Prajapati, Nik Ambrosch, Parth Wazurkar, Péter Kókai,
Terez Nemes, Victor Ma, Zoltan Pallagi.

Assets 3

27 Aug 10:39

alltilla

syslog-ng-3.20.1

c89908b

syslog-ng-3.20.1

3.20.1

Features

Add persist-tool (#2511)
Collectd destination (#1433)
Config reload status feedback (#2367)
Netskope parser (#2541)
Websense parser (#2471)
Json list support (#2536)
Xml-parser: add list support (#2544)

Bugfixes

DNS memory leak/segfault fix (#2500)
Elasticsearch2: fix bulk send for client-mode("http") (#2478)
Few leak fix (#2563)
Filter-re: ref/unref NVTable around regex eval (#2494)
Fix allowing negative version number in config (#2548)
Fix app-parser() per reload memory leak (#2469)
Fix non-reliable disk-queue truncating problem on load (#2406)
Fix threaded source/destination crash when reverting configuration (#2555)
Http: add missing free for self->body_template (#2558)
Java, elasticsearch2, explicit unsupport for named templates (#2534)
Loggen: parameter handling (#2477)
Python-source: fix crash when posting from python thread (#2533)
Read acknowledgments send by Riemann (#2523)
Redis, Riemann: fix ref/unref-ing templates (#2514, #2530)
Syslog-ng@default: use pid file location on control socket (#2489)
Threaded-dest: Fix integer overflow (#2512)
Threaded-dest: make persist_name local (#2516)
Xml/geoip2: make prefix optional (#2538)

Other changes

Autotools, cmake: add detection for pylint, nosetests (#2537,#2564)
Autotools: relaxing python dependency requirement (#2472)
Cfg-parser: printing error positions in case of parse failure (#2455)
Cmake: add missing detections (#2510)
Collect-cov.sh: make coverage should work with lcov in bionic (#2515)
Configure: fix "--with-docbook*" option usage (#2465)
Custom LGTM.com query for detecting uses of gmtime that are not covered by a lock (#2413)
Fix static uClibc-ng support (#2501)
Format non-literal fixes (#2567)
Grammar: remove the pragma less include (#2550)
Http-worker: refactor & fix curl_easy_getinfo error message (#2527)
Lib/compat: replace strcasestr() implementation (#2482)
Libtest: Adding stopwatch.h into libtest_HEADERS list (#2553)
Packaging: fix the description of mod-examples (#2522)
Python-debugger: fix macro completion (#2439)
Python, java destination add already introduced retry (#2559)
Remove elastic v1 support (#2554)
Simplify libtest queue utils (#2556)
Syslog-ng.8.xml: remove unneeded default-modules section (#2475)
Travis: use the latest Bison version (macOS) (#2529)
Various fixes for issues reported by LGTM (#2524)

Notes to the developers

Example-msg-generator: num option (#2565)
Own grammar support in generator plugin (#2552)
ProtoClient: provide process_in function to logwriter (#2468)
Pytest_framework: eliminating __registered_instances, exposing SyslogNgCtl to user api (#2503)
Pytest_framework: MessageReader: Explain the local context around python asserts (#2507)
Pytest_framework: support for implicit groups in config.create_logpath (#2490)
Pytest: Renaming pytest_framework to python_functional (#2542)
Python-destination: send can return worker_insert_result_t, flush support (#2487)
Python: internal() source exposed via syslogng.Logger (#2505)
Remove unused submodules (#2525)
Simpler names for WORKER_INSERT_RESULT_T in language bindings (#2506)
Split xml-parser into xml-parser and xml-scanner (#2459)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:
Andras Mitzki, Andrej Valek, Antal Nemes, Attila Szakacs, Balazs Scheidler,
Bas van Schaik, Fᴀʙɪᴇɴ Wᴇʀɴʟɪ, Gabor Nagy, Laszlo Boszormenyi, Laszlo Budai,
Lorand Muzamel, László Várady, Mehul Prajapati, Naveen Revanna, Peter Czanik,
Peter Kokai, Romain Tartière, Stephen, Terez Nemes, Norbert Takács,
Soubhik Chakraborty, NottyRu, Chris Packham.

Assets 3

Releases: syslog-ng/syslog-ng

syslog-ng-3.29.1

3.29.1

Highlights

Features

Bugfixes

Packaging

Notes to developers

Other changes

Credits

Uh oh!

syslog-ng-3.28.1

3.28.1

Highlights

Features

Bugfixes

Other changes

Notes to developers

Credits

Uh oh!

syslog-ng-3.27.1

3.27.1

Highlights

Features

Bugfixes

Packaging

Notes to developers

Uh oh!

syslog-ng-3.26.1

3.26.1

Highlights

Features

Bugfixes

Packaging

Notes to developers

Uh oh!

syslog-ng-3.25.1

3.25.1

Highlights

Features

Bugfixes

Other changes

Credits

Uh oh!

syslog-ng-3.24.1

3.24.1

Highlights

Features

Bugfixes

Other changes

Notes to the developers

Credits

Uh oh!

syslog-ng-3.23.1

3.23.1

Features

Bugfixes

Other changes

Notes to the developers

Credits

Uh oh!

syslog-ng-3.22.1

3.22.1

Highlights

Features

Bugfixes

Other changes

Notes to the developers

Credits

Uh oh!

syslog-ng 3.21.1

3.21.1

Highlights

Features

Support for non-syslog or non-standard formats in SCL

Bugfixes

Other changes

Notes to the developers

Credits

Uh oh!