lib: chunkio: file: fix mmap leak due to missing mremap on syncing (f…

…luent#1037)

When a file is synced, the memory map size can change since there is
always extra space available in the region. The current implementation
did not adjusted the memory mapped size leading to keep mapped regions.

This patch make sure to adjust memory map size if required.

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

bitbake: add recipe for v1.0.3 release

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

filter_parser: fix error message (fluent#1008)

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

filter: compose safety Tag and fix write offset (fluent#993)

Filter was trusting that input plugins was ingesting a NULL terminated
value, but if the data comes from Forward this will not be the case and
when calling the Routing will not match any rule.

This patch add a safety tag composer plus fix a problem when setting up
the write position when data has been removed.

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

storage: remove unused variables

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

output: on proxy plugin init, validate return value (fluent#949)

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

in_systemd: improve performance when packaging fields

When processing records, each one have multiple fields and due to
serialization when converting from Systemd format to Fluent Bit, we
need to know beforehand how many fields a record contains before
do the packaging. Systemd C API do not expose a function to get the
number of fields contained in a record, so the solution was to iterate
all fields and do the proper counting. This procedure is not optimal
for performance since once the fields are counted, we need to iterate
all fields again to do the packaging.

I've opened a RFE with Systemd team asking for such missing feature:

  systemd/systemd#10906

In the meanwhile, this patch workaround the problem in the following way:

  - Pack (register) the maximum number of fields allowed by 'max_fields'
    property.
  - Iterate, count and package all Systemd fields from the record.
  - Do a manual adjustment of the msgpack Map size header.

With this approach there is a performance improvement:

  - before => in_systemd_collect():  78.39%
  - after  => in_systemd_collect():  73.90%

this improvement represents a ~4.5% gain in performance.

note: the test was done using Valgrind (--tool=callgrind) processing 4k of
records coming from the Journal.

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

worker: initialize event structure

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

tests: internal: pack: use new flb_pack_json() prototype

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

filter_throttle: fix leak on cleanup (fluent#868)

The hash table and window were leaked in the destructor.

Signed-off-by: Don Bowman <don@agilicus.com>