Skip to content

Fix for getting the right log entries via Thruk #14

@GuusH

Description

@GuusH

Hi,

I've been having issues with the reporting functionality in Thruk. See shinken-solutions/shinken#1566 for my original bug report. After digging I found it's actually a small problem in this module that's causing the problem. Naparuba's statement that getting logs via the livestatus module will blow up if you have a very large number of logs is very true, however currently this module gets all logs instead of the relevant logs when using the reporting functionality in Thruk, which makes that more or less useless.

Details:
This is the relevant part of the livestatus command that's being issued by Thruk for an "SLA Hostgroup" report:

Filter: class = 2
Filter: message ~~ TIMEPERIOD TRANSITION: 24x7
Filter: type = TIMEPERIOD TRANSITION
Filter: message ~~ 24x7
And: 2
Or: 2
Or: 3

The make_mongo_filter code translates that into (nice piece of coding btw, changing the postfix livestatus protocol into the prefix mongo filter!):

[{ '$or' : 
[{ '$or' : 
[{ '$and' : 
[{ 'time' : { '$exists' : True } }, 
{ 'type' : 'TIMEPERIOD TRANSITION' }]
}, 
{ 'time' : { '$exists' : True } }
] 
},

The 2 'message' filter attributes are gone and replaced by these dummy 'time' entries. These are always true and cause the filter to get all logs. This causes my "report from hell" to request 7 GB of logs and then churns the broker to death in about 5 minutes by eating all memory and eating 100% cpu. And that's just our test environment: our production environment has 10 times as much log entries...
So I added the 'message' attribute to the 'good_attributes' and 'string_attributes' lists in the make_mongo_filter function, et voila, now this is the result:

[{ '$or' : 
[{ '$or' : 
[{ '$and' : 
[{ 'message' : { '$regex' : '24x7', '$options' : 'i' } }, 
{ 'type' : 'TIMEPERIOD TRANSITION' }] 
}, 
{ 'message' : { '$regex' : 'TIMEPERIOD TRANSITION: 24x7', '$options' : 'i' } }
] 
},

That's more like it! After this fix the report completes in about a minute and does not consume huge amounts of memory or cpu.
Here's the pull request: #13

I hope you'll merge it.

PS: there's something wrong with the Travis integration, since that fails on PRs for this module on stuff that wasn't even touched. See also my other PR (#11) that updates this module to be MongoDB 3 compatible :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions