[opennms-discuss] Syslog no longer matching post upgrade from 19.0.x to 20.0.1

[Madden, Joe]

Hi All,

We use a lot of syslog messages which we matching on process match, and Severity.

These configurations worked on v19 but not v20. We did update to 20.0.1 to fix the syslogd-configuration.xml re-ordering but the matches which worked before, no longer work.

Please see an example syslog message (Below and attached as image):

<14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017 14:31:51.25


Our syslog configuration is like so:

<?xml version="1.0"?>

<syslogd-configuration>
<configuration
syslog-port="10514"
new-suspect-on-message="false"
parser="org.opennms.netmgt.syslogd.CustomSyslogParser"
forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"
matching-group-host="6"
matching-group-message="8"
discard-uei="DISCARD-MATCHING-MESSAGES"
/>

<import-file>syslog/Custom.syslog.xml</import-file>
<import-file>syslog/ApacheHTTPD.syslog.xml</import-file>
<import-file>syslog/LinuxKernel.syslog.xml</import-file>
<import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>
<import-file>syslog/OpenSSH.syslog.xml</import-file>
<import-file>syslog/OpenWrt.syslog.xml</import-file>
<import-file>syslog/Procmail.syslog.xml</import-file>
<import-file>syslog/Postfix.syslog.xml</import-file>
<import-file>syslog/Sudo.syslog.xml</import-file>



</syslogd-configuration>

File: syslog/Custom.syslog.xml

<syslogd-configuration-group>
<ueiList>
<ueiMatch>
<process-match expression="^HAL_ASE$" />
<match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
<uei>mottmac.com/syslog/Logstash/informational</uei>
<severity>Info</severity>
</ueiMatch>
</ueiList>
</syslogd-configuration-group>


Any ideas why these would no longer match?

Thanks

Joe

[Madden, Joe]

Hi All,

I am still trying to get to the bottom of this if anyone has any ideas.


Cheers

Joe.

From: Madden, Joe [mailto:***@mottmac.com]
Sent: 18 July 2017 16:11
To: General OpenNMS Discussion <opennms-***@lists.sourceforge.net>
Subject: [opennms-discuss] Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Hi All,

We use a lot of syslog messages which we matching on process match, and Severity.

These configurations worked on v19 but not v20. We did update to 20.0.1 to fix the syslogd-configuration.xml re-ordering but the matches which worked before, no longer work.

Please see an example syslog message (Below and attached as image):

<14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017 14:31:51.25


Our syslog configuration is like so:

<?xml version="1.0"?>

<syslogd-configuration>
<configuration
syslog-port="10514"
new-suspect-on-message="false"
parser="org.opennms.netmgt.syslogd.CustomSyslogParser"
forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"
matching-group-host="6"
matching-group-message="8"
discard-uei="DISCARD-MATCHING-MESSAGES"
/>

<import-file>syslog/Custom.syslog.xml</import-file>
<import-file>syslog/ApacheHTTPD.syslog.xml</import-file>
<import-file>syslog/LinuxKernel.syslog.xml</import-file>
<import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>
<import-file>syslog/OpenSSH.syslog.xml</import-file>
<import-file>syslog/OpenWrt.syslog.xml</import-file>
<import-file>syslog/Procmail.syslog.xml</import-file>
<import-file>syslog/Postfix.syslog.xml</import-file>
<import-file>syslog/Sudo.syslog.xml</import-file>



</syslogd-configuration>

File: syslog/Custom.syslog.xml

<syslogd-configuration-group>
<ueiList>
<ueiMatch>
<process-match expression="^HAL_ASE$" />
<match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
<uei>mottmac.com/syslog/Logstash/informational</uei>
<severity>Info</severity>
</ueiMatch>
</ueiList>
</syslogd-configuration-group>


Any ideas why these would no longer match?

Thanks

Joe

[Cyrille Bollu]

Hi Joe,

Usually, turning syslogd's log level to TRACE (in file log4j2.xml) shows
enough information to debug such issue.

Br,

Cyrille

Post by Madden, Joe
Hi All,
I am still trying to get to the bottom of this if anyone has any ideas.
Cheers
Joe.
*Sent:* 18 July 2017 16:11
*Subject:* [opennms-discuss] Syslog no longer matching post upgrade from
19.0.x to 20.0.1
Hi All,
We use a lot of syslog messages which we matching on process match, and Severity.
These configurations worked on v19 but not v20. We did update to 20.0.1 to
fix the syslogd-configuration.xml re-ordering but the matches which worked
before, no longer work.
<14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017 14:31:51.25
<?xml version="1.0"?>
<syslogd-configuration>
<configuration
syslog-port="10514"
new-suspect-on-message="false"
parser="org.opennms.netmgt.syslogd.CustomSyslogParser"
forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\
2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"
matching-group-host="6"
matching-group-message="8"
discard-uei="DISCARD-MATCHING-MESSAGES"
/>
<import-file>syslog/Custom.syslog.xml</import-file>
<import-file>syslog/ApacheHTTPD.syslog.xml</import-file>
<import-file>syslog/LinuxKernel.syslog.xml</import-file>
<import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>
<import-file>syslog/OpenSSH.syslog.xml</import-file>
<import-file>syslog/OpenWrt.syslog.xml</import-file>
<import-file>syslog/Procmail.syslog.xml</import-file>
<import-file>syslog/Postfix.syslog.xml</import-file>
<import-file>syslog/Sudo.syslog.xml</import-file>
</syslogd-configuration>
File: syslog/Custom.syslog.xml
<syslogd-configuration-group>
<ueiList>
<ueiMatch>
<process-match expression="^HAL_ASE$" />
<match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
<uei>mottmac.com/syslog/Logstash/informational</uei>
<severity>Info</severity>
</ueiMatch>
</ueiList>
</syslogd-configuration-group>
Any ideas why these would no longer match?
Thanks
Joe
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
http://www.opennms.org/index.php/Mailing_List_FAQ
opennms-discuss mailing list
To *unsubscribe* or change your subscription options, see the bottom of
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

[Seibold, Michael]

Hi Joe,

maybe you will find some hints in the release notes - I believe there was a major change, probably there is a new syslog parser configured by default and some possibility to change back to the old one.

-Michael


Von: Madden, Joe [mailto:***@mottmac.com]
Gesendet: Sonntag, 23. Juli 2017 11:16
An: General OpenNMS Discussion <opennms-***@lists.sourceforge.net>
Betreff: Re: [opennms-discuss] Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Hi All,

I am still trying to get to the bottom of this if anyone has any ideas.


Cheers

Joe.

From: Madden, Joe [mailto:***@mottmac.com]
Sent: 18 July 2017 16:11
To: General OpenNMS Discussion <opennms-***@lists.sourceforge.net>
Subject: [opennms-discuss] Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Hi All,

We use a lot of syslog messages which we matching on process match, and Severity.

These configurations worked on v19 but not v20. We did update to 20.0.1 to fix the syslogd-configuration.xml re-ordering but the matches which worked before, no longer work.

Please see an example syslog message (Below and attached as image):

<14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017 14:31:51.25


Our syslog configuration is like so:

<?xml version="1.0"?>

<syslogd-configuration>
<configuration
syslog-port="10514"
new-suspect-on-message="false"
parser="org.opennms.netmgt.syslogd.CustomSyslogParser"
forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"
matching-group-host="6"
matching-group-message="8"
discard-uei="DISCARD-MATCHING-MESSAGES"
/>

<import-file>syslog/Custom.syslog.xml</import-file>
<import-file>syslog/ApacheHTTPD.syslog.xml</import-file>
<import-file>syslog/LinuxKernel.syslog.xml</import-file>
<import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>
<import-file>syslog/OpenSSH.syslog.xml</import-file>
<import-file>syslog/OpenWrt.syslog.xml</import-file>
<import-file>syslog/Procmail.syslog.xml</import-file>
<import-file>syslog/Postfix.syslog.xml</import-file>
<import-file>syslog/Sudo.syslog.xml</import-file>



</syslogd-configuration>

File: syslog/Custom.syslog.xml

<syslogd-configuration-group>
<ueiList>
<ueiMatch>
<process-match expression="^HAL_ASE$" />
<match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
<uei>mottmac.com/syslog/Logstash/informational</uei>
<severity>Info</severity>
</ueiMatch>
</ueiList>
</syslogd-configuration-group>


Any ideas why these would no longer match?

Thanks

Joe

[Seth Leger]

Hi Joe,

I looked at the changes that I made to the syslog parser and couldn't
see any particular reason why your config would not work. There were a
variety of bugfixes that went into 20.0.0.

As far as the new parser, there is a new parser (RadixTreeSyslogParser)
but it is not configured as the default yet. However, it is more
functional than the other parsers so it may be switched to the default
in a future release.

I would follow Cyrille's advice and turn the logging up, it should give
you more details about the parsing inside CustomSyslogParser. Or you
could give the new RadixTreeSyslogParser a whirl. :)

Seth Leger
The OpenNMS Group

Post by Madden, Joe
Hi All,
I am still trying to get to the bottom of this if anyone has any ideas.
Cheers
Joe.
*Sent:* 18 July 2017 16:11
*Subject:* [opennms-discuss] Syslog no longer matching post upgrade from
19.0.x to 20.0.1
Hi All,
We use a lot of syslog messages which we matching on process match, and Severity.
These configurations worked on v19 but not v20. We did update to 20.0.1
to fix the syslogd-configuration.xml re-ordering but the matches which
worked before, no longer work.
<14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017 14:31:51.25
<?xml version="1.0"?>
<syslogd-configuration>
<configuration
syslog-port="10514"
new-suspect-on-message="false"
parser="org.opennms.netmgt.syslogd.CustomSyslogParser"
forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"
matching-group-host="6"
matching-group-message="8"
discard-uei="DISCARD-MATCHING-MESSAGES"
/>
<import-file>syslog/Custom.syslog.xml</import-file>
<import-file>syslog/ApacheHTTPD.syslog.xml</import-file>
<import-file>syslog/LinuxKernel.syslog.xml</import-file>
<import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>
<import-file>syslog/OpenSSH.syslog.xml</import-file>
<import-file>syslog/OpenWrt.syslog.xml</import-file>
<import-file>syslog/Procmail.syslog.xml</import-file>
<import-file>syslog/Postfix.syslog.xml</import-file>
<import-file>syslog/Sudo.syslog.xml</import-file>
</syslogd-configuration>
File: syslog/Custom.syslog.xml
<syslogd-configuration-group>
<ueiList>
<ueiMatch>
<process-match expression="^HAL_ASE$" />
<match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
<uei>mottmac.com/syslog/Logstash/informational</uei>
<severity>Info</severity>
</ueiMatch>
</ueiList>
</syslogd-configuration-group>
Any ideas why these would no longer match?
Thanks
Joe
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
http://www.opennms.org/index.php/Mailing_List_FAQ
opennms-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss