[opennms-discuss] syslog-configuration issue

[Madden, Joe]

Hi All,

I have an issue with syslog configuration. We have a customized syslog setup which was working on V19 but after an update to v20 it is now broken.

<ueiMatch>
<process-match expression="^HAL_ASE\\DbServer" />
<match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
<severity>Critical</severity>
<uei>mottmac.com/syslog/HAL_ASE/critical</uei>
</ueiMatch>

Internal Exception: org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'severity'. One of '{"http://xmlns.opennms.org/xsd/config/syslog":uei}' is expected.]

I therefore reordered it as Seth suggested in the OpenNMS chat to


<ueiMatch>
<process-match expression="^HAL_ASE\\DbServer" />
<match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
<uei>mottmac.com/syslog/HAL_ASE/critical</uei>
<severity>Critical</severity>
</ueiMatch>


However I now get the following error:

Invalid content was found starting with element 'severity'. One of '{"http://xmlns.opennms.org/xsd/config/syslog":parameter-assignment}' is expected.

I added <parameter-assignment matching-group="0" parameter-name="Match" /> Thinking it was complaing that this was missing but it doesn't seem to make any difference.

Does anyone have any suggestions on what I would need to do to get around this issue. I was starting to wonder if it was complaining about the use of Critical in <severity>Critical</severity>


Cheers

Joe

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

[Seibold, Michael]

Hi Joe,

I don't have v20 installed so I can't check it ad hoc - just look into the /opt/opennms/share/xsds/syslogd-configuratiopn-types.xsd (at least it's this one in meridian). The syntax of the config file should be declared there.

In Meridian I found for the severity

<element minOccurs="0" maxOccurs="unbounded" name="severity">
<annotation>
<documentation>The name of a syslog severity. If present, the severity of
an incoming message must match one of the severities named
by an instance of this tag within the ueiMatch.
</documentation>
</annotation>
<simpleType>
<restriction base="string">
<pattern value="[Ee]mergency|[Aa]lert|[Cc]ritical|[Ee]rror|[Ww]arning|[Nn]otice|[Ii]nfo|[Dd]ebug"/>
</restriction>
</simpleType>
</element>

If it's not declared there than something went wrong...


One hint for performance: if you know the syslog facility for the incoming syslogs then use the facility as the first matching rule and the severity as second matching rule.
Reason: comparing those values is probably realized as an integer compare and should be MUCH faster then a complex regex comparison. Using those two matching rules in front of the regex should probably avoid for the majority of syslogs to ever reach the regex comparison.

We had some SEVERE performance/availability problems when -due to some bigger "events"- a bunch of syslogs where coming in in a short time. After reconfiguring the syslog as described above the performance/availability problems where gone.

-Michael

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

[Madden, Joe]

Hi All,

Sorry for taking so long to reply.

I don't have the file: /opt/opennms/share/xsds/syslogd-configuration-types.xsd (I assume the p was a typo)

To be honest we use Logstash to send a few messages a day the performance issue is unlikely to be a problem for us (And it was all working before v20 upgrade) I suspect something in the V20 change has broken it.


<syslogd-configuration-group>
<ueiList>
<ueiMatch>
<process-match expression="^HAL_ASE\\DbServer" />
<match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
<uei>mottmac.com/syslog/HAL_ASE/critical</uei>
<severity>Critical</severity>
</ueiMatch>
</ueiList>
</syslogd-configuration-group>

I can't see anything obviously missing/incorrect from this configuration compared to the XSD.

Are we support to have the XSD in /opt/opennms/share/xsds/ I assumed the ones that were used would be in a jar somewhere.

Thanks

Joe.

-----Original Message-----
From: Seibold, Michael [mailto:***@gkvi.de]
Sent: 03 July 2017 12:29
To: General OpenNMS Discussion <opennms-***@lists.sourceforge.net>
Subject: Re: [opennms-discuss] syslog-configuration issue

Hi Joe,

I don't have v20 installed so I can't check it ad hoc - just look into the /opt/opennms/share/xsds/syslogd-configuratiopn-types.xsd (at least it's this one in meridian). The syntax of the config file should be declared there.

In Meridian I found for the severity

<element minOccurs="0" maxOccurs="unbounded" name="severity">
<annotation>
<documentation>The name of a syslog severity. If present, the severity of
an incoming message must match one of the severities named
by an instance of this tag within the ueiMatch.
</documentation>
</annotation>
<simpleType>
<restriction base="string">
<pattern value="[Ee]mergency|[Aa]lert|[Cc]ritical|[Ee]rror|[Ww]arning|[Nn]otice|[Ii]nfo|[Dd]ebug"/>
</restriction>
</simpleType>
</element>

If it's not declared there than something went wrong...


One hint for performance: if you know the syslog facility for the incoming syslogs then use the facility as the first matching rule and the severity as second matching rule.
Reason: comparing those values is probably realized as an integer compare and should be MUCH faster then a complex regex comparison. Using those two matching rules in front of the regex should probably avoid for the majority of syslogs to ever reach the regex comparison.

We had some SEVERE performance/availability problems when -due to some bigger "events"- a bunch of syslogs where coming in in a short time. After reconfiguring the syslog as described above the performance/availability problems where gone.

-Michael

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

[Seibold, Michael]

Hi Joe,

Post by Madden, Joe
I don't have the file: /opt/opennms/share/xsds/syslogd-configuration-types.xsd
Are we support to have the XSD in /opt/opennms/share/xsds/ I assumed the ones that were used would be in a jar somewhere.

In the past we had sometimes a strange behaviour after upgrading because our xsds-directory is in another place (linked via softlink) than opennms installer assumes, and he installed the new files in "his" place. So we sometimes had older xsd files in the directory opennms uses and the GUI somtimes produced corrupt config files (by example when editing thresholds).
That said, it should always be checked that those files correspond with the running version of opennms.

Post by Madden, Joe
(And it was all working before v20 upgrade) I suspect something in the V20 change has broken it

Benjamin Reed wrote in "This Week in OpenNMS: July 5th, 2017" :
"I fixed the syslogd-configuration.xml parser to honor out-of-order ueimatch fields, like Castor did."

Unfortunally the corresponding bug number is not mentioned.

So probably you are running into this bug or something similar?

-Michael


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

[Madden, Joe]

Yeah - I found that bug and reported it via the OpenNMS Chat.

I suspect its related to that - Can I import the updated xsd somehow or can I update to 20.0.1? - It's not on the repo yet.

I reordered it to get around the ordering bug - But maybe this error is from the same thing still?

Joe.
-----Original Message-----
From: Seibold, Michael [mailto:***@gkvi.de]
Sent: 10 July 2017 11:49
To: General OpenNMS Discussion <opennms-***@lists.sourceforge.net>
Subject: Re: [opennms-discuss] syslog-configuration issue

Hi Joe,

Post by Madden, Joe
/opt/opennms/share/xsds/syslogd-configuration-types.xsd
Are we support to have the XSD in /opt/opennms/share/xsds/ I assumed the ones that were used would be in a jar somewhere.

In the past we had sometimes a strange behaviour after upgrading because our xsds-directory is in another place (linked via softlink) than opennms installer assumes, and he installed the new files in "his" place. So we sometimes had older xsd files in the directory opennms uses and the GUI somtimes produced corrupt config files (by example when editing thresholds).
That said, it should always be checked that those files correspond with the running version of opennms.

Post by Madden, Joe
(And it was all working before v20 upgrade) I suspect something in the
V20 change has broken it

Benjamin Reed wrote in "This Week in OpenNMS: July 5th, 2017" :
"I fixed the syslogd-configuration.xml parser to honor out-of-order ueimatch fields, like Castor did."

Unfortunally the corresponding bug number is not mentioned.

So probably you are running into this bug or something similar?

-Michael


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

[Seibold, Michael]

Post by Madden, Joe
Can I import the updated xsd somehow

Maybe those xsds files are in /var/opennms/xsds on your system - it may depend on the type of OS

Michael


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss