Discussion:
[opennms-discuss] XML collection not working
JohnD Blackburn
2017-07-17 04:32:10 UTC
Permalink
Hi all,

I have been trying to implement a custom monitor for CUPS printers using the XML collector, but I'm having problems getting it to collect any data.

Can someone please confirm that my configuration is correct, or let me know what I've missed.

We're running opennms 20.0.1



I have a webserver set up that returns the metrics I care about in xml format:

[***@aue-s-opennms01 etc]# curl http://10.1.2.3:8000/printers.xml
<?xml version="1.0" encoding="UTF-8"?>
<printers>
<printer name="Printer1">
<njobs>5</njobs>
</printer>
<printer name="Printer2">
<njobs>0</njobs>
</printer>
<printer name="Printer3">
<njobs>0</njobs>
</printer>
</printers>


Here is (I think) all the relevant configuration bits:

xml-datacollection-config.xml:

<xml-collection name="ABBCS-CUPS">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<xml-source url="http://{ipaddr}:8000/printers.xml">
<import-groups>xml-datacollection/CUPS-Printers.xml</import-groups>
</xml-source>
</xml-collection>


xml-datacollection/CUPS-Printers.xml:

<xml-groups>
<xml-group name="CUPS-Printer-stats" resource-type="CUPSPrinters"
resource-xpath="/printers/printer"
key-xpath="@name">
<xml-object name="printerName" type="string" xpath="@name" />
<xml-object name="njobs" type="GAUGE" xpath="njobs" />
</xml-group>
</xml-groups>

collectd-configuration.xml:

<package name="ABBCS-CUPS">
<filter>(IPADDR != '0.0.0.0')</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
<service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
<parameter key="collection" value="ABBCS-CUPS"/>
<parameter key="handler-class" value="org.opennms.protocols.xml.vtdxml.DefaultVTDXmlCollectionHandler"/>
</service>
</package>

...

<collector service="ABBCS-CUPS" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>


datacollection-configuration.xml:

<?xml version="1.0"?>
<datacollection-config rrdRepository="/opt/opennms/share/rrd/snmp/">
...
<snmp-collection name="ABBCS" snmpStorageFlag="select">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:8928</rra>
<rra>RRA:AVERAGE:0.5:12:8784</rra>
<rra>RRA:AVERAGE:0.5:480:366</rra>
<rra>RRA:MIN:0.5:12:8784</rra>
<rra>RRA:MAX:0.5:12:8784</rra>
</rrd>
...
<include-collection dataCollectionGroup="ABBCS-CUPS"/>
</snmp-collection>
</datacollection-config>



datacollection/ABBCS-CUPS.xml:

<datacollection-group name="ABBCS-CUPS">
<resourceType name="CUPSPrinters" label="CUPS Printers"
resourceLabel="Printer ${printerName}" >
<persistenceSelectorStrategy
class="org.opennms.netmgt.collection.support.PersistAllSelectorStrategy"/>
<storageStrategy
class="org.opennms.netmgt.collection.support.IndexStorageStrategy"/>
</resourceType>
</datacollection-group>


poller-configuration.xml:

<package name="ABBCS-CUPS">
<filter>((IPADDR != '0.0.0.0') &amp; (catincServers))</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
<parameter key="retry" value="1"/>
<parameter key="timeout" value="3000"/>
<parameter key="port" value="8000"/>
<parameter key="url" value="/printers.xml"/>
<parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
<parameter key="rrd-base-name" value="cups" />
<parameter key="response" value="200-202,299"/>
<parameter key="response-text" value=".*printer.*"/>
</service>
<downtime begin="0" end="300000" interval="30000"/>
<downtime begin="300000" end="43200000" interval="300000"/>
<downtime begin="43200000" end="432000000" interval="600000"/>
<downtime begin="432000000" delete="true"/>
</package>

...

<monitor service="ABBCS-CUPS" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor"/>


Any assistance greatly appreciated.

Regards,
John Blackburn.
Ronald Roskens
2017-07-18 13:31:11 UTC
Permalink
Does it work if you take out the handler-class parameter in the collectd-configuration.xml? The size of your XML is so small, that I don’t think VTD is going to make that much of a difference in speed for parsing it, so lets try with the default handler first. (You didn’t list which packages you have installed. The VTD handler isn’t installed by default and you need an extra package for it.
Post by JohnD Blackburn
Hi all,
I have been trying to implement a custom monitor for CUPS printers using the XML collector, but I’m having problems getting it to collect any data.
Can someone please confirm that my configuration is correct, or let me know what I’ve missed.
We’re running opennms 20.0.1
<?xml version="1.0" encoding="UTF-8"?>
<printers>
<printer name="Printer1">
<njobs>5</njobs>
</printer>
<printer name="Printer2">
<njobs>0</njobs>
</printer>
<printer name="Printer3">
<njobs>0</njobs>
</printer>
</printers>
<xml-collection name="ABBCS-CUPS">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<xml-source url="http://{ipaddr}:8000/printers.xml">
<import-groups>xml-datacollection/CUPS-Printers.xml</import-groups>
</xml-source>
</xml-collection>
<xml-groups>
<xml-group name="CUPS-Printer-stats" resource-type="CUPSPrinters"
resource-xpath="/printers/printer"
<xml-object name="njobs" type="GAUGE" xpath="njobs" />
</xml-group>
</xml-groups>
<package name="ABBCS-CUPS">
<filter>(IPADDR != '0.0.0.0')</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
<service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
<parameter key="collection" value="ABBCS-CUPS"/>
<parameter key="handler-class" value="org.opennms.protocols.xml.vtdxml.DefaultVTDXmlCollectionHandler"/>
</service>
</package>


<collector service="ABBCS-CUPS" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>
<?xml version="1.0"?>
<datacollection-config rrdRepository="/opt/opennms/share/rrd/snmp/">


<snmp-collection name="ABBCS" snmpStorageFlag="select">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:8928</rra>
<rra>RRA:AVERAGE:0.5:12:8784</rra>
<rra>RRA:AVERAGE:0.5:480:366</rra>
<rra>RRA:MIN:0.5:12:8784</rra>
<rra>RRA:MAX:0.5:12:8784</rra>
</rrd>


<include-collection dataCollectionGroup="ABBCS-CUPS"/>
</snmp-collection>
</datacollection-config>
<datacollection-group name="ABBCS-CUPS">
<resourceType name="CUPSPrinters" label="CUPS Printers"
resourceLabel="Printer ${printerName}" >
<persistenceSelectorStrategy
class="org.opennms.netmgt.collection.support.PersistAllSelectorStrategy"/>
<storageStrategy
class="org.opennms.netmgt.collection.support.IndexStorageStrategy"/>
</resourceType>
</datacollection-group>
<package name="ABBCS-CUPS">
<filter>((IPADDR != '0.0.0.0') &amp; (catincServers))</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
<parameter key="retry" value="1"/>
<parameter key="timeout" value="3000"/>
<parameter key="port" value="8000"/>
<parameter key="url" value="/printers.xml"/>
<parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
<parameter key="rrd-base-name" value="cups" />
<parameter key="response" value="200-202,299"/>
<parameter key="response-text" value=".*printer.*"/>
</service>
<downtime begin="0" end="300000" interval="30000"/>
<downtime begin="300000" end="43200000" interval="300000"/>
<downtime begin="43200000" end="432000000" interval="600000"/>
<downtime begin="432000000" delete="true"/>
</package>


<monitor service="ABBCS-CUPS" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor"/>
Any assistance greatly appreciated.
Regards,
John Blackburn.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________>
http://www.opennms.org/index.php/Mailing_List_FAQ <http://www.opennms.org/index.php/Mailing_List_FAQ>
opennms-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/opennms-discuss <https://lists.sourceforge.net/lists/listinfo/opennms-discuss>
JohnD Blackburn
2017-07-19 02:09:02 UTC
Permalink
Thankyou for your response.

I installed the VTD required packages before I tried setting it as a handler.

As suggested, I’ve removed the handler class entry from collectd-configuration.xml, but it was still not working.

I have just now added the following to the default-foreign-source.xml

<detector name="ABBCS-CUPS" class="org.opennms.netmgt.provision.detector.simple.HttpDetector">
<parameter key="url" value="/printers.xml"/>
<parameter key="port" value="8000"/>
</detector>

Which seems to have fixed the issue, but then I found I also needed to add a “~” to the “response-text” parameter in poller-configuration.xml:

<parameter key="response-text" value="~.*printer.*"/>


The documentation for the XML collector doesn’t mention default-foreign-source.xml at all, so is this something that I shouldn’t really need, and I’ve missed something else in my configuration, or is this just an oversight, and should be in the doco?










[ABB logo]

John Blackburn


Solution Architect - Cloud Services


Enterprise Software





Level 1, 757 Ann Street


Fortitude Valley, Queensland, 4006
AUSTRALIA
Phone: +61 7 33033219


Mobile: +61 403 344 524
E-Mail: ***@au.abb.com<mailto:***@au.abb.com>
www.abb.com<http://www.abb.com/>




From: Ronald Roskens [mailto:***@elfin.net]
Sent: Tuesday, 18 July 2017 11:31 PM
To: General OpenNMS Discussion <opennms-***@lists.sourceforge.net>
Subject: Re: [opennms-discuss] XML collection not working

Does it work if you take out the handler-class parameter in the collectd-configuration.xml? The size of your XML is so small, that I don’t think VTD is going to make that much of a difference in speed for parsing it, so lets try with the default handler first. (You didn’t list which packages you have installed. The VTD handler isn’t installed by default and you need an extra package for it.

On Jul 17, 2017, at 12:32 AM, JohnD Blackburn <***@au.abb.com<mailto:***@au.abb.com>> wrote:

Hi all,

I have been trying to implement a custom monitor for CUPS printers using the XML collector, but I’m having problems getting it to collect any data.

Can someone please confirm that my configuration is correct, or let me know what I’ve missed.

We’re running opennms 20.0.1



I have a webserver set up that returns the metrics I care about in xml format:

[***@aue-s-opennms01 etc]# curl http://10.1.2.3:8000/printers.xml
<?xml version="1.0" encoding="UTF-8"?>
<printers>
<printer name="Printer1">
<njobs>5</njobs>
</printer>
<printer name="Printer2">
<njobs>0</njobs>
</printer>
<printer name="Printer3">
<njobs>0</njobs>
</printer>
</printers>


Here is (I think) all the relevant configuration bits:

xml-datacollection-config.xml:

<xml-collection name="ABBCS-CUPS">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<xml-source url="http://{ipaddr}:8000/printers.xml<http://%7bipaddr%7d:8000/printers.xml>">
<import-groups>xml-datacollection/CUPS-Printers.xml</import-groups>
</xml-source>
</xml-collection>


xml-datacollection/CUPS-Printers.xml:

<xml-groups>
<xml-group name="CUPS-Printer-stats" resource-type="CUPSPrinters"
resource-xpath="/printers/printer"
key-xpath="@name">
<xml-object name="printerName" type="string" xpath="@name" />
<xml-object name="njobs" type="GAUGE" xpath="njobs" />
</xml-group>
</xml-groups>

collectd-configuration.xml:

<package name="ABBCS-CUPS">
<filter>(IPADDR != '0.0.0.0')</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
<service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
<parameter key="collection" value="ABBCS-CUPS"/>
<parameter key="handler-class" value="org.opennms.protocols.xml.vtdxml.DefaultVTDXmlCollectionHandler"/>
</service>
</package>




<collector service="ABBCS-CUPS" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>


datacollection-configuration.xml:

<?xml version="1.0"?>
<datacollection-config rrdRepository="/opt/opennms/share/rrd/snmp/">


<snmp-collection name="ABBCS" snmpStorageFlag="select">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:8928</rra>
<rra>RRA:AVERAGE:0.5:12:8784</rra>
<rra>RRA:AVERAGE:0.5:480:366</rra>
<rra>RRA:MIN:0.5:12:8784</rra>
<rra>RRA:MAX:0.5:12:8784</rra>
</rrd>


<include-collection dataCollectionGroup="ABBCS-CUPS"/>
</snmp-collection>
</datacollection-config>



datacollection/ABBCS-CUPS.xml:

<datacollection-group name="ABBCS-CUPS">
<resourceType name="CUPSPrinters" label="CUPS Printers"
resourceLabel="Printer ${printerName}" >
<persistenceSelectorStrategy
class="org.opennms.netmgt.collection.support.PersistAllSelectorStrategy"/>
<storageStrategy
class="org.opennms.netmgt.collection.support.IndexStorageStrategy"/>
</resourceType>
</datacollection-group>


poller-configuration.xml:

<package name="ABBCS-CUPS">
<filter>((IPADDR != '0.0.0.0') &amp; (catincServers))</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
<parameter key="retry" value="1"/>
<parameter key="timeout" value="3000"/>
<parameter key="port" value="8000"/>
<parameter key="url" value="/printers.xml"/>
<parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
<parameter key="rrd-base-name" value="cups" />
<parameter key="response" value="200-202,299"/>
<parameter key="response-text" value=".*printer.*"/>
</service>
<downtime begin="0" end="300000" interval="30000"/>
<downtime begin="300000" end="43200000" interval="300000"/>
<downtime begin="43200000" end="432000000" interval="600000"/>
<downtime begin="432000000" delete="true"/>
</package>




<monitor service="ABBCS-CUPS" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor"/>


Any assistance greatly appreciated.

Regards,
John Blackburn.



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org<http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Ronald Roskens
2017-07-19 13:25:49 UTC
Permalink
I can understand the confusion. Services need to be discovered (either manually or automatically) first before you can either poll them or collect data from them.

What documentation or tutorials have you read through so far? I’d like to know what was missed so that this can be improved for the next person who tries to do this.

Ron
Post by JohnD Blackburn
Thankyou for your response.
I installed the VTD required packages before I tried setting it as a handler.
As suggested, I’ve removed the handler class entry from collectd-configuration.xml, but it was still not working.
I have just now added the following to the default-foreign-source.xml
<detector name="ABBCS-CUPS" class="org.opennms.netmgt.provision.detector.simple.HttpDetector">
<parameter key="url" value="/printers.xml"/>
<parameter key="port" value="8000"/>
</detector>
<parameter key="response-text" value="~.*printer.*"/>
The documentation for the XML collector doesn’t mention default-foreign-source.xml at all, so is this something that I shouldn’t really need, and I’ve missed something else in my configuration, or is this just an oversight, and should be in the doco?
<image001.gif>
John Blackburn
Solution Architect - Cloud Services
Enterprise Software
Level 1, 757 Ann Street
Fortitude Valley, Queensland, 4006
AUSTRALIA
Phone: +61 7 33033219
Mobile: +61 403 344 524
www.abb.com <http://www.abb.com/>
Sent: Tuesday, 18 July 2017 11:31 PM
Subject: Re: [opennms-discuss] XML collection not working
Does it work if you take out the handler-class parameter in the collectd-configuration.xml? The size of your XML is so small, that I don’t think VTD is going to make that much of a difference in speed for parsing it, so lets try with the default handler first. (You didn’t list which packages you have installed. The VTD handler isn’t installed by default and you need an extra package for it.
Hi all,
I have been trying to implement a custom monitor for CUPS printers using the XML collector, but I’m having problems getting it to collect any data.
Can someone please confirm that my configuration is correct, or let me know what I’ve missed.
We’re running opennms 20.0.1
<?xml version="1.0" encoding="UTF-8"?>
<printers>
<printer name="Printer1">
<njobs>5</njobs>
</printer>
<printer name="Printer2">
<njobs>0</njobs>
</printer>
<printer name="Printer3">
<njobs>0</njobs>
</printer>
</printers>
<xml-collection name="ABBCS-CUPS">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<xml-source url="http://{ipaddr}:8000/printers.xml <http://{ipaddr}:8000/printers.xml>">
<import-groups>xml-datacollection/CUPS-Printers.xml</import-groups>
</xml-source>
</xml-collection>
<xml-groups>
<xml-group name="CUPS-Printer-stats" resource-type="CUPSPrinters"
resource-xpath="/printers/printer"
<xml-object name="njobs" type="GAUGE" xpath="njobs" />
</xml-group>
</xml-groups>
<package name="ABBCS-CUPS">
<filter>(IPADDR != '0.0.0.0')</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
<service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
<parameter key="collection" value="ABBCS-CUPS"/>
<parameter key="handler-class" value="org.opennms.protocols.xml.vtdxml.DefaultVTDXmlCollectionHandler"/>
</service>
</package>


<collector service="ABBCS-CUPS" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>
<?xml version="1.0"?>
<datacollection-config rrdRepository="/opt/opennms/share/rrd/snmp/">


<snmp-collection name="ABBCS" snmpStorageFlag="select">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:8928</rra>
<rra>RRA:AVERAGE:0.5:12:8784</rra>
<rra>RRA:AVERAGE:0.5:480:366</rra>
<rra>RRA:MIN:0.5:12:8784</rra>
<rra>RRA:MAX:0.5:12:8784</rra>
</rrd>


<include-collection dataCollectionGroup="ABBCS-CUPS"/>
</snmp-collection>
</datacollection-config>
<datacollection-group name="ABBCS-CUPS">
<resourceType name="CUPSPrinters" label="CUPS Printers"
resourceLabel="Printer ${printerName}" >
<persistenceSelectorStrategy
class="org.opennms.netmgt.collection.support.PersistAllSelectorStrategy"/>
<storageStrategy
class="org.opennms.netmgt.collection.support.IndexStorageStrategy"/>
</resourceType>
</datacollection-group>
<package name="ABBCS-CUPS">
<filter>((IPADDR != '0.0.0.0') &amp; (catincServers))</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
<parameter key="retry" value="1"/>
<parameter key="timeout" value="3000"/>
<parameter key="port" value="8000"/>
<parameter key="url" value="/printers.xml"/>
<parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
<parameter key="rrd-base-name" value="cups" />
<parameter key="response" value="200-202,299"/>
<parameter key="response-text" value=".*printer.*"/>
</service>
<downtime begin="0" end="300000" interval="30000"/>
<downtime begin="300000" end="43200000" interval="300000"/>
<downtime begin="43200000" end="432000000" interval="600000"/>
<downtime begin="432000000" delete="true"/>
</package>


<monitor service="ABBCS-CUPS" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor"/>
Any assistance greatly appreciated.
Regards,
John Blackburn.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________>
http://www.opennms.org/index.php/Mailing_List_FAQ <http://www.opennms.org/index.php/Mailing_List_FAQ>
opennms-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/opennms-discuss <https://lists.sourceforge.net/lists/listinfo/opennms-discuss>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________>
http://www.opennms.org/index.php/Mailing_List_FAQ <http://www.opennms.org/index.php/Mailing_List_FAQ>
opennms-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/opennms-discuss <https://lists.sourceforge.net/lists/listinfo/opennms-discuss>
JohnD Blackburn
2017-07-20 00:32:32 UTC
Permalink
The main document I used for setting up the XML collection was this one:

https://wiki.opennms.org/wiki/XML_Collector

which makes no reference to default foreign source definitions or discovery.

I think it needs a section on discovery pointing people to documentation on how to discover the service and/or what methods of discovery would be valid. Also, it is not obvious that XML collection is (potentially) a foreign source so it never occurred to me to search for doco on configuring the default foreign source.

Thanks again for your response.

Regards,
John









[ABB logo]

John Blackburn


Solution Architect - Cloud Services


Enterprise Software





Level 1, 757 Ann Street


Fortitude Valley, Queensland, 4006
AUSTRALIA
Phone: +61 7 33033219


Mobile: +61 403 344 524
E-Mail: ***@au.abb.com<mailto:***@au.abb.com>
www.abb.com<http://www.abb.com/>




From: Ronald Roskens [mailto:***@elfin.net]
Sent: Wednesday, 19 July 2017 11:26 PM
To: General OpenNMS Discussion <opennms-***@lists.sourceforge.net>
Subject: Re: [opennms-discuss] XML collection not working

I can understand the confusion. Services need to be discovered (either manually or automatically) first before you can either poll them or collect data from them.

What documentation or tutorials have you read through so far? I’d like to know what was missed so that this can be improved for the next person who tries to do this.

Ron

On Jul 18, 2017, at 10:09 PM, JohnD Blackburn <***@au.abb.com<mailto:***@au.abb.com>> wrote:

Thankyou for your response.

I installed the VTD required packages before I tried setting it as a handler.

As suggested, I’ve removed the handler class entry from collectd-configuration.xml, but it was still not working.

I have just now added the following to the default-foreign-source.xml

<detector name="ABBCS-CUPS" class="org.opennms.netmgt.provision.detector.simple.HttpDetector">
<parameter key="url" value="/printers.xml"/>
<parameter key="port" value="8000"/>
</detector>

Which seems to have fixed the issue, but then I found I also needed to add a “~” to the “response-text” parameter in poller-configuration.xml:

<parameter key="response-text" value="~.*printer.*"/>


The documentation for the XML collector doesn’t mention default-foreign-source.xml at all, so is this something that I shouldn’t really need, and I’ve missed something else in my configuration, or is this just an oversight, and should be in the doco?










<image001.gif>

John Blackburn



Solution Architect - Cloud Services



Enterprise Software






Level 1, 757 Ann Street



Fortitude Valley, Queensland, 4006
AUSTRALIA
Phone: +61 7 33033219



Mobile: +61 403 344 524
E-Mail: ***@au.abb.com<mailto:***@au.abb.com>
www.abb.com<http://www.abb.com/>





From: Ronald Roskens [mailto:***@elfin.net]
Sent: Tuesday, 18 July 2017 11:31 PM
To: General OpenNMS Discussion <opennms-***@lists.sourceforge.net<mailto:opennms-***@lists.sourceforge.net>>
Subject: Re: [opennms-discuss] XML collection not working

Does it work if you take out the handler-class parameter in the collectd-configuration.xml? The size of your XML is so small, that I don’t think VTD is going to make that much of a difference in speed for parsing it, so lets try with the default handler first. (You didn’t list which packages you have installed. The VTD handler isn’t installed by default and you need an extra package for it.

On Jul 17, 2017, at 12:32 AM, JohnD Blackburn <***@au.abb.com<mailto:***@au.abb.com>> wrote:

Hi all,

I have been trying to implement a custom monitor for CUPS printers using the XML collector, but I’m having problems getting it to collect any data.

Can someone please confirm that my configuration is correct, or let me know what I’ve missed.

We’re running opennms 20.0.1



I have a webserver set up that returns the metrics I care about in xml format:

[***@aue-s-opennms01 etc]# curl http://10.1.2.3:8000/printers.xml
<?xml version="1.0" encoding="UTF-8"?>
<printers>
<printer name="Printer1">
<njobs>5</njobs>
</printer>
<printer name="Printer2">
<njobs>0</njobs>
</printer>
<printer name="Printer3">
<njobs>0</njobs>
</printer>
</printers>


Here is (I think) all the relevant configuration bits:

xml-datacollection-config.xml:

<xml-collection name="ABBCS-CUPS">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<xml-source url="http://{ipaddr}:8000/printers.xml<http://%7bipaddr%7d:8000/printers.xml>">
<import-groups>xml-datacollection/CUPS-Printers.xml</import-groups>
</xml-source>
</xml-collection>


xml-datacollection/CUPS-Printers.xml:

<xml-groups>
<xml-group name="CUPS-Printer-stats" resource-type="CUPSPrinters"
resource-xpath="/printers/printer"
key-xpath="@name">
<xml-object name="printerName" type="string" xpath="@name" />
<xml-object name="njobs" type="GAUGE" xpath="njobs" />
</xml-group>
</xml-groups>

collectd-configuration.xml:

<package name="ABBCS-CUPS">
<filter>(IPADDR != '0.0.0.0')</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
<service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
<parameter key="collection" value="ABBCS-CUPS"/>
<parameter key="handler-class" value="org.opennms.protocols.xml.vtdxml.DefaultVTDXmlCollectionHandler"/>
</service>
</package>




<collector service="ABBCS-CUPS" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>


datacollection-configuration.xml:

<?xml version="1.0"?>
<datacollection-config rrdRepository="/opt/opennms/share/rrd/snmp/">


<snmp-collection name="ABBCS" snmpStorageFlag="select">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:8928</rra>
<rra>RRA:AVERAGE:0.5:12:8784</rra>
<rra>RRA:AVERAGE:0.5:480:366</rra>
<rra>RRA:MIN:0.5:12:8784</rra>
<rra>RRA:MAX:0.5:12:8784</rra>
</rrd>


<include-collection dataCollectionGroup="ABBCS-CUPS"/>
</snmp-collection>
</datacollection-config>



datacollection/ABBCS-CUPS.xml:

<datacollection-group name="ABBCS-CUPS">
<resourceType name="CUPSPrinters" label="CUPS Printers"
resourceLabel="Printer ${printerName}" >
<persistenceSelectorStrategy
class="org.opennms.netmgt.collection.support.PersistAllSelectorStrategy"/>
<storageStrategy
class="org.opennms.netmgt.collection.support.IndexStorageStrategy"/>
</resourceType>
</datacollection-group>


poller-configuration.xml:

<package name="ABBCS-CUPS">
<filter>((IPADDR != '0.0.0.0') &amp; (catincServers))</filter>
<include-range begin="1.1.1.1" end="254.254.254.254"/>
<include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
<parameter key="retry" value="1"/>
<parameter key="timeout" value="3000"/>
<parameter key="port" value="8000"/>
<parameter key="url" value="/printers.xml"/>
<parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
<parameter key="rrd-base-name" value="cups" />
<parameter key="response" value="200-202,299"/>
<parameter key="response-text" value=".*printer.*"/>
</service>
<downtime begin="0" end="300000" interval="30000"/>
<downtime begin="300000" end="43200000" interval="300000"/>
<downtime begin="43200000" end="432000000" interval="600000"/>
<downtime begin="432000000" delete="true"/>
</package>




<monitor service="ABBCS-CUPS" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor"/>


Any assistance greatly appreciated.

Regards,
John Blackburn.



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org<http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org<http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Ronald Roskens
2017-07-20 16:05:10 UTC
Permalink
I’ve added a note under the configuration basics mentioning that you need to add the service to the node either using a provisioning foreign source or as managed service in a provisioning requisition.

I also updated the wiki page to note you don’t need an additional package installed under OpenNMS Horizon 20.

Ron
https://wiki.opennms.org/wiki/XML_Collector <https://wiki.opennms.org/wiki/XML_Collector>
which makes no reference to default foreign source definitions or discovery.
I think it needs a section on discovery pointing people to documentation on how to discover the service and/or what methods of discovery would be valid. Also, it is not obvious that XML collection is (potentially) a foreign source so it never occurred to me to search for doco on configuring the default foreign source.
Thanks again for your response.
Regards,
John
JohnD Blackburn
2017-07-21 05:24:35 UTC
Permalink
Thanks for that Ronald, excellent work!

That bit about not requiring the extra package in V20 had me confused for a little bit when I couldn’t find the package to install ☺


John

From: Ronald Roskens [mailto:***@elfin.net]
Sent: Friday, 21 July 2017 2:05 AM
To: General OpenNMS Discussion <opennms-***@lists.sourceforge.net>
Subject: Re: [opennms-discuss] XML collection not working

I’ve added a note under the configuration basics mentioning that you need to add the service to the node either using a provisioning foreign source or as managed service in a provisioning requisition.

I also updated the wiki page to note you don’t need an additional package installed under OpenNMS Horizon 20.

Ron

On Jul 19, 2017, at 8:32 PM, JohnD Blackburn <***@au.abb.com<mailto:***@au.abb.com>> wrote:

The main document I used for setting up the XML collection was this one:

https://wiki.opennms.org/wiki/XML_Collector

which makes no reference to default foreign source definitions or discovery.

I think it needs a section on discovery pointing people to documentation on how to discover the service and/or what methods of discovery would be valid. Also, it is not obvious that XML collection is (potentially) a foreign source so it never occurred to me to search for doco on configuring the default foreign source.

Thanks again for your response.

Regards,
John

Loading...