Discussion:
[opennms-discuss] DNS Provisioning causes flood of nodeUpdated events
Jonathan Heard
2017-06-30 16:14:55 UTC
Permalink
Hello all,

I've been experimenting with using DNS as a source for Provisiond as
per https://wiki.opennms.org/wiki/DNS_Importing, which in our
environment would be an excellent way to keep OpenNMS up to date and
it's immediately updated when new VMs are provisioned. I've got it
working, and by using a foreign source and expressions in the DNS URL in
provisiond-configuration.xml it's happily populating my requisitions and
discovering host services - e.g.

<rescan-existing>dbonly</rescan-existing>
<requisition-def import-name="test"
import-url-resource="dns://ns01/my.zone/test/?expression=^test-.*">
<cron-schedule>* 5/30 * * * ? *</cron-schedule>
</requisition-def>
<requisition-def import-name="admin"
import-url-resource="dns://ns01/my.zone/admin/?expression=^admin-.*">
<cron-schedule>* 1/30 * * * ? *</cron-schedule>
</requisition-def>

I want it to scan DNS very regularly (possibly even more than every
30mins a above) but at the moment every new import from DNS causes
approximately 10 uei.opennms.org/nodes/nodeUpdated events for every
host, even though those hosts are already in the requisition and have
not changed :-(

I've tried disabling the following in opennms.properties too:

org.opennms.provisiond.scheduleRescanForExistingNodes=false
org.opennms.provisiond.scheduleRescanForUpdatedNodes=false

From a look through the DNS provisioning code it looks to me like it
always builds the whole requisition from scratch (i.e. although it uses
an IXFR 'incremental' DNS zone transfer, the SOA serial number is
hardcoded to 0 so it gets the whole zone every time), but the
Hostname/IP and Foreign Source+ID never changes. (Sidenote: The Foreign
IDs assigned by DNS are just an integer and many of them are a negative
number - Is this correct behaviour?)

What I can't figure out as yet is why so many nodeUpdated events are
triggered for each host (Anyone know if this is normal?), and whether
there's a way and I can force OpenNMS to only treat the node as
'updated' if something about that node has actually changed?

Thanks in advance for any advice.

Regards

Jonathan


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Jonathan Heard
2017-07-05 13:17:59 UTC
Permalink
I've raised https://issues.opennms.org/browse/NMS-9492 for this problem
and submitted a patch, which cures this for me. This is my first journey
into the OpenNMS code and development practices so I hope it's up to
scratch :-)
Post by Jonathan Heard
Hello all,
I've been experimenting with using DNS as a source for Provisiond
as per https://wiki.opennms.org/wiki/DNS_Importing, which in our
environment would be an excellent way to keep OpenNMS up to date and
it's immediately updated when new VMs are provisioned. I've got it
working, and by using a foreign source and expressions in the DNS URL
in provisiond-configuration.xml it's happily populating my
requisitions and discovering host services - e.g.
<rescan-existing>dbonly</rescan-existing>
<requisition-def import-name="test"
import-url-resource="dns://ns01/my.zone/test/?expression=^test-.*">
<cron-schedule>* 5/30 * * * ? *</cron-schedule>
</requisition-def>
<requisition-def import-name="admin"
import-url-resource="dns://ns01/my.zone/admin/?expression=^admin-.*">
<cron-schedule>* 1/30 * * * ? *</cron-schedule>
</requisition-def>
I want it to scan DNS very regularly (possibly even more than every
30mins a above) but at the moment every new import from DNS causes
approximately 10 uei.opennms.org/nodes/nodeUpdated events for every
host, even though those hosts are already in the requisition and have
not changed :-(
org.opennms.provisiond.scheduleRescanForExistingNodes=false
org.opennms.provisiond.scheduleRescanForUpdatedNodes=false
From a look through the DNS provisioning code it looks to me like it
always builds the whole requisition from scratch (i.e. although it
uses an IXFR 'incremental' DNS zone transfer, the SOA serial number is
hardcoded to 0 so it gets the whole zone every time), but the
Hostname/IP and Foreign Source+ID never changes. (Sidenote: The
Foreign IDs assigned by DNS are just an integer and many of them are a
negative number - Is this correct behaviour?)
What I can't figure out as yet is why so many nodeUpdated events are
triggered for each host (Anyone know if this is normal?), and whether
there's a way and I can force OpenNMS to only treat the node as
'updated' if something about that node has actually changed?
Thanks in advance for any advice.
Regards
Jonathan
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
http://www.opennms.org/index.php/Mailing_List_FAQ
opennms-discuss mailing list
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

Loading...