[opennms-discuss] RRD vs Jrobin

[rmg_opennms]

Hey guys,

Been a while since I¹ve been on. We implemented 1.2.9 a year and half ago
and it¹s been going strong. With 1.3.3 just out now I¹m making the big
jump. So as I¹m playing around with it I noticed that Jrobin is now the
default file type for graphing. I was wondering if this is due to the goal
of releasing opennms as jar files in the future or if it was done for
performance reasons? I¹ve been requested to switch to rrd because the
graphs look different then the existing install (seriously, this was a
request). So I¹m just wondering if I can tell them it¹s for performance or
anything like that. Looks real good so far, had it up and running in about
3 hours with JVM collection and diskio collection (though I¹m currently
working through diskio collection issue)

Thx,
Ed

[Mike Huot]

Post by rmg_opennms
Hey guys,
Been a while since I’ve been on. We implemented 1.2.9 a year and
half ago and it’s been going strong. With 1.3.3 just out now I’m
making the big jump. So as I’m playing around with it I noticed
that Jrobin is now the default file type for graphing. I was
wondering if this is due to the goal of releasing opennms as jar
files in the future or if it was done for performance reasons?
I’ve been requested to switch to rrd because the graphs look
different then the existing install (seriously, this was a
request). So I’m just wondering if I can tell them it’s for
performance or anything like that. Looks real good so far, had it
up and running in about 3 hours with JVM collection and diskio
collection (though I’m currently working through diskio collection
issue)

Ed -

Jrobin was initially introduced to allow for a safe way to run
multiple threads for writing for rrd data. One of the biggest
performance issues with OpenNMS has been data collection as part of
that solution we needed a way to improve the write process and jrobin
helped. A side benefit is that it eliminates one more dependency for
installing OpenNMS.

Mike

[Mark Tuempfel]

One of the biggest performance issues
with OpenNMS has been data collection as part of that solution we needed
a way to improve the write process and jrobin helped.

half the truth. Have a look at topic "RRD write performance", I think most gain
comes from changing the sampling rate of RRD!

Mark

[Mark Tuempfel]

Post by Mark Tuempfel

One of the biggest performance issues
with OpenNMS has been data collection as part of that solution we needed
a way to improve the write process and jrobin helped.

half the truth. Have a look at topic "RRD write performance", I think most gain
comes from changing the sampling rate of RRD!
Mark

also mentioned by taurus in his blog entry "OpenNMS 1.3.3 Released"
http://blogs.opennms.org/?p=112

[Les Mikesell]

Post by Mark Tuempfel

One of the biggest performance issues
with OpenNMS has been data collection as part of that solution we needed
a way to improve the write process and jrobin helped.

half the truth. Have a look at topic "RRD write performance", I think most gain
comes from changing the sampling rate of RRD!

Has anyone come up with a way to convert old history to the new format?

[Fraley, Will]

http://www.opennms.org/index.php/FAQ-Install#Q:_How_do_I_convert_an_rrdt
ool_rrd_to_a_jrobin_rrd.3F

Jrobin will read an xml rrdtool dump, so I assume that is what this tool
does. I've parsed xml dumps without any problems using jrobin.

-Will

-----Original Message-----
From: opennms-discuss-***@lists.sourceforge.net
[mailto:opennms-discuss-***@lists.sourceforge.net] On Behalf Of Les
Mikesell
Sent: Friday, June 22, 2007 1:53 PM
To: General OpenNMS Discussion
Subject: Re: [opennms-discuss] RRD vs Jrobin

One of the biggest performance issues
with OpenNMS has been data collection as part of that solution we

needed > a way to improve the write process and jrobin helped.
half the truth. Have a look at topic "RRD write performance", I think
most gain comes from changing the sampling rate of RRD!

Has anyone come up with a way to convert old history to the new format?
--
Les Mikesell
***@gmail.com


------------------------------------------------------------------------
-
This SF.net email is sponsored by DB2 Express Download DB2 Express C -
the FREE version of DB2 express and take control of your XML. No limits.
Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of
this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

[Les Mikesell]

Post by Fraley, Will
http://www.opennms.org/index.php/FAQ-Install#Q:_How_do_I_convert_an_rrdt
ool_rrd_to_a_jrobin_rrd.3F
Jrobin will read an xml rrdtool dump, so I assume that is what this tool
does. I've parsed xml dumps without any problems using jrobin.

Actually the things I'd like to keep are already in jrobin format from a
pre-release trunk build of 1.3.3. I'd just like to convert to the new
layout with less full-resolution history and smaller files.

[Fraley, Will]

Ahhh gotcha. I want to do the exact same thing (and am planning on it
when I have time).

I haven't looked at what is involved in doing so, but I'm assuming it
will be somewhat painful.

-----Original Message-----
From: opennms-discuss-***@lists.sourceforge.net
[mailto:opennms-discuss-***@lists.sourceforge.net] On Behalf Of Les
Mikesell
Sent: Friday, June 22, 2007 2:46 PM
To: General OpenNMS Discussion
Subject: Re: [opennms-discuss] RRD vs Jrobin

Post by Fraley, Will
http://www.opennms.org/index.php/FAQ-Install#Q:_How_do_I_convert_an_rr
dt
ool_rrd_to_a_jrobin_rrd.3F
Jrobin will read an xml rrdtool dump, so I assume that is what this
tool does. I've parsed xml dumps without any problems using jrobin.

Actually the things I'd like to keep are already in jrobin format from a
pre-release trunk build of 1.3.3. I'd just like to convert to the new
layout with less full-resolution history and smaller files.
--
Les Mikesell
***@gmail.com

------------------------------------------------------------------------
-
This SF.net email is sponsored by DB2 Express Download DB2 Express C -
the FREE version of DB2 express and take control of your XML. No limits.
Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of
this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

[Mark Tuempfel]

Post by Fraley, Will
I haven't looked at what is involved in doing so, but I'm assuming it
will be somewhat painful.

I think best is to migrate from JRobin, although
http://oss.oetiker.ch/rrdtool/prog/index.en.html
list bindings for python and perl, but ONMS is Java!

It should be easy to write a java cli tool that converts from old to new format.
You can use JRobin API to do the math behind (resolution changing).

Mark

[David Hustace]

Post by Mark Tuempfel
half the truth. Have a look at topic "RRD write performance", I think most gain
comes from changing the sampling rate of RRD!

Actually quite a bit more than half the truth. We implemented JRobin
about 3 years ago now and one of the main reasons we did was to
attempt to improve performance due to the fact that the RRDTool
libraries are not thread-safe and requires us to synchronize (block)
on when entering the JNI Interface:

public static synchronized native String[] launch(String cmd);


Not only did this allowed us to have multiple threads writing RRD
data, synchronizing, in general, adds significant overhead, so this
has been a great performance improvement. BTW: I understand that
some recent RRDTool releases are thread-safe.

You are right, however, that one of the most significant performance
improvements has been the RRD structure change. It even improves the
RRDTool performance.

If you have a large number of RRDs that you are updating, I would try
switching to RRDTool and see if there is any difference in
performance (if you can spare the time and disk space).



David Hustace
The OpenNMS Group, Inc.

[rmg_opennms]

Thanks guys for the advice and hints. The main reason we're upgrading is
because our primary installation is pulling about 170k individual rrd files
and yeah, it's become an issue in terms of performance.

I'm playing with 1.3.3 now, have left it as jrobin and enabled the threading
of jrobin to 10 for now. Also just converted to the use grouping of values
into single files for the group.

It seems to be working great at the moment. Since this is still at a test
level I may do a little benchmarking of the system:

Dell 1850
2gb memory (soon to have 4)
RH4EL
2 Dual Core Xeon 3.20GHz

Should be useful to give people an idea of what the improvements are doing.

Thx,
Ed

Date: Fri, 22 Jun 2007 14:00:35 -0500
Conversation: [opennms-discuss] RRD vs Jrobin
Subject: Re: [opennms-discuss] RRD vs Jrobin
http://www.opennms.org/index.php/FAQ-Install#Q:_How_do_I_convert_an_rrdt
ool_rrd_to_a_jrobin_rrd.3F
Jrobin will read an xml rrdtool dump, so I assume that is what this tool
does. I've parsed xml dumps without any problems using jrobin.
-Will
-----Original Message-----
Mikesell
Sent: Friday, June 22, 2007 1:53 PM
To: General OpenNMS Discussion
Subject: Re: [opennms-discuss] RRD vs Jrobin

One of the biggest performance

issues

with OpenNMS has been data collection as part of that solution we

needed > a way to improve the write process and jrobin helped.
half the truth. Have a look at topic "RRD write performance", I think
most gain comes from changing the sampling rate of RRD!

Has anyone come up with a way to convert old history to the new format?
--
Les Mikesell
------------------------------------------------------------------------
-
This SF.net email is sponsored by DB2 Express Download DB2 Express C -
the FREE version of DB2 express and take control of your XML. No limits.
Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
http://www.opennms.org/index.php/Mailing_List_FAQ
opennms-discuss mailing list
To *unsubscribe* or change your subscription options, see the bottom of
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
http://www.opennms.org/index.php/Mailing_List_FAQ
opennms-discuss mailing list
To *unsubscribe* or change your subscription options, see the bottom of this
https://lists.sourceforge.net/lists/listinfo/opennms-discuss