RRDtool: moving data between 32bit and 64bit architectures

— 12,876 views — 21 Comments

When you mix architectures (or are moving from 32bit to 64bit like me) and use RRDtool for statistics, you might run into this problem:

ERROR: This RRD was created on another architecture

You can solve this, to dump the RRD file on the server that created it to XML, like this:

rrdtool dump stats.rrd > stats.xml

Then transfer it to the new server, and convert it back to RRD:

rrdtool restore -f stats.xml stats.rrd

The  other server can now read the file. You can test it with the info command:

rrdtool info stats.rrd

Hope this helps someone!

PS: If you want to convert a whole directory to XML, use this little bash oneliner I created for this job:

for f in *.rrd; do rrdtool dump ${f} > ${f}.xml; done

Comments

21 comments, archived as they were. New comments are closed.

Dago

Thanks … this was very useful for me, specially the oneliner bash …. and for the same reason, I’ll share my restore oneliner bash.

for f in *.xml; do rrdtool restore ${f} `echo ${f} | cut -f1 -d .`.rrd; done

Bye

Remi Bergsma

Great to hear Dago, and thanks for sharing! 🙂

Justin

Just to pull up some skeletons, I’d suggest using parameter expansion instead of the backticks, something like:

time for f in *.rrd; do rrdtool dump ${f} > ${f%%.rrd}.xml; done
time for f in *.xml; do rrdtool restore ${f} ${f%%.xml}.rrd; done

Cheers!

bodhix

I should have read the comments first…I did come up with a oneliner too 🙂

for f in *.xml; do g=`basename ${f} “.rrd.xml”` ; echo “processing $f $g”; rrdtool restore -f ${f} ${g}.rrd; done

bernard

Hi, i have converted rrd files to xml on my old debian 6 32 bits machine and transfered them to the new debian 6 64 bits machine.
when i tried to restored them to rrd, i received lot of errors and most of them where not converted.
I can see some graphs displayed but not all of them.
Also my poller on the new machine is not working at all knowing that the cron file is exactly the same as the old machine.
i had rebuild poller cash and in the cacti log file i see nothing.
please help.

bernard

this is the error i am receiving:
root@cacti:/var/lib/cacti/rra# for f in *.xml; do rrdtool restore ${f} `echo ${f} | cut -f1 -d .`.rrd; done
idirect-nms_rxtcp_63.rrd.xml:4661: parser error : Comment not terminated
<!– 201
^
I think after this error the bash stops restoring the rest of the xml files.

your help is highly appreciated.

Remi Bergsma

Hi,

Could you show the contents of the directory? Are there any files with spaces or so?

bernard

Hi Remi,
Can you please send me your email in order to attach the folder since it contains lot of files.
please see also the cacti crontab job for the poller:
MAILTO=root
*/1 * * * * www-data php /usr/share/cacti/site/poller.php >/dev/null 2>/var/log/cacti/poller-error.log
it is exactly the same as on the old machine but i see no logs in the cacti as if it is not polling.
So far i was able to transfer cacti config files and cacti database to the new machine, i can see that the settings are the same as the old one.
the only problem is the graphs and the poller not working.
PS: i didnt mentioned any file with spaces in the xml files.

regards

bernard

Hi Remi
update:
now the poller (spine) is working and the graphs are graphing.
i had to do the following:
mysql> grant all on cacti.* to cacti@localhost identified by ‘passwrd’;
However i still have this issue regarding the error when converting from xml to rra.
so the graphs that were not converted back are now opening but i think they are created from scratch and history on those graphs will not be displayed.

regards

NanoCube (@nanocube)

Brilliant; migrating munin data from 32 to 64 bits; needed this.
Only caveat is that I needed to modify Dago’s restore string somewhat since the database files from munin contain lots of periods in their filenames; which broke the ‘cut’ statement in there. I simply used:

for f in *.xml; do rrdtool restore ${f} `echo ${f} | sed s/\.xml//`; done

Claudio Kuenzler

Thanks for sharing. I’ve known the dump and restore commands but I never came across this error (cross architecture) before. The important info was, that the dump must be done on the server which created the rrd files. In my case I had to use a similar server with the same 32bit architecture as the creating server.

Sander

thank you for posting this, was driving me nuts for like 2 minutes until I found this post.

Robert

Thanks for this!

thepoonaset

I had to convert x64 .rrd’s to an i386 machine and my RRD’s were in sub-directories under /tmp/data.
Instead of trying to tell bash how many sub-dirs deep to look, just use find.
This will prevent guessing how many sub-dirs deep to search with eyesores such as:
subdir()
{
for f in $1; do
if [[ -d $f ]]; then
(($2 ${f}.xml; done

Convert back to .rrd on the i386 machine:
find /tmp/data -type f -iname ‘*.xml’ -print0|while IFS= read -r -d ” f; do rrdtool restore ${f} `echo ${f} | cut -f1 -d .`.rrd -f; done

redguy

Maybe old topic, but… be sure you have same locale on both machines as it influences xml serialization/deserialization. You may use: LANG=C before every rrdtool call like:

for f in *.rrd; do LANG=C rrdtool dump ${f} > ${f}.xml; done

with different locale you may have scaled/quantized results as floats from xml are misinterpreted.

Emil

Even older but… Thank you VERY much for taking time to make that comment!!

I can hardly believe it is not mentioned in the man pages.

mysterfr

Great article, saved my life !

However I see that no one knows shell parameter expansion…

Backup :
for i in *.rrd; do rrdtool dump $i > ${i%.*}.xml ; done
=> Dumps file.rdd to file.xml

Restore :
for i in *.xml; do rrdtool restore -f $i > ${i%.*}.rrd ; done
=> Restore file.xml to file.rrd

mysterfr

Woops, restore line should be : (damn copy/paste)
Great article, saved my life !

Backup :
for i in *.rrd; do rrdtool dump $i > ${i%.*}.xml ; done
=> Dumps file.rdd to file.xml

Restore :
for i in *.xml; do rrdtool restore -f $i ${i%.*}.rrd ; done
=> Restore file.xml to file.rrd

Bastian Kainz

Hi,

I run into an error like

“ERROR” […] get_xml_double

while restoring. Setting

LANG=C

before and everything works fine.

Mc.Sim

Hey Remi,
Thank you for that nice article.
You can add another for-loop to restore rrd-data from dumped xml.

for f in *.xml; do dst=`echo ${f} | sed s/\.xml$//` ; rrdtool restore -r -f ${f} ${dst} ; done