[GE users] inserting large accounting file into ARCo database

jana jana.olivova at sun.com
Thu Apr 8 09:53:25 BST 2010


Hi,

You can use the this utility to import accounting file into ARCo, it is a script that transforms accounting file into a reporting file format.

http://arco.sunsource.net/arco_faq.html

Test it first, there have been different number of fields in some of the reporting file lines, but I think the dbwriter is able to handle less fields now, ti just inserts 0, or null, this is mostly related to the Advanced Reservation that has not been in previous SGE releases. If it does not work you will just need to tweak the script. If you dig trough the archives, you might find an email where I already send this info.

Regards,

Jana Olivova


On 04/07/10 23:39, wagoodman wrote:

Thank you Daniel,

I guess I would have to start fresh, The users will still be able to use
qacct
(for the archived accounting file) and we will continue archive the
accounting
files. Thanks a lot for all your help.

Bill

-----Original Message-----
From: Daniel Templeton [mailto:daniel.templeton at oracle.com]
Sent: Wednesday, April 07, 2010 5:28 PM
To: Goodman, William
Cc: users
Subject: Re: [GE users] inserting large accounting file into ARCo
database

The accounting file is used by qacct.  The reporting file is used by
ARCo.  The information in the accounting file is a subset of what's in
the reporting file.  There is no tool to insert an accounting file into
the ARCo database.

Daniel

On 04/07/10 14:25, Goodman, William wrote:


O.k. I'm confused now, I thought that the reporting is generated from
the accounting file.
I will not have any reporting file until I install arco (which I


haven't


done yet), Does
SGE have a tool to dump the accounting data directly into the ARCo
database?

Bill

-----Original Message-----
From: Daniel Templeton [mailto:daniel.templeton at oracle.com]
Sent: Wednesday, April 07, 2010 5:16 PM
To: users
Cc: Goodman, William
Subject: Re: [GE users] inserting large accounting file into ARCo
database

Sadly, no.  The accounting file is only for qacct.  ARCo only reads


the


reporting file.  You'll need to either convert the accounting file
format into the reporting file format or write a utility to dump the
accounting data directly into the ARCo database.  I'd probably try to
former.  Check out the accounting and reporting man pages for format
details.

Daniel

On 04/07/10 14:11, wagoodman wrote:


Daniel,

The file is an archived accounting file. Under
$SGE_ROOT/default/common/accounting.
SGE is currently writing to that file. We archive the accounting


file(s)


when it reaches
100 MB (accounting.1.gz, accounting.2.gz and accounting.3.gz


etc..etc.


I


have
concatenated them to a file called accounting.orig that is 4 GB. So


what


I want to do
is "cat $SGE_ROOT/default/common/accounting accounting.orig>
accounting" then fire up
DB writer and I should have all the data collected in a calendar year


in


my ARCo database,
would that work?

Bill

-----Original Message-----
From: templedf [mailto:daniel.templeton at oracle.com]
Sent: Wednesday, April 07, 2010 4:22 PM
To: users at gridengine.sunsource.net<mailto:users at gridengine.sunsource.net>
Subject: Re: [GE users] inserting large accounting file into ARCo
database

When you say it's an accounting file, do you mean it's the file


called


"accounting", or is it the "reporting" file?

If the later, there's not much you need to do.  Just install ARCo and
fire up the DB writer.  It will slog through your data and dump it


into


the database.  If the former, you'll need to write a utility to


either


transform the accounting file into a reporting file or to add it to


the


DB directly.

Daniel

On 04/07/10 13:07, wgoodman wrote:


I have been testing the SGE ARCo database accounting and reporting


software for about 2 months, now I'm ready to bring it into my
production environment. However I want to keep all the data that I


have


collected for about a year. I have a 4 GB accounting flat file that I
want to incorporate into the ARCo database. So when I install ARCo, I
would have a calendar year. How do I import that 4 GB accounting file
into the ARCo  database?


Bill

------------------------------------------------------



http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessage


Id=252596


To unsubscribe from this discussion, e-mail:


[users-unsubscribe at gridengine.sunsource.net<mailto:users-unsubscribe at gridengine.sunsource.net>].

------------------------------------------------------



http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessage


Id=252597

To unsubscribe from this discussion, e-mail:
[users-unsubscribe at gridengine.sunsource.net<mailto:users-unsubscribe at gridengine.sunsource.net>].

------------------------------------------------------



http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessage


Id=252605


To unsubscribe from this discussion, e-mail:


[users-unsubscribe at gridengine.sunsource.net<mailto:users-unsubscribe at gridengine.sunsource.net>].




------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=252610

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net<mailto:users-unsubscribe at gridengine.sunsource.net>].





More information about the gridengine-users mailing list