[GE users] SDM 6.2U3 won't start

cbyun cbyun at ll.mit.edu
Wed Jul 1 15:37:56 BST 2009


It seems that during the SDM installation, the Java policy template was copied into the SDM spool directory as is:

# diff /opt/sdm/util/templates/java.policy.template /var/spool/sdm/sdm62u3/security/java.policy

What is the expected behavior here?

- Chansup

> -----Original Message-----
> From: cbyun [mailto:cbyun at ll.mit.edu]
> Sent: Wednesday, July 01, 2009 9:28 AM
> To: users at gridengine.sunsource.net
> Subject: RE: [GE users] SDM 6.2U3 won't start
>
> Hi Richard,
>
> I'm using exactly the same JVM:
> # java -version
> java version "1.6.0_13"
> Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02, mixed mode)
>
>
> I'm running the CentOS 5.3 built by Rocks 5.2.
>
> # cat /etc/*release
> CentOS release 5.3 (Final)
> Rocks release 5.2 (Chimichanga)
> # uname -a
> Linux llgriddev 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009
> x86_64 x86_64 x86_64 GNU/Linux
>
> However, majority of MBean policy was set correctly in my java.policy:
>
> # grep MBean /var/spool/sdm/sdm62u3/security/java.policy
>      permission javax.management.MBeanTrustPermission "register";
>    permission javax.management.MBeanServerPermission "createMBeanServer";
>    permission javax.management.MBeanTrustPermission "register";
>    permission javax.management.MBeanPermission "*", "registerMBean";
>    permission javax.management.MBeanPermission "*", "unregisterMBean";
>    permission javax.management.MBeanPermission "*", "getDomains";
>    permission javax.management.MBeanPermission "*", "getObjectInstance";
>    permission javax.management.MBeanPermission "*", "queryMBeans";
>    permission javax.management.MBeanPermission "*", "queryNames";
>    permission javax.management.MBeanPermission "*", "getAttribute";
>    permission javax.management.MBeanPermission "*", "getMBeanInfo";
>    permission javax.management.MBeanPermission "*",
> "addNotificationListener";
>    permission javax.management.MBeanPermission "*",
> "removeNotificationListener";
>    permission javax.management.MBeanPermission "*", "isInstanceOf";
>    permission javax.management.MBeanPermission "*", "getObjectInstance";
>    permission javax.management.MBeanPermission "*", "invoke";
>      /* permission javax.management.MBeanPermission "*", "runtime"; */
>    permission javax.management.MBeanServerPermission "createMBeanServer";
>    permission javax.management.MBeanTrustPermission "register";
>    permission javax.management.MBeanPermission "*", "registerMBean";
>    permission javax.management.MBeanPermission "*", "unregisterMBean";
>    permission javax.management.MBeanPermission "*", "getDomains";
>    permission javax.management.MBeanPermission "*", "getObjectInstance";
>    permission javax.management.MBeanPermission "*", "queryMBeans";
>    permission javax.management.MBeanPermission "*", "queryNames";
>    permission javax.management.MBeanPermission "*", "getAttribute";
>    permission javax.management.MBeanPermission "*", "getMBeanInfo";
>    permission javax.management.MBeanPermission "*",
> "addNotificationListener";
>    permission javax.management.MBeanPermission "*", "isInstanceOf";
>    permission javax.management.MBeanPermission "*", "getObjectInstance";
>    permission javax.management.MBeanPermission "*", "invoke";
>    permission javax.management.MBeanPermission
> "com.sun.grid.grm.ui.ConfigurationService#run", "getMBeanInfo";
>    permission javax.management.MBeanPermission
> "com.sun.grid.grm.ui.ConfigurationService#run", "invoke";
>    permission javax.management.MBeanPermission
> "com.sun.grid.grm.ui.ConfigurationService#state", "getAttribute";
>
> Even if I commented out the one line which  you recommended, cs_vm still
> doesn't start.
>
> Any further suggestions?
>
> Thanks,
> - Chansup
>
>
>
> > -----Original Message-----
> > From: Richard.Hierlmeier at sun.com [mailto:Richard.Hierlmeier at sun.com]
> > Sent: Wednesday, July 01, 2009 3:19 AM
> > To: users at gridengine.sunsource.net
> > Subject: Re: [GE users] SDM 6.2U3 won't start
> >
> > Hi Chansup,
> >
> > SDM uses its own java policy file. After the installation it is stored
> at
> >
> >    <local_spool_dir>/security/java.policy
> >
> > It seems that there is something wrong with this policy file. The action
> > "runtime" for the javax.management.MBeanPermission does not exist.
> >
> > Please remove the following line from the java.policy file. You can
> > comment it
> > out with C-style comments:
> >
> > <    permission javax.management.MBeanPermission "*", "runtime";
> >  > /* permission javax.management.MBeanPermission "*", "runtime"; */
> >
> > You can also remove/comment out this line in the
> > <dist-dir>/util/templates/java.policy.template. This file is used as
> > source for
> > the java.policy file during the installation.
> >
> > The strange thing is that the problem does not occur in our environment.
> > We
> > tested it also with java6u13. On my opensolaris box I tested it with
> > 1.6.0_13-b03:
> >
> > % java -version
> > java version "1.6.0_13"
> > Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
> > Java HotSpot(TM) Server VM (build 11.3-b02, mixed mode)
> >
> > What is the exact version of your jvm?
> >
> > Richard
> >
> >
> > cbyun wrote:
> > > I found the following err file on /tmp:
> > >
> > > # cat err.2730.44
> > > java.security.policy: error adding Permission,
> > javax.management.MBeanPermission:
> > >         java.lang.IllegalArgumentException: Invalid permission:
> runtime
> > >
> > > This might be the cause of the start up failure.
> > > Anyway, Below is the stack trace that I got:
> > >
> > > # /bin/sh -x /opt/sdm/bin/sdmadm -d -s sdm62u3 suj
> > > + PATH=/bin:/usr/bin:/sbin:/usr/sbin
> > > + CLASS=com.sun.grid.grm.cli.SdmAdm
> > > ++ dirname /opt/sdm/bin/sdmadm
> > > + BASEDIR=/opt/sdm/bin
> > > ++ cd /opt/sdm/bin
> > > ++ pwd
> > > + BASEDIR=/opt/sdm/bin
> > > ++ dirname /opt/sdm/bin
> > > + SDM_DIST=/opt/sdm
> > > + JVM_ARGS=
> > > + for arg in '"$@"'
> > > + '[' -d = -D ']'
> > > + for arg in '"$@"'
> > > + '[' -s = -D ']'
> > > + for arg in '"$@"'
> > > + '[' sdm62u3 = -D ']'
> > > + for arg in '"$@"'
> > > + '[' suj = -D ']'
> > > + '[' /usr/java/default = '' ']'
> > > + JAVA=/usr/java/default/bin/java
> > > + '[' '!' '(' -x /usr/java/default/bin/java -a '!' -d
> > /usr/java/default/bin/java ')' ']'
> > > ++ /opt/sdm/util/arch
> > > + ARCH=lx26-amd64
> > > + export ARCH
> > > + case $ARCH in
> > > + ARCH=lx-amd64
> > > + JVM_ARGS=' -Djava.library.path=/opt/sdm/lib/lx-amd64'
> > > + JVM_ARGS=' -Djava.library.path=/opt/sdm/lib/lx-amd64 -
> > Djava.endorsed.dirs=/opt/sdm/lib/ext/endorsed'
> > > + JVM_ARGS=' -Djava.library.path=/opt/sdm/lib/lx-amd64 -
> > Djava.endorsed.dirs=/opt/sdm/lib/ext/endorsed -
> > Dcom.sun.grid.grm.management.connectionTimeout=20'
> > > + /usr/java/default/bin/java -Djava.library.path=/opt/sdm/lib/lx-amd64
> -
> > Djava.endorsed.dirs=/opt/sdm/lib/ext/endorsed -
> > Dcom.sun.grid.grm.management.connectionTimeout=20 -jar /opt/sdm/lib/sdm-
> > starter.jar com.sun.grid.grm.cli.SdmAdm -d -s sdm62u3 suj
> > > jvm         host            result message
> > > -----------------------------------------------------------------
> > > cs_vm       llgriddev.local ERROR  JVM: cs_vm died during
> > >                                    startup.
> > > com.sun.grid.grm.GrmException: JVM: cs_vm died during startup.
> > >         at
> com.sun.grid.grm.bootstrap.ParentStartupService.start(Unknown
> > Source)
> > >         at
> com.sun.grid.grm.ui.component.StartJVMCommand.execute(Unknown
> > Source)
> > >         at com.sun.grid.grm.ui.impl.CommandServiceImpl.execute(Unknown
> > Source)
> > >         at
> > com.sun.grid.grm.cli.cmd.components.StartJVMCliCommand.execute(Unknown
> > Source)
> > >         at com.sun.grid.grm.cli.AbstractCli.run(Unknown Source)
> > >         at com.sun.grid.grm.cli.SdmAdm.main(Unknown Source)
> > >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >         at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> > 39)
> > >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
> > pl.java:25)
> > >         at java.lang.reflect.Method.invoke(Method.java:597)
> > >         at
> com.sun.grid.grm.util.MainWrapper$SystemRunThread.run(Unknown
> > Source)
> > > executor_vm llgriddev.local ERROR  Timeout. Pid file:
> > >                                    /var/spool/sdm/sdm62u3/run/exe
> > >                                    cutor_vm at llgriddev.local for
> > >                                    JVM: executor_vm not found.
> > > com.sun.grid.grm.GrmException: Timeout. Pid file:
> > /var/spool/sdm/sdm62u3/run/executor_vm at llgriddev.local for JVM:
> > executor_vm not found.
> > >         at
> com.sun.grid.grm.bootstrap.ParentStartupService.start(Unknown
> > Source)
> > >         at
> com.sun.grid.grm.ui.component.StartJVMCommand.execute(Unknown
> > Source)
> > >         at com.sun.grid.grm.ui.impl.CommandServiceImpl.execute(Unknown
> > Source)
> > >         at
> > com.sun.grid.grm.cli.cmd.components.StartJVMCliCommand.execute(Unknown
> > Source)
> > >         at com.sun.grid.grm.cli.AbstractCli.run(Unknown Source)
> > >         at com.sun.grid.grm.cli.SdmAdm.main(Unknown Source)
> > >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >         at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> > 39)
> > >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
> > pl.java:25)
> > >         at java.lang.reflect.Method.invoke(Method.java:597)
> > >         at
> com.sun.grid.grm.util.MainWrapper$SystemRunThread.run(Unknown
> > Source)
> > > rp_vm       llgriddev.local ERROR  JVM: rp_vm died during
> > >                                    startup.
> > > com.sun.grid.grm.GrmException: JVM: rp_vm died during startup.
> > >         at
> com.sun.grid.grm.bootstrap.ParentStartupService.start(Unknown
> > Source)
> > >         at
> com.sun.grid.grm.ui.component.StartJVMCommand.execute(Unknown
> > Source)
> > >         at com.sun.grid.grm.ui.impl.CommandServiceImpl.execute(Unknown
> > Source)
> > >         at
> > com.sun.grid.grm.cli.cmd.components.StartJVMCliCommand.execute(Unknown
> > Source)
> > >         at com.sun.grid.grm.cli.AbstractCli.run(Unknown Source)
> > >         at com.sun.grid.grm.cli.SdmAdm.main(Unknown Source)
> > >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >         at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> > 39)
> > >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
> > pl.java:25)
> > >         at java.lang.reflect.Method.invoke(Method.java:597)
> > >         at
> com.sun.grid.grm.util.MainWrapper$SystemRunThread.run(Unknown
> > Source)
> > > Error: com.sun.grid.grm.GrmException: Command has generated error.
> > >         at
> > com.sun.grid.grm.cli.cmd.components.StartJVMCliCommand.execute(Unknown
> > Source)
> > >         at com.sun.grid.grm.cli.AbstractCli.run(Unknown Source)
> > >         at com.sun.grid.grm.cli.SdmAdm.main(Unknown Source)
> > >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >         at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> > 39)
> > >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
> > pl.java:25)
> > >         at java.lang.reflect.Method.invoke(Method.java:597)
> > >         at
> com.sun.grid.grm.util.MainWrapper$SystemRunThread.run(Unknown
> > Source)
> > > + exit 2
> > >
> > >
> > > Also, I changed the logging level but I don't get cs_vm log.
> > >
> > > Here is what I have in the logging.properties:
> > >
> > > # Default global logging level.
> > > # Loggers and Handlers may override this level
> > > .level=FINE
> > > javax.level=SEVERE
> > >
> > > com.sun.grid.grm.service.level=FINE
> > >
> > > # --- grid.grm logging level ---
> > > com.sun.grid.grm.level=FINE
> > >
> > > # --- FileHandler ---
> > > java.util.logging.FileHandler.level=ALL
> > >
> >
> java.util.logging.FileHandler.formatter=com.sun.grid.grm.util.GrmFormatter
> > > java.util.logging.FileHandler.pattern=${log_dir}/${jvm_name}-%g.log
> > > java.util.logging.FileHandler.append=true
> > > <cure here>
> > >
> > > # find /var/spool/sdm/sdm62u3/log -ls
> > > 4406418    4 drwxr-xr-x   2 sge      root         4096 Jun 30 11:24
> > /var/spool/sdm/sdm62u3/log
> > > 4406510    0 -rw-r--r--   1 root     root            0 Jun 30 11:09
> > /var/spool/sdm/sdm62u3/log/executor_vm.stderr
> > > 4406511    0 -rw-r--r--   1 root     root            0 Jun 30 11:09
> > /var/spool/sdm/sdm62u3/log/executor_vm-0.log.lck
> > > 4406516    4 -rw-r--r--   1 root     root          330 Jun 30 11:08
> > /var/spool/sdm/sdm62u3/log/executor_vm-0.log.1
> > > 4406515    0 -rw-r--r--   1 root     root            0 Jun 30 11:07
> > /var/spool/sdm/sdm62u3/log/executor_vm-0.log.1.lck
> > > 4406512   96 -rw-r--r--   1 root     root        90359 Jun 30 11:10
> > /var/spool/sdm/sdm62u3/log/executor_vm-0.log
> > > 4406505    0 -rw-r--r--   1 root     root            0 Jun 30 11:09
> > /var/spool/sdm/sdm62u3/log/executor_vm.stdout
> > >
> > > There is not much valuable information in executor_vm-0.log. It is
> > waiting for cs_vm and failed.
> > >
> > > Any further suggestions are welcome.
> > >
> > > Thanks,
> > > - Chansup
> > >
> > >
> > >
> > >> -----Original Message-----
> > >> From: Michal.Bachorik at sun.com [mailto:Michal.Bachorik at sun.com]
> > >> Sent: Tuesday, June 30, 2009 10:06 AM
> > >> To: users at gridengine.sunsource.net
> > >> Cc: Byun, Chansup
> > >> Subject: Re: [GE users] SDM 6.2U3 won't start
> > >>
> > >> also, you should increase the logging level (in
> > >> <sdp_spool>/logging.properties).
> > >>
> > >> regards,
> > >>
> > >> m.
> > >>
> > >> easymf wrote:
> > >>> chansup,
> > >>>
> > >>>
> > >>> try to start it with "sdmadm -d -s sdm62u3 suj". it will print a
> > >>> stacktrace .. and also, look at the log files in sdm spool dir.
> > >>> as all of your jvms has "died" (refused to start), it seems to be a
> > >>> problem with installation or configuration.
> > >>>
> > >>> regards,
> > >>>
> > >>> michal
> > >>>
> > >>> cbyun wrote:
> > >>>
> > >>>> Hi,
> > >>>>
> > >>>> I just installed SDM from SGE 6.2u3 release without any errors.
> > >>>> Then, I tried to start JVMs but none of them started with the
> > following
> > >> error:
> > >>>> # sdmadm -s sdm62u3 suj
> > >>>> jvm         host            result message
> > >>>> -----------------------------------------------------------------
> > >>>> cs_vm       llgriddev.local ERROR  JVM: cs_vm died during
> > >>>>                                    startup.
> > >>>> executor_vm llgriddev.local ERROR  Timeout. Pid file:
> > >>>>                                    /var/spool/sdm/sdm62u3/run/exe
> > >>>>                                    cutor_vm at llgriddev.local for
> > >>>>                                    JVM: executor_vm not found.
> > >>>> rp_vm       llgriddev.local ERROR  JVM: rp_vm died during
> > >>>>                                    startup.
> > >>>> Error: Command has generated error.
> > >>>>
> > >>>> I used Java 1.6.
> > >>>>
> > >>>> # echo $JAVA_HOME
> > >>>> /usr/java/latest
> > >>>>
> > >>>> # /usr/java/latest/bin/java -version
> > >>>> java version "1.6.0_13"
> > >>>> Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
> > >>>> Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02, mixed mode)
> > >>>>
> > >>>> Any suggestions for further troubleshooting?
> > >>>>
> > >>>> Thanks,
> > >>>> - Chansup
> > >>>>
> > >>>> ------------------------------------------------------
> > >>>>
> > >>
> >
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId
> > >> =204401
> > >>>> To unsubscribe from this discussion, e-mail: [users-
> > >> unsubscribe at gridengine.sunsource.net].
> > >>>>
> > >>> ------------------------------------------------------
> > >>>
> > >>
> >
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId
> > >> =204552
> > >>> To unsubscribe from this discussion, e-mail: [users-
> > >> unsubscribe at gridengine.sunsource.net].
> > >> ------------------------------------------------------
> > >>
> >
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId
> > >> =204605
> > >>
> > >> To unsubscribe from this discussion, e-mail: [users-
> > >> unsubscribe at gridengine.sunsource.net].
> > >
> > > ------------------------------------------------------
> > >
> >
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId
> > =204625
> > >
> > > To unsubscribe from this discussion, e-mail: [users-
> > unsubscribe at gridengine.sunsource.net].
> >
> >
> > --
> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> -
> > -
> > Richard Hierlmeier           Phone: ++49 (0)941 3075-223
> > Software Engineering         Fax:   ++49 (0)941 3075-222
> > Sun Microsystems GmbH
> > Dr.-Leo-Ritter-Str. 7      mailto: richard.hierlmeier at sun.com
> > D-93049 Regensburg           http://www.sun.com/grid
> >
> > Sitz der Gesellschaft:
> > Sun Microsystems GmbH, Sonnenallee 1, D-85551 Kirchheim-Heimstetten
> > Amtsgericht Muenchen: HRB 161028
> > Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Dr. Roland Boemer
> > Vorsitzender des Aufsichtsrates: Martin Haering
> >
> > ------------------------------------------------------
> >
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId
> > =204763
> >
> > To unsubscribe from this discussion, e-mail: [users-
> > unsubscribe at gridengine.sunsource.net].
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId
> =204803
>
> To unsubscribe from this discussion, e-mail: [users-
> unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=204807

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list