[GE users] Workload management and virtualization

Ignacio Martin Llorente llorente at dacya.ucm.es
Mon Nov 10 20:38:45 GMT 2008


Hi,



On 10/11/2008, at 20:31, rayson wrote:

> On 11/10/08, Ignacio Martin Llorente <llorente at dacya.ucm.es> wrote:
>> Of course, you could modify SGE to be a VM distributed manager, but
>> my position is that you have to modify many internals because the
>> efficient management of VMs is different from efficient management of
>> jobs. In other words, You can not have a technology being able to
>> efficiently managed both jobs and VMs.
>
> Starting and stopping VMs should not be a lot more difficult than
> starting and stopping normal jobs. However, execd and shepherd will
> not be able to monitor processes if they don't run in the same VM
> started for the job, yet running in the same VM would create new
> problems... this for sure will create some challenge. But another
> question is, do we really need to monitor each process of the job
> running in the VM, or we can take the shortcut and just monitor the VM
> as a whole.


>
>
> So I think we may need to think more about job accounting in the VM
> world. With a normal job, getrusage() is sufficient (and PDC is a
> bonus), but in the virtualization world, we will need to call special
> APIs to get the resource usage of the whole VM started for the job.

The answer to this question depends on the integration level between  
SGE and VMs. If you aim is to use SGE to manage VMs, it is assumed  
that you do not have access to the VMs, the VM SGE will only manage  
(control/monitor) VMs as black boxes-
>
>
>
>> We use libvirt for our driver to access KVM, and we also provide a  
>> new
>> libvirt interface on top of OpenNebula (http://trac.opennebula.org/wiki/LibvirtOpenNebula
>> ), yes, on top of a distributed infrastructure.
>
> Interesting that it uses libvirt... however, I am not familiar with
> OpenNebula, can you add it to:
>
> http://wiki.gridengine.info/wiki/index.php/Virtualization_and_Grid_Engine

Sure
>
>
>
>> BTW, we would like to know if somebody is interested in VirtualBox  
>> for
>> the virtualization of computing clusters, because we are evaluating
>> the development of drivers for this virtualization platform.
>
> While VirtualBox is very popular in the desktop world, HPC requires
> different requirements and VirtualBox still needs to address those.
> For example, VirtualBox cannot create an SMP VM.

OK, I see,

Thanks
>
>
> Rayson
>
>
>
>>
>> Regards
>>
>>>
>>>
>>> Daire
>>>
>>> ------------------------------------------------------
>>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=88353
>>>
>>> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net
>>> ].
>>
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=88410
>>
>> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net 
>> ].
>>
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=88425
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net 
> ].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=88428

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list