XtremWeb-HEP 8

XtremWeb-HEP Gets Support for Virtual Appliance

The new release of XtremWeb-HEP 8 is a breakthrough in the world of Desktop Grid Computing.

For the first time, DG users are able to register their own Virtual Images for their preferred OS and applications and remotely deploy them on Volunteer PC.

Thanks to the embedded IBIS Smartsockets technology, users can directly and securely connect (through ssh) to the remote VM even if the PC is located behind firewall.

The convergence of Desktop Grid Computing and Cloud Computing is there.

On January 20th, 2012, the middeware is deployed at LAL for our final tests. The middleware will be released when these tests are successfully passed.

A new web site has been set up : Flying-Grid.org.

The documentation has been updated.

 Jun 7th, 2013 : XWHEP 8.2.2

• Corrections
  1. When inserting a new session, we must force owneruid if not set
  2. When trying to insert a new object not subject to be inserted by the client (task, host), we must throw an exception and return an error to the client
  3. Some code has been rewritten in a more object oriented way. IdRpc and XWTag now have embedded memthods to create (repectively) XMLRPCCommands and TableInterfaces. This permits to avoid the usage of switch(idrpc) and switch(xwtag)
  4. A bug corrected on client side, in the macro parser
• New features
  1. n/a

 April 16th, 2013 : XWHEP 8.2.1

• Corrections
  1. character encoding forced to UTF-8
  2. an inter-thread synchonization bug corrected in server
• New features
  1. n/a

 Feb 26th 2013 : XWHEP 8.2.0

• Corrections
  1. LiveCD are now configured as follow
      the login is now "vmuser" in place of "xwuser"
      the script to create new LiveCD is in /usr/local/sbin/ (ubuntu_crealtelivecd.sh and/or sl5_createlivecd.sh)
      the user "vmuser" is in /etc/sudoers to be able to execute /sbin/poweroff and /usr/local/sbin/ (ubuntu_crealtelivecd.sh and/or sl5_createlivecd.sh)
      dirsetup is not allowed any more: it was a security hole
      context.sh is executed at the end of the boot process
  2. Executor : a bug corrected regarding cleaning processes
  3. Web Interface v0.2 :
      help migrated into the "XtremWeb-HEP User Guide"
      introducing Web workers
  4. Virtualization contextualization nows follow the HEPiX Virtualization Working Group recommandations (http://grid.desy.de/vm/hepix/vwg/doc/html/index.shtml)
  5. bugs corrected in shell scripts
  6. code cleaned for a better usage of the Jetty library
  7. in order to integrate jopenid, several libraries upgraded
      jetty from 6.1.7 to 8.1.8
      log4j from 1.2.9 to 1.2.17
      slf4j from 1.5.3 to 1.7.2
      servlet api from 2.5 to 3.0
• New features
  1. introducing the WALLCLOCKTIME mechanism
  2. introducing RESULTREQUEST status
  3. introducing FORCENEWUID in the worker configuration file to ease deployment over clouds
  4. we can now retrieve jobs per status
  5. introducing openid usage to ease platform usage

 June 25th 2012 : XWHEP 8.1.2

 Corrections
1. on server side, the scheduling improved : hosts.incomingconnection is now taken into account to ensure a server like job is not sent to a volunteer resource where incoming connections are not allowed
2. on server side, the scheduling improved : restoring a feature from XtremWeb 1.8.0 by INRIA : "expected host" so that the end user can specify at submission time the expected volunteer resource to run the job. Note, that if the expected host does not request a job or does not match the requirements (OS, CPU...), the job is never computed.
3. the DG to SG bridge is now compatible with shared applications so that we can use VirtualBox on EGI resources
4. some bugs corrected on client side regarding application registration and task management
5. a bug corrected on temporary file usage: they are now cleaned at software exit
6. the REST interface has been completely rewritten. The <xwhep></xwhep> root element MUST be used. Examples:
    to send a new application https://server/sendapp/?XWPARAM=<xwhep version="8.0.2-flyinggrid"><app name="pouet" /></xwhep>
    to retrieved registered applications https://server/getapps
7. a bug corrected in HTTP data download
8. SmartSocket usage corrected; inter node communications fully functional now; thank to the support of Jason Maassen from the Netherlands eScience Center in Amsterdam

 New features

1. the creation of new Live CD can be customized by providing several optional files:
   • a text file ’user.packages’ may be provided to install custom packages; this file must contain a space separated packages list

   a text file ’user.hostname’ may be provided to customize the LiveCD hostname; this file must contain the host name only

   • any package files are installed on the fly (*.rpm or *.deb, depending on the LiveCD OS)
2. the client can now be used to create SmartSockets end point, independently of any task. This may be useful to create SmartSockets tunnel from a running VM to the client PC (e.g. mount the client FS inside the running VM)
3. a new Mac OS X package "xwhep.vworker" to deploy the middleware inside a VM
4. the server contains an embedded Web service interface available through HTTPS

 May 3rd, 2012 : XWHEP 8.0.2

 Corrections
1. on server side, the scheduling improved : hosts.incomingconnection is now taken into account to ensure a server like job is not sent to a volunteer resource where incoming connections are not allowed
2. on server side, the scheduling improved : restoring a feature from XtremWeb 1.8.0 by INRIA : "expected host" so that the end user can specify at submission time the expected volunteer resource to run the job. Note, that if the expected host does not request a job or does not match the requirements (OS, CPU...), the job is never computed.
3. the DG to SG bridge is now compatible with shared applications so that we can use VirtualBox on EGI resources
4. some bugs corrected on client side regarding application registration and task management
5. a bug corrected on temporary file usage: they are now cleaned at software exit
6. the REST interface has been completely rewritten. The end user MUST have a valid X509 certificate. The <xwhep></xwhep> root element MUST be used.
   e.g.:
   • https://server/sendapp/?XWPARAM=<xwhep version="8.0.2-head"><app uid="2853ae1e-3711-4412-b77f-acea8d9b54ee"  name="pouet" /></xwhep>: to send a new application
   • https://server/getapps : to retrieved registered applications
   • https://server/getusers: to retrieved registered users
   • etc.
   • https://server/get/objectUID : to retrieved a specific registered object


 New features

1. the creation of new Live CD can be customized. This is done by providing several optional files:
   • a text file ’user.packages’ may be provided to install custom packages this file must contain a space separated packages list
   • a text file ’user.hostname’ may be provided to customize the LiveCD hostname this file must contain the host name only
   • any package files are installed on the fly (*.rpm or *.deb, depending on the LiveCD OS)
2. the client can now be used to create SmartSockets end point, independently of any task. This may be useful to create SmartSockets tunnel from a running VM to the client PC (e.g. mount the client FS inside the running VM)
3. a new Mac OS X package "xwhep.vworker" to deploy the middleware inside a VM

 Feb 20th, 2012 : XWHEP 8.0.1

 Corrections
1. a bug corrected in Socket Time out management: reconnection must be automatic
2. the client now compressed any data, if necessary
3. worker must not use the "connected" mode introduced in 7.4.0
4. worker and dispatcher must not use any device at all : setProperty("java.awt.headless", "true") (See http://java.sun.com/developer/technicalArticles/J2SE/Desktop/headless/)
5. a bug corrected on URI usage: ampersand are now accepted if written as "&s;"
6. on client side, bugs corrected on XML file usage (whith "—xwxml")
7. on client side, a bug corrected when retrieving user group


 New features

1. worker definition has new attributes:
   • shared applications : comma separated list of application names shared as volunteer sharing
   • shared library : comma separated list of library names shared as volunteer sharing
   • shared data : comma separated list of data names shared as volunteer sharing
   • incomingconnections : volunteer resoure may accept communication tunneling. The middleware does not modify the resource firewall in any manner
2. usage of IBIS Smartsockets to interconnect jobs running on worker side (to each other and to/from client)
3. a new CommandLineOption : KEEPZIP (for debug purposes)
4. work definition has new attributes:
   • readydate : when the work has been downloaded by the worker
   • datareadydate : when all data of the work has been downloaded by the worker
   • compstartdate : when the worker has started the job
   • compenddate : when the worker has ended the job
5. tasks have their own UID now (it used to be their related work one) this aims to keep track of all event over the platform we can now retrieve all run informations (all start dates and host informations per task)
6. a work can be modified by its owner or the administrator this is needed by SpeQulOS (EDGI JRA2)
7. we can force the cache content path for any entry this is needed by Bitdew/XWHEP interconnection (still under development)