HomeContactBlogMember Area

See Cloud for more details.

Report Cloud Workshop Thursday, March 25, 2010

10:00am - 4:00pm Location:Camino Real, Hyatt Hotel EclipseCon/OSGi DevCon, Santa Clara, CA

Report

The OSGi Alliance held a joint workshop March 25, 2010 in the Santa Clara Hyatt. The goal of the workshop was to find out what role OSGi could play in the cloud. Obviously OSGi is very well suited for remote deployment, a characteristic that seems extremely applicable for cloud applications. The day started with an introduction, several companies like Paremus, Rackspace, SAP, VMWare, and TIBCO gave a short presentation or overview. We then split up into  discussion groups until lunch to discuss the topics that are relevant on the boundary of clouds and OSGi. After lunch we reconvened and created a shared list of topics and merged as many as possible to get to a number of these topics. These topics were then divided and discussed and smaller groups to find the requirements. After the tea break we discussed the topics and how to make progress. Along the way we lost a few people to the ongoing conference but the list of requirements was still quite long. There was even sufficient energy in the room to a subgroup to work  further on the discovered requirements. The workshop leader (Peter Kriens) will create an OSGi mailing list for  this workgroup where this report will be discussed. It seems very likely that there will be enough member companies available to start a Cloud Expert Group.

Discovered Topics

The workgroups provided a number of topics. These topics were grouped in the following items:

  • Provisioning/Configuration - A primary aspect of the cloud is the remoteness of the servers. OSGi is already very well suited for remotely provisioning applications to servers and configuring them. A new aspect that clouds bring to this aspect is the sheer magnitude of servers that must be provisioned combined with the size of today's applications. In such an environment it is likely that instances need to be provisioned on the fly depending on the dynamic load. The provisioning must handle new instances of machines as well as new processes.
  • OSGi Cloud View Management - Just like provisioning, the key aspect that needs attention is the sheer size of the cloud. Thousands of instances require a different management model than 5-10 servers. A novel aspect is also the changing number of instances. Special software will be required to make the instances more autonomous and goal driven so that the dependency on central management can be minimized.
  • Environment. Awareness/Service Level Agreement/Resource Mgmt - A key aspect of cloud based computing is the autonomy of the instances in the cloud. This aspect makes it necessary for the software to be aware of the environment so local decisions can be made.
  • Efficiency / Data Affinity - Starting and stopping instances is relatively easy in the cloud. However, the location of the instance can influence the efficiency of the computation. For example, data located in the same location will be faster to access than data in another datacenter. Cloud middleware will require knowledge of these affinities.
  • Post Mortem/Debugging - Murphy requires that failures are debuggable. This will require access to post-mortem dumps when a process failed. It will also be necessary to debug a series of systems, both alone and in concert.
  • Discovery - The autonomy of the systems combined with the size of these systems will require models of automatic discovery of instances, process, and resources.
  • Migration/Forking - Failures and increased loading of the system will require the migration of parts of the application to other instances/processes as well as forking processes to other instances.
  • Isolation/Application Model - A model to describe an "application"

Requirements

  • Better way to do filter selection - LDAP filters not powerful enough. This requirement needs use cases because it is not clear what is meant.
  • Resource usage properties/capability requirements - The systems must become reflective in every aspect.
  • Provide information about the env., system events -
  • Services need to be able to specify their qualities, need to be dynamic -
  • Possibility to record changes while the fw runs (install/update/etc) - Allows replay for debugging and post mortem analysis'
  • Allow save and restore the system bundle and all bundles to instantiate in another frameworks - Make it easy to migrate/fork
  • Abstracting resources to be migrated
  • Snapshot of the framework + bundles - A snapshot so the system can be debugged and migrated/forked.
  • A broker service that knows about latency, cost, location, etc. - A complex large scale system will require making many trade offs, requiring a broker abstraction
  • Application must allow running an app in separate frameworks (isolation). -
  • Application must be able to put requirements on the environment (CPU, etc) in a declarative way -
  • Legacy OSGi app support, must be possible to run apps in and out of the cloud -
  • Support for native cloud apps to local.
  • Modularity of JVM languages
  • Possible to make queries of the cloud “services”, notifications
  • Discovering of running instances of the same app
  • Query about the location (database in the eu use case) - The eu use case is the problem that the EU puts strict privacy requirements on the use of personal data, the location of the server is included.

Moderator's Interpretation

After discussing OSGi in the Cloud for a full day and many offline discussions I think the OSGi provisioning model is an excellent base for the cloud. A base, because there are number of unique aspects to the cloud that are currently not addressed in a standardized way. The unique aspects of the cloud are the sheer number of instances involved as well as the need to move functionality around based on load and other dynamic aspects of a large system. The large number of instances requires that they are much easier to manage than the way systems are managed today. It is clear that such systems cannot be managed by shells, however powerful. It is likely that systems must become more autonomous and goal driven. Autonomous systems will require extensive up-to-date information about the components that make up their operating environment to make correct decisions. Autonomous systems will also require more predictability. The OSGi model based on dynamic resolving might be too simple for quickly firing up instances. Instead a model where a framework is configured, its state frozen, and then recreated might be more predictable than model where each instances resolves on its own with the change to resolve slightly different because of timing and state differences.

A cloud based system changes the picture of how we view applications. Applications are no longer monoliths that can be replicated, instead a picture emerges allows functional parts to grow and shrink independently based on the dynamic load and failures. The OSGi service based model is an excellent programming model for this because it allows bundles (and their dependencies) to be moved to other systems where a service is used to abstract the location of the executing bundle. Especially important here is asynchronous messaging because it seems clear that asynchronous messaging is required for any serious requirements.

Obviously, the capability of being able to move functions around the cloud will require the up-to-date knowledge of the instances in the cloud that are being active. Though central management through JMX could be made to work it seems that there is a need for a more efficient, dynamic and peer-to-peer model. Products like ZooKeeper provide a very interesting model that allows a large number of instances to reliably collaborate under the duress of changing loads and failures.

Even though the existing OSGi specifications are an excellent base there is still work left with the existing provisioning model. Several aspects are under work in the Enterprise Expert Group like the Application Model, Asynchronous messaging, and OBR.

For the Application model it will be instructive to take cloud requirements into account. What is an application in the cloud? From my experience one of the key aspects in the cloud is to have the capability to scale aspects of the application while other aspects will not require scaling. For example a complex function might require lots of instances while the basic web front could end up to only require a few instances. It will be interesting to see how the traditional application view changes taking these scaling requirements into account.

Asynchronous messaging is one of the few patterns in software that is relatively easy to scale if the processing of the messages does not require persistent state. JMS provides a model of messaging that is already popular. What is missing is a standardized management model that allows the dynamic reconfiguration of the topology. One of the disadvantages of asynchronous messaging is the more awkward way to interact with messages in comparison to normal API calls. Asynchronous messaging is actively investigated in the EEG.

OBR (bundle repository) seems to be positioned to take an important role in any cloud based system. Provisioning without such a model will be very hard and will require too much user interaction. OBR is very well suited to provision (and de-provision) bundles from frameworks.

The cloud is a computing model that shares many aspects with the more traditional data centers and middleware that will be developed for the cloud is likely to be very applicable in large enterprise systems; there is no dichotomy here. Actually it might be better to rename any effort in this area to "large scale", "scalable", "elastic" computing to make it clear that most if not all of the functions necessary for the cloud are just as applicable for data centers of enterprises.

Next Step

It seems clear that there are many opportunities to create specifications for cloud based computing. An RFP is prepared by Peter Kriens that is circulated among the people that signed up for the work. This RFP will be the basis for the decision to form a new EG, make it part of the EEG or execute no further action. Member companies that want to participate should contact .

Click here to download RFP133.


Home | Site Map | Trademark Policy | Privacy Policy

Copyright © 2013 OSGi™ Alliance. Comments about the site?  Send them to: OSGi Alliance WebMaster.