Other OSGi blogs |
The OSGi Extender Model
2007-02-23
Installing and upgrading software is a complicated process that is very error prone in today’s operating system. In Windows, there is a whole industry for installers that take (some of) the pain out of installing a windows application. In Linux RPM package management is famous.
These installers are necessary because applications in operating systems touch many other subsystems. For example, to install an address book application in Linux, it is necessary to have at least the following things done:
An RPM script for the GNU Indent program looks like:
Notice how many hard coded details are specified in this script. Scripts like these are highly sensitive to small changes in the deployment environment.
The model is that the application developer/deployer has to know it all. There is a good reason why installation scripts so often fail: coupling. Installation scripts require deep knowledge of the structure and setup of the target system. The installation script writer is one of the few persons that has to understand all aspects of the application to configure and initialize the environment. Uninstallation is even harder because it is not easy to know what files are created by the applications that should be removed. It is the unfortunate truth that after a year or two, my Windows operating system needs to be reformatted to get rid of the applications that are no longer removable because their installation script fails, a simple missing file can already cause such a script/installer to fail.
Installation scripts are so hard, that one day a friend of mine promised to eat the installation floppy (yes this was a long time ago) if it failed one more time. Alas, we served the floppy with a fresh salad to make it easier to digest.
Now once there was an operating system that did not require installation scripts at all: The Macintosh. When I moved form the Mac to the PC I was utterly confused with installers. On a Mac, installation means dragging a file to any folder on the Mac. That’s it. All OS related installation is performed automatically when an executable is copied, renamed, or deleted. The reason that this can magically (for PC users) work is because a Mac executable contained a simple database with installation information (the so called resources). When the executable is copied, the MacOS is called with a handle to the executable. Associations, icons, help, and many more resources are stored in this flexible database. Moving the file to another location? No problem, this is handled by the OS. Uninstalling? Simple, just delete the file.
With hindsight, it was obvious that my first week on the PC would be a disaster. After installing a number of applications, I decided to rearrange the folder structure, something I often did on the Mac without thinking about. Yes I got this pesky warning but what is a registry??? … A reformat was the only solution in the end.
Let us inspect the differences between the Mac and PC/Unix a bit deeper. An example of information that needs to be installed on both the Mac and the PC are the file associations. When a text file is double clicked, an editor needs to be opened, when an html file is clicked, we want to see the browser. In the PC this association is made by the file extension of the file. For example: .html, .txt, .doc. The association extension-to-application is made in the Windows Registry (Yes, I am now thoroughly familiar with it). On the Mac, the association is stored in the desktop database and copied from the resources of the executable.
It is interesting to compare the two strategies to set this information:
The Mac model has many advantages and few drawbacks. The biggest drawback is maybe psychological, the programmers feel less in control because so little installation work is left and delegated to other subsystems (job security!). Subsystems they do not have direct control over. However, this lack of control is maybe one of its greatest assets in my opinion. Over the years, the Macintosh was able to get more and more functionality that was immediately applicable to old applications. For example, the desktop database could be totally restructured because normal applications had no knowledge whatsoever about this database. Restructuring the Windows registry … eh, well that would be a major undertaking because any useful application on Windows has intricate knowledge of large parts of the registry.
Nowadays, many Mac applications also have installers because they have become dependent on many other components that are installed on the system. E.g. many applications use MySQL, a web server, and other components. In today’s operating systems, MySQL needs to be configured and initialized for a new application that uses it.
Looking at the advantages of the declarative model that was used in the Mac for file associations, would it not be nice if the other subsystems could work similarly?
For example, if the MySQL component could see that a new application was installed, it could look in the application resources and see if there was installation information for this MySQL component. If so, it could use this information to setup the tables and permissions. If the file was standardized (lets say standard SQL), it might even be picked up by another database like Progress, DB2 or Oracle, without the application being aware of this. Updates and installation could work in similar vein if the subsystem (in this case the database subsystem) could get notifications of updates and uninstallations.
An interesting aspect of this extendeer model is the robustness. Quite often, applications are uninstalled because they have become unreliable; their state is messed up. Relying on scripts that run in the context of this messed up state is not good, often creating a larger mess by not finishing and leaving expensive resources lingering on the system. If a messed up application gets uninstalled, the chance that the subsystems can clean up for the application is much higher because they are more likely to be in a healthy state.
If life could only be so good … But it can! This is exactly that the OSGi Alliance enables and promotes. Meet the Synchronous Bundle Listener.
The Life Cycle layer in the OSGi Framework allows bundles to listen to installations, updates and uninstallations of other bundles. A normal Bundle Listener will handle those events asynchronous. That is, when you get the event, the cause has moved on and has not waited for you to look at it. The Synchronous Bundle Listener works differently, it will handle the same event, you guessed it, synchronously. With the Synchronous Bundle Listener you get a chance to do your thing before the cause of the event has passed. This allows the subsystem to perform its initialization before the application gets control.
In the Synchronous Bundle Listener event call back, the subsystem can look at the resources of the bundle and check if there is anything of its liking. If it is, it can use this resource to perform the necessary initialization for the new bundles.
This is all rather abstract so let us work out a simple example.
If you write a servlet for an OSGi service platform than you must get the Http Service and register your servlet with it. This is boilerplate code, not very interesting. An extender could simplify this model by removing all the cruft for the servlet programmer. Whenever a bundle gets started, the servlet extender would inspect a header in the bundle about to be started. If this header was set, it would load the servlet class from the bundle and register it with the Http Service.
This model is easily programmed, you can find an example on the snippets site. This snippet discusses some of the implementation issues around extenders.
No model is perfect and obviously the extender model is no exception. There are a number of issues that may impact the suitability.
Extenders do not have access to the private area of a bundle. This area is reserved for the bundle itself and should not be touched by other bundles. Any party that could access this area makes it impossible for a bundle to reason about its private data area. For example, locking is not necessary in the private area to prevent access from other parties.
If a bundle needs its private data area initialized, it can do this lazily in the start method. If it is necessary to initialize this data after install, the extender method is not suitable.
Extenders run with their own permissions. An extender should therefore be careful not to perform operations on behalf of the bundle that the bundle would have no authority for. For example, a malicious bundle should not be able to wreak havoc because the extenders did things on its behalf without checking.
Then again, there is an advantage that an extender could do more privileged operations than the bundle itself is allowed. For example, an extender could create a database for a bundle without the bundle having permission to create it (obviously it requires permission to use it).
Extenders could scope the operations they have by using Permission Admin to get the permissions of the bundle and using a doPrivileged call with those permissions on the stack. For example, an extender could get the permissions of the signer and scope all actions with these permissions.
This issue needs to be analyzed in detail for each type of extender. In most cases a solution can be found but in certain cases extenders are not suitable.
Synchronous Bundle Listeners are, well, synchronous. They are called in the middle of an installation process. If they do not return, the installation cannot be finished. Good frameworks will handle hanging synchronous bundle listeners, but it is not good when it happens.
Extenders are likely to be system components and should therefore have extra quality requirements. For this reason, a synchronous bundle listener must have AdminPermission[bundle,LISTENER]. The model is therefore less suitable when the initialization is done by untrusted components.
The life cycle events of the OSGi Framework enable a much more friendly installation model in comparison to traditional scripts and installers. By delegating application setup and initialization to the subsystem applications will require less, highly volatile, knowledge of the deployment platform. Further, subsystems have more possibilities to clean up and adapt to changes in the applications.
The single most important drawback is that the model is less familiar to Windows and Unix (but not Mac) programmers. In these environments, the lack of life cycle events and heavy reliance on the file system structure makes the described model impossible. This lack of familiarity by the programmer makes the OSGi model look like there is no control, a feeling many programmers distinctly dislike.
However, the OSGi is a component model where it is impossible to know all the life cycle events. By delegating the installation task to the different subsystems, the overall system becomes significantly more robust, reliable, future proof, and simpler to maintain.
Peter Kriens
These installers are necessary because applications in operating systems touch many other subsystems. For example, to install an address book application in Linux, it is necessary to have at least the following things done:
- Testing if the dependencies are fulfilled
- Setup an SQL database and tables in MySQL
- Place the executables and files in an appropriate place
- Link the documentation and servlets to an Apache web server
- Link to the commands from the application from the appropriate bin directory
- Setup the permissions
- Etc.
An RPM script for the GNU Indent program looks like:
Name: indent
Version: 2.2.6
Release: 1
Source0: %{name}-%{version}.tar.gz
License: GPL
Group: Development/Tools
%description
The GNU indent program reformats C code to any of a
variety of formatting standards, or you can define your
own.
%setup
-q
%build
./configure
make install
%defattr(-,root,root)
/usr/local/bin/indent
%doc /usr/local/info/indent.info
%doc
%attr(0444,root,root) /usr/local/man/man1/indent.1
%doc COPYING AUTHORS README NEWS
Notice how many hard coded details are specified in this script. Scripts like these are highly sensitive to small changes in the deployment environment.
The model is that the application developer/deployer has to know it all. There is a good reason why installation scripts so often fail: coupling. Installation scripts require deep knowledge of the structure and setup of the target system. The installation script writer is one of the few persons that has to understand all aspects of the application to configure and initialize the environment. Uninstallation is even harder because it is not easy to know what files are created by the applications that should be removed. It is the unfortunate truth that after a year or two, my Windows operating system needs to be reformatted to get rid of the applications that are no longer removable because their installation script fails, a simple missing file can already cause such a script/installer to fail.
Installation scripts are so hard, that one day a friend of mine promised to eat the installation floppy (yes this was a long time ago) if it failed one more time. Alas, we served the floppy with a fresh salad to make it easier to digest.
Now once there was an operating system that did not require installation scripts at all: The Macintosh. When I moved form the Mac to the PC I was utterly confused with installers. On a Mac, installation means dragging a file to any folder on the Mac. That’s it. All OS related installation is performed automatically when an executable is copied, renamed, or deleted. The reason that this can magically (for PC users) work is because a Mac executable contained a simple database with installation information (the so called resources). When the executable is copied, the MacOS is called with a handle to the executable. Associations, icons, help, and many more resources are stored in this flexible database. Moving the file to another location? No problem, this is handled by the OS. Uninstalling? Simple, just delete the file.
With hindsight, it was obvious that my first week on the PC would be a disaster. After installing a number of applications, I decided to rearrange the folder structure, something I often did on the Mac without thinking about. Yes I got this pesky warning but what is a registry??? … A reformat was the only solution in the end.
Let us inspect the differences between the Mac and PC/Unix a bit deeper. An example of information that needs to be installed on both the Mac and the PC are the file associations. When a text file is double clicked, an editor needs to be opened, when an html file is clicked, we want to see the browser. In the PC this association is made by the file extension of the file. For example: .html, .txt, .doc. The association extension-to-application is made in the Windows Registry (Yes, I am now thoroughly familiar with it). On the Mac, the association is stored in the desktop database and copied from the resources of the executable.
It is interesting to compare the two strategies to set this information:
The Mac model has many advantages and few drawbacks. The biggest drawback is maybe psychological, the programmers feel less in control because so little installation work is left and delegated to other subsystems (job security!). Subsystems they do not have direct control over. However, this lack of control is maybe one of its greatest assets in my opinion. Over the years, the Macintosh was able to get more and more functionality that was immediately applicable to old applications. For example, the desktop database could be totally restructured because normal applications had no knowledge whatsoever about this database. Restructuring the Windows registry … eh, well that would be a major undertaking because any useful application on Windows has intricate knowledge of large parts of the registry.
Nowadays, many Mac applications also have installers because they have become dependent on many other components that are installed on the system. E.g. many applications use MySQL, a web server, and other components. In today’s operating systems, MySQL needs to be configured and initialized for a new application that uses it.
Looking at the advantages of the declarative model that was used in the Mac for file associations, would it not be nice if the other subsystems could work similarly?
For example, if the MySQL component could see that a new application was installed, it could look in the application resources and see if there was installation information for this MySQL component. If so, it could use this information to setup the tables and permissions. If the file was standardized (lets say standard SQL), it might even be picked up by another database like Progress, DB2 or Oracle, without the application being aware of this. Updates and installation could work in similar vein if the subsystem (in this case the database subsystem) could get notifications of updates and uninstallations.
An interesting aspect of this extendeer model is the robustness. Quite often, applications are uninstalled because they have become unreliable; their state is messed up. Relying on scripts that run in the context of this messed up state is not good, often creating a larger mess by not finishing and leaving expensive resources lingering on the system. If a messed up application gets uninstalled, the chance that the subsystems can clean up for the application is much higher because they are more likely to be in a healthy state.
If life could only be so good … But it can! This is exactly that the OSGi Alliance enables and promotes. Meet the Synchronous Bundle Listener.
Synchronous Bundle Listener
The Life Cycle layer in the OSGi Framework allows bundles to listen to installations, updates and uninstallations of other bundles. A normal Bundle Listener will handle those events asynchronous. That is, when you get the event, the cause has moved on and has not waited for you to look at it. The Synchronous Bundle Listener works differently, it will handle the same event, you guessed it, synchronously. With the Synchronous Bundle Listener you get a chance to do your thing before the cause of the event has passed. This allows the subsystem to perform its initialization before the application gets control.
In the Synchronous Bundle Listener event call back, the subsystem can look at the resources of the bundle and check if there is anything of its liking. If it is, it can use this resource to perform the necessary initialization for the new bundles.
This is all rather abstract so let us work out a simple example.
A Servlet Helper
If you write a servlet for an OSGi service platform than you must get the Http Service and register your servlet with it. This is boilerplate code, not very interesting. An extender could simplify this model by removing all the cruft for the servlet programmer. Whenever a bundle gets started, the servlet extender would inspect a header in the bundle about to be started. If this header was set, it would load the servlet class from the bundle and register it with the Http Service.
This model is easily programmed, you can find an example on the snippets site. This snippet discusses some of the implementation issues around extenders.
Issues
No model is perfect and obviously the extender model is no exception. There are a number of issues that may impact the suitability.
No Access to private area of bundle
Extenders do not have access to the private area of a bundle. This area is reserved for the bundle itself and should not be touched by other bundles. Any party that could access this area makes it impossible for a bundle to reason about its private data area. For example, locking is not necessary in the private area to prevent access from other parties.
If a bundle needs its private data area initialized, it can do this lazily in the start method. If it is necessary to initialize this data after install, the extender method is not suitable.
Permissions
Extenders run with their own permissions. An extender should therefore be careful not to perform operations on behalf of the bundle that the bundle would have no authority for. For example, a malicious bundle should not be able to wreak havoc because the extenders did things on its behalf without checking.
Then again, there is an advantage that an extender could do more privileged operations than the bundle itself is allowed. For example, an extender could create a database for a bundle without the bundle having permission to create it (obviously it requires permission to use it).
Extenders could scope the operations they have by using Permission Admin to get the permissions of the bundle and using a doPrivileged call with those permissions on the stack. For example, an extender could get the permissions of the signer and scope all actions with these permissions.
This issue needs to be analyzed in detail for each type of extender. In most cases a solution can be found but in certain cases extenders are not suitable.
Hanging Bundle Listeners
Synchronous Bundle Listeners are, well, synchronous. They are called in the middle of an installation process. If they do not return, the installation cannot be finished. Good frameworks will handle hanging synchronous bundle listeners, but it is not good when it happens.
Extenders are likely to be system components and should therefore have extra quality requirements. For this reason, a synchronous bundle listener must have AdminPermission[bundle,LISTENER]. The model is therefore less suitable when the initialization is done by untrusted components.
Conclusions
The life cycle events of the OSGi Framework enable a much more friendly installation model in comparison to traditional scripts and installers. By delegating application setup and initialization to the subsystem applications will require less, highly volatile, knowledge of the deployment platform. Further, subsystems have more possibilities to clean up and adapt to changes in the applications.
The single most important drawback is that the model is less familiar to Windows and Unix (but not Mac) programmers. In these environments, the lack of life cycle events and heavy reliance on the file system structure makes the described model impossible. This lack of familiarity by the programmer makes the OSGi model look like there is no control, a feeling many programmers distinctly dislike.
However, the OSGi is a component model where it is impossible to know all the life cycle events. By delegating the installation task to the different subsystems, the overall system becomes significantly more robust, reliable, future proof, and simpler to maintain.
Peter Kriens
I have a look at the sample code given for this extender pattern. There you have looked for the ServletMap manifest header in the bundles which are either in STARTING or ACTIVE bundle. Can we use the RESOLVED state for this? If not can you please tell me the reason?
Thanks in Advance.
Sameera
So from an OSGI perspective, never use a bundle unless it is ACTIVE. Unfortunately, this is not done in Eclipse for historical reasons.
I hit a design a issue, when using the ACTIVE state. Here is my scenario. Bundle A depends on the OSGi service x. Service x can be exposed by several bundles in the system during the initial startup. For bundle A to work properly, it needs to obtain all the x services in the system and then only bundle A can perform its normal operation.
If bundle A starts after all the bundles which register the x service, then it is fine. But now we are depending on the ordering of bundles. This is also not good.
We also can use ServiceTracker to track x services. But how do we know exactly that we obtained all the x services without depending on the start levels?
Your comments on this issue are greatly appreciated.
It seems to me that you have not well defined your dependencies but just want to get bundle start ordering in disguise?
What is that x does that you depend on? Why could bundle A not work with only one x?
You can always wait for "n" services x, but that is usually very brittle, because quite often it is nice to run the system with a bundle disabled, or enabled or add another bundle that provides new functionality.
I need much more information about your application to really give an advise because now it sounds you just want bundle start ordering, which will never work in OSGi because of the dynamism. You must find the real dependencies and model those, that usually works very nicely.
Kind regards,
Peter Kriens