OSGi and Spring
2006-08-29
Update: JSR 232 is approved in the final ballot!
Last week I visited the BEA offices for a meeting with the Spring-OSGi group. This project started a few months ago when Interface21 (the shepherds of Spring), specifically Adrian Colyer, contacted us to see if we could make the Spring- and OSGi technology compatible. At first I had to look a bit deeper into Spring because I had only heard of the technology. I quickly found out that it is highly popular in a diverse range of Java applications. So what does Spring do?
The core responsibility of Spring is configuration. One of the key software lessons our industry learned in the last decade is that you want your software modules to be as cohesive and uncoupled as possible. The more uncoupled software is, the easier it is to reuse it. However, in the end you need some place where the modules are wired together and their options set. It is a like squeezing a balloon, you can never get rid of the coupling, you can only concentrate it in as few places as possible. This is a similar model that is promoted for OSGi applications; your bundle activator should configure a number of POJOs that finally do the real work. Spring's reason for being is based on this software model.
The Spring core interprets an XML file that describes how a set of beans must be wired together. A bean is a POJO that is configured with get and set methods. Beans are accessed using reflection but can optionally participate by implementing a number of Spring interfaces. The Spring XML looks a lot like a script language (something XML is extremely ill suited for, but that is another story, and yes, I know there is an alternative) but actually creates a graph and can therefore correctly handle dependencies and ordering.
On top of this configuration code, the Spring library has a large number of nicely decoupled libraries that can be used to decorate the beans. Do you need for JTA transaction support, but your POJO did not know anything about JTA? No problem, with Spring you can provide the required transaction calls around your methods without changing the source code. Spring uses two forms of aspect oriented programming to achieve the decoration. You can use events around the method calls defined in an interface, or you can use full blown AspectJ programming. There is also a library that provides an shell around JDBC, removing any implementation intricacies. And there is of course much, much, more. Overall pretty cool stuff.
Oracle and BEA were also very interested in providing Spring capabilities on top of the OSGi Service Platform. Hal Hildebrand (Oracle) and Andy Piper (BEA) were part of the team doing the work. On the mailing list, Richard Hall and Jeff McAffer also tuned in. With these people, you can be guaranteed of lots of interesting discussions on the Spring-OSGi mailing list; I learned a lot about features required to integrate larger enterprise applications. There is a lot of messy code out there! It is amazing how many home grown mechanisms application programmers have invented to get some kind of extensibility. Many of these mechanisms have peculiar ideas about class loading that do not always mesh well with the OSGi rather strict rules. One of the key focus of OSGi R5 will likely be addressing how this legacy code will run unmodified on OSGi service platforms.
A couple of weeks ago I decided to plummet in the deep and see what all the fuzz was all about so I wrote a simple application using the Spring-OSGi prototype. Adrian had written a very nice draft of a spec so it was not hard to get started. After a couple of hours the application was running and I seemed to be able to use many of the facilities. From an OSGi perspective, I had some qualms. I thought some of the solutions were to cumbersome and there was too much choice in certain areas. I wrote a review report from my experiences, which I had to redo because it got lost in the mail ... I think the project was going very well but I felt we needed to do a hack session together. I did not have enough Spring experience to change their code, and I think they could use some OSGi experience to improve their code. We therefore decided to meet in London. Olivier Gruber and BJ Hargrave from IBM also attended because of their
OSGi expertise and interest in Spring.
The group had become a bit too big to do any real hacking but that turned out to be ok. We spent most of the first day going through Adrian's excellent specification text. We were incredibly productive that day, we killed so many features! With lots of intense discussions we could significantly simplify the model. It turned out that the OSGi service registry and the Spring application context could really work together very well. The tricky part is of course the dynamics, the OSGi specifications seem to remain unique in their focus on dynamicity. Interestingly, the end result looks like a specification that can take over or merge with our Declarative Services in the long run because it is now a super-set. Another item to discuss for OSGi R5.
Overall, this is a very interesting project that has a lot of industry attention: two very cool technologies that are being married. I hope that the group can turn the prototype into the mainstream Spring code. Adrian told me they were thinking of Spring 2.1. As a bonus, Spring 2.1 will deliver all its libraries as bundles, that is, all libraries will have the correct OSGi headers.
I am convinced that the OSGi Enterprise workshop will discuss this project, for example, should OSGi standardize the OSGi layer of Spring?
Peter Kriens
P.S. You did RSVP did you? Realize that we will have limited seating based on the people that have RSVP'ed. Hope to see you there!
Last week I visited the BEA offices for a meeting with the Spring-OSGi group. This project started a few months ago when Interface21 (the shepherds of Spring), specifically Adrian Colyer, contacted us to see if we could make the Spring- and OSGi technology compatible. At first I had to look a bit deeper into Spring because I had only heard of the technology. I quickly found out that it is highly popular in a diverse range of Java applications. So what does Spring do?
The core responsibility of Spring is configuration. One of the key software lessons our industry learned in the last decade is that you want your software modules to be as cohesive and uncoupled as possible. The more uncoupled software is, the easier it is to reuse it. However, in the end you need some place where the modules are wired together and their options set. It is a like squeezing a balloon, you can never get rid of the coupling, you can only concentrate it in as few places as possible. This is a similar model that is promoted for OSGi applications; your bundle activator should configure a number of POJOs that finally do the real work. Spring's reason for being is based on this software model.
The Spring core interprets an XML file that describes how a set of beans must be wired together. A bean is a POJO that is configured with get and set methods. Beans are accessed using reflection but can optionally participate by implementing a number of Spring interfaces. The Spring XML looks a lot like a script language (something XML is extremely ill suited for, but that is another story, and yes, I know there is an alternative) but actually creates a graph and can therefore correctly handle dependencies and ordering.
On top of this configuration code, the Spring library has a large number of nicely decoupled libraries that can be used to decorate the beans. Do you need for JTA transaction support, but your POJO did not know anything about JTA? No problem, with Spring you can provide the required transaction calls around your methods without changing the source code. Spring uses two forms of aspect oriented programming to achieve the decoration. You can use events around the method calls defined in an interface, or you can use full blown AspectJ programming. There is also a library that provides an shell around JDBC, removing any implementation intricacies. And there is of course much, much, more. Overall pretty cool stuff.
Oracle and BEA were also very interested in providing Spring capabilities on top of the OSGi Service Platform. Hal Hildebrand (Oracle) and Andy Piper (BEA) were part of the team doing the work. On the mailing list, Richard Hall and Jeff McAffer also tuned in. With these people, you can be guaranteed of lots of interesting discussions on the Spring-OSGi mailing list; I learned a lot about features required to integrate larger enterprise applications. There is a lot of messy code out there! It is amazing how many home grown mechanisms application programmers have invented to get some kind of extensibility. Many of these mechanisms have peculiar ideas about class loading that do not always mesh well with the OSGi rather strict rules. One of the key focus of OSGi R5 will likely be addressing how this legacy code will run unmodified on OSGi service platforms.
A couple of weeks ago I decided to plummet in the deep and see what all the fuzz was all about so I wrote a simple application using the Spring-OSGi prototype. Adrian had written a very nice draft of a spec so it was not hard to get started. After a couple of hours the application was running and I seemed to be able to use many of the facilities. From an OSGi perspective, I had some qualms. I thought some of the solutions were to cumbersome and there was too much choice in certain areas. I wrote a review report from my experiences, which I had to redo because it got lost in the mail ... I think the project was going very well but I felt we needed to do a hack session together. I did not have enough Spring experience to change their code, and I think they could use some OSGi experience to improve their code. We therefore decided to meet in London. Olivier Gruber and BJ Hargrave from IBM also attended because of their
OSGi expertise and interest in Spring.
The group had become a bit too big to do any real hacking but that turned out to be ok. We spent most of the first day going through Adrian's excellent specification text. We were incredibly productive that day, we killed so many features! With lots of intense discussions we could significantly simplify the model. It turned out that the OSGi service registry and the Spring application context could really work together very well. The tricky part is of course the dynamics, the OSGi specifications seem to remain unique in their focus on dynamicity. Interestingly, the end result looks like a specification that can take over or merge with our Declarative Services in the long run because it is now a super-set. Another item to discuss for OSGi R5.
Overall, this is a very interesting project that has a lot of industry attention: two very cool technologies that are being married. I hope that the group can turn the prototype into the mainstream Spring code. Adrian told me they were thinking of Spring 2.1. As a bonus, Spring 2.1 will deliver all its libraries as bundles, that is, all libraries will have the correct OSGi headers.
I am convinced that the OSGi Enterprise workshop will discuss this project, for example, should OSGi standardize the OSGi layer of Spring?
Peter Kriens
P.S. You did RSVP did you? Realize that we will have limited seating based on the people that have RSVP'ed. Hope to see you there!
Add a comment
OSGi Enterprise Workshop
2006-08-16
It is rare that a technology developed for embedded devices ends up at the core of enterprise applications. The constraints and optimization axes of these two worlds are, ehhh, worlds apart. Still, it looks like the OSGi Alliance is succeeding in this remarkable feat as demonstrated by the increasing interest of enterprise software vendors in OSGi technology. Initially, some vendors voiced concerns about potential patents held by the founding members, but the by now famous patent pledge made those worries go away.
Enterprise software is not a well defined area. We know it is usually big, requires lots of people to work on it, and it is invariably expensive. Web servers are of course a big aspect of Enterprise software as well as tight integration with databases. Java made big inroads in this world because it provided a uniform environment for applications. Its type safety reduced the number of errors during the integration phases and this made it smoother to integrate different applications.
J2EE was SUN’s attempt to capture the communality of Enterprise applications in a framework. The idea behind a framework is that it reduces the application specific code and the resulting application more robust. It is a wonderful appealing idea and object oriented theory gave birth to too many frameworks in the nineties. I even build one for Ericsson in 94! Frameworks are incredibly appealing but they are so hard to get right. They key problem is the natural focus on the advantages of the framework and total disregard for the increased complexity and learning curve that a framework will cause. For example, if we add a function for authentication, we needlessly increase the complexity for people that do not need authentication at all. Just the increase in the API and manual is negative. The battle field of software is therefore littered with failed frameworks (alas, including mine). Only frameworks that can restrict themselves to core functionality and fanatical focus on decoupling stand a chance to succeed.
I do not think SUN and compatriots heeded those lessons from the nineties when they started with J2EE. From the start, J2EE became a monster in deployment complexity and intrusiveness. 5 years ago I bought a book about J2EE and spent a week developing applications, just to understand what the hype was. Well, I utterly disliked the intrusiveness of the code and the many, seemingly unnecessary, restrictions. This dislike happened ever before I found out about the XML situps that one needs to perform before you get your “Hello World” running. Look at the funny comparison between web application development environments from JPL to demonstrate my dislike. I understand that the EJB 3 JSR is providing more focus on simplicity, but I will hold my judgment until I have played with it.
The success of Spring is caused by the intrusiveness of J2EE into applications and its inclusion of many technologies that are not always necessary, or in other words, the classic Framework syndrome. Developers love frameworks but they are not very faithful; this is where the POJOs are coming from. POJOs are successful because the coupling between the units of design are minimized, but more important, they are coupled in the right direction. Let the framework call you, not you calling the framework. The result is that the developers can focus on their domain area, not on learning Yet Another Framework.
The OSGi Framework is successful because it was designed to decouple, which provides many benefits during development and deployment. The OSGi Framework could not care less about what your components do and how they do it. It only provides the primitives to deploy your components and to let them collaborate in a dynamic way. Collaboration is done through service interfaces, which makes the components easy to test without complicated harnesses. Components can be build on top of POJOs, very little code has to be coupled to the OSGi Framework. Standardizing this component layer will create universal middleware. Middleware developed in different industries that can be deployed on any platform. In the end, application development will be mixing and matching universal middleware.
The OSGi development model is so attractive to the Enterprise world because it is sometimes amazing how often I see the wheels being reinvented. Often just because the existing wheels were connected to axes that the developer had no use for. Though the OSGi Framework is quite mature for the embedded, vehicle, home automation, and mobile markets, we need to work on the issues that are crucial for the enterprise markets. What functionality is missing in R4 that is important for enterprise applications?
Next month I will lead an OSGi workshop in the bay area where we will discuss these issues with a number of experts. This is an open workshop but seats are limited. If your company is into making Enterprise applications, please attend (register ASAP with rranck@inventures.com, we have limited number of seats). The signs are that OSGi technologies will play a major role in server side applications over the coming years. If you can not attend the workshop, do not hesitate to add comments to this blog or just email me.
Peter Kriens
Enterprise software is not a well defined area. We know it is usually big, requires lots of people to work on it, and it is invariably expensive. Web servers are of course a big aspect of Enterprise software as well as tight integration with databases. Java made big inroads in this world because it provided a uniform environment for applications. Its type safety reduced the number of errors during the integration phases and this made it smoother to integrate different applications.
J2EE was SUN’s attempt to capture the communality of Enterprise applications in a framework. The idea behind a framework is that it reduces the application specific code and the resulting application more robust. It is a wonderful appealing idea and object oriented theory gave birth to too many frameworks in the nineties. I even build one for Ericsson in 94! Frameworks are incredibly appealing but they are so hard to get right. They key problem is the natural focus on the advantages of the framework and total disregard for the increased complexity and learning curve that a framework will cause. For example, if we add a function for authentication, we needlessly increase the complexity for people that do not need authentication at all. Just the increase in the API and manual is negative. The battle field of software is therefore littered with failed frameworks (alas, including mine). Only frameworks that can restrict themselves to core functionality and fanatical focus on decoupling stand a chance to succeed.
I do not think SUN and compatriots heeded those lessons from the nineties when they started with J2EE. From the start, J2EE became a monster in deployment complexity and intrusiveness. 5 years ago I bought a book about J2EE and spent a week developing applications, just to understand what the hype was. Well, I utterly disliked the intrusiveness of the code and the many, seemingly unnecessary, restrictions. This dislike happened ever before I found out about the XML situps that one needs to perform before you get your “Hello World” running. Look at the funny comparison between web application development environments from JPL to demonstrate my dislike. I understand that the EJB 3 JSR is providing more focus on simplicity, but I will hold my judgment until I have played with it.
The success of Spring is caused by the intrusiveness of J2EE into applications and its inclusion of many technologies that are not always necessary, or in other words, the classic Framework syndrome. Developers love frameworks but they are not very faithful; this is where the POJOs are coming from. POJOs are successful because the coupling between the units of design are minimized, but more important, they are coupled in the right direction. Let the framework call you, not you calling the framework. The result is that the developers can focus on their domain area, not on learning Yet Another Framework.
The OSGi Framework is successful because it was designed to decouple, which provides many benefits during development and deployment. The OSGi Framework could not care less about what your components do and how they do it. It only provides the primitives to deploy your components and to let them collaborate in a dynamic way. Collaboration is done through service interfaces, which makes the components easy to test without complicated harnesses. Components can be build on top of POJOs, very little code has to be coupled to the OSGi Framework. Standardizing this component layer will create universal middleware. Middleware developed in different industries that can be deployed on any platform. In the end, application development will be mixing and matching universal middleware.
The OSGi development model is so attractive to the Enterprise world because it is sometimes amazing how often I see the wheels being reinvented. Often just because the existing wheels were connected to axes that the developer had no use for. Though the OSGi Framework is quite mature for the embedded, vehicle, home automation, and mobile markets, we need to work on the issues that are crucial for the enterprise markets. What functionality is missing in R4 that is important for enterprise applications?
Next month I will lead an OSGi workshop in the bay area where we will discuss these issues with a number of experts. This is an open workshop but seats are limited. If your company is into making Enterprise applications, please attend (register ASAP with rranck@inventures.com, we have limited number of seats). The signs are that OSGi technologies will play a major role in server side applications over the coming years. If you can not attend the workshop, do not hesitate to add comments to this blog or just email me.
Peter Kriens
Why is Software So Brittle?
2006-08-03
The past few days I acted as a software user instead of software programmer: I needed a wiki for an OSGi project. How hard can that be? After a bit of Googling I found Ward Cunningham’s site (he is the Wiki inventor) and he is hosting a wiki about wikis (This text will have a weird number of w words). Well, that site contained a very long list of wikis! I guess a wiki is on the sweet spot of being relative easy to make but with a big visual impact.
How should one choose from so many wikis? Silly enough, the implementation language seemed to be the most logical selection criteria. As you might have guessed, Java is one of my favorite languages and was therefore an obvious choice. Eagerly checking the list for OSGi based implementations turned out, however, to be a disappointment. Many wikis point out how extensible they are but none had chosen the OSGi service platform to implement their plugins; they all developed plugins in a proprietary way. There is a lot of evangelizing left to do.
Anyway, after looking at some Java implementations I found a wiki that looked very promising: FitNesse. This is not a simple wiki, it also contains an extensive testing framework. However, the wiki looked really good (not a common feature for a lot of wikis) and the code looked interesting. When I inspected the code, my hands itched … if I had been a wise men, I would have slapped them and continued. The code looked so clean and so easy to bundlefy that I could not resist. Turning it into a bundle was indeed trivial; the makers of FitNesse provided a very clean configuration and start/stop interface. I only had a slight problem with resource loading. They used the system class loader for resource loading instead of their own class loading, can’t see why, but if that was all I could fix it. Still optimistic at that moment in time.
We have an OSGi framework running that hosts the Bundle Repository, so it was attractive to install this wiki bundle on the same machine, sharing the same VM for these applications is a serious memory saver. The FitNesse bundle ran its own web server (Oh, why don’t they use OSGi technology?). This required me to map their web server through our standard web server so that we could handle authentication in one place. The standard OBR web server runs behind a firewall but is proxied through our normal web server. After trying this same trick for the wiki, it turned out that the FitNesse wiki code assumed it had the whole web server on its own; it could not handle a prefix path. I was getting this sinking feeling that I might have taken the wrong path. As a solution, I could of course run the web server directly out in the open. Unfortunately, this would require me to write the authentication code. The sinking feeling changed into hitting rock bottom. Maybe it was time to turn around and see if there were easier ways of getting this done. If I only had been a wise man …
I talked to BJ Hargrave and he recommended the MediaWiki, the wiki that powers the WikiPedia. This wiki is PHP based and run by hundreds of sites. How hard can it be to install this? Well, hard. It turned out we were running PHP 4 on our server and MediaWiki runs on PHP 5. Sigh. Well, we have rpm installed on our Linux system so we just download PHP 5 and we are on our way. Well, that is the theory. The sinking feeling returned. PHP 5 required a long list of additional uploads. Carefully trying to install a few dependent packages quickly showed that the dependency tree closely resembled the big bang. BJ had tole me there was also a PHP 4 version of the MediaWiki so I felt myself floating to the surface again, only to sink at record speed again after installing this version. Yes, it requires version 4, which happened to be a different version 4 than our version 4. The dependency fan out was not as enormous as the PHP 5, but still impressive enough for me to give up.
Simple and small became my main criteria now. So I finally found a simple and very small PmWiki based on PHP that actually had the functionality I needed, and nothing more. It was a breeze to install and did not have any nasty dependencies.
So what did I learn? Well, besides not trying to be clever, I learned that we are an industry with a problem. The last couple of years we have spent a tremendous amount of energy in formalizing the dependencies of our software artifacts; assuming that this would solve the problems of unreliable software. However, it looks like we have created another problem which is the dependency fan out. Unfortunately, virtually all known dependency mechanisms are based on versioned artifacts. Artifacts are usually a collection of functions that are not always needed. Often there is a core of highly coupled packages and a periphery of utilities and mappers to other artifacts. For example, many tool vendors today provide a mapper to run their tools under ant or maven; implicitly coupling their artifact to maven and ant.
We need to make the use of software less painful. Though we can extend our current way of working with big-bang releases (integrating many different products in a single build), in the end we need to find a way that minimizes dependencies between artifacts. The OSGi service registry with its service oriented architecture is clearly in the right direction. However, we need more. Tools that can visualize the dependencies and that can help us reduce the dependencies. Yes, with today’s tools we can actually manage the dependencies, but that does not imply that dependencies have become a positive thing. I think the OSGi is on the right way, but a lot of work is left to do.
How should one choose from so many wikis? Silly enough, the implementation language seemed to be the most logical selection criteria. As you might have guessed, Java is one of my favorite languages and was therefore an obvious choice. Eagerly checking the list for OSGi based implementations turned out, however, to be a disappointment. Many wikis point out how extensible they are but none had chosen the OSGi service platform to implement their plugins; they all developed plugins in a proprietary way. There is a lot of evangelizing left to do.
Anyway, after looking at some Java implementations I found a wiki that looked very promising: FitNesse. This is not a simple wiki, it also contains an extensive testing framework. However, the wiki looked really good (not a common feature for a lot of wikis) and the code looked interesting. When I inspected the code, my hands itched … if I had been a wise men, I would have slapped them and continued. The code looked so clean and so easy to bundlefy that I could not resist. Turning it into a bundle was indeed trivial; the makers of FitNesse provided a very clean configuration and start/stop interface. I only had a slight problem with resource loading. They used the system class loader for resource loading instead of their own class loading, can’t see why, but if that was all I could fix it. Still optimistic at that moment in time.
We have an OSGi framework running that hosts the Bundle Repository, so it was attractive to install this wiki bundle on the same machine, sharing the same VM for these applications is a serious memory saver. The FitNesse bundle ran its own web server (Oh, why don’t they use OSGi technology?). This required me to map their web server through our standard web server so that we could handle authentication in one place. The standard OBR web server runs behind a firewall but is proxied through our normal web server. After trying this same trick for the wiki, it turned out that the FitNesse wiki code assumed it had the whole web server on its own; it could not handle a prefix path. I was getting this sinking feeling that I might have taken the wrong path. As a solution, I could of course run the web server directly out in the open. Unfortunately, this would require me to write the authentication code. The sinking feeling changed into hitting rock bottom. Maybe it was time to turn around and see if there were easier ways of getting this done. If I only had been a wise man …
I talked to BJ Hargrave and he recommended the MediaWiki, the wiki that powers the WikiPedia. This wiki is PHP based and run by hundreds of sites. How hard can it be to install this? Well, hard. It turned out we were running PHP 4 on our server and MediaWiki runs on PHP 5. Sigh. Well, we have rpm installed on our Linux system so we just download PHP 5 and we are on our way. Well, that is the theory. The sinking feeling returned. PHP 5 required a long list of additional uploads. Carefully trying to install a few dependent packages quickly showed that the dependency tree closely resembled the big bang. BJ had tole me there was also a PHP 4 version of the MediaWiki so I felt myself floating to the surface again, only to sink at record speed again after installing this version. Yes, it requires version 4, which happened to be a different version 4 than our version 4. The dependency fan out was not as enormous as the PHP 5, but still impressive enough for me to give up.
Simple and small became my main criteria now. So I finally found a simple and very small PmWiki based on PHP that actually had the functionality I needed, and nothing more. It was a breeze to install and did not have any nasty dependencies.
So what did I learn? Well, besides not trying to be clever, I learned that we are an industry with a problem. The last couple of years we have spent a tremendous amount of energy in formalizing the dependencies of our software artifacts; assuming that this would solve the problems of unreliable software. However, it looks like we have created another problem which is the dependency fan out. Unfortunately, virtually all known dependency mechanisms are based on versioned artifacts. Artifacts are usually a collection of functions that are not always needed. Often there is a core of highly coupled packages and a periphery of utilities and mappers to other artifacts. For example, many tool vendors today provide a mapper to run their tools under ant or maven; implicitly coupling their artifact to maven and ant.
We need to make the use of software less painful. Though we can extend our current way of working with big-bang releases (integrating many different products in a single build), in the end we need to find a way that minimizes dependencies between artifacts. The OSGi service registry with its service oriented architecture is clearly in the right direction. However, we need more. Tools that can visualize the dependencies and that can help us reduce the dependencies. Yes, with today’s tools we can actually manage the dependencies, but that does not imply that dependencies have become a positive thing. I think the OSGi is on the right way, but a lot of work is left to do.
Comments
| Previous Posts | Archives |
|---|---|