|
|||||||
| Blog | / | OSGi Alliance Blog: 200804 | |||||
|
KISS Principle2008-04-28
Wikipedia defines this principle as:
The KISS principle (acronym for "Keep It Simple, Stupid") states that design simplicity should be a key goal and unnecessary complexity avoided. It serves as a useful principle in a wide array of disciplines, such as software development, animation, photography, engineering, and strategic planning. Today Google Alerts dumped me into two discussions where OSGi seems to be on the wrong side of the divide regarding the KISS principle. The first is an extensive debate in the dev@tomcat.apache.org mailing list, the second is the JSR 277 mailing list. The Tomcat developers have been asked to osgify their popular web server. This resulted in a discussion that was very interesting to read. Several people seem to have a disdain for the OSGi zealots (which I guess is me) but there seems to be a general uneasiness with this request. However, my key trigger for this blog was that in this discussion I noticed that several people regard OSGi as overly complex and heavy. The JSR 277 discussion raised the same problem from another direction. From the JSR 291 interoperation discussion: "I wanted something in Java that was simpler and cleaner that OSGi-users could leverage if they wanted.". Is OSGi already ossified? The key problem to decide this question is, as so often the case, requirements. Is Java complex? Hard to answer, if you want to write a "Hello World" it is humongous overkill. If you are writing a large scale web application it is nice to have a solid foundation. The key phrase in the definition of KISS is unnecessary complexity. Complexity is in the eye of the beholder. First, the OSGi API is very small by any measure. There is only one mandatory package that only has 26 class files, including the security and exception classes. And you can already leverage OSGi without writing Java code. Can you call it intrusive when you can turn a JAR into a bundle by adding one or two manifest lines? All the complexity people complain about is optional. That said, I do agree that even optional things create some kind of conceptual complexity. So maybe there is a lot of cruft in the options? Let us take a feature like import packages which is often considered hard. Most dependency mechanisms like Maven, Ivy, and also the original JSR 277 are based on the Require-Bundle concept. I.e. you declare a dependency on the wrapper. Wrapper based dependency models are simple to understand for us humans. It would work actually very well in practice if JARs would be highly cohesive and not change their constituent packages over time. Which raises the question: Do you want the woman, or the dress? We developers have discovered the wonders of refactoring and things do change. OSGi is a module system, it is about being able to change software while not breaking the deployments. Though import packages might be harder to understand initially than Require-Bundle, Import-Package does create more resilient systems because it minimizes coupling. That is why people that have used both systems usually decide to standardize on Import-Package (and bnd makes this quite easy). Similar to the uses: directive. In the Tomcat discussion it was stated that the uses directives made the manifest unreadable. It was much nicer to create a manifest by hand, it looked better. I do agree from an aesthetic point of view but who reads a manifest? The manifest is intended to be read by the OSGi Framework. The uses directive is an incredible important concept to maintain classpace consistency when the same class can be loaded multiple times through different class loaders. If you have lived in a world where this model is supported, you understand that uses have a very high value because they minimize runtime problems. I do not say that OSGi has no unnecessary complexity at all. Obviously, the specification has been evolving over almost ten years. We have to tried to keep it is simple as possible, but we are as fallible as all of us. However, requirements have always come first in OSGi and fulfilling requirements just forbid you to make things too simple because they force you to understand the problem. Obviously the key question is, are many of these requirements highly specialized cases not useful for the majority. Or are they just solving problems that awaits anybody that ventures into the wonderful land of modularity? Peter Kriens CommentsFirst book on OSGi at JAX 2008!2008-04-25
Last evening I got back from a very pleasant trip to JAX 2008 in Wiesbaden. I was asked to give the best practice and the OSGi modularity presentations. Rod Johnson had already warmed the audience for OSGi with the result that we had to close the room when people where sitting, crawling, and standing everywhere even though there were still 40-50 people standing outside. Good news for OSGi, but I had to give the presentation for a second time that evening at 21.15, where I was surprised that there were still 50-60 people present that late in the evening. The next day was even better because I got a much bigger room that was also filled. Feels good after gotten used to preaching to audiences of 10 people for so many years.
I also got handed a copy of the first official OSGi book in the world. The book, written by Gerd Wütherich, Matthias Lübken, Nils Hartmann, and Bernd Kolb.This is a thorough book on OSGi (though Equinox biased) that I recommend to anyone who is using OSGi or wants to know more about it. You can buy it at d.punkt. I hope they translate it in English soon. I already read most of the text during my Christmas vacation to help them with a review. Gerd had already shown me on the first day the thank you section. They had been really nice, and made the thank you in Dutch! Great work! It is great to finally being able to point to a good OSGi book. JAX is a great conference, very well organized. Well, there is so much to do that I'll keep it short. Though there is one observation, unrelated to OSGi, I can't help myself making. On the way back with Ryan Air I observed that people can be incredibly rude at jumping the queue, and not just the one that look like it, even elegantly looking older people showed an incredible rudeness. Though I admire Ryan Air for many things, their queue management is non existent, seeming to call out the worst behavior in people. I figured people behaved so badly so that to get the best seats in the plane. However, when I boarded at the end of the queue the emergency seats in the middle were still mostly free. So why are people so much in a hurry that they are jumping the queue? One day I have to debug this, but now back to the build! Peter Kriens P.S. Talking about books, you can now also order the specification (core + compendium) in printed form for a special price. CommentsHellblazer : breskeby :
Hi Peter, I have to thank your for your sessions. It was really inspiring. Also the session about classloading and visibility by martin lippert with your first level support ;-). I had the chance to read some lines of the book at the conference and it seems to be a good completion to the osgi reference documentation.
cheers, René Gröschke Ps.: Dealing with a defect air conditioning and overcrowded trains, our homeyard journey wasn't much better than yours ;-) Is XML Part of the Problem?2008-04-19 Last week I had a great time in Stockholm working two days at ComActivity AB. They are reworking their applications so they can take advantage of OSGi in their server applications. In this process they are investigating lots of technologies. And as is so often the case, one runs in modularity problems while using legacy libraries. Ok, they are not all legacy libraries, they are often brand new and provide valuable functionality, and so often for free as well. What a world!However, integrating libraries into an OSGi based system is often fraud with class loader problems. In this case we took an in-depth look at the Cinnamon library from www.spicefactory.org. This library is a Remote Procedure Call (RPC) library for Flex based applications using the AMF protocol. AMF is an Adobe (publicized) standard for serializing ActionScript objects from Flash. At the start of the visit I was shown a very nice demo of the application. Fast, powerful, and looking good despite the austere theme they were using. As can be expected, the application took advantage of the OSGi architecture. A top level application used the service registry to discover modules that implemented a specific application functionality. This module provided the Flex flash SWF file and provided the receiving end of the RPC interface. It was kind of cool to see how they made new bundles with bnd, deployed them with File Install and during the whole morning never restarted their Tomcat that was hosting their OSGi WAR file. During the demo we went through the design by looking at the source code, modifying things, and then testing it. One thing I noticed was the large amount of configuration and specialized classes to make this application work with Cinnamon. I once write a tiny webrpc application and did not think it had to be that hard. (I am an optimist). One of the key issues was configuration. The Cinnamon library had two types of configuration: Spring based and a private XML configuration format. The key problem is that these text based configuration files are forced to use dynamic class loading. If these classes must come from other bundles, then OSGi (rightfully) barks. The whole purpose of the OSGi modularity is to have controlled class loading, taking care of versions and other constraints. Doing Class.forName blissfully ignores such mundane details. A large part of the custom code was necessary to find the right classes at the right time. Obviously, this was working. However, cruft frustrates me and I convinced them we should try to get rid of it. So at the last two hours of the second day we created a new project and started from scratch. No Spring XML and no Cinnamon XML. It was already at the end of the second day so we had little time but it turned out to be surprisingly simple to use the Cinnamon library from Java only, taking advantage of the facilities of the service registry (and that without even reading the manual or javadoc). Still, by getting rid of the XML configuration things became significantly easier. All of the class loading problems disappeared as by magic because from the Java code we had direct access to the right types without requiring dynamic loading. By single-stepping through the Cinnamon library I also noticed how much code was spent on making the library flexible and adaptable. As so many libraries today, it was doing impressive class loader gymnastics to implement yet another plugin system. I think the library could be significantly simplified if it had been using OSGi to provide this plugin mechanism with bundles and services. It is sad to see how many time the same wheels are invented over and over again. Anyway, I also remembered how I had a similar experience with Hibernate last year. In an effort to use Hibernate from an OSGi based system, I switched from the XML configuration to do the configuration in Java code. Also in that case, a lot of things that caused class loading issues just disappeared because classes could be manipulated as classes (with the proper versioning) and not as strings. Different, but still related, the day before we had another experience where we significantly simplified life. They had used Spring to define global configuration for database connections. This is obviously highly specific information for each of their customers. With Spring they could provide this configuration through a bundle. However, if they wanted to change the configuration, they had to update this bundle. Looking closer at the configuration file it was clear that they were specifying several connections to databases. This looked like a clear case for the Configuration Admin's Managed Service Factory. We modified their pool manager to use the Configuration Admin by registering a Managed Service Factory service. This was only a few lines of code. They were already using File Install, and File Install makes it easy to set new configurations by placing a property file in the watched directory. It was kind of sad to see how small and readable the property file was in comparison with the XML configuration it replaced. Now do not get me wrong. I am very impressed with the Spring, Cinnamon, and Hibernate libraries and I do see the value of centralizing configuration files. Maybe OSGi changes the rules a bit and this XML based model works better in monolithic applications. Lets face it, what is the difference of changing XML or Java code in Eclipse? However, last weeks experience makes me wonder if we should not be a little bit more critical before using XML to solve problems that are much cleaner and better solved in plain old Java? Peter Kriens CommentsHellblazer :
This is why Spring has annotation driven configuration. Google Guice uses it exclusively, eschewing XML completely. It's really a toss up, personally.
Chris Brind :
My colleagues and I have been saying this for more than 5 years. There seemed to be a point where people were rushing to drop properties files in favour of XML and this just added complexity to something that should be very simple. As a result, even though we went through the same hype cycle, we soon abandoned XML files where properties files were already adequate.
waterflower :
yes, annotation great, but it's cannot replace the position of configuration files.
So back to .ini again? or osgi.ini style extension? Indeed, since configuration files is a must thing, so how do we configure? We need a need format and must a oo style configuration. I do an ObjectNotation at on.dev.java.net. And now, I make it support map directly. And it's so simple as configuration should, nature as the Nature be.:) At future, seems ON is real. JSR 294 Superpackages No More2008-04-08 Alex Buckley posted a mail to the JSR 294 list with was more or less superpackages meeting kryptonite. Though in this case, Alex is in my opinion the good guy! He basically abandons all the unnecessary complexity of the earlier proposal, see my blog about this subject, and comes up with a guiding principle: ''simpler is better''.The proposal definitely follows that principle. A source file is marked with the word ''module'', just like a package. And that is basically it.
The compiler encodes the module membership in the class file and the VM can enforce that any member attached with the module keyword is not accessible from another member unless it is in the same module. I like it! This can actually be quite useful, also for some of the work we are currently doing in the OSGi Alliance concerning modularization. More detail is needed, but this significant simplification is absolutely in the right direction. There are some rough edges obviously, and I like to weigh my input on them. Module membership should not be on the source but on the package because it is ''inherently'' safer. Alex argues in his blog that from a technical point of view the best place for module membership declaration is the package (package-info.java), but that from a ''moral'' point of view the class file is better because it allows one to introspect a class file and have all information together. Moral in this case should be seen as the Java philosophy, which he defines as focus on understandability by not letting the compiler infer information, but requiring that the programmer states his intentions clearly, even if this is redundant. I.e.
Is for me harder to read than:
Because I can trust the compiler to do the right thing and I trust it more with these things than my reading capabilities. I guess this was also attempted in the JLS, which is also highly redundant. Personally I think this decreases readability because you tend to glaze over sections you already have read 5 times in the previous page. However, this makes it too easy to miss small, highly important differences. Anyway, I guess consistency is important (though I strongly believe Java would be more readable with ''less'' verbosity, less is more!) but I still very much prefer inherent safe models instead of tool enforced safe models. Why would you introduce a data structure that can be wrong if you can also do it in a way that does not allow those errors? Let me explain this a bit more. Both Alex and I seem to agree wholeheartedly that packages are first class citizens and that you can not have modules criss-cross through packages. I.e. each package member must have the same module membership. That is, you can not have two classes in the same package that would not belong to the same module. So module membership is therefore not on the type but on its package. Placing the module membership in the source of the type just allows you to create invalid configurations when two sources in the same package would have a different module membership, which will just require more tools to fix them. Putting the membership on the package would also be a good reason to overhaul the reflection methods of Package. Packages have always been second class citizens in the Java VM, for example you can not find its members. By placing the module membership on the package we have a perfectly good reason to fix these deficiencies. Additionally, I do have some more wishes for the EDR2. I'll go through them shortly.
M1: Import-Module: M2
From an OSGi perspective I think modules as proposed by Alex will be a welcome addition to the language. I think it addresses some of our uses cases we are working on. Nice work! Peter Kriens CommentsHoward Lovatt : Behrang Saeedzadeh :
Hi Peter,
I have a question regarding JSR 294: 1- Are there any benefits in extending Java's syntax to support modules, etc. syntactically instead of relying on APIs, etc? Last time I remember, extending Java's syntax was not a very successful operation: Generics have a lot of ugly edge cases. Even if introduced without any ugly side effects and edge cases, are there any benefits in supporting modules syntactically? Peter :
I do think that a module keyword is a good idea because it does provide value. Adding a keyword is possible as long as it is not made a keyword that can appear anywhere in source form. It is quite easy to only interpret the module keyword in places where only keywords are used, thereby not classing with existing code.
Problems with generics go deeper but they were mostly caused (I was told) because they added wildcards late in the game. However, this should be a warning for JSR 277 (which has 294 folded in) to be extremely careful in the design. Kind regards, Peter Kriens
|
||||||
First, let's be honest: only trivial bundles require "one or two manifest lines." Real-world scenarios demand a tool like bnd to dig through our code and figure out the complexities of required dependencies, exported packages, etc., and most of my manifests wind up over fifty lines long.
Bnd is an excellent tool, but it still requires integration into the build process. In our team environment, at least, that's much, much easier said than done, and is complicated a hundredfold by the vast and disparate wasteland of Java build tools that were never designed to handle corner cases. (For starters, try integrating projects using bnd with Eclipse RCP applications.) It's hardly the fault of OSGi or bnd, but it's still something that haunts me daily.
One of the primary reasons for this difficulty is the reliance on the Java manifest. Simply put: the file is used for too many things. Code signing, OSGI configuration and a plethora of extensions all crowd together to form what we would call very low cohesion if we were describing a class. I've wished from the beginning that OSGi used a separate configuration file.
The ramifications for this are surprisingly widespread. For instance, I once tried to create an annotation library that could be used to automatically generate OSGi configuration. Unfortunately, even though I could create a correct manifest (thanks in no small part to your bnd library), every single build system required a separate and distinct step to get that manifest incorporated into the final archive. Throw in even one more unexpected step - code signing, for instance - and the whole thing fell apart. I have since abandoned the idea.
The same goes for extension information. Since I find Declarative Services to be one of the most useful facets of OSGi, I wrote (and regularly use) an annotation library that can automatically generate Declarative Services configuration files. With the Java 6 annotation processing API, they can even be generated automatically without any intervention from the developer. But the spec still requires a Service-Component line in the manifest, and the only way to do that turns out to be manually. Refactor one class and - kaboom! - the whole thing breaks. I've been thinking of creating my own version for a while now that is identical to the Declarative Services specification but, rather than relying on the Service-Component attribute of the manifest, simply looks up every XML file in, say, a META-INF/SCR directory.
It seems to me that OSGi was built by brilliant individuals that understood the best practices of software engineering quite well. Nevertheless, the configuration process neglects so many of those same practices. There is low cohesion, with too many unrelated things thrown into the manifest. There are tight coupling and duplication, with class names repeated in un-refactorable (any chance that is a word?) text files. Etc.
Forgive me if I've gone on a bit of a tangent or hijacked your blog - and feel free to edit or remove this post if you so desire. My message to you is this: OSGi is really, really close. Simplify configuration with the same best practices that you have already obviously mastered and R5 will be fantastic.
The problems you signal is a build problem. Though we can do some things (we are talking now about allowing a Service-Component header with wildcards, and maybe we should have a usable default), build systems are clearly out of scope for OSGi right now.
Interestingly, I like having things in the manifest instead of the plethora of XML files one find in other systems. The manifest describes the contents, the fact that a bundle can contain so many different content type does not make this uncohesive imho because it is all metadata.
You can instruct Eclipse to refactor class names in text files as well. Anyway, any configuration with a text file has a similar problem.
I have personally started working on a build system because I recognize many of your issues. However, after already spending quite a bit of time I started to realize that even when I would build the best possible build system in the world, it would be unlikely I could ever get paid for it because it is hard to compete with free products like maven and ant.
Anyway, if you have complexity, it is bound to show up somewhere. The point of my blog was to talk about unnecessary complexity. Do you think all your problems would disappear if we had had a separate configuration file? In that case, I'd love to see an example of this.
Peter Kriens
At the moment, I'm using an in-house Maven plugin to create my manifest. It simply adds Require-Bundle entries for nearest (traverses dependency tree) bundle dependencies. The nearest non-bundle dependencies are embedded and added to the Bundle-ClassPath.
----------
e.g.
Bundle-ClassPath: .,
dependency/xml-apis/xml-apis/1.0.b2/xml-apis-1.0.b2.jar,
dependency/xom/xom/1.1/xom-1.1.jar
----------
The embedded jars are then programmatically added as resources to the MavenProject instance. Export-Package is configured with regex include/exclude filters.
Anyway, I'd really like to incorporate Bnd in my plugin so I can eliminate the use of Require-Bundle. During that transition, I'd also like to get rid of as much manual configuration as possible. I plan to use annotations and apt wherever possible to dynamically configure Bnd. I know you've expressed some reservations about annotation-based manifest configuration in the past [1].
It would really help me if you could give some guidance:
Is something like @NonPublicAPI acceptable for filtering out packages/classes? The annotation would be @Documented so non-OSGi consumers could understand the restriction.
Which parts of configuration could/should be annotation driven?
Can you direct me to any existing projects which might help me?
Best Regards,
Luke
[1] - http://www.aqute.biz/Blog/2007-05-21
You're absolutely right: so many of the problems stem from build systems. I'll also agree with you that there isn't enough money in build systems to buy half a sandwich. That said, if you have some ideas for an improved Java build system, I would eagerly contribute in any way possible on a volunteer basis. It may not pay well, yet I am still a young man and plan to be using Java for quite some time; a better build system is number one on my wish list.
Getting back on topic, there have been two times that stand out when I've wished that OSGi used a different configuration file than the manifest. The first was when I started using OSGi and was trying the most basic tutorials out there. Nothing worked. The problem was that I was using Maven and placing the manifest in the src/main/resources/META-INF folder, which is supposed to get copied verbatim to the final artifact. However, Maven will automatically overwrite that manifest with it's own! Default NetBeans project? You have to know to switch to file view and edit the pre-existing manifest in the root folder. Eclipse? The manifest needs to be in a META-INF folder under the root. Of all necessary complexity, a novice OSGi developer shouldn't have to fight that battle.
The second time was when I tried to create an annotation-driven OSGi configuration tool, as mentioned previously. Again, the build system, coupled with limitations of annotation processors, gets in the way. Generating configuration into non-manifest files, however, is a snap.
Ultimately, each build system and IDE treats the manifest as a special file. And yes - that means that it really is the fault of the build system. Nevertheless, this is the world we live in. Moving OSGi configuration out of the manifest would be a big step towards decoupling it from the surrounding environment. It's the same basic principle that OSGi promotes so well: decouple separate parts in order to better manage the unexpected.
To answer your final question directly: no, I don't think every problem would disappear just by moving configuration to a different file, I only think it would be one helpful step. For what it's worth, I think OSGi has done a great job managing complexity, and I certainly prefer working with the current format over the mounds of XML configuration found elsewhere.
Luke,
It seems like we are coming from a similar place with annotations. I started a project a while ago on Java.net and published my code there (https://naked-osgi.dev.java.net/). The goal was to configure exports, activators, bundle name, version and Declarative Services using annotations and bnd-style parsing for imports. As you'll note from the project page, however, I've since abandoned the concept due to limitations of the Java annotation processing API and the aforementioned troubles with Java build systems. Currently, I'm trying to work with a more limited scope, generating Declarative Services configurations, and have had much greater success. If you are interested in building off what I have done or just picking my brain, feel free to contact me via the project owner email link on the project page. (Fair warning: the code base is not in the best of shape since I have changed directions so many times to try and get around limitations. Code cleanup is my primary goal before a 1.0 release, followed closely by documentation.)
- Evan