Other OSGi blogs |
OSGi DevCon 2009
2008-10-30
It is that time of the year again. While the rain (at least in the northern hemisphere) rattles on the windows, the fireplace is buzzing, one has to think about that sunny place Santa Clara where EclipseCon and OSGi DevCon will be held next year. Yes, it is again time for submissions.
In the last year the OSGi Alliance has seen an accelerating adoption of its technology. We therefore expect an avalanche of interesting papers, presentations, demos, and tutorials. If you use OSGi technology in an interesting way, if you have done research in an OSGi related area, if you can teach others about OSGi, then get your virtual pen and submit it to the submission system. You have until November 24, but as we all know: Time flies.
I am looking forward to lots of exciting submissions!
Peter Kriens
In the last year the OSGi Alliance has seen an accelerating adoption of its technology. We therefore expect an avalanche of interesting papers, presentations, demos, and tutorials. If you use OSGi technology in an interesting way, if you have done research in an OSGi related area, if you can teach others about OSGi, then get your virtual pen and submit it to the submission system. You have until November 24, but as we all know: Time flies.
I am looking forward to lots of exciting submissions!
Peter Kriens
Add a comment
Closures
2008-10-20
As an old Smalltalker the addition of closures to the Java language sounds awfully good. Not a coding day goes by or I have a tinge of frustration when I have to do extraneous typing that would have been unnecessary in Smalltalk, where closure are the bread and butter of programming. So you would expect that I would be thrilled. However, I am having some reservations because of a very non-functional aspect of the two main proposals (BGCA and FCM): class loading. Let me elaborate.
About two years ago I played with Groovy and very much liked the builder pattern. With the builder pattern you can create code that looks like HTML but has access to all of Groovy's extensive features. This was great! A friend and I created some examples from an existing web application and it looked wonderful. Unfortunately, it turned out that there was a huge penalty because every closure was compiled into a class. With thousands of closures, class loading quickly became a performance issue. Just imagine heapspaces of many gigabytes (most of them strings, perm spaces that grow to hundreds of megabytes or more).
Knowing closures, I am convinced that untold developers will enthusiastically use them and also create thousands of closures in an average application. This will add a significantly to the number of classes in an application. And though I am the first to admit that class loading is fast, (and even faster in OSGi), the cost is absolutely not zero. And this worries me because when I look at the big vendors in the J2EE application server market than I know that their code bases are already creaking at the seams. You would be impressed if you knew the tricks that were being played to keep the system from grinding to a halt.
Until now, we have been more or less saved by computers becoming faster and faster, and disks growing bigger and bigger. However, the news I read today tells me those good days are over. Yes, we will get more and more cores, but each core will become slower. Though you can do some parallelization of class loading, there are many serial bottlenecks that are unavoidable.
The worst thing of all is that I do not fully understand why the proposals could not compile to methods. Methods are much cheaper than classes because the overhead of class loading is mitigated over all the methods in a class.
To make one thing clear, OSGi class loading is much more efficient than other class loading models due to its network nature instead of hierachy. An OSGi class loader knows authoritively who to consult for a class. OSGi based systems will have the least number of problems.
I do realize that it is usually not a good idea to let a high level design be influenced by low level details like class loading. However, I am ringing the alarm bell in this case. I am seeing too many problems at the bottom of the system that are caused by the people at the top that seem oblivious of the problems they are causing downstream. When you share resources, which is what you do in an application server, you must accept some responsibility for the commons or in the end face reduced performance.
Both JRuby and Groovy now compile closures to methods. Hey! If the dynamic guys can do it, then shouldn't we be able to do this as well in Java?
Peter Kriens
About two years ago I played with Groovy and very much liked the builder pattern. With the builder pattern you can create code that looks like HTML but has access to all of Groovy's extensive features. This was great! A friend and I created some examples from an existing web application and it looked wonderful. Unfortunately, it turned out that there was a huge penalty because every closure was compiled into a class. With thousands of closures, class loading quickly became a performance issue. Just imagine heapspaces of many gigabytes (most of them strings, perm spaces that grow to hundreds of megabytes or more).
Knowing closures, I am convinced that untold developers will enthusiastically use them and also create thousands of closures in an average application. This will add a significantly to the number of classes in an application. And though I am the first to admit that class loading is fast, (and even faster in OSGi), the cost is absolutely not zero. And this worries me because when I look at the big vendors in the J2EE application server market than I know that their code bases are already creaking at the seams. You would be impressed if you knew the tricks that were being played to keep the system from grinding to a halt.
Until now, we have been more or less saved by computers becoming faster and faster, and disks growing bigger and bigger. However, the news I read today tells me those good days are over. Yes, we will get more and more cores, but each core will become slower. Though you can do some parallelization of class loading, there are many serial bottlenecks that are unavoidable.
The worst thing of all is that I do not fully understand why the proposals could not compile to methods. Methods are much cheaper than classes because the overhead of class loading is mitigated over all the methods in a class.
To make one thing clear, OSGi class loading is much more efficient than other class loading models due to its network nature instead of hierachy. An OSGi class loader knows authoritively who to consult for a class. OSGi based systems will have the least number of problems.
I do realize that it is usually not a good idea to let a high level design be influenced by low level details like class loading. However, I am ringing the alarm bell in this case. I am seeing too many problems at the bottom of the system that are caused by the people at the top that seem oblivious of the problems they are causing downstream. When you share resources, which is what you do in an application server, you must accept some responsibility for the commons or in the end face reduced performance.
Both JRuby and Groovy now compile closures to methods. Hey! If the dynamic guys can do it, then shouldn't we be able to do this as well in Java?
Peter Kriens
Comments
Ed Merks
:
Peter, both the links point to the same thing: the FCM proposal.
It's interesting that making Java yet more complex will actually make it simpler for users...
It's interesting that making Java yet more complex will actually make it simpler for users...
JodaStephen
:
As co-author of FCM closures, I would be quite happy to see our proposal compile to methods where appropriate. For me, the main point is to get the semantics right, particularly wrt return.
Neil Bartlett
:
Call me a hopeless idealist but surely this in an implementation problem rather than a fundamental flaw with the specifications of either BGGA or FCM?
Also since this is a Java 7 change, there should be the opportunity for whatever VM changes might be necessary to support the Java language changes (ao long as those changes are made such that they are compatible with JSR 291!).
Also since this is a Java 7 change, there should be the opportunity for whatever VM changes might be necessary to support the Java language changes (ao long as those changes are made such that they are compatible with JSR 291!).
Ismael Juma
:
An implementation based on method handles would be interesting.
The alternative would be to fix anonymous inner classes so that they're not so wasteful. This would benefit existing systems too.
Ismael
The alternative would be to fix anonymous inner classes so that they're not so wasteful. This would benefit existing systems too.
Ismael
Boris Bokowski
:
It'll be interesting to see if Closures will even make it into Java 7, see for example Alex Miller's predictions: http://tech.puredanger.com/2008/08/02/java7-prediction-update/
I wouldn't be surprised if Java (the language) became less important over time, but am hoping that Java (the VM) will stay and remain important for a long time to come. Which means that language issues are not as important as issues of JVM mechanics, such as being able to efficiently implement Closures. If Groovy and JRuby are happy with the current JVM capabilities, then that's a very good sign.
I wouldn't be surprised if Java (the language) became less important over time, but am hoping that Java (the VM) will stay and remain important for a long time to come. Which means that language issues are not as important as issues of JVM mechanics, such as being able to efficiently implement Closures. If Groovy and JRuby are happy with the current JVM capabilities, then that's a very good sign.
Ismael Juma
:
Boris, I don't think Groovy and JRuby are happy with the JVM as it is, but the improvements in the MLVM should make them happier. Members of both projects have been contributing to the design and implementation discussions. Part of the MLVM work is being standardised through the invokedynamic JSR so it will not be restricted to HotSpot.
Peter Kriens
:
Ed: thanks, fixed it.
Neil: When I look at the proposals the inner classes seem an intrinsic part of their specifications. It looks like the Generic Java type system lacks the constructs to handle the type constraints that are needed for generically typing methods. As said in the blog, dynamic languages do not have this problem so it is straightforward to compile to methods.
I wish it was an implementation detail but combined with my Groovy experience and what I see happening in app server land I think we are heading for a future crash if closures are compiled to classes due to the large number of closures that will appear and the incredible number of apps people nowadays seem to run on a single VM ...
I hope I am wrong, but I think it is something to consider. As stated, I think we are running out of faster processors ...
Neil: When I look at the proposals the inner classes seem an intrinsic part of their specifications. It looks like the Generic Java type system lacks the constructs to handle the type constraints that are needed for generically typing methods. As said in the blog, dynamic languages do not have this problem so it is straightforward to compile to methods.
I wish it was an implementation detail but combined with my Groovy experience and what I see happening in app server land I think we are heading for a future crash if closures are compiled to classes due to the large number of closures that will appear and the incredible number of apps people nowadays seem to run on a single VM ...
I hope I am wrong, but I think it is something to consider. As stated, I think we are running out of faster processors ...
pniederw
:
Peter Kriens
:
At the time I played with it they did and I complained about this. The JRuby guys weighted in and explained how they compiled to methods. I recall reading they then decided to go to methods in the next major release, but I can't find any real reference except my original post: http://markmail.org/message/edyinfu62exrnexi
Then again, I did not spent a lot of time searching ...
Kind regards,
Peter Kriens
Then again, I did not spent a lot of time searching ...
Kind regards,
Peter Kriens
OSGi BundleFest
2008-10-15
We are now three days in the BundleFest and the spirits are still high after all the long hours of work put in. The group is pretty much settled in and we all know the routine. We start at 9 am (though the first walk in around 8 am). The first thing we do is have a meeting where we report where we are, what we want to do, and any road blocks. This is supposed to be a general (short) meeting but this morning I had to switch off the network, which I could easily do because my mac provides the routing functionality. However, it costed me some serious points with some of the attendents.We then work until around 11 for coffee. It is intersting to see how smal groups form and people work together and then continue working highly concentrated. It is a perfect forum to learn, every time I sit next to a person I learn new shortcuts in Eclipse. Though Roman Roelofsen (ProSyst) gave me a hard time because he typed faster than I could read.
Today I worked mostly on bnd, it had a rather nasty bug that it got confused when the Elipse workspace differed from the bnd workspace. Anyway, with Tim Diekmann's (TIBCO) and Victor Ransmayr help I figured out the problem and after that the solution was easy. It is amazing how many easy to invalidate assumptions one makes in code.
I almost missed lunch (which is excellent here) because I had to pick up Andreas Kraft (Deutsche Telekom) from the airport while dropping off Sergey Beryozkin (IONA), but the airport is so close that some fries and mussels were left (not many though!).
In the afternoon we had an interesting discussion about the TCK for RFC 119 distributed computing. Should this TCK test for real distribution (i.e. between VMs or even machines), or do we only have to test the assertions that are mentioned in the specification. This was quite a heated discussion that was loudly continued over tea.
I even get some help with the older TCKs. By moving the OSGi build on bnd we had the problem that all the test cases need to be redone. OSGi test cases are quite complicated because they have to install and install hundreds of bundles. In our build, we have about a 110 projects but we generate over 900 bundles, many of those bundles are embedded. Stoyan Boshev (ProSyst) actually is moving along very fast through one of the most complicated test suites we have now. I am happy I do not have to move that one, though it means he pushes me hard to get the functionality in the OSGi build right.
Probably the coolest thing about this meeting is that you can shout a question and there are three people that have a well-informed opinion and sometimes factual information. For me it is sometimes a bit getting used to, because most of my time I spent my time working alone, having so many people around is overwhelming sometimes. But it is fun.
The worst thing, for me, is the food. This place is having copious lunches that are of excellent quality but contain way too many calories. Oh well, next week no food I guess :-(
Well, off to the next item ...
Peter Kriens
OSGi BundleFest
2008-10-13
This week we are doing a really interesting experiment. We've organized a week of coding around the next release of OSGi. So, this morning (Monday 13/10/2008) we all joined in the Mercure hotel in La Grande Motte (on the Mediterranean coast). Though the weather wasn't too good (we actually got some rain), the temperature is nice, we can work with the glass doors open. And tomorrow looks very nice.It is kind of hectic for me because we are using a new build based on OSGi. This of course affects everybody so I am running around trying to solve problems. It is amazing how many variations you can have in Eclipse! But it is a fantastic sight to see this very experienced group hacking away at bundles. Though it takes some mental gymnastics to go from a bnd problem, to a RFC 119 property, a new method on Deployment Admin, a network sharing problem, getting the right food for our vegetarian, and a JUnit problem within 10 minutes. But man, it is fun to see the enthusiasm and energy.
One of the key goals is to kick start the different Reference Implementations and Compatibility Test suites. So, today we saw lots of discussions and concentrated faces. It looks like this week is going to be highly productive for OSGi, it is a pity that we miss some really important people!
Peter Kriens