JSR 294 SuperPackages Revisited
2007-12-12
Some of my questions about JSR 294 have been answered by Alex Buckley from Sun. I have updated the existing blog to not hide that I had some misunderstanding. If you're interested in 294 it might be interesting to take a look at these changes, I definitely started to understand 294 better, though I am afraid to say that it did not improve my feeling about it. So please read again and watch the [] for changes.
JSR 294 SuperPackages
Peter Kriens
JSR 294 SuperPackages
Peter Kriens
Add a comment
JCP, or What?
2007-12-03
In my vanity I subscribe to Google Alerts on my name. This morning it pointed me to the blog of Stephen Colebourne. Stephen argues against the JCP for many of the same reasons I have, (guess that is why he quoted me so he popped up in my vanity alert). Stephen makes some very sensible proposals. However, some comments posted on the blog are a bit worrying. Yes, I agree the current JCP is bad because its owner (Sun) is more interested in its own interest than the interest of the Java community. Ok, it is also flawed because the process has no requirements phase and is often lead by the commercial interests of one party. However, at the core of some of the comments is a discussion of what is more relevant: an implementation or a specification?
This discussion has been raised inside the OSGi Alliance as well. Why not pick one framework implementation and put a stamp on it and name it the standard? Obviously a single implementation is simpler, what do we get from a time consuming specification process that forces people to compromise?
Well, just as obviously, people's requirements differ. Equinox is optimized for very large systems, uses lots of caching, but is therefore much bigger than Knopflerfish or Apache Felix. The best part about OSGi technology is that it still strives to write applications once and run anywhere, and it is really getting closer. An independent specification allows different implementations that optimize for a certain area.
The second advantage is neutrality. Comments on Stephen's blog make it sound that today most Java innovation takes place on the OpenJDK mailing list. I am not sure if this will create a coherent API in the long run though I am pretty sure it will create bloat because one man's holy grail is another man's unnecessary complexity. The specification process allows the balancing of the interest of several parties before it is cast. Note that the implementation freedom inherent in specifications is a good way to keep the specifications aligned, just implement if differently.
The third advantage is time. Though the slow process of standardization is often used as a negative, I think it is actually a positive (within limits). Last year we started the requirements process for Enterprise OSGi and one of the main aspects is Distributed OSGi. In the past 6 months you can see how the different parties started to understand each other's use cases and requirements and how they are not trying to find solutions that are much broader than if any of them had hacked together a solution on their own. I really think that most of the OSGi specifications actually reflect this focus on industry wide solutions and not trying to solve a small acute problem under great time pressure.
The fourth advantage is the whole struggle with intellectual property rights (IPR). Life might be beautiful if everything was free, but it isn't. We live in a world where companies have legitimate rights to the fruits of their labor. Interestingly, the most free for all movement (GNU) creates the biggest problem of all because of its viral nature, affecting anything it touches. It turns out a lot easier handle the IPR issues in an implementation than in a specification because there is much less IPR involved (and therefore much less touching).
In a way, the world of creating implementations and specifications is similar to classes and interfaces. I am pretty sure no Java programmer wants to give up interfaces, the reuse of Java code would be significantly harder. Therefore, could Java be maintained by an open source community as some think? I do not think so. The language would quickly drift in a direction the hardest pushers push it into, it will bloat and become unusable for many applications.
What we need is a strong modularity layer so different solutions can compete evenly. The market can then decide which solutions work best. However, to evolve the core platform Java will need a community process, just one that is not dominated by the interest of a single commercial party.
Peter Kriens
P.S. One of the comments states that OSGi is more complex than JSR 277. Sigh. The OSGi framework API is significantly smaller counted as methods and classes/interfaces. We just have more good documentation. And obviously we cover the cases that JSR 277 will discover in the next 5 years.
This discussion has been raised inside the OSGi Alliance as well. Why not pick one framework implementation and put a stamp on it and name it the standard? Obviously a single implementation is simpler, what do we get from a time consuming specification process that forces people to compromise?
Well, just as obviously, people's requirements differ. Equinox is optimized for very large systems, uses lots of caching, but is therefore much bigger than Knopflerfish or Apache Felix. The best part about OSGi technology is that it still strives to write applications once and run anywhere, and it is really getting closer. An independent specification allows different implementations that optimize for a certain area.
The second advantage is neutrality. Comments on Stephen's blog make it sound that today most Java innovation takes place on the OpenJDK mailing list. I am not sure if this will create a coherent API in the long run though I am pretty sure it will create bloat because one man's holy grail is another man's unnecessary complexity. The specification process allows the balancing of the interest of several parties before it is cast. Note that the implementation freedom inherent in specifications is a good way to keep the specifications aligned, just implement if differently.
The third advantage is time. Though the slow process of standardization is often used as a negative, I think it is actually a positive (within limits). Last year we started the requirements process for Enterprise OSGi and one of the main aspects is Distributed OSGi. In the past 6 months you can see how the different parties started to understand each other's use cases and requirements and how they are not trying to find solutions that are much broader than if any of them had hacked together a solution on their own. I really think that most of the OSGi specifications actually reflect this focus on industry wide solutions and not trying to solve a small acute problem under great time pressure.
The fourth advantage is the whole struggle with intellectual property rights (IPR). Life might be beautiful if everything was free, but it isn't. We live in a world where companies have legitimate rights to the fruits of their labor. Interestingly, the most free for all movement (GNU) creates the biggest problem of all because of its viral nature, affecting anything it touches. It turns out a lot easier handle the IPR issues in an implementation than in a specification because there is much less IPR involved (and therefore much less touching).
In a way, the world of creating implementations and specifications is similar to classes and interfaces. I am pretty sure no Java programmer wants to give up interfaces, the reuse of Java code would be significantly harder. Therefore, could Java be maintained by an open source community as some think? I do not think so. The language would quickly drift in a direction the hardest pushers push it into, it will bloat and become unusable for many applications.
What we need is a strong modularity layer so different solutions can compete evenly. The market can then decide which solutions work best. However, to evolve the core platform Java will need a community process, just one that is not dominated by the interest of a single commercial party.
Peter Kriens
P.S. One of the comments states that OSGi is more complex than JSR 277. Sigh. The OSGi framework API is significantly smaller counted as methods and classes/interfaces. We just have more good documentation. And obviously we cover the cases that JSR 277 will discover in the next 5 years.
Comments
robilad
:
As you point out, the ability to work together on specifications, implementations and corresponding test suites has many advantages for everyone. And of course, having modularity out of the box in Java 7 will decrease the pressure to ram APIs into the core platform, so there will be less of a pressing need to have JSRs bundled in the platform purely for deployment reasons. So I think we need the JCP now, and will continue to need it in the future.
Otoh, currently, the JCP is too subservient to the interests of the proprietary software vendors, starting from the 'confidential by default' mode of operation, that makes it very hard to positively influence JSRs as an outsider of the inner circle of companies wanting to push it through. You surely know what that feels like. ;)
But that's a problem the JCP shares with OSGi, where I can't participate without being on the payroll of a member company (or shelling out 20K USD for the right to give away my own ideas in the spec process, yay!), so I can't actually fix the problems arising from OSGi's denial of source code being the fundamental currency unit of software exchange. ;)
Another problem the JCP shares with OSGi is that it allows specifications for 'open' standards to be encumbered with proprietary components, be they RIs, TCKs, etc. That effectively allows members to extract a royalty from implementors of the 'open' standards through a side channel. The sad fact that neither the OSGi TCK is open source, nor are any major Java platform TCKs, is evidence of organizations rigged to the favor of the proprietary vendors at the expense of the user's need to verify for themselves in their own deployment environment, if a vendor's solution meets the requirements laid out in the standard.
From my outsider perspective, both the JCP and OSGi are rather far from being an optimal standards org, with transparency, open source by default, etc. But both are worth having.
The interesting question is: how do we fix them? At least the JCP seems to allow for individuals to band together and change the rules to improve the system for everyone, while OSGi seems to require an expensive initiation rite to give one a voice within the system. So I think that we can fix the JCP sooner than we can fix OSGi.
Otoh, currently, the JCP is too subservient to the interests of the proprietary software vendors, starting from the 'confidential by default' mode of operation, that makes it very hard to positively influence JSRs as an outsider of the inner circle of companies wanting to push it through. You surely know what that feels like. ;)
But that's a problem the JCP shares with OSGi, where I can't participate without being on the payroll of a member company (or shelling out 20K USD for the right to give away my own ideas in the spec process, yay!), so I can't actually fix the problems arising from OSGi's denial of source code being the fundamental currency unit of software exchange. ;)
Another problem the JCP shares with OSGi is that it allows specifications for 'open' standards to be encumbered with proprietary components, be they RIs, TCKs, etc. That effectively allows members to extract a royalty from implementors of the 'open' standards through a side channel. The sad fact that neither the OSGi TCK is open source, nor are any major Java platform TCKs, is evidence of organizations rigged to the favor of the proprietary vendors at the expense of the user's need to verify for themselves in their own deployment environment, if a vendor's solution meets the requirements laid out in the standard.
From my outsider perspective, both the JCP and OSGi are rather far from being an optimal standards org, with transparency, open source by default, etc. But both are worth having.
The interesting question is: how do we fix them? At least the JCP seems to allow for individuals to band together and change the rules to improve the system for everyone, while OSGi seems to require an expensive initiation rite to give one a voice within the system. So I think that we can fix the JCP sooner than we can fix OSGi.
Peter
:
When I started out with the OSGi I thought very similar to you. We should be grateful for people those individuals that are willing to spent all that time writing specifications! However, over time I learned that the physical work is only one aspect of it.
For a specification to succeed it is paramount that key players are standing behind it. Nobody gives a rat's ass about a Peter Kriens' JSR 420, however perfect it might technically be. For a specification to succeed it is a paramount that some of the big players in the market heed attention and connect their name to it. A specification is not just the technical content, it is also about the fact that major players agreed on something. Now, this agreement should be on technical quality in the ideal case, and in the OSGi Alliance, I think it is, at least we try very hard.
The other aspect is just quality. Fred Brooks' seminal Book "The Mythical Man Month" describes in detail the difference between an individual's effort versus a professional product. There are so many aspects in the production of a good specification that are just beyond the realm of individuals. The OSGi specifications starts with a requirements document (RFP), then an implementation proposal (RFC), then a professional spec writer in parallel with the construction of a Reference Implementation and a Test suite. Many meetings are necessary to create the consensus between the key players that is at the core of a good spec. Doing this right is really expensive and beyond what an individual can do. The $20K is dwarfed in comparison what it costs to go to the meetings and provide the required resources. And what is the motivation to spent this kind of money for an individual? You can also look at the $20k as a threshold to ensure only companies (or groups) that are willing to put serious effort in it to participate. If there is no business case in the end, things tend to go nowhere.
That said, I think we have shown in the past that we are listening. Anybody can submit RFPs and they are taken very, very, seriously. However, in the end you need a number of companies that are serious enough to invest in addressing these requirements.
TCKs. The JCP and the OSGi Alliance have a completely different model. The JCP is owned by Sun and licenses are paid directly to the license holder which is a commercial company (mostly Sun, but there are many others as well).
The OSGi Alliance, in contrast, is the owner of the RI and TCK and provides all rights indiscriminately to all its members which can basically use it in any way they see fit. So for $20K you not only get the right to give your ideas :-), you also get some source code and TCK back. The crucial difference between JCP and OSGi is that the majority of members decide and the bylaws have no provision from rejection a membership application. All members are really treated equal. I think it makes a huge difference if you are at the whim of a competitor or that decisions come from a non-profit organization that tries to create a market for component based systems ruled by real by-laws, non-profit, and anti kartel rules. The transparency lies in the fact that anybody can become a member. Yes, $20k is a lot for you and me, but if it is a road block for a commercial company than they just think it is not important enough to bother ...
Concrete. You say you can fix the JCP before the OSGi because the JCP has free memberships. I am a JCP individual member. There is no public mailing list of JCP members, and the denial of my participation in JSR 277 was a nice example where several EC members tried to help me but still failed.
Now, your comment is nicely not about technology. So where did we screw up that you are so eager to change? I have not seen any RFPs from you?
Kind regards,
Peter Kriens
P.S. Watch the news ... you might like the new plans the OSGi Alliance has
For a specification to succeed it is paramount that key players are standing behind it. Nobody gives a rat's ass about a Peter Kriens' JSR 420, however perfect it might technically be. For a specification to succeed it is a paramount that some of the big players in the market heed attention and connect their name to it. A specification is not just the technical content, it is also about the fact that major players agreed on something. Now, this agreement should be on technical quality in the ideal case, and in the OSGi Alliance, I think it is, at least we try very hard.
The other aspect is just quality. Fred Brooks' seminal Book "The Mythical Man Month" describes in detail the difference between an individual's effort versus a professional product. There are so many aspects in the production of a good specification that are just beyond the realm of individuals. The OSGi specifications starts with a requirements document (RFP), then an implementation proposal (RFC), then a professional spec writer in parallel with the construction of a Reference Implementation and a Test suite. Many meetings are necessary to create the consensus between the key players that is at the core of a good spec. Doing this right is really expensive and beyond what an individual can do. The $20K is dwarfed in comparison what it costs to go to the meetings and provide the required resources. And what is the motivation to spent this kind of money for an individual? You can also look at the $20k as a threshold to ensure only companies (or groups) that are willing to put serious effort in it to participate. If there is no business case in the end, things tend to go nowhere.
That said, I think we have shown in the past that we are listening. Anybody can submit RFPs and they are taken very, very, seriously. However, in the end you need a number of companies that are serious enough to invest in addressing these requirements.
TCKs. The JCP and the OSGi Alliance have a completely different model. The JCP is owned by Sun and licenses are paid directly to the license holder which is a commercial company (mostly Sun, but there are many others as well).
The OSGi Alliance, in contrast, is the owner of the RI and TCK and provides all rights indiscriminately to all its members which can basically use it in any way they see fit. So for $20K you not only get the right to give your ideas :-), you also get some source code and TCK back. The crucial difference between JCP and OSGi is that the majority of members decide and the bylaws have no provision from rejection a membership application. All members are really treated equal. I think it makes a huge difference if you are at the whim of a competitor or that decisions come from a non-profit organization that tries to create a market for component based systems ruled by real by-laws, non-profit, and anti kartel rules. The transparency lies in the fact that anybody can become a member. Yes, $20k is a lot for you and me, but if it is a road block for a commercial company than they just think it is not important enough to bother ...
Concrete. You say you can fix the JCP before the OSGi because the JCP has free memberships. I am a JCP individual member. There is no public mailing list of JCP members, and the denial of my participation in JSR 277 was a nice example where several EC members tried to help me but still failed.
Now, your comment is nicely not about technology. So where did we screw up that you are so eager to change? I have not seen any RFPs from you?
Kind regards,
Peter Kriens
P.S. Watch the news ... you might like the new plans the OSGi Alliance has
robilad
:
You didn't screw up, no worries. I'm just comparing both institutions and pointing out how they both currently fail my 'requirements' for an open, community standards development body. Those requirements are rather simple:
* no walled garden
* no NDAs
* communication relating to the development of a standard between the expert group must be publicly viewable
* all components of a standard must be unconditionally available under liberal terms to everyone
* i.e. no click through licenses on specs
* no proprietary software in RIs, TCKs, etc.
* patent covenants for all implementations
Now, I understand that a lot of proprietary software vendors will need to be dragged kicking and screaming to allow something like that to happen. :) But that's no reason not to make it happen. Opening up the Java platform wasn't popular at all when we started working on it, and it took years for IBM, Intel, ASF, Sun, etc. to jump on the bandwagon and realize that the benefits of that move were larger than the potential drawbacks.
So, how do I submit an RFP to change OSGi to meet the above requirements, to get that process rolling? We might as well try to improve both OSGi & the JCP while we are at it, after all.
The JCP can be fixed pretty easily, by electing individuals who care about fixing it into the ECs, rather than those that have been in power so far, without improving the system. If one has the majority in the JCP, for example, it's trivial to stop any JSR with proprietary components in its tracks, before it does any damage to downstream users.
Meanwhile, I'm looking forward to the new plans from the OSGi alliance, and I hope to catch up with you at JavaPolis!
* no walled garden
* no NDAs
* communication relating to the development of a standard between the expert group must be publicly viewable
* all components of a standard must be unconditionally available under liberal terms to everyone
* i.e. no click through licenses on specs
* no proprietary software in RIs, TCKs, etc.
* patent covenants for all implementations
Now, I understand that a lot of proprietary software vendors will need to be dragged kicking and screaming to allow something like that to happen. :) But that's no reason not to make it happen. Opening up the Java platform wasn't popular at all when we started working on it, and it took years for IBM, Intel, ASF, Sun, etc. to jump on the bandwagon and realize that the benefits of that move were larger than the potential drawbacks.
So, how do I submit an RFP to change OSGi to meet the above requirements, to get that process rolling? We might as well try to improve both OSGi & the JCP while we are at it, after all.
The JCP can be fixed pretty easily, by electing individuals who care about fixing it into the ECs, rather than those that have been in power so far, without improving the system. If one has the majority in the JCP, for example, it's trivial to stop any JSR with proprietary components in its tracks, before it does any damage to downstream users.
Meanwhile, I'm looking forward to the new plans from the OSGi alliance, and I hope to catch up with you at JavaPolis!
Peter
:
If you send me a mail on Peter.Kriens@osgi.org, I'll send you the template for the RFP. I'd love to see your input
However, I still think you do not understand how the JCP works. Sun is in full control at any time. There is no separate organization with bylaws and board. The ECs can provide inputs but the agenda is fully controlled by Sun. I am not sure how you can change this model, unless Sun changes its way voluntarily.
See you at Javapolis! Kind regards,
Peter Kriens
However, I still think you do not understand how the JCP works. Sun is in full control at any time. There is no separate organization with bylaws and board. The ECs can provide inputs but the agenda is fully controlled by Sun. I am not sure how you can change this model, unless Sun changes its way voluntarily.
See you at Javapolis! Kind regards,
Peter Kriens
robilad
:
Sun isn't in the kind of control that can't be worked around, there is just no one interested in pushing for changing the JCP on the ECs, which is entirely possible with the current setup.
All Sun can do with their legendary 'veto power' is to prevent Java-language and core Java platform JSRs from passing (A.5.9 of JCP 2.6). That kind of veto power has been irrelevant in the past 5 years, afaict, and will likely continue to be irrelevant.
The real power is with the EC, that, too, can through their vote prevent JSRs from passing. As we'll elect ourselves in the coming years a JCP EC that's representative of the needs of the Free Software Java developers, we'll be able to raise the standards on specifications bearing the 'community' label of the JCP, by voting down JSRs that don't follow the requirements outlined above, so that obvious mistakes, like passing the OSGi JSR along with a proprietary TCK, (or most Java platform JSRs so far) stop happening.
All Sun can do with their legendary 'veto power' is to prevent Java-language and core Java platform JSRs from passing (A.5.9 of JCP 2.6). That kind of veto power has been irrelevant in the past 5 years, afaict, and will likely continue to be irrelevant.
The real power is with the EC, that, too, can through their vote prevent JSRs from passing. As we'll elect ourselves in the coming years a JCP EC that's representative of the needs of the Free Software Java developers, we'll be able to raise the standards on specifications bearing the 'community' label of the JCP, by voting down JSRs that don't follow the requirements outlined above, so that obvious mistakes, like passing the OSGi JSR along with a proprietary TCK, (or most Java platform JSRs so far) stop happening.
Peter
:
| Previous Posts | Archives |
|---|---|