Btool, the OSGi Bundle Maker
2005-11-24
Bundles are plain old JAR files, which in their turn are plain old ZIP files. The JAR files are a convenient container for code and resources, grouped in packages. Additionally, a manifest is provided that provides bits and pieces of information about the contents. The common way to make a bundle is to create a directory structure that mirrors the desired structure and then use the jar-tool of the JDK to create the JAR file.
Unfortunately, this way of working is error prone because the contents of the JAR are closely related to the manifest. The manifest must specify the packages that can be exported (available to others) as well as which packages must be imported. This is not trivial, especially when bundles are designed as small, self contained, cohesive units, for example, OSGi RI and TCK consist over 700 bundles, which have a myriad of relations among each other.
Another problem is that making the intermediate directory structure is usually done in highly project dependent ant or make files. These scripts are hard to read, which makes it difficult to get an overview how the JAR file is structured. A particularly tricky problem is when a JAR file needs a copy of a package that is on the class path, this is almost impossible without special support.
Meet Btool. Btool started its life in 1998 when I got so frustrated with making bundles using only the jar-tool as support. Tracking import- and exported packages and their versions are not my description of fun. I was working for Ericsson Research at that time and had to help a lot of people making bundles.
The first version of Btool was given a root class, which it then recursively parsed for all dependencies. Command line parameters were used to indicate which packages should packed in the JAR. From this information, Btool could calculate the import and export clauses. Versions were derived from a file called packageinfo, which we maintain in each package directory. The packageinfo file is also packed in the JAR.
Btool generates the manifest from the generated JAR and a raw-manifest file that can contain property references. An example raw-manifest file:
Manifest-Version: 1.0
Bundle-Name: ${p}
Bundle-Activator: ${p}.Activator
Bundle-Description: The %24{p} variable is
replaced with the project name ${p}
Comment: $(DATE)
$(IMPORT-PACKAGE)
$(EXPORT-PACKAGE)
$(PRIVATE-PACKAGE)
The manifest file is pre-processed and the property references are replaced with the appropriate information. Btool supports an extensive number of macros like substitute, prefix/suffix, sort, filter, and more.
Since then a lot has changed, we are currently at generation 3. Btool is now an ant task and completely driven by a properties file. Eclipse drove the use of the build.properties file (which is standard in Eclipse) but in the end they have become incompatible. A typical build.properties file looks like:
jars.compile.order=${p}.jar
expand.${p} = [com.acme.mybundle], \
com.acme.util.*
include.mybundle.jar = [doc/license.html], \
doc/help.html, \
[README.txt]
The expand property indicates the packages that should be stored in the JAR, which come from the project’s class path. If the package is enclosed with square brackets ([]), it means the package is to be exported. Package can contain wildcards, which will include all packages starting with that prefix. The project’s class path can be set with the Eclipse IDE (we use the .classpath file) or the classpath property can be set to a list of directories or JAR files. The include property will copy files into the JAR. If a file is enclosed with square brackets, it will be pre-processed just like the manifest.
While creating the JAR file, Btool can store all the sources in the OPT-INF/src directory, calculate the Name sections, create files for initial provisioning, and can do many more functions. However, crucial is the analysis phase at the end. When the JAR file is created, Btool can open it again and analyze it for consistency. It will check the import/export sections, checks if the Bundle Activator is in the JAR or imported, looks at the Bundle-Classpath, and does many more checks. The analysis has saved me from endless grief.
I am so used to Btool that for me personally it is hard to see how you can make bundles without it. However, must people are doing well without it looking at the large number of bundles that are created. However, there is work going on. In the Oscar/Felix project there is mangen that creates manifests for a set of bundles. There is also the OSGi plugin for the Maven project, Eclipse is working hard on bundle tools, and there are plugins. However, when you are member of OSGi, take a look at the OSGi repository on membercvs and see how Btool can be used to simplify your projects.
Unfortunately, this way of working is error prone because the contents of the JAR are closely related to the manifest. The manifest must specify the packages that can be exported (available to others) as well as which packages must be imported. This is not trivial, especially when bundles are designed as small, self contained, cohesive units, for example, OSGi RI and TCK consist over 700 bundles, which have a myriad of relations among each other.
Another problem is that making the intermediate directory structure is usually done in highly project dependent ant or make files. These scripts are hard to read, which makes it difficult to get an overview how the JAR file is structured. A particularly tricky problem is when a JAR file needs a copy of a package that is on the class path, this is almost impossible without special support.
Meet Btool. Btool started its life in 1998 when I got so frustrated with making bundles using only the jar-tool as support. Tracking import- and exported packages and their versions are not my description of fun. I was working for Ericsson Research at that time and had to help a lot of people making bundles.
The first version of Btool was given a root class, which it then recursively parsed for all dependencies. Command line parameters were used to indicate which packages should packed in the JAR. From this information, Btool could calculate the import and export clauses. Versions were derived from a file called packageinfo, which we maintain in each package directory. The packageinfo file is also packed in the JAR.
Btool generates the manifest from the generated JAR and a raw-manifest file that can contain property references. An example raw-manifest file:
Manifest-Version: 1.0
Bundle-Name: ${p}
Bundle-Activator: ${p}.Activator
Bundle-Description: The %24{p} variable is
replaced with the project name ${p}
Comment: $(DATE)
$(IMPORT-PACKAGE)
$(EXPORT-PACKAGE)
$(PRIVATE-PACKAGE)
The manifest file is pre-processed and the property references are replaced with the appropriate information. Btool supports an extensive number of macros like substitute, prefix/suffix, sort, filter, and more.
Since then a lot has changed, we are currently at generation 3. Btool is now an ant task and completely driven by a properties file. Eclipse drove the use of the build.properties file (which is standard in Eclipse) but in the end they have become incompatible. A typical build.properties file looks like:
jars.compile.order=${p}.jar
expand.${p} = [com.acme.mybundle], \
com.acme.util.*
include.mybundle.jar = [doc/license.html], \
doc/help.html, \
[README.txt]
The expand property indicates the packages that should be stored in the JAR, which come from the project’s class path. If the package is enclosed with square brackets ([]), it means the package is to be exported. Package can contain wildcards, which will include all packages starting with that prefix. The project’s class path can be set with the Eclipse IDE (we use the .classpath file) or the classpath property can be set to a list of directories or JAR files. The include property will copy files into the JAR. If a file is enclosed with square brackets, it will be pre-processed just like the manifest.
While creating the JAR file, Btool can store all the sources in the OPT-INF/src directory, calculate the Name sections, create files for initial provisioning, and can do many more functions. However, crucial is the analysis phase at the end. When the JAR file is created, Btool can open it again and analyze it for consistency. It will check the import/export sections, checks if the Bundle Activator is in the JAR or imported, looks at the Bundle-Classpath, and does many more checks. The analysis has saved me from endless grief.
I am so used to Btool that for me personally it is hard to see how you can make bundles without it. However, must people are doing well without it looking at the large number of bundles that are created. However, there is work going on. In the Oscar/Felix project there is mangen that creates manifests for a set of bundles. There is also the OSGi plugin for the Maven project, Eclipse is working hard on bundle tools, and there are plugins. However, when you are member of OSGi, take a look at the OSGi repository on membercvs and see how Btool can be used to simplify your projects.
Add a comment
Why Did We Let Jini Out of OSGi R4
2005-11-17
The OSGi Service Platform Release 4 was the first release where we actually dropped a specification. In R3, there was a special section that we named “Recommended”, clearly indicating that they were no specifications. It contained the UPnP™, JINI, Namespace, and Initial Provisioning recommendations.
So in R4 we needed to make the decision to elevate these recommendations to specification status. The UPnP and Initial Provisioning recommendations were obvious candidates for elevation while the Name Space recommendation was just as obvious for removal. The Jini recommendation was a harder because Jini itself was not completely dead. So after some soul searching we decided to remove the Jini recommendation for the following reasons:
So where the UPnP recommendation had several members highly interested, no such defenders could be found for the Jini recommendation. Also looking at the outside world, it is clear that the UPnP adoption is gaining momentum, in contrast with Jini.
Personally, I absolutely loved the Jini specifications, never read a more well written architecture specification. However, after I spent some serious time with it, which was in 1997 when I first met Jon Bostrom selling Jini together with Billy Joy, I was very disappointed. The implementation was so bad, that I have a hard time believing they have been able to solve all the problems in these 8 years.
I must admit I am surprised it keeps popping up in different places, but it somehow refuses to die, regardless of the number of places where it fails. Ever seen a consumer Jini device?
Interestingly, Service Location Protocol (SLP) seems to be popping up its pretty head again. This is intriguing because SUN, as author of SLP and JINI, politically decided to assassinate SLP in favor of Jini. SLP (as I recall) supports a mode where it carries a JAR file, which is of course a perfect match for OSGi. And SLP is basically the Jini network discovery protocol without the additional the requirements for Java and its security implications.
Maybe the OSGi should take a look at SLP? Let me know.
Peter Kriens
OSGi Evangelist
So in R4 we needed to make the decision to elevate these recommendations to specification status. The UPnP and Initial Provisioning recommendations were obvious candidates for elevation while the Name Space recommendation was just as obvious for removal. The Jini recommendation was a harder because Jini itself was not completely dead. So after some soul searching we decided to remove the Jini recommendation for the following reasons:
- None of the members showed any interested in this recommendation or defended it.
- Jini was considered more or less on life-support if not already declared dead.
- Security implications that are caused by Jini were not compatible with the strong security model of the OSGi specifications.
- Complex interactions between class loaders made the use of Jini cumbersome.
- The problem of loading ill-defined code in your VM
So where the UPnP recommendation had several members highly interested, no such defenders could be found for the Jini recommendation. Also looking at the outside world, it is clear that the UPnP adoption is gaining momentum, in contrast with Jini.
Personally, I absolutely loved the Jini specifications, never read a more well written architecture specification. However, after I spent some serious time with it, which was in 1997 when I first met Jon Bostrom selling Jini together with Billy Joy, I was very disappointed. The implementation was so bad, that I have a hard time believing they have been able to solve all the problems in these 8 years.
I must admit I am surprised it keeps popping up in different places, but it somehow refuses to die, regardless of the number of places where it fails. Ever seen a consumer Jini device?
Interestingly, Service Location Protocol (SLP) seems to be popping up its pretty head again. This is intriguing because SUN, as author of SLP and JINI, politically decided to assassinate SLP in favor of Jini. SLP (as I recall) supports a mode where it carries a JAR file, which is of course a perfect match for OSGi. And SLP is basically the Jini network discovery protocol without the additional the requirements for Java and its security implications.
Maybe the OSGi should take a look at SLP? Let me know.
Peter Kriens
OSGi Evangelist
2005-11-14
OSGi Test Suites
I am currently working on the most important, but also very boring, part of the obligations we have in the OSGi Alliance: the test suites. This time around it was a bit different though, we had a single company writing most of the test suites. Normally this is more dispersed. CESAR is a Brazilian research institute that was hired by IBM and Motorola to provide many of the required test suites, and they have done a thorough job. The requirements of the test suites were higher for this release because we will license the test suites to any company that wants to be compliant with JSR 232. The test suites are of course free for any OSGi Alliance member.
The test suites are based on the nowadays standard JUnit model. The key difference is that we deploy them as bundles. We include a facility that allows test suites to run on the target machine and a facilitating machine, this setup makes it impossible to use a single bundle for a test suite. The facilitating machine sets up the environment and then downloads a testcase to the target. The target executes the testcases that can interact with the facilitating machine. This makes it possible to minimize the requirements on the target machine and removes the need for manual interaction. This has turned out to be a very elegant model, we are very happy with it.
But these weeks we have to get all the parts together, coming from several continents, and quite a lot of people. Lot of work, but fortunately, it looks like we are getting there.
There are discussion going on to make the OSGi test tooling available outside the OSGi Alliance. If you are interested, let us know.
Peter Kriens
OSGi Evangelist
I am currently working on the most important, but also very boring, part of the obligations we have in the OSGi Alliance: the test suites. This time around it was a bit different though, we had a single company writing most of the test suites. Normally this is more dispersed. CESAR is a Brazilian research institute that was hired by IBM and Motorola to provide many of the required test suites, and they have done a thorough job. The requirements of the test suites were higher for this release because we will license the test suites to any company that wants to be compliant with JSR 232. The test suites are of course free for any OSGi Alliance member.
The test suites are based on the nowadays standard JUnit model. The key difference is that we deploy them as bundles. We include a facility that allows test suites to run on the target machine and a facilitating machine, this setup makes it impossible to use a single bundle for a test suite. The facilitating machine sets up the environment and then downloads a testcase to the target. The target executes the testcases that can interact with the facilitating machine. This makes it possible to minimize the requirements on the target machine and removes the need for manual interaction. This has turned out to be a very elegant model, we are very happy with it.
But these weeks we have to get all the parts together, coming from several continents, and quite a lot of people. Lot of work, but fortunately, it looks like we are getting there.
There are discussion going on to make the OSGi test tooling available outside the OSGi Alliance. If you are interested, let us know.
Peter Kriens
OSGi Evangelist
2005-11-10
The UPnP and OSGi Specifications
A common misconception is that OSGi and UPnP are competing standards. Well, the good news is that they are definitely not in competition! Actually, they work surprisingly well together. That is good news for us because more and more devices are UPnP enabled. Interestingly, very few devices actually are labeled with the UPnP Forum brand, but it is introduced in virtually any networked device, which are in increasing number of consumer devices. For example, when I start an OSGi Framework at home, I can see the registration of:
A common misconception is that OSGi and UPnP are competing standards. Well, the good news is that they are definitely not in competition! Actually, they work surprisingly well together. That is good news for us because more and more devices are UPnP enabled. Interestingly, very few devices actually are labeled with the UPnP Forum brand, but it is introduced in virtually any networked device, which are in increasing number of consumer devices. For example, when I start an OSGi Framework at home, I can see the registration of:
- A Belkin Wireless Network Router
- A Linksys network storage unit
- A Go Video Networked DVD Player,
- Two Xerox Printers
- A Media-Player on another PC
- A Wireless camera
This silent revolution is very good news for OSGi users, UPnP addresses the key issue of how to communicate with CE devices, something that is beyond the scope of the OSGi Alliance. But what does the OSGi Alliance then provides? The OSGi provides the secure place to run all those programs that provide UPnP devices (like Media Players, for example) and programs that control those UPnP devices.
There is a big need for the flexibility that an OSGi Platform provides because not everything is rosy in the UPnP world. The specification is in certain areas rather loose, and too many programmers are rather lax, which means that there are many interoperability problems. I actually have to run two different media servers to server my two media players. However, these incompatibilities are an opportunity for OSGi based platforms. Additionally, not all devices have the resources to provide UPnP interfaces; a perfect opportunity for an OSGi Platform to bridge this device so that it can be controlled by UPnP remote controllers. The managed application is well suited to a world where many mediations between devices are necessary.
UPnP enabled devices are a very interesting opportunity for OSGi platforms. I know that many people are working on it. This will be big in the coming years, mark my words!
Peter Kriens
OSGi Evangelist
2005-11-09
Welcome to the OSGi Alliance Blog. As the OSGi evangelist, I have the honor to fill this space on a regular basis. I will provide insights in how OSGi technology is used, inform you about events and shows that attract many OSGi adopters, or tell you about interesting deployments and developments in the market. If you have a story, a product, a rumor, or a company that should be discussed here. Send it to the OSGi Evangelist.
A complaint I often get is hard to understand what OSGi is all about. For them, it is hard to understand the excitement that people have for the OSGi Alliance. To ease the transition to this enthusiasm, we have written two whitepapers.
Business oriented people should read the first, while the technical people should read the second.
Today's tidbits
A complaint I often get is hard to understand what OSGi is all about. For them, it is hard to understand the excitement that people have for the OSGi Alliance. To ease the transition to this enthusiasm, we have written two whitepapers.
Business oriented people should read the first, while the technical people should read the second.
Today's tidbits
- Yesterday the Mobile Expert Group delivered a new draft of the Mobile Specification. The draft must not be approved by the board, which is planned for tonight, so we can make it available to you as soon as possible.
- November 15 the Vehicle Expert Group meets in Böblingen, Germany to discuss the Vehicle release. This is a members only meeting.
Comments
| Previous Posts | Archives |
|---|---|