Do You Know What Libraries You’re Using? OSGi Makes it Easy to Find the Answers
By Graham Charters
Some shortcomings of Java and Java EE
The rapid growth in open source projects means that more and more enterprise applications make use of third-party libraries, provided as one or more JAR files providing some runtime capability – libraries for MVC, persistence, dependency injection, logging, testing, Web services, the list goes on. As organizations seek to improve operational effectiveness and agility, inefficiencies in collaboration around — and re-use of — the assets they develop gain greater focus.
Many organizations use Java EE for deployment of their enterprise applications. The Java EE approach to deployment requires an assembler to package all the modules (JAR files) used by the application into an EAR or WAR file. This means that the development, test and operations teams have a single artifact to ship around, but there are still drawbacks. The deployment artifact is essentially opaque. Any analysis is an exercise in archaeology; it involves looking inside the deployment artifact to see what JARs it contains, then trying to understand which JARs correspond to which libraries.
What follows are some of the questions that commonly arise when working with Java EE applications. We then describe a progression of OSGi adoption that addresses the issues outlined.
What libraries does my application package and use?
Many companies deploying enterprise applications have rules governing the use of third-party libraries. These rules can cover areas such as:
- pedigree - is the code truly owned by the project?
- quality - is the code of a suitable quality?
- license - is the code provided under a license that permits it to be used in the way the organization wants to use it?
- openness – is the code from a project that is open to requirements and contributions to enable an organization to contribute to it, should the need arise?
For any reasonably-sized Java EE IT operation, it can be very difficult to keep on top of all the dependencies they consume. The opaque nature of the deployment artifacts (EAR and WAR) means organizations don’t necessarily have a good grasp of what they are deploying. Not having a good understanding of what is being deployed can lead to mistakes and inefficiencies in governing third-party libraries. Some may be covered multiple times, or worse still, they may be missed completely, resulting in deploying something of dubious heritage or inappropriate license which can have serious business implications.
Which applications are deploying this library?
It is common for multiple applications within an organization to be developed using the same libraries and frameworks: Teams work on multiple applications or share knowledge and skills to achieve improved efficiency and serviceability. However, because Java EE requires libraries to be packaged inside the deployment artifact, each application contains duplicates. This results in duplication of disk space, but it also potentially results in duplication of effort if those libraries have to go through some review processes to ensure they’re appropriate for the organization to use. Worse still, if an organization needs to apply a critical fix to a library, then they need to find each copy and repackage/redeploy each application. As we have already seen, this is difficult because of the opaque nature of the deployment artifact and the fact that mapping JAR names to libraries and versions (i.e. a specific snapshot or release) may not be obvious.
Why is my application using an old version of this library when I packaged a newer one with it?
Java EE defines class loading to follow a parent delegation model. What this means is that rather than load classes from the module you deployed, the runtime first delegates to the parent class loaders to see if they have the classes available and only if classes are not available does it load them from the deployed module.
Referring back to the “rapid growth in open source”, many Java EE runtimes also use the same open source projects in their implementation and so it is not uncommon for applications to pick up classes from the runtime that they expected to get from their module. If the runtime uses a different version of the library from the one packaged in the application, then this can lead to problems that are difficult to track down. One explanation may be that an application that worked for a long time breaks simply because a service upgrade to the runtime updated the runtime’s copy of a library to a newer version.
What version of this library is my application using?
The previous questions have alluded to library “versions”, but versions are not a native Java concept. It is therefore often difficult to determine what version of a library an application is using. One library may use a naming convention to describe its version, whereas another uses a file in the JAR. The JAR files themselves are just packaging artifacts and therefore cannot be relied upon from one release to the next. In fact, the JAR files can be different for different copies of the same library. In the “critical fix” scenario we mentioned earlier, it is not sufficient to find all copies of a particular library as the fix may only apply to a specific version of the library.
How WebSphere OSGi applications addresses these shortcomings
What follows are a few pointers regarding how OSGi — and the way its use has been enabled in WebSphere and Rational products — can help address the problems outlined above. It’s worth noting that this is not a Java EE vs. OSGi discussion; OSGi leverages many Java and Java EE technologies, such as servlets, JSP, JPA, JMX while addressing the issues outlined above.
OSGi gives your artifacts identity
OSGi is the dynamic module system for Java, defined by the OSGi Alliance. One of the key concepts of OSGi is the Bundle. An OSGi Bundle is essentially a JAR file plus some module meta-data. That meta-data is held in the JAR manifest and is described using headers defined by the OSGi Alliance.
|
|
Example OSGi Bundle Manifest for a Logging library
|
Two headers to note here are the Bundle-SymbolicName and Bundle-Version. Together these identify the bundle, so it should never be the case that two JARs with the same symbolic name and version are different (i.e. contain different code or resources).
The first thing we can immediately see is that with OSGi, we can uniquely identify what’s contained in the JARs that make up the libraries that we use. We can therefore answer any questions regarding the identity of the modules packaged in the application by looking inside each module’s manifest.
OSGi applications are assembled based on identities
Giving your artifacts identity does not address the problem that Java EE applications are still defined in terms of a zip of packaging artifacts (JARs or WARs within an EAR), rather than the real module identities that they are logically made up of. What WebSphere’s OSGi application model does is define an application in terms of a set of requirements for modules based on their identity. Better still, these identity requirements can allow for ranges of versions, thus enabling fixes to be applied to an application without having to create a whole new application definition. You no longer need to create a new application that is functionally identical to the existing one just to apply a fix. Below is an example OSGi application definition. This shows that applications have an identity, defined by the symbolic name and version. The application contents are also listed and allow a range of versions to be used (signified by the OSGi version range syntax “[x, y]”).
|
|
Example OSGi Application Definition
|
Once your applications are defined in terms of module identities, you can administer them around these identities and have a far better understanding of what you’re using. The screenshot below shows an example of a deployed OSGi application that lists the identities of the deployed content bundles.
 |
| Example OSGi application content bundle identities |
We can see from the above screenshot that when using the OSGi application model, it is very easy to determine the libraries and their versions that an application uses in its content. The screenshot shows how it is also possible to update the versions of the modules being used by using a “pull-down” to select version 1.3.1 of the logging bundle.
Bundle repositories enable a single source
Having an application definition that refers to identities rather than contained artifacts means you can be flexible when choosing where you source the actual artifacts. You may opt to include them in an application archive (OSGi applications use a zip file with an .eba extension), or, instead you could devolve responsibility for owning the artifacts to some other entity in the deployment process, such as a repository. By not requiring all application archives to contain the artifacts, you open up the possibility to have a single place from which to source each artifact. This means that you don’t need to ship around duplicate copies of the same artifact with each application that uses it.
WebSphere’s OSGi application support comes with a built-in internal bundle repository capability and also allows configuration of external repositories, based around the OSGi Bundle Repository (OBR) specification. When an OSGi application is deployed to WebSphere, these repositories are used to find the corresponding JARs for the module identity requirements listed in the application.
 |
Identification of JARs to deploy for an OSGi application
(Click to enlarge image) |
WebSphere keeps track of the applications that use bundles from bundle repositories. This information is held in its bundle cache. This gives administrators a place to immediately determine which repository bundles are in use and by which applications. Now if you need to apply a fix to an application, it is extremely easy to tell which applications need to be updated, and putting the fixed bundle in a repository provides a single place from which to apply that fix.
 |
Administrative view of bundles in use by applications
(Click to enlarge image)
|
 |
Administrative view showing the applications that use a
specific library bundle
(Click to enlarge image) |
Bundle repositories enable governance and collaboration
Earlier, I described the common need for third-party library governance to ensure libraries are appropriate for an organization to use. I also explained how having duplicate copies of these libraries scattered across a number of opaque applications can make it very difficult to ensure these governance rules are adhered to. I also described how this duplication results in wasted effort because different application teams perform due diligence on the same libraries, or ones which are subtly different when the same one would suffice.
Making bundle repositories available to the application development and operations teams helps address the duplication of effort but also provides a focal point around which governance can be applied and teams can collaborate. Bundles can be added to a repository and then taken through a life-cycle that indicates whether the bundles are considered experimental, pending approval, approved for production…or basically whatever process suits the way your organization works. Users of the repository can also collaborate around the bundles. For example, they could rate the bundles, request enhancements, report problems, ask and answer questions, and so on.
Enabling this form of governance and collaboration increases re-use by enabling the discovery of assets and the conversations that help shape their future. Increased re-use results in improved quality because fewer components are used by more consumers. It also reduces duplication of effort because increased awareness and the ability to influence existing libraries means teams are far less likely to create their own.
Rational Asset Manager (RAM) is on a path to provide such a governed, collaborative bundle repository. RAM 7.5.1 (Milestone 3 at the time of writing) has added OSGi bundles as one of the types of assets it understands. These bundle assets inherit the governance and collaboration capabilities that RAM provides for all asset types. RAM also has client integration into Eclipse and Rational Application Developer (RAD), enabling developers to search for bundles stored in RAM and download them, all from within their development environment. Finally, RAM 7.5.1 has added the ability to generate OBR repository definitions and can therefore be configured into WebSphere as an external bundle repository for application provisioning.
The diagram below shows an example RAM repository populated with a number of OSGi bundles. You can see that each bundle has a life-cycle state, a version, and can be rated. Package and service assets are also shown. These are automatically found in the bundles when they are added to the repository and linked to their providing bundle. This makes it very easy to search for packages and services to use (during development or deployment) and then find a corresponding bundle that provides them. RAM also provides the ability to comment on assets and create discussion forums (not shown in the diagram).
 |
RAM 7.5.1 populated with OSGi Bundles
(Click to enlarge image) |
Summary
In the introduction I identified a number of questions that development and operations teams often need to answer, as well as how Java and Java EE’s development and deployment models make answers to these questions difficult to find. I have explained how OSGi gives libraries an identity and how WebSphere’s OSGi application support leverages these identities to bring clarity and flexibility to application development and deployment. Finally, I have shown you how bundle repositories enable governance and collaboration around shared modules, thus improving quality, removing duplication of resources, and improving time-to-value.
So, if you’re deploying Java EE applications, struggling with development and operational inefficiencies, and not able to easily answer questions about the libraries you use, maybe it’s time to take a look at OSGi.
 Graham Charters is a Senior Technical Staff Member in the IBM WebSphere Application Server development organization. He is responsible for the OSGi Applications feature of the Application Server and a committer and PMC member of the Apache Aries OSGi programming model project. He is also the IBM technical lead in the OSGi Alliance Enterprise Expert Group.
|
Claus Jensen is a Senior Technical Staff Member on the Business Process Optimization (BPO) Foundation team and Chief Architect for SOA-BPM-EA technical strategy, driving their alignment and integration both conceptually and practically. The BPO Foundation team has architectural responsibility for all of IBM's software products to ensure that they support the key principles of SOA, Business Process Management (BPM) and Business Agility from a client perspective. 
