Web Services and IT Management

August 18, 2005
Volume 3, issue 6

Download PDF version of this article PDF

Web services aren’t just for application integration anymore.

PANKAJ KUMAR, HEWLETT-PACKARD

Platform and programming language independence, coupled with industry momentum, has made Web services the technology of choice for most enterprise integration projects. Their close relationship with SOA (service-oriented architecture) has also helped them gain mindshare. Consider this definition of SOA: “An architectural style whose goal is to achieve loose coupling among interacting software agents. A service is a unit of work done by a service provider to achieve desired end results for a service consumer. Both provider and consumer are roles played by software agents on behalf of their owners.”1 Although SOA doesn’t mandate Web services, its emphasis on loose coupling requires use of something with the characteristics of Web services.

The real power of SOA and Web services becomes apparent when various constituents are added, removed, replaced, or upgraded without adversely impacting the whole system. This is just not possible when each part of the architecture relies on intimate knowledge of the inner workings of every other part and shares code, in the form of language-specific libraries, for processing messages.

IT management is a problem well suited for SOA- and Web services-based solutions that will allow products from different vendors, running on different platforms, to work together. Despite attempts to standardize on a small number of hardware and software systems, a typical IT infrastructure is sufficiently heterogeneous to make management a horrendously complex task. Standard management interfaces such as SNMP help in health monitoring and simple control tasks, but fall short in addressing complex management tasks such as configuring and controlling complex software applications. SNMP’s shortcomings include its iterative approach to access management data (using GET and GET NEXT), causing excessive network traffic and performance bottlenecks; lack of support for sophisticated system configuration and control tasks; inability to specify filtering at the data collection site; low-level programming required for instrumentation; and so on. Although a number of solutions, both standards-based and proprietary, have emerged to fix these problems, none has been widely adopted.

Alternatives such as a single product or a suite of products from a single vendor that addresses all management needs (such as Microsoft Office) are not feasible within the complex IT infrastructure of today’s enterprises.

Current Work in the Industry

At the basic level, Web services technologies—HTTP(S), XML, XML-Schema, SOAP, WSDL, and, to a lesser extent, UDDI—include standards for transport, messaging, interface description, and discovery. These standards do not define programming language APIs but enable exchange of information by defining syntax and semantics of messages that flow over the wire or are stored on disk.

A typical IT management system consists of one or more managers (in this context, manager is not a person, but a software entity) talking to a number of agents. The agents either interact with the resource to be managed or are embedded within the resource, as shown in figure 1.

Within SOA for IT management, it is reasonable to think of each managed resource to be proxied by an identifiable Web service and expose the manageability interface as a Web service interface. Within this framework, an agent is responsible for hosting the Web service representation of the resource. This architecture is very similar to accessing a resource (i.e., a Web page) where the Web server provides transparent access to uniquely identifiable resources.

Although the basic standards referred to earlier are adequate for a variety of SOA solutions, IT management has some special needs that these standards do not meet:

The Web service representing an IT resource should be able to send alerts or notifications to those managers that have expressed interest. Another way of saying this is that these Web services should be able to support not only a request-response pattern but also a publish-subscribe pattern of interaction.
It should be possible for a manager to retrieve the state of the underlying IT resource, represented by a collection of named and typed properties, from the corresponding Web service in an efficient manner.
Traditionally, the Web services have been considered permanent, meaning their lifetime is essentially infinite and the clients do not have to worry about when a service comes into existence or when it goes away. This means there is no standard way of notifying clients of life-cycle events such as creation, upgrade, and destruction. Real IT resources, however, are commissioned and decommissioned on a need basis; hence, their Web service representations also have to be ephemeral.

These aspects became clear during discussions by the WSDM-TC (Web Services Distributed Management Technical Committee), an OASIS group chartered to create specifications for management using Web services and management of Web services.2 Note the use of using and of in the previous sentence; we will revisit this in what follows. It was also clear that, although related, management of Web services is very distinct from management using Web services. In fact, the WSDM-TC eventually came out with two specifications: WSDM-MUWS (Management Using Web Services) and WSDM-MOWS (Management of Web Services).

This article is primarily concerned with WSDM-MUWS.

Before addressing how this specification satisfies the IT management requirements, it is worthwhile to take a look at parallel, but very similar, developments in the grid community,3 where the primary motivation was to develop a set of specifications to provision and use resources from a semi-autonomous pool or grid. Although initial grid development predated popular Web service specifications, subsequent work in Web services has converged Web services and grid services in the WSRF (Web Services Resource Framework).4 This was accomplished by defining precise methods to deal with state and lifetimes. WSRF is a set of five different, but interlocking, specifications. Additionally, there was a set of new specifications called WSN (Web Services Notification)5 to support the publish-subscribe pattern of message exchange.

Note that WSRF, with its support for exposing properties of a resource and life-cycle operations, and WSN, with its support for subscription-based notifications, meet the IT management requirements identified earlier.

The development of WSRF and WSN was prompted as much by the desire to make grid services become normal Web services as by the need to create a Web service infrastructure layer that can be used by other specifications such as WSDM. In fact, the development of these specifications was highly influenced by IT management requirements worked out by WSDM-TC, further proof that the requirements of exposing state, life-cycle, and supporting publish-subscribe were not unique to the grid.

WSDM-MUWS uses WSRF to expose resource state and life cycle, and WSN for asynchronous notifications. Furthermore, it defines a number of IT management-specific resource properties and topics that can be used for exposing a manageability interface, as illustrated in figure 2.

Relationship with Other Standards

A number of IT management standards have been developed over time, besides the venerable granddaddy of network management standards, SNMP. These include CIM (Common Information Model)6 and JMX (Java Management Extension)7, among many others. CIM defines abstractions for modeling IT systems and has created models for a number of IT domains. JMX defines the management architecture for Java-based systems and MBeans, an API for instrumenting Java applications.

How does WSDM-MUWS relate to these standards? JMX and CIM individually define complete management technology stacks, whereas WSDM-MUWS is concerned only with what goes on the wire. The expectation is that one would be able to use WSDM-MUWS along with pieces of other standards to create complete solutions. For example, it should be possible to expose a CIM-specified model through WSDM-MUWS. Similarly, it should be possible to use JMX as the instrumentation API for Java programs. In fact, work is going on in both the DMTF8 and Java Community Process9 to make such integration a reality.

The previous discussion should not imply that any CIM model or JMX-instrumented system could automatically be exposed using WSDM-MUWS without loss of information or capability. CIM models are object-oriented, and these model objects do not always map well to services, the unit of management in WSDM-MUWS.

A different kind of issue arises in exposing JMX MBeans through WSDM-MUWS. JMX specifies only an API, not elements of a model. For example, you could create an MBean with an attribute to represent number of requests processed so far, but you can’t mark it as a metric with information about the last reset time. As a result, it is much harder to construct WSDM-MUWS-compliant manageability interfaces in an automated way.

Because of the way object-oriented software principles are used to develop Web services, it is possible to use CIM and JMX for WSDM-MUWS-based IT management solutions. This usage, however, is not about automatic transformation of any existing CIM model or JMX-instrumented application. That would be akin to converting objects to Web services, and we all know that such conversion, although feasible, doesn’t result in many benefits of service orientation. What makes sense is to:

Use CIM abstractions to create a model that is best suited for WSDM-MUWS, and, hence, fit to be an SOA component. For existing models, one could create adapters around objects or aggregations of objects that qualify to be exposed as Web services. These adapters could either be model-specific or generic, taking model-specific transformation rules as input.
Use JMX API to create MBeans that are best suited for Web service representations of resources. Similar to the approach for CIM models, existing JMX MBeans may be adapted through suitable wrappers to create WSDM-MUWS-compliant Web services.

Of course, many people would like existing instrumentation to be automatically converted, on the fly, to be WSDM-MUWS-compliant. Complex technical issues make such automatic conversion very hard, at least with the current state of various specifications. Future developments should make it much simpler to use parts of different technologies for creating solutions.

LESSons learned

A number of core ideas that have shaped WSRF, WSN, and WSDM specifications came from early grid work and WSMF (Web Services Management Framework), a set of specifications developed by HP for management of, and using, Web services.10 WSMF has been used to create HP OpenView Smart Plug-ins with a number of software applications from vendors such as Tibco and BEA.11,12

At HP we are also working on a number of Apache incubation projects to create open source implementations of WSRF, WSN, and WSDM specifications.13,14,15

Our involvement in these Web services-based IT management projects has led to a set of best practices. These do not need to be adhered to under all circumstances, but they are principles that deserve some attention.

Service orientation has to be properly thought through. Doing a blind conversion of existing APIs to WSDL-specified Web services may not be adequate or desirable in all cases. For example, while creating the Smart Plug-in for BEA WebLogic Integration, we started with the premise that each WebLogic Integration MBean can be mapped to a WSDM WS-Resource. These MBeans, however, were designed to serve as an object-oriented API for building a management console and didn’t always correspond to resources we wanted to manage. Also, the MBean methods used reference to remote objects, making mechanical mapping of MBean to WS-Resource very hard.

This is not to say that replacing a platform- or language-specific remoting technology, such as RMI, DCOM, or CORBA, with Web services has no advantage. In fact, it is the first step and ensures that interacting components are not tightly coupled and can be written in any language and run on any operating system. To achieve real service orientation, it is important to make sure that:

Messages represent access of service (and not low-level interactions).
Component interactions are coarse-grained.
Components do not participate in ACID (atomicity, consistency, isolation, and durability) transactions.

Design for evolution. A number of Web service specifications are still evolving and you must account for this during design and implementation. WSDM 1.0 and a number of specifications that it refers to are likely to be revised. The pragmatists among us may want to wait until the dust settles. For innovators and early adopters, however, waiting for a specification to gain wide adoption is often not a choice. The best thing is to account for the fact that the specifications will change after implementation and be ready for the eventual change.

An important aspect is to allow a client program to work with multiple protocols and even different versions of the same protocol. We learned this lesson the hard way. Our WSMF-based implementations required a good amount of reengineering to support WSDM 1.0.

Now that we have learned this lesson, the WSDM 1.0-based implementations would be able to support later versions with very little architectural change.

Write wrappers (rather than doing a complete reimplementation). This goes along with the previous point about design and implementation that can be quickly adapted for a change in specifications. Writing wrappers gives the flexibility to replace them as things change or to support multiple interfaces with multiple wrappers. These wrappers could be in-proc or out-proc, depending upon the particulars of the component.

We adopted this approach for creating WS-Resources corresponding to managed resources of WebLogic Integration as wrappers around a collection of existing MBeans. This significantly reduced the development time, compared with a native instrumentation by modifying WebLogic Integration sources, albeit at the cost of some runtime performance overhead.

Handle failure conditions gracefully. A Web services client cannot assume that all services it interacts with are available all the time or can service its request in a timely manner. Hence, its design must account for the fact that services can come and go at arbitrary times or take a long time to service a particular request. Even in such scenarios, the client should be able to work, either by interacting with other services or at reduced functionality. It is not acceptable for a client to wait indefinitely, or hang, while expecting a response from a particular service.

When a Web browser times out fetching a Web page or stumbles upon a dead URL, there is a human user to recognize the situation and take a corrective action. Not so with a Web services client. The recovery logic must be built into the client itself.

Create models. Components participating in IT management solutions often exchange a variety of information with each other. It is important for this information to conform to information models that are well understood by all those who need to process it. Wherever possible, it is preferable to use existing models, such as those created by DMTF or other such organizations.

Also note that WSDM-MUWS, with its emphasis on representing managed resources as individually addressable and identifiable Web services, does an acceptable job of representing the model of all such managed resources, their properties, and relationships, but offers little for exchanging models held within a managed resource.

To understand this, let us go back to the WebLogic Integration example. In this solution we treated business process types as managed resources, each type corresponding to a WS-Resource. The process nodes and their relationships defining the control flow within a process type were represented through a custom XML document.

Realize that you are an early adopter of technology. Web services have a lot of potential for solving IT management problems, but parts of the technology have yet to mature. Therefore, it is important to set the expectations appropriately and apply the technology to areas where the advantages outweigh the risks.

We recommend areas where traditional solutions fall short and Web services offer significant advantages: distributed applications with proprietary management interfaces; multilanguage, multivendor IT management integration projects; and so on.

Not surprisingly, most of the lessons learned appear to be the same ones that apply to any bleeding-edge software development/deployment project. After all, why should IT management software be different from any other kind of enterprise software?

References

He, H. 2003. What is service-oriented architecture? http://webservices.xml.com/.
WSDM-TC; http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsdm.
Global Grid Forum; http://www.ggf.org/.
WSRF-TC; http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsrf.
WSN-TC; http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsn.
DMTF; http://www.dmtf.org/.
Java Management; http://java.sun.com/products/JavaManagement/.
WS-CIM; http://www.dmtf.org/newsroom/newsletter/2004/08/page6.
JSR addressing WSDM adapter for JMX; http://www.jcp.org/en/jsr/detail?id=262.
WSMF specifications; http://devresource.hp.com/drc/specifications/wsmf/index.jsp.
HP OpenView Smart Plug-in for Tibco; http://www.managementsoftware.hp.com/products/spi/spi_tibco/.
HP OpenView Smart Plug-in for BEA WebLogic Integration; http://www.managementsoftware.hp.com/products/spi/spi_beawi/index.html.
Apache Project for WSRF implementation; http://incubator.apache.org/apollo/.
Apache Project for WSN implementation; http://incubator.apache.org/hermes/.
Apache Project for WSDM-MUWS implementation; http://incubator.apache.org/muse/.

PANKAJ KUMAR is a software architect and technical lead for Management Software Business at Hewlett-Packard and works with partners on creating HP OpenView-based management solutions using Web services. He has more than 16 years of industry experience and is author of the book J2EE Security for Servlets, EJBs, and Web Services (Prentice Hall, 2003). He has a B.Tech. in computer science from Indian Institute of Technology, Kanpur.

Originally published in Queue vol. 3, no. 6—
Comment on this article in the ACM Digital Library