Category Archives: webmethods

Webmethods running linux commands as a java service

Recently a friend asked me how in webmethods, she could fire linux specific utility tasks. She wanted to find out if the disk space is full on linux. This can be done very easily by the “df- kh” command. But how to invoke this through webmethods?

Solution: Follow the steps below.

Step 1: Create a Java Service with the following input and output

webmethods-java-service-1

Step 2: Paste in the following code:

//—- BEGIN OF CODE
//Author: Hussain Fakhruddin
//Date: 20090203
IDataCursor cursor = pipeline.getCursor();
//Return if no command is specified…
if (!cursor.first(“command”)) return;
String output = “”;
try {
String command = (String)cursor.getValue();
String line=””;

//Creating a Process Instance and executing the command..
Process p = Runtime.getRuntime().exec(command);

// Capturing the output.
BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream()));
while ((line = input.readLine()) != null) {
output = output + line + “\n”;
}
input.close();
}
catch (Exception err) {
// Do something on error… may be throw a ServiceException
}
// setting the outputMsg for whatever output the command has given.
cursor.insertAfter(“outputMsg”, output);

//—–END OF CODE————

webmethods-run-external-command-or-programs

Step 3: Run the program and pass the unix/linux command

Note: DO NOT use commands which are not exiting after giving an output. For example “tail -f” . This service is only for commands like “df -hk” or “cp file1 file2” or “pwd” or “ls -lrt“.

If you face some issues, do feel free to write back.

Webmethods to TIBCO Migration

I’ve been getting a lot of hits and requests to post something about Webmethod to Tibco migration. As discussed in many of my previous posts, Tibco is highly marketing is products and particularly asking organizations to migrate from WebMethods!  You should see the way Tibco sales guys talk! They are technically convincing!

SO lets try to discuss if its really a good decision to migrate from Webmethods to TIBCO

Reason

You must have a reason to migrate. Don’t migrate just because someone told you. Or you are getting good Tibco resource available. There are plenty of resources for Webmethods too.

You must migrate because you find something missing in WebMethods. Or something which takes a lot of time in Webmethods than Tibco.

You must migrate if supporting an application in Webmethods is more difficult than webmethods.

Problems & Strategy

So lets say you have found your big reason to actually migrate from Webmethods to TIBCO. So what are the next steps:

1. Study the architecture. There is a huge change from Hub and Spoke architecture to Bus architecture. Webmethods runs an integration server. You can run each packages / services in the single server. But Tibco is Bus Architecture. Each package/service will run as a separate BWEngine. That means you’ll end up having a lot of system level processes on the server.

2. There will be no centralised management. In webmethods, you can share a lot of common services and call them from inside. But in Tibco you need to include the utility services as part of your EAR. Please note, while running things from Tibco designer, this is not the case.

This results in usage of extra memory. Lets say you have a lot of common business services, Its absolutely impossible to include each one of them as a library in your final EAR. Hence people tend to deploy these libraries as a separate BWEngine. Again a bigger problem is to do interprocess communication. For this EMS is used. A typical example is the TIBCO CUF(Common Utility framework ). All this becomes too complex for something simple.

Tibco Administrator is again single point of management for all TIBCO BWEngines and its again a good one but not quite complex as WebMethods’ Admin panel.

What TIBCO doesn’t give you: The JMS doesn’t even have a web based EMS Queue viewer… (I plan to write it someday! Or Tibco Guys should pay me for that! !)

Tibco Admion also doesn’t give you statistics about how much CPU is consumed or how much memory is being used.

However there is a workaround for that by using Tibco HAWK. Again another product that you may have to buy.

Supporting

While supporting Tibco applications, one cannot debug it unless the code is taken from the repository or installed on your system.
You will have to manage your Tibco code through your own repository. Applying patches in Tibco is also not very matured as Webmethods.

My Choice

I would still be reluctant to migrate to TIBCO. Not because I know Webmethods More than I know TIBCO, but because I feel webmethods is quite a good product. Tibco is more of a hype(I’m sure Tibco guys will hate me for saying this). But facts are facts. I get more stuff working and in an easy way in webmethods than TIBCO!

Reader’s Choice:

I would request the readers to post their own comments.

Webmethods:How to Kill a Running Service from the Integration Server

Many of you have wondered how to kill a WebMethods service while its running on the Integration Server.

Its become a Pet Interview Question for many but the interviewer also expects one of the following facts which is not the most recommended method to kill a running service.

Here are some facts:

1. If you are running the service on debug mode, the service will get killed if you kill your developer.
2. If you are running the service on RUN mode, the service runs on the Integration Server and will not get killed.
3. If your client has activated the service but terminated the session, the service will still run on the IS.
4. If your webmethod service is hung, You will try to Reload the entire package. But this will NOT help.
5. You will also try to unload the package but this will NOT help too.

What you need to do is on a Thread Level. The idea is quite simple: Just get the list of all the current running threads and kill the service which you need.

I will try to explain this in a step by step manner. Developers please do try it out on your dev boxes as well.

Step 1:
Create a Flow service like this:

wm1

Step 2:
Lets create a Flow service which will get ALL running threads. This should take NO input but lets output a String containing all running threads on that Integration Server.

w2

Step 3:

Insert the following Java Code in your WebMethods Flow Service that you just created. The code is commented well enough.

//—BEGIN CODE—–
// Author: Hussain Fakhruddin (hussulinux@gmail.com)
// Date: 20090128
//Get the Current Thread
Thread currentThread = Thread.currentThread();
//Get all Thread Groups.
ThreadGroup rootThreadGroup = currentThread.getThreadGroup();
//Traverse to the Top Thread Level
while (rootThreadGroup.getParent() != null) {
rootThreadGroup  = rootThreadGroup.getParent();
}
//Getting all Threads
Thread [] threads = new Thread[1000];
int threadCount = rootThreadGroup.enumerate(threads, true);
String threadList = “”;
// Now traverse through all the Threads and use the ServerThread API from webmethods to get all the thread detail.
int i=0;
for (; i<threadCount ; ++i){
if (threads[i] instanceof com.wm.app.b2b.server.ServerThread) {
//Casting a raw thread into ServerThread type
com.wm.app.b2b.server.ServerThread serverThread = (com.wm.app.b2b.server.ServerThread) threads[i];
//Getting the service Name for which the thread belongs to
if (serverThread.getState().getService() != null){
threadList = threadList + serverThread.getState().toString() ;
java.util.Stack threadStack = (java.util.Stack) serverThread.getState().getCallStack();
for (Iterator iter=threadStack.iterator(); iter.hasNext();) {
Object threadObj = iter.next();
threadList = threadList + “\n ” + threadObj.getClass().getName();
threadList = threadList + “, ” + threadObj.toString();
}
}
}
}
IDataCursor cursor = pipeline.getCursor();
cursor.insertAfter(“threads”, threadList);
cursor.destroy();

//—–END OF CODE—–

Step 4: Now lets create another flow service “killThread” with the following inputs and outputs

wm3

Step 5: Now copy paste the following code. We’re going to traverse through the entire list and call the .interrupt() method to kill the process. Note: you need to provide FULL path of the service which you want to kill. You can get the full path from the webmethods admin panel (under package management).

//—BEGIN CODE—–
// Author: Hussain Fakhruddin(hussulinux@gmail.com)
// Date: 20090128
IDataCursor cursor = pipeline.getCursor ();
if (cursor.first (“serviceName”)) {
String serviceName = (String) cursor.getValue ();
Thread current = Thread.currentThread ();
ThreadGroup root = current.getThreadGroup ();
while (root.getParent () != null) {
root = root.getParent ();
}
Thread [] threads = new Thread [1000];
int count = root.enumerate (threads, true);
String sb = “Not Found”;
StringBuffer killedList = new StringBuffer ();
for (int i=0; i<count; i++) {
if (threads[i] instanceof com.wm.app.b2b.server.ServerThread) {
com.wm.app.b2b.server.ServerThread serverThread = (com.wm.app.b2b.server.ServerThread) threads[i];
if (serverThread.getState ().getService () != null) {
java.util.Stack stack = (java.util.Stack) serverThread.getState ().getCallStack ();
for (Iterator iter=stack.iterator (); iter.hasNext ();) {
Object obj = iter.next ();
String name = obj.toString ();
if (name.trim().equals (serviceName)) {
threads[i].interrupt ();
sb = “Interrupted”;
killedList.append (stack.toString ()).append (“\n”);
}
}
}
}
}
cursor.insertAfter (“status”, sb);
cursor.insertAfter (“killedList”, killedList.toString ());
cursor.destroy ();
}
//—–END OF CODE—–

That’s it folks! Save and run it.. Come back to me if you face problems..

WebMethods Document Tracker for Broker analysis

Did you ever have the need to retrieve the documents from the broker without creating another client just for debugging? Try the WebMethods Document Tracker. Its simple and sweet!

It will just take the settings of your broker server and which publisher to subscribe to.

Its quite strong and can handle large documents at ease.

Some common features of the document tracker is as follows:

It can publish a document even with a complex data structure.

It has the ability to export all or selected documents from outgoing or Incoming panel.

It can give you published or delivery reports. Also it can give you statistics of how many documents published in a single go. Or how many currently getting published and delay between two publishes.

Its multi-threaded. This means that publishing and delivery is now in separate threads. This also means you have the ability to browse Incoming documents while doing a pub/sub. Further adding to this, we can even stop the document to be published in the middle. (Highly recommended for debugging/issue analysis at the broker level)

The interface is quite easy to use, although its java Swing with some MFC for windows.

Here are some screen shots for your eyes!

webmethods-document-tracker-pic1

The above screen shows the webmethods document tracker’s main screen. I have highlighted the panel where you can check out the incoming and outgoing documents. The subscribe tab allows you to subscribe to a document.

webmethods-document-tracker-select-broker

This one shows how we can select a broker to monitor. You can even have a secure connection to the broker by providing a certificate file.

WebMethods and TIBCO CUF – Common Utility Framework Services

The Common Utility Framework or CUF is a set of utility services created by any enterprise for their internal usage.

These utility services form the backbone and will provide a set of APIs to the applications which will be written for any enterprise solution.

The major advantage of using a CUF in your enterprise solution is to have a common standard implemented with respect to change of human resource in any organization.

Basic Features of CUF

A CUF should be responsible for everything which is used again and again and a programmer doesn’t like to code for it. All extra work which is most necessary but one doesn’t want to waste time in implementing it should go inside CUF.

CUF services would include:

Auditing (Transaction Logging)
Say you have a long process; you break it down and want to insert logging of the process as checkpoints. You really need some good utility to log all of this. Apache log4j is a good thing, but it needs to get into your CUF so that all other application can access it easily.

One can even log the following:
Complete Message Traceability : Track the entire message
Performance – Time spent in each transaction

Logging (General Logging)
You just need some general debug logging, or any other log.

You may want to put the logs on files or database. It depends on your organization.

There should have varied levels of log. How much data you want to capture?

This CUF service should also be responsible to change the levels of the logs at runtime

Exception Handling
There will always be happy path and sad path, what to do in that case? A common framework to handle common exception scenarios.

You may want to log these errors.

The best practice is to create a generic handler for all common exception.

Error Notification and Escalation
System Errors –Internal Errors, Mis-configuration, Database errors, Timeouts etc.
Business Errors – Component or Application Errors related to resources, processes, static data etc.

One should also keep in mind the following

Error levels – Critical, High, Medium, Low etc.
Alert levels – Whom to escalate, what level?

Escalations procedures – FIX, Root Cause and Prevention mechanism.

Message Repair:
Correct Missing Data – Human is notified, fixing and reprocessing work flow afterward

Reprocessing:
CUF should ideally be responsible or rather say intelligent enough to have reprocess logic
There should be different levels of retry:
For example –Retry from start, retry from last error checkpoint etc.

Medium:
What medium to use for notification?
Emails, Dashboards, alert screens, SMS
This should be configurable as per the interface
It should also be reliable

Initialization and Cleanup

Initial setup on start up
To reserve some memory
To initialize a data feed.
To initialize a pool of objects etc.

Locking mechanism: A set of common locking mechanism for shared services.
To ensure only one instance of the service is running
To ensure concurrency at service call level.

Cleanup of memory
To ensure your service terminates properly
To ensure you release all garbage data and clear unused memory

Custom garbage collectors.
May be a custom garbage collector on the service level.
For example, if a service has been running for more than X hours. Where X is max limit, it should get killed.

Shared Resources:

Database Pooling:
A common database pool of database and adapters which can be used as shared resources.

Middleware:
A message driven middleware or JMS/EMS shared adapters to PUB-SUB messages.

Service Management:
Manage service calls, Thread Management, Interrupt and reset shared object pools.

As a conclusion –A CUF should be responsible to make life easy.

There are hard and fast rules or concrete standards that organizations should follow but the above points should help you getting started to have CUF implemented at your enterprise.

webMethods Vs TIBCO

WebMethods and Tibco are the two most popular Enterprise Application Integration tools in the market today.  Lets look at some technical details to bring out the differences between them.

If you understand EAI, you’ll probably understand Hub and Spoke Architecture and Bus architecture.

1. Hub and Spoke Vs Bus Architecture

WebMethods is Hub and Spoke Architecture and Tibco is Bus architecture.
Hub and spoke is some thing where you have a centralized server, and all the data lines extending from it like spokes whereas

Bus architecture is like a data bus connecting and serving data to all the applications connected to it. One has to imagine many practical scenarios before one can perfectly avoid confusion between the two.

2. Broker Vs EMS
WebMethod has a product called Broker Server. This is their way of implementing JMS ‘kind of stuff’. I say ‘kind of’ because it can be used easily only with webMethods Integration server. Other data can be exchanged only through some extra wrappers/adapters.

On the other hand Tibco came out with a complete 100% JMS implementation which they say is written in C(and they also claim that because its written in C, its faster – I have doubts). The product is called EMS. One can create queues and publish and subscribe to it.

3. Document Vs XML

WebMethods internally stores each structure as a document. The broker can understand only documents. This means that if you have an XML , you’d first need to create a document out of it and then will be able to use it on their brokers(off course they provide API for conversion)

Tibco is quite good at these. You can send and receive just any XML or data by defining a schema. Hats off on that..

4. Packages Vs EARs

In Webmethods, each module or project is represented as a package. Each package may have functions called flow services. These can be invoked by other packages residing on the same Integration server. A package can contain their own flow language code(which is in XML). This gets converted internally into Java code. A Java service will have pure Java code.

Tibco deployment is more like a Enterprize archive(EAR). You need to pack all your compiled code and then deploy to the server(called BusinessWorks Engine).
This can sometimes be problematic because we have to have our own version control system, whereas webmethods has it inbuilt.

5. Integration Server Vs BusinessWorks

Webmethods has a centralized server where all the code is deployed. This is called an Integration Server. An integration server will also contain their built in services.
Tibco on the other hand has the product called BusinessWorks. EAR files are deployed on the BW engines to work out the trick.
At the end, I feel both are nothing but tweaked tomcat servers!!!

6. Services Vs Pallets.

A service in WebMethod , be it built-in or user built is a small function of a package(think of it as a class). There is a set of complete startup utilities which WebMethods provides. These are called Built-in Servcies

Tibco has a concept of pallets. These pallets are also some small function. But the implemennntation is done in just one city!

7. Triggers Vs Queues

Whenever a data comes to the Broker from an Integration server or rather say, webmethods Broker receieves a Queue, then a Trigger is called. These triggers in WebMethods do nothing but call another service[built-in or custom built].

JMS implementation of TIBCO called EMS does a delivery of the message(JMS message -Remember it can be anything!) to all the clients who have subscribed to the Queue.

8. Developer Vs Designer

Developer is for Webmethods. Its a java (old fashioned look, revamped in new version) based IDE which connects to IntegrationServer and allows you to ‘drag-n-drop-programming’. It gives you options to watch variables, debug, step through etc.

Tibco Designer – This is also a Java based tool which allows you to create your processes. You have have predefined template (basically a folder structure) for a new project and build everything accordingly. It gives you options for testing/debugging with breakpoints etc.

9. Flow Language Vs Flow Charting

Webmethods uses a language called Flow Language. A flow language consists of simple programming constricts like looping, branching, breaking etc. A flow service is written in Flow Language and comprises of Flow Steps(instructions ). Everything is drag and drop with some basic property setting such as Input/Output and pipeline variables.

Tibco implements everything like a flow chart. There is a canvas where start and stop end points are given. We can drag and drop existing processes and connect them with flow lines. These flow lines carry data with them in its pipeline kind of implementation.

It has to be noted that Webmethod’s earlier version (say around 3-4) had a similar interface what Tibco has it today. Further to that, Webmethods 7 interfaces have changed a lot from webmethods 6.

10. Software AG Vs Tibco Software

Software AG acquired Webmethods and has a revenue around 200mn. Whereas Tibco has a revenue of 500mn. (Figures from wikipedia).
I personally believe that Tibco markets its product quite well. They also have a good grip on open source community by sponsoring DWR and TIBCO GI projects. Current trend goes towards Tibco because of their wide range of products and excellent marketing. But webmethods is not a bad software too!  There are more resources available for webmethods (advantage website) than Tibco. I strongly believe one doesn’t need an official training to get started(offcourse yes! for advance stuff) but you’ll find more webmethods trainers than Tibco in the market.
Support wise, I feel both are same – somewhat useless. Support now a days is just the name! But both release patches etc etc.
Implementation – Depends on how complicated you want things to be! (So, equal points to both here as well)
Cost – Depends again on the deal your crack with those guys. They don’t release the prices on their sites, which means they see you first(check your spending capability) and name the price!!

Overall , both middle wares are quite a good competition to each other. There are no translators available between the two for migration. For your company to choose which one to implement: Consider the following factors:

1. Cost – The deal you crack with them?
2. Products needed?
3. XML based or can work with Documents ?

Factors which shouldn’t affect your choice.

1. Cool and convincing marketing
2. Myths like Tibco is next gen tool
3. Myths like WebMethods is age old and time tested
4. One performs better than other – Trust me , Ive worked with both and performance is programmer’s responsibility not framework’s!