Logic

Of all Mathematics I learned at my undergraduate courses I value most the simple but powerful two logics which I consider are very useful in understanding the world.
They are,

1. x ⇒ y (x implies y or if x then y)
2. x ⇔ y ( x if and only if y)

These two logics apply in most of the things we see/hear in our every day life. To understand them one need not have great learning on Mathematics or Logic. It is very intuitive, yet very subtle in their application. Some times it may seem too simple. But when you think carefully you understand that you need to be very careful in applying them with facts.

First simply mean that if x is true then y must be true. It is that simple. The only fact we know is that if x is true then y must be true. Take x and y to be any two facts that you face in life. For example let x be the fact that you buying product O from company M and y be the fact that you get discount in buying O.

So here our expressed logic is

If you buy O from M then you will get discount.

This could be any marketing propaganda that you are targeted at in your every day life. So what do you think?. What if you don’t buy O?. Then you won’t get discount?. The answer is, If you don’t buy from M, there is no more implication about your discounts with regard to buying from M. Of course you may still get discount by buying from some other company. May be not. We cannot come to any precise conclusion about your discounts from the given logic.

x ⇒ y is not equalent to (not x) ⇒ (not y). But sadly this is something that most people take without giving proper heed to the facts.

But here is an equivalent logic

x ⇒ y  is equivalent to (not y) ⇒  (not x)

So under the context of our example we can say that

If you are not getting discount in buying your O  then you haven’t bought O from M. Yes this is true.

So what if x ⇒ y and (not x) ⇒ (not y) are both given as true logic?

Look carefully. (not x) ⇒ (not y) is equivalent to

(not (not y)) ⇒ (not (not x)). What does that mean?. It is simply y ⇒ x.

So we have both x ⇒ y and y ⇒ x. In other words x if and only if y. This logic is defined with the symbol x ⇔ y.

So when this apply to our example it says

If you buy O from M then you get discount and if you get discount then you have bought it from M. So in this case it is to be understood that only M company gives discount for O and no other company in this world give a discount for O. If some company express such logic it should be taken as very strong claim. You get discount for product O if and only if you get O product from our company.

When reading anything, hearing anything I tend to think precisely along these logics. In that way I believe I get no more than it really mean, and no less than it really mean. This result in very little chance of getting cheated at least.

Plot diagrams with gnuplot

I needed a good tool to easily map the results from my performance tests into human readable graphs. After much R&D I finally settled with the tool called gnuplot. Earlier I used openoffice chart tool for drawing simple charts. But it has limited capabilities and not so easy to customize. With gnuplot I could easily draw my histgrams just by giving all data in a file and giving all commands in a separate file. Following I explain how.
First install gnuplot on your working machine. In ubuntu

apt-get install gnuplot

Then I created my command file called data.p

# set terminal png transparent nocrop enhanced font arial 8 size 420,320
set output 'mygraph.png'
set xlabel "Message Type"
set ylabel "Requests per second"
set boxwidth 0.9 absolute
set style fill solid 1.00 border -1
set style histogram clustered gap 1 title offset character 0, 0, 0
set datafile missing '-'
set style data histograms
set xtics border in scale 1,0.5 nomirror rotate by -45 offset character 0, 0, 0
set xtics ("echoDoubles" 0.00000, "echoInts" 1.00000, "echoMeshInterfaceObjects" 2.00000, "echoSimpleEvents" 3.00000, "echoStrings" 4.00000)
set title "Large Data Sets(500 elements)\nResults from Apache Bench"
set yrange [ 0.00000 : 700. ] noreverse nowriteback
plot 'mydata.dat' using 2:xtic(1) ti col, '' u 3 ti col

It easy to understand the commands by following gnuplot tutorial

Then I created my data file called mydata.dat

# Large Data Sets(500 elements)
#
Region Axis2/C Waspc
echoDoubles 340.07 273.04
echoInts 452.17 318.69
echoMeshInterfaceObjects 162.90 79.40
echoSimpleEvents 101.16 68.78
echoStrings 542.96 270.54

Note that commented lines are interpreted as comments by gnuplot. Data columns are separated by tabs or spaces but not by colons or any other delimiter.
Now execute command gnuplot and go into the gnuplot command line interface. Now from within gnuplot execute

gnuplot> load ‘./mydata.p’

which will draw the following histogram

This approach is very easy because each time I update my performance test results I just need to change the data in my .dat file and after that just execute gnuplot to redraw the graphs.

WSO2 Web Services Framework for C++ (WSO2 WSF/C++) 2.0.0 Released

I’m very much pleased to acknowledge the WSF/C++ release from WSO2. This is a long felt need for the WSO2 web services stacks in C/C++. As I have pointed out in my article comparing WSF/C and gSOAP as well as article compaing WSF/C and RogueWave’s HydraExpress the main minus point for WSF/C stack was its lacking of C++ API level support. Even at the time I was compairing these stacks WSF/C++ 1.0.0 was available but without code generation support and server side support. WSF/C++ is a fully featured release with API’s for writing services as well as complete code generation support. So now I can speak of WSF/C++ as a complete web services stack for C/C++ web services development with the added advantage of providing C++ support over the already feature rich WSF/C stack.
While using the C++ API’s for writing web services, developers can always exploit the underline WSF/C core API’s to his advantage.

When faced with flat tire

How many of you like to have your tire completely flat at an isolated area with 15km back to the previous town and 15km to the next town?. Well it happened to me yesterday on my bike where there is no spare wheel for rescue. I inspected the flat tire but could not find any nail or nothing struct to the tire.
For a long time I kept a flat mate can in my traveling bag in addition to the necessary tool set. I have never tried it and in fact did not have much faith in it. Since there was no alternative I tried it. Bravo, it fixed the thing. It emits a white creamy rubber solution into the tire while inflating the tire with pressure. The cream is supposed to fix all the punctures.
The instruction on the bottle says to travel slowly and fix the puncture within next 15-20km. In the next town I stopped at a tire service place and they said no need to do a separate patch and asked me to continue without any doubt. I traveled about another 30km in high speed and still the tire goes strong. I bought the can for about 800Rupees and it rewarded me at last. Even in a car this is good thing to keep stand by because it prevents you from all the hassle of changing wheels until you go to next service station.

Benchmark testing with httperf

In a previous blog I discussed benchmark testing tools I used for some benchmark perposes. At that time I really used Apache Bench(ab) for the purpose. However when I am again doing some bench mark testing I stumbled upon a new tool(at least for me) httperf which seems to be more advanced and reliable than ab. Although there were ample guidance on how to use this for http GET requests I could not find any easy to read guide on how to use httperf with http POST requests. Following I try to explain how httperf could be used in testing my services.

httperf --server localhost --port 9090 --uri /axis2/services/weather
     --num-conn 4000 --num-call 16 --rate 64 --timeout 5 > report

Above command is a simple scenario where I request a HTTP get request on a servic running on context /axis2/services/weather on localhost.

It is important to understand the  options for the command.

1. –server – ip of the machine service is running

2. –port – port the service is running

3. –uri – The context path of the service on the server

4. –num-con – Number of test calls made to the service

5. –num-call – Number of calls per tcp connection.

6. –rate – Number of connections created per second to make requests to the service.

7. –time-out – This is the maximum time httperf wait for a successful response.

It is important to understand the correct balance between num-conn, num-call and rate options.

The maximum number of requests the httperf would generate = num-call * rate

Clearly your num-conn could be reasonably higher than this number. You could experiment between num-call and rate to produce a good load to the server. The actual number of requests to the server you will see in the generated report could be less than this number depending on the saturation of the server, client or network bandwidth. If such thing happen you need to reduce your num-call or rate options and find out at what point exactly saturation happen. If your num-conn is very large your server may exaust all servicing processes after some time.

Now let’s look at a more advance scenario. Here I am sending a http post request with some payload defined in the file name inputfile.

httperf --server localhost --port 80 --uri /axis2/services/weather --hog --method POST
    --add-header="Content-type:application/soap+xml;charset=UTF-8``\n``"
    --wsesslog=4000,0,inputfile --max-piped-calls 16 --rate 64 --timeout 5  > result

In adition to the payload you can also specify the http method and the server context uri in this input file. For more descriptions on writing this input file read [1]. Following is the content of the inputfile I used.

/axis2/services/weather method=POST contents="<soapenv:Envelope xmlns:soapenv=
\"http://www.w3.org/2003/05/soap-envelope\"><soapenv:Header xmlns:wsa=
\"http://www.w3.org/2005/08/addressing\"><wse:Identifier xmlns:wse=
\"http://schemas.xmlsoap.org/ws/2004/08/eventing\">
urn:uuid:57becf58-0fc3-1de1-24ad-0012f0b857ac</wse:Identifier>
<wsa:To>http://localhost:9090/axis2/services/weather</wsa:To>
<wsa:Action>http://schemas.xmlsoap.org/ws/2004/08/eventing/GetStatus</wsa:Action>
<wsa:MessageID>urn:uuid:14d256a2-0f95-1de1-2b1a-0012f0b857ac</wsa:MessageID>
</soapenv:Header><soapenv:Body><wse:GetStatus xmlns:wse=
\"http://schemas.xmlsoap.org/ws/2004/08/eventing\"></wse:GetStatus>
</soapenv:Body></soapenv:Envelope>"

The important additional options of the above command is
1.–wsesslog – This value for this option is a three comma separated parameters. First one is the number of request sessions you need to make to the server. Second is used to emulate a real user delay between sessions. Third is the inputfile.

2. –max-piped-calls – Number of calls need in one tcp connection(in http 1.1). This is used instead of num-call here.

[1] http://www.xenoclast.org/doc/benchmark/HTTP-benchmarking-HOWTO/node6.html

What is good for Axis2/C pthreads or cbthreads?

Dinesh Premalal in his blog post on Axis2/C with cbthread explain that using cbthreads he ran about 100,000 concurrent requests using Axis2/C client. The reason for this achievement is because of the non-preemptive and continuation-based threads implemented with cbthreads. However in his next blog Axis2/C with pthreads he goes on explaining how he used similer Axis2/C client with pthreads and could only run about 300 concurrent requests. This is predictable because each thread in pthreads consume considerable amount of memory in it’s stack-based nature implementation. In adition Dinesh’s program for pthreads does not use a thread pool which cause the the program to create threads until system resources exhausted.

In cbthreads home page it is mentioned

“A downside of having a non-preemptive thread system is that you cannot assign threads to blocking system calls as you can with pthreads, and have threads run while the system call blocks. If you do, the entire process blocks.”

This reason highly discourage using cbthreads for Axis2/C. Besides pthreads is provenly portable among numerous platforms. In my opinion what Axis2/C needs is a good thread pool implementation using pthreads.

In search of a good thread pool using pthreads I came across this one which seems to be a good candidate. I wrote a similer client as Dinesh wrote for testing cbthreads using this pthreads thread pool.

Axis2/C client With pthread threadpool for 10,000 request
real 1m9.974s
user 0m34.890s
sys 0m9.297s

Axis2/C client with cbthreads for 10,000 requests
real 3m29.835s
user 0m13.621s
sys 0m8.857s

This shows that using pthreads with a good thread pool is the right way to use pthreads for Axis2/C with it’s all proven features.

To try this with Axis2/C echo.cpp client program do the following steps.

1. Download the pthread pool from here and build the source by executing make in the root source folder.

2. Install Axis2/C. I assume you installed it into /axis2c/deploy.(You need Axis2/C 1.6 or build it from svn source)

3. Copy the following echo.cpp source file and build.sh build script into the same source folder.

4. Execute build.sh and run the exectuable program.

One thing to notice about the echo.cpp code is creating service client for each thread. Main overhead of creating service client is associated with creating the Axis2/C main cofiguration context. This is reduced by creating each service client passing the same configuratin context as following.

conf_ctx = axis2_svc_client_get_conf_ctx(svc_client, env);
svc = axis2_svc_client_get_svc(svc_client, env);
svc_client = axis2_svc_client_create_with_conf_ctx_and_svc(env, client_home, conf_ctx, svc);

echo.cpp

#include <TThreadPool.hh>
#include <stdio.h>
#include <stdio.h>
#include <axiom.h>
#include <axis2_util.h>
#include <axiom_soap.h>
#include <axis2_client.h>
#include <axis2_conf_ctx.h>
#include <axis2_svc.h>

#define NTHREADS 10000
#define POOLSIZE 100

axiom_node_t *
build_om_payload_for_echo_svc(const axutil_env_t * env);

typedef struct _svcinfo {
    const axutil_env_t *env;
    axis2_svc_client_t *svc_client;
    axis2_char_t *address;
}svc_info_t;

static void svc_client_send_request (void *v);

class echo : public TThreadPool::TJob
{
public:
   echo ( int id )
       : TThreadPool::TJob( id )
   {
   }

public:
  void run ( void * arg )
  {
      svc_client_send_request(arg);
  }
};

int
main(int argc, char **argv)
{
    const axutil_env_t *env = NULL;
    const axis2_char_t *address = NULL;
    axis2_endpoint_ref_t *endpoint_ref = NULL;
    axis2_options_t *options = NULL;
    const axis2_char_t *client_home = NULL;
    axis2_svc_client_t *svc_client = NULL;
    int i = 0;
    TThreadPool * pool = NULL;
    axis2_conf_ctx_t *conf_ctx = NULL;
    axis2_svc_t *svc = NULL;

    /* Set up the environment */
    env = axutil_env_create_all("echo.log", AXIS2_LOG_LEVEL_TRACE);

    /* Set end point reference of echo service */
    address = "http://localhost:9091/axis2/services/echo";
    if (argc > 1)
    {
        if (axutil_strcmp(argv[1], "-h") == 0)
        {
            printf("Usage : %s [endpoint_url]\n",
                   argv[0]);
            printf("use -h for help\n");
            return 0;
        }
        else
        {
            address = argv[1];
        }
    }
    printf("Using endpoint : %s\n", address);

    /* Set up deploy folder. It is from the deploy folder, the configuration is picked up
     * using the axis2.xml file.
     * In this sample client_home points to the Axis2/C default deploy folder. The client_home can
     * be different from this folder on your system. For example, you may have a different folder
     * (say, my_client_folder) with its own axis2.xml file. my_client_folder/modules will have the
     * modules that the client uses
     */
    client_home = AXIS2_GETENV("AXIS2C_HOME");
    if (!client_home || !strcmp(client_home, ""))
        client_home = "../..";

    /* Create service client */
    svc_client = axis2_svc_client_create(env, client_home);
    if (!svc_client)
    {
        printf
            ("Error creating service client, Please check AXIS2C_HOME again\n");
        AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI,
                        "Stub invoke FAILED: Error code:" " %d :: %s",
                        env->error->error_number,
                        AXIS2_ERROR_GET_MESSAGE(env->error));
        return -1;
    }

    /* Set service client options */
    axis2_svc_client_set_options(svc_client, env, options);

    /* Engage addressing module */
    axis2_svc_client_engage_module(svc_client, env, AXIS2_MODULE_ADDRESSING);

    conf_ctx = axis2_svc_client_get_conf_ctx(svc_client, env);
    svc = axis2_svc_client_get_svc(svc_client, env);
    pool = new TThreadPool( POOLSIZE );

    for (i = 0; i < NTHREADS; i++)
    {
        echo *job = new echo(i);
        axis2_svc_client_t *t_client = NULL;
        svc_info_t *svcinfo = NULL;

        svcinfo = (svc_info_t *) malloc (sizeof (svc_info_t));
        if (!svcinfo)
        {
            fprintf (stderr, "no enough memory\n");
            exit (1);
        }
        svcinfo->env = env;
        t_client = axis2_svc_client_create_with_conf_ctx_and_svc(env, client_home, conf_ctx, svc);
        svcinfo->svc_client = t_client;
        svcinfo->address = (axis2_char_t *) address;
        pool->run( job, (void *) svcinfo, true );
    }

    pool->sync_all();
    axis2_svc_client_free(svc_client, env);
    printf("success\n");
    return 0;
}

/* build SOAP request message content using OM */
axiom_node_t *
build_om_payload_for_echo_svc(
    const axutil_env_t * env)
{
    axiom_node_t *echo_om_node = NULL;
    axiom_element_t *echo_om_ele = NULL;
    axiom_node_t *text_om_node = NULL;
    axiom_element_t *text_om_ele = NULL;
    axiom_namespace_t *ns1 = NULL;
    axis2_char_t *om_str = NULL;

    ns1 =
        axiom_namespace_create(env, "http://ws.apache.org/axis2/services/echo",
                               "ns1");
    echo_om_ele =
        axiom_element_create(env, NULL, "echoString", ns1, &echo_om_node);
    text_om_ele =
        axiom_element_create(env, echo_om_node, "text", NULL, &text_om_node);
    axiom_element_set_text(text_om_ele, env, "Hello World!", text_om_node);
    om_str = axiom_node_to_string(echo_om_node, env);

    if (om_str)
    {
        AXIS2_FREE(env->allocator, om_str);
        om_str = NULL;
    }
    return echo_om_node;
}

static void
svc_client_send_request (void *v)
{
    svc_info_t *svcinfo;
    const axutil_env_t *env;
    axis2_svc_client_t *svc_client;
    axiom_node_t *payload = NULL;
    axiom_node_t *ret_node;
    axis2_options_t *options = NULL;
    axis2_endpoint_ref_t *endpoint_ref = NULL;
    axis2_char_t *address = NULL;

    svcinfo = (svc_info_t *)v;
    env = svcinfo->env;
    svc_client = svcinfo->svc_client;
    address = svcinfo->address;

    options = (axis2_options_t *) axis2_svc_client_get_options(svc_client, env);
    if(!options)
    {
        options = axis2_options_create(env);
        axis2_svc_client_set_options(svc_client, env, options);
    }

    endpoint_ref = axis2_endpoint_ref_create(env, address);

    axis2_options_set_to(options, env, endpoint_ref);
    axis2_options_set_action(options, env,
                             "http://ws.apache.org/axis2/c/samples/echoString");

    /* Build the SOAP request message payload using OM API. */
    payload = build_om_payload_for_echo_svc(env);

    /* Send request */
    ret_node = axis2_svc_client_send_receive(svc_client, env, payload);

    if (ret_node)
    {
        axiom_node_free_tree(payload, env);
        axiom_node_free_tree(ret_node, env);
        ret_node = NULL;
        printf("Invoke SUCCESSFUL\n");
    }
    else
    {
        AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI,
                        "Stub invoke FAILED: Error code:" " %d :: %s",
                        env->error->error_number,
                        AXIS2_ERROR_GET_MESSAGE(env->error));
        printf("echo client invoke FAILED\n");
    }

    if(svcinfo)
    {
        AXIS2_FREE(env->allocator, svcinfo);
        svcinfo = NULL;
    }
    axis2_svc_client_free(svc_client, env);
}

build.sh

gcc *.cpp -oecho -I./src libthrpool.a -lstdc++ -lpthread -L/axis2c/deploy/lib -lm -laxis2_engine -laxutil -laxis2_axiom -I. -I/axis2c/deploy/include/axis2-1.6.0/

Memory Handling conventions in Axis2/C

I found no documentation on basic memory handling conventions in Axis2/C. Until this content appear under a developer guide for Axis2/C, here are some facts that could be useful in preparing such document. Here I don’t talk about advance facilities provided regarding memory management in Axis2/C. However I’ll provide links below for such documents. I just talk about few points that could be useful for service and client developers in handling memory which are not included in the above mentioned documents.

Memory handling in services

Axis2/C services should be written implementing the API defined in axis2_svc_skeleton.h. Basically if your service name is echo you will have two implementation files echo_skeleton.c and echo.c. echo_skeleton.c will contain the functions implementing the API including life-cycle management functions of the service. They are echo_create(), echo_init(), echo_free, echo_invoke and echo_on_fault(). Within the create function you will instantiate axis2_svc_skeleton_t stuct and assign your services operation structure to the svc_skeleton_t structures operation structure so that your functions will be called from withing the Axis2/C engine.
Within free function you will free the sekelton structure you allocated in create function. When the service unloaded this function is called by the engine to free the service resources. If you allocate any additional stuff withing the service create function it is your responsibility to free them then and there.
In invoke function you can check the operation requested and call the appropriate service function from echo.c where all the service logic functions are implemented. The response nodes if any are created withing these service functions will be freed by the Axis2/C engine. Also any property you set there into Axis2/C configuration context structures will be garbaged by the Axis2/C engine according to their life scope. Any additional stuff you created there must be handled by your self.

In the on_fault function you can create a fault node which could be inserted as detail element in creating soap faults by the engine when something goes wrong when invoking your service. It is the responsibility of the Axis2/C engine to free such nodes.

Memory handling in clients

Most of the memory allocated in the client side (including SOAP request and reply messages), should be cleaned in the client side itself. Memory allocated for properties are taken care of by the SOAP engine. SOAP engine reclaims the memory allocated for properties based on the scope (Request, Session or Application) you set for each property.

You can read more advance information regarding memory management in Axis2/C in the following articles. However some of these articles are old so if you find it is difficult to understand them with the current Axis2/c code base don’t hesitate to ask in the Axis2/C developer list.

Overcoming Memory Related Issues in Axis2/C

Memory Management with Apache Axis2/C

Writing robust programs in C

The draft named Robust design techniques for C Programs by David Turner introduce a modern languages like approach for exception handling in C. It speaks about four approaches for error handling in C.

1. Being extremely paranoid.

2. Implementing transactions which in concept similer to commit/rollback approach in databases.

3. Structured Exception Handling(SEH)

4. Cleanup stack exception handling(CSEH)

Of the four he recommends 4 th approach as the best. He expect that this paper will stir interest in this technique and motivate enough people to introduce such features in the libraries and programs they write..

I recently wrote an article for WSO2 Oxygentank called “Using WSF/C extended error handling mechanism in your own modules and applications.” which basically explain how WSF/C stack handle errors. From the above four approaches what WSF/C has taken is paranoid approach like many popular libraries programmed in C. However I belive that WSF/C could also greatly benefit from CSEH approach if carefully adopted. Infact James Clerk hinted about this approach in the early days of Axis2/C design discussions.

Jeep Trek through Bambarakanda

This time we decided to go on a trip on jeeps. Sanjaya and I prepared our jeeps for the journey. The route was decided to be the well known jeep trek from Kalupahana(On Colombo-Haputale road) to Ohiya through Bambarakanda falls. Participants were Sanjaya, Janapriya, Amila, Nandika, Dumindu, Dushyantha and his wife, Diluka, Me and my wife.
We started from the office at about 11am in the morning and reached Kalupahana at about 3.30pm. From Kalupahana we did a sharp left turn towards Bambarakanda falls. We travelled upwards about 4 kilometers on a narrow road which has some 45 degree bends towards the end of that distance and then suddenly we were in the vicinity of the awe-inspiring view of the Bambarakanda falls.

Bambarakanda falls as first seen when we travel from Kalupahana

We stayed the night at Bambarakanda Rest managed by a Lady called Sera. There are 3 peaks that surround the view from Bambarakanda Rest. One of them is Vangedigala Kanda and the one next to it is called Gobbelikanda. Gonamale Kanda is next to it. They are one moutain range. We were informed that if we trek to these mountains we can have a good view of the country. So we set out in the morning to climb Vangedigala and then Gobbelikanda. Vangedigala and Gobbelikanda peak are joined through a narrow path through the mountain range. For a serious trekker the climb from Rest house to Vangedigala is about two hour climb. But it took for our team about 4 hours to reach the Vangedigala. We traveled through the edge of a pine forest which extend from the root to the top. Just before the peak there was a very steep climb through the rock which posed to be very dangerous specially when it rains. Just as we reached there heavy downpour wahsed all of us making the climb extremely difficult. From Vangedigala to Gobbelikanda kanda it is only about 1/2 hour walk through the narrow gap I mentioned. The areas is covered with thick grass upto the Gobbelikanda. Once we reached the Gobbelikanda we could see that it strech further in a similar grass covered path towards Gonamale Kanda. However we could see that after some distance the grass path ends and there starts the thick jungle which could take another day to reach it. The view from GobbeliKanda is extremely beautiful and we were lucky to have clear surronding with sun shine which helped to view the vast country around. It is my advice that if you plan on climbing a mountain start it early in the morning in a sunny day. Because after the noon normally the mist would cover up the surrounding and it would be only mist nothing else you would see which is just the thing happened in our laster year journey to Namunukula Kanda. In a way it is also nice experience like in mid of a floating cloud.

View of Gobbelikanda peak from Vangedigala peak

Towards one side of the moutain(in the direction of the Bambarakanda falls) we could see only big mountain which extends towards Worlds end. We could see the origins of Bambarakanda falls from there through which our path next day towards Horton plains could be seen as a long winding serpant. On the other side of the mountain there strecthed the vast country extending towards down south through Balangoda area. We found that certain posts like road side milestone posts which within a gap about 300 meters between any two of them stretched from the start of our path  to the top of the mountain and strectching even further. This coud indeed be used as a good way of finding the correct path for a new comer. Janapriya has a good panoramic view from the peak (soon to be published) in his blog. When you view this panorama you could experience the surrounding view from mountain as if you were really there.

The back journey through the pine forest was extremely difficult after the rain. Rubber slippers were extremely  unsuitable for such descends when it is rainy. Some kind of moutain boots are the best. We reached the rest at about 8pm.

Next day we set out for Ohiya through the well known jeep trek. The cllimb was extremely difficult for a non- 4*4 vehicle. But since both of us had vehicles best suited for the journey(Mitsubishi 4DR5 and Suzuki Jimney) it caused us no problem. The first half of the steep climb end at about 2 hours of climb at a pass called V-Cut. The road is strewn with various size/shape of stones which posed extreme danger for the tires. We stopped at V-Cut which is supposed to be in the area just on the top of Bambarakanda falls. The view from there towards the path we travelled so far could make the weaker heart tremble.

Path we travelled from Bambarakanda rest as seen from V-Cut pass

The v-cut pass is situated in the gap between two mountains. From now on it is a second mountain we start to climb. After few distance we crossed a ford which is the stream that feed water to Bambarakanda falls. After some more distance our way laid through a tea estate and some estate settlements. After yet another rigorous climb we reached the top of that mountain from which started a steep descent called Devils staircase. Devil’s staircase is some three hairpin bends descending down into the opposite side of the mountain. This path is concrete paved extremely easing the descent, which could otherwise be disasterous. When you look ahead from Devil’s staircase you can see some long distance in the to be travelled road where you can see a bend in the road from where the mountain you are currently in is separated from another mountain through which the road ahead lies. Just after this bend you can see that there is a sharp climb to the left in a sharp left bend from the main road to Ohiya. We learnt that this road leads to Horton Plains through some sharp bends and climbs. But we were also informed by the same people that it won’t be as difficult as the road we travelled so far. Howerver we choose to follow our original path to Ohiya.

Few distance from the above mentioned junction towards Ohiya we found a place in the road from which we had the best view so far in our journey. Lo and Behold. This is one of the best mountain scenes that I have ever experienced anywhere in our beautiful  hill country.

View after some distance passing Devil’s staircase

From that point on we travelled some further distance through unmaintained difficult roads passing Estate Superintendent’s bungalow and Udaveriya tea estate factory. At some point we found concrete paved road the kind of which I have never seen anywhere else. The road is concreted only along the two wheel tracks. Middle is left unattended. I had to drive extremely carefully less the wheel fall off the cemented track and jolt uncomfortably. After some more drive we reached well paved narrow road which led us to the junction where it meet the Ohiya-Pattipola road through horton plains. When you travel from Ohiya to Horton plains you will find this junction just after you pass the villages and enter the wild life reserve.

From this point on we drove through Horton plains, Pattipola, Ambewela, Blackpool stopping just for tea and by the time we reached Colombo it was 12pm.

Notes:

(1)This travel note from Lakdasun helped us in preparation for the journey

(2)Sanjaya and damitha have uploaed more photos on this journey

Try Open Source SOA!

Good read, Mike Kavis explains the strength of open source SOA. In this article he also explain the WSO2 SOA stack.

http://www.cio.com/article/440370/Tight_Budgets_Try_Open_Source_SOA_

You can find news release for the case study which is mentioned in Mike’s article here.  You can download the case study here.

Configuring Apache2 to produce core dumps

Even after setting

ulimit -c unlimited in the shell I could not find a core dump file when my apache2 server crashed.  In my Ubuntu system I enabled core dump in /etc/default/apport. Still I could not see core dumps. But I could produce a core dump for a simple C program. But still not for Apache2. Finally I found that I need to configure Apache2 for core dumps from here

http://www.cyberciti.biz/tips/configure-apache-web-server-for-core-dump.html

Also following link could be useful for Ubuntu users

https://wiki.kubuntu.org/Apport

Restful PHP web services

I finished reading Samisa’s recently published Restful PHP Web services book. Normally when it comes to technical books I don’t read every bit of it. My understanding is it is not necessary to do so.  Because you have the book you can always refer to it whenever necessary. However my initial reading aim at fulfilling one task. That’s to understand how to use the book quickly when need arise. It’s kind of indexing the book in memory for faster reference later. Also interesting bits that attract me most will be covered withing this initial iteration of reading.

But with this book I read it from start to end without missing a single word of it. It is so compact, so easy to read. One reason for this change is my curiosity about the content of the book. That curiosity is aroused by several reasons. For one thing I often hear the technical jargons contained in the book within the office environment I live. My colleges at WSO2 developed WSF/PHP which support RESTful as well as SOAP web services in the same stack.  This WSF/PHP use WSF/C as it’s web services core on which development team I’m a developer at WSO2.  For some time I wanted to know what they do in these PHP web services stuff but could not find time/desire/way to do it fast. This book provided me all that.

Second I share the belief with many that WSF/PHP and the rest of the scripting web services stacks like WSF/Ruby, WSF/Python  from WSO2 have big potential within the web services market in near future. So gaining knowledge in those areas is always a good thing to do.

Chapter 6 nicely explained the use of MVC architecture in Zend framework for supporting restful web services with step by step easy to understand examples.

In addition to explaining the facts related to the subject matter Samisa has nicely used his knowledge on Software development which he gathered through experience as a developer as well as through years of teaching at the University of Moratuwa Sri Lanka. Throughout the book he explains best practices and how to avoid pitfalls(especially in chapter 7)  which are useful hints for any developer.

There are extensive code samples which could be a + point and at the same time – point depending the reader. However even a non-php programmer could easily understand those code excerpts without much effort(I could and I’m not a PHP programmer he he)

I wish if there was a section comparing positive and negative aspects of REST and SOAP web service approaches. May be an design approach which could take best from both worlds(I don’t know).

Deepal’s Axis2 Book

After delaying for several months after purchasing the book I finally found time to read Deepal’s new Axis2 book. But once I started to read it is one go read.

Nice thing about the book is that for a person who has fair knowledge of web services concepts the book immediately make him familier with Axis2 in a very short time. If a developer need to use Axis2 for his web services project this is the best way to get him started because he can grasp the essentials within a days’ time by reading the book.

Although the book says that it is for Java developers it is good read for Axis2/C developers as well because it describes basically the same architecture(Of course there are differences but when it is taken as a whole it is still very useful). In tern it is good read for all who are interested in Axis2/C related/based stacks as well like WSF/C, WSF/PHP, WSF/Ruby, WSF/Python and WSF/Perl.

When I purchased the book I have following things in mind.

1. Axis2/C is based on Axis2/Java architecture. At the time we developed Axis2/C, Axis2/Java is already passed through it’s initial developer versions and we learned the architecture with discussions with the community as well as by directly diving in to the code base. But once Axis2/C is matured we rarely looked back into the Axis2/Java deeply. But we know that there is major changes going on through discussions with community. So I needed to update easily on new changes.

2. To share the views of a major Axis2/Java contributor.

3. Just because it is on Axis2 and by Deepal !!!.

But my aims 1 and 2 are not fully satisfied because it turned out to be very introductory book and I found anything new rarely. Also there are no views just facts in compact order on Axis2. But no worries. I really enjoyed reading the book because of its easy flowing style.

There should be a sequel to this book preferably an advanced book, a kind of a mastering version. These days the interest is not basically on the core but on many projects based around Axis2 like Synapse, WSO2/ESB, WSO2/Mashup server etc etc and there should be books on each of these subjects as well.

Actually Samisa has actually started the trend by writing a nice book called RESTful PHP Web Services which I believe is based on WSF/PHP. Very much keen on having a good read of the book and will tell about it in future.

Finally the book’s design is very attractive and invites for reading. However there were errors I found on the book which does not affect the overall aspect of the book.

Access secure enabled web services from command line

In my article on WSF/C wsclient command line tool I have explained briefly on how Rampart/C is used to provide security for messages sent using wsclient. Here I would like to explain in detail some examples.

I assume you have installed Rampart/C and wsclient as explained in respective documentations. When you install WSF/C both of them get automatically installed.

Service used is the sec_echo sample service which is deployed when you install Rampart/C.

Also you need to set your WSFC_HOME variable to your repository location.

Now execute the script providing the port you wish to run simple axis2 server.

$ cd WSFC_HOME/bin/samples/wsclient

sh sec_echo.sh 9091

Let me explain what happen when you execute this script.

First you need to change the sec_echo/services.xml according to the policy you wish it to have. So the script will copy a services.xml file with the desired policy and restart the server. After that it execute the following command.

$WSFC_HOME/bin/wsclient –soap –no-mtom –user alice –digest –password password –timestamp –sign-body –key /axis2c/deploy/bin/samples/rampart/keys/ahome/alice_key.pem –certificate /axis2c/deploy/bin/samples/rampart/keys/ahome/alice_cert.cert –recipient-certificate /axis2c/deploy/bin/samples/rampart/keys/ahome/bob_cert.cert encrypt-signature –encrypt-payload http://localhost:9090/axis2/services/sec_echo < $WSFC_HOME/bin/samples/wsclient/data/echo.xml

When you closely examine the above command you can see that user alice send a echo message to the service which is signed and encrypted. To sign the body of the message she need her private key which she provide through –key option. To encrypt the body of the message she need the recipients public key which she provide through –recipient-certificate option.  You need to provide public key of alice using option –certificate which is required for verification process. Note that service policy requires you to sign the signature and timestamp the message. The default behaviour of wsclient is to sign the message before encrypting it. If you need to change this behaviour(encrypt before sign) use –encrypt-before-signing option.

Now if you need to run the sample by providing a policy file run the following sample.

$ sh sec_echo_with_policy.sh 9091

The ability to provide a policy file enable the wsclient to provide fine grained security policies required by the service.  Following is the wsclient command used to send the request.

$WSFC_HOME/bin/wsclient –soap –no-mtom –user alice –digest –password password –key /axis2c/deploy/bin/samples/rampart/keys/ahome/alice_key.pem –certificate /axis2c/deploy/bin/samples/rampart/keys/ahome/alice_cert.cert –recipient-certificate /axis2c/deploy/bin/samples/rampart/keys/ahome/bob_cert.cert –policy-file $WSFC_HOME/bin/samples/wsclient/data/policy.xml http://localhost:9090/axis2/services/sec_echo <$WSFC_HOME/bin/samples/wsclient/data/echo.xml

Note that –policy-file option is used to provide the policy xml file. Also since now security policy is provided by policy xml file you don’t need to use wsclient specific options like –timestamp, –sign-body, –encrypt-signature and –encrypt-payload

Beautiful Anjuta

Anjuta is now wonderful. Those of you seeking a modern C/C++ ide environment Anjuta could be the best choice. Although vi/gdb/valgrind trio is still  my favourite development environment, now and then I look at ide’s whether they are matured enough for my taste. About 3 years back when I had a look at Anjuta I thought that it is promising but it still did not invited me to adopt. At that time I tried with Kdeveloper and Eclipse cdt plugin environment as well which were lagging far behind as robust ide environments at that time.

Today I tried to build Axis2/C in cdt environment. It seemed improved and just as I wanted to try it on my machine but it crashed while doing very simple task. That’s end of cdt with me at least for some time to come.

But this marvelous Anjuta. It is unbelievable how beautiful and matured she has become. It has whole set of features that I could not imagine three years back. If you are really interested, do some playing with her.

Using it for Axis2/C is very simple.

You can build anjuta from apt repositories in dabian environments. Make sure that you also build autogen project. Then after starting Anjuta just create a new project for Axis2/C. What you need to do is, in new menu select ‘Project from existing sources’ sub menu. That’s it. Rest is very intuitive.

But I found that still Unjuta is lacking with good context sensitive menu popup.

Building Axis2/C from source

If you need to use Axis2/C and related project for serious work then you might need to understand the internals of its build system. Axis2/C uses GNU auto build tools to build the project libraries. A simple and good reading on the subject can be found here.

If the usual configure, make, make install does not work for your platform when installing Axis2/C it is advisable to download and install autotools for your system before installing Axis2/C.

In a dabian system what you need to install from apt repositories are

libtool

autoconf

automake

After installing these tools running the build.sh found in your Axis2/C source distribution(You can find this script only in the Axis2/C svn repository) will run these tools to generate your Configure and Makefiles.

ccache is another tool that you can find in apt repositories which would greatly increase your recompile performance.

Also make sure that you have installed zlib1g-dev which is available from apt repositories

Indigenous Music

Senanayaka Weraliyadda’s creation ‘Mal Paba’ is really interesting piece of music(youtube contain a sunflower version which is totally ridiculus). I love to hear this song so much. To me the music in that song is what I can call is real indigenous music. I can cleary understand the music emanated from real indeginous music instruments like Davul, Bera, thammattam, thalampota etc etc. Althought Thabla is very interesting instrument I believe it is not used in our ancient music and I don’t hear it in this song. To me this man is really talented musician who deserve more recognition.

It is amazing to see how those simple instruments that we hear played in Pirith gedara, Dane gedara, Perahara could produce such great music.

It is very unlucky of me that I can not enjoy the music produces by modern musicians. The last generation of musicians who produced music that I could enjoy are no more seen to be creating new music.

Installing WSF/PHP on Linux

To build apache2 with debug and apr support

./configure –enable-modules=all –enable-so –enable-maintainer-mode –with-included-apr

This will install apache2 in /usr/local/apache2 (You may use export CFLAGS=”-O0 -g -ggdb3″ before configuring)

To install php with apache2

After installing apache2 as above then download and install php as below.

./configure –with-apxs2=/usr/local/apache2/bin/apxs
sudo make install

This will add the following entry to httpd.conf
LoadModule php5_module modules/libphp5.so

and put libphp5.so in /usr/local/apache2/modules/

Now put the followng entries to httpd.conf
AddHandler php5-script php
AddType text/html php

Now create the following script and put it into apache2/htdocs directory.
<?php
phpinfo();
?>

Start apache2 and run the script in the browser.
This will show you where your php.ini file should be copied to(/usr/loal/lib/)

Now copy the php.ini-recommended file which can be found in your php download, into /usr/local/lib/ as php.ini and add following lines into php.ini

extension=wsf.so
extension_dir =”/usr/local/lib/php/20060613+lfs”
wsf.home=”/usr/local/lib/php/20060613+lfs/wsf_c”
wsf.log_level = 4

You can find above information like extension_dir by executing

php-config

Now you can download and install wsf/php as follows

./configure –with-wsf –enable-openssl –with-axis2=`pwd`/wsf_c/axis2c/include –enable-tests=no –enable-wsclient=no –enable-savan=no –prefix=`php-config –extension-dir`/wsf_c && make -j 100

make

As root

make install

This will install your wsf/php extension into /usr/local/lib/php/20060613+lfs.

Copy the samples directory from your wsf/php download into apache2/htdocs directory.

Restart apache2 and try your wsf/php samples

Using valgrind to debug/profile Axis2/C

For Axis2/C developers in linux platform valgrind is a indispensable tool. It helps you debug and profile your services and modules.

The gdb/valgrind is very powerful combination for debugging your applications in linux.

Before you use these tools make sure that you have built your applications and Axis2/C with debugging enabled. This is a simple check on seeing whether the configure.ac in Axis2/C root source folder has -g entry in CFLAGS as in

CFLAGS=”$CFLAGS -ansi -Wall -Werror -Wno-implicit-function-declaration -g -D_GNU_SOURCE”

You can debug Axis2/C clients by

valgrind ./echo 2> result

This will show any context errors associated with your client code. Context errors simply says that your code is broken at the specified places. If you need to further investigate these points you can use gdb with the specified source files line numbers. If you find zero context errors in your code then you are lucky. Then you can further investigate by

valgrind –leak-check=full ./echo 2> result

which show you all the memory leaks you have in your code. However this will hide the repetitions. To see them also you can use.

valgrind –leak-check=full –leak-resolution=high 2> result

You can debug server side similarly.

valgrind –leak-check=full ./axis2_http_server 2> result

or if you want to debug with Axis2/C apache module

valgrind –leak-check=full ./httpd -X 2> result

You can use result file to investigate on context errors and leaks as in the client side.

However there is a caveat when it comes debugging Axis2/C modules and Axis2/C services. valgrind could not show leaks in modules and services. I suspect this is because of dll loading libraries. The solution is to build your service/module statically with Axis2/C engine. Currently Axis2/C does not provide any facility to do this(I suggest this for future releases). So we need to do a small hack to get this done. I will explain here how I did the hack in order to debug Sandesha2/C module.

Say you need to debug your in flow path. I created the in handler header as following and put it in axis2c/src/core/engine folder

#ifndef SANDESHA2_IN_HANDLER_H
#define SANDESHA2_IN_HANDLER_H

/**
* @file sandesha2_sender.h
* @brief Sandesha Sender Interface
*/

#include <axutil_allocator.h>
#include <axutil_env.h>
#include <axutil_error.h>
#include <axutil_string.h>
#include <axutil_utils.h>
#include <axis2_conf_ctx.h>

#ifdef __cplusplus
extern “C”
{
#endif

axis2_status_t AXIS2_CALL
sandesha2_in_handler_invoke(
struct axis2_handler *handler,
const axutil_env_t *env,
struct axis2_msg_ctx *msg_ctx);

/** @} */
#ifdef __cplusplus
}
#endif
#endif /* SANDESHA2_IN_HANDLER_H */

Now go and include that file in phase.c file. Also add the following code into phase_invoke function

if(!axutil_strcmp(handler_name, “SandeshaGlobalInHandler”) || !axutil_strcmp(
handler_name, “SandeshaInHandler”))
{
AXIS2_LOG_DEBUG(env->log, AXIS2_LOG_SI,
“dam_handler_name %s. dam_phase_name %s”, handler_name, phase->name);
if(!axutil_strcmp(handler_name, “SandeshaGlobalInHandler”))
{
status = sandesha2_global_in_handler_invoke(phase->first_handler, env, msg_ctx);
}
if(!axutil_strcmp(handler_name, “SandeshaInHandler”))
{
status = sandesha2_in_handler_invoke(phase->first_handler, env, msg_ctx);
}
}
else
status = axis2_handler_invoke(handler, env, msg_ctx);
if (!status)
{
AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI,
“Handler %s invoke failed within phase %s”, handler_name, phase->name);
return status;
}

Also link to sandesha2/C module library in the axis2c/src/core/engine/Makefile.am and don’t forget to include your module include files also there in.

That’s just the trick. Now you can build Axis2/C and your module and use valgrind as explained before.

At this point I would like to add that adding static module build support for Axis2/C is a great thing to do. Adding modules to Axis2/C is not a frequent phenomena. Actually we currently have only few modules like Rampart, Sandesha2/C and Savan/C. Making these statically built with Axis2/C might give us some performance advantage. We can give an Axis2/C configuration option to build certain standard modules statically with Axis2/C.

To do profiling Axis2/C modules/services with valgrind see previous entry here.

Axis2/C Performance

Axis2/C benchmark performance test results are published here

lighttpd module for Axis2/C

Dinesh has developed an Axis2/C module for lighttpd (pron. lighty) a light weight web server is designed and optimized for high performance environments. It claims to power Youtube. Also see.

I mentioned that Axis2/C version 1.5 reached 25K tps performance margin running on Apache2. I hope Axis2/C running as a lighttpd module will further extend this margin. Still there is some work to be done before it could be brought to production use. But the important thing is the module is working now.

Few words about Dinesh’s effort on the work. He started on this module two weeks before he left WSO2 for his higher studies in USA(And he married last month !!). In the middle of all his preparation for new life he had time and will to bring this work to a working state. At this time of his parting I would like to remind his great contributions to Axis2/C project. He workied on lot of ares’s in Axis2/C code. He initiated Guththila Stax parser, XMPP transport and contributed to wsclient command line tool to name a few. I think he is the one so far to be the release manager for most number of Axis2/C releases. Kudos Dinesh for your great work.

Axis2/C and Rampart/C is used in Eucaplyptus

Axis2/C and Rampart/C is used as main technologies in an open source EC2 cloud computing implementation. See the main technologies used in this document and this. See this infoQ vidio for a presentation of Eucaplytus by its lead.

Apache Axis2/C Programming Model

I would like to talk about our programming model when it comes to Axis2/C Apache2 module. As discussed in a earlier blog suppose two requests come to Axis2/C sequentially. Say the first request is served by one apache process and the next request is served by another process. Then problem is you cannot access the previous requests configuration context from your next request because each process has it’s own configuration context. We recently solved this problem using shared memory. But as should be expected this is extremely inefficient.

The other and more effiecient solution is using a database that could be shared among your processes. Sandesha2/C uses this approach. To solve concurrency issues when using this approach we can use Apache2 golbal mutex which could be used when accessing global resources like system files and databases. But the problem is how we could pass this global mutext from Axis2 apache module to be accessible from other parts of the code, for example from a Axis2/C module. We currently pass Apache2 pools through axis2_env_t environment. My understanding is that we could use a similar method to solve the problem.

Another XT225 journey

Interesting bike journey, again using XT225 serow

Axis2/C Apache2 Module. Can it be further improved?

According to my performance testing on Axis2/C 1.5(Soon to be released) I could reach 25K tps with keep alive on. When tested with a empty module instead of Axis2/C where no handling of soap data is done but just set a HTTP_OK and return it, reached 40Ktps mark(This is also similar to index page request as might be expected). This means there is still room for 15K improvement. Profiling tools show that mostly the improvements needs on parser.

Also when I experiments with using a localpool instead of the request pool of apache the performance degraded by about 1000 tps. This further shows that there could be further performance gains that could be achived perhaps by improving apache2 module. Has anybody thought of implementing Axis2/C module as a apache2 filter or using filters instead as an apache2 content generator handler?. In apache2 documents it says that

‘Any form processing application is normally be implemeneted as a handler-perticulary those that accept POST data.’

However in the same documentation it says that markup processing libraries better be implemented as filters or using filters.  It also says that parsers like expat and libxml2 which have parseChunk APIs work well with apache2 filters. Working with filters give direct access to bucket brigades which gives the control into the filter developer for data manipulations.

I heard somebody is developing a expat parser plugging for Axis2/C. It is a good thing to do a performance comparision with that also.

These directions are for future considerations only. Axis2/C is already ready for  performance hungry  deployments, may be ahead of other competing  web services engines. I have actually plans for doing some performance benchmarks using Axis2/C as a soap gateway in an EC2 cloud.

Profiling tools that I used to profile Axis2/C

Now that I have done some performance tests(Results will be published soon. In summary Axis2/C 1.5 which will be released within next week achieved 25K tps benchmark) I thought of using a profiler to see which part of codebase need further attention for improvement. I profiled with google proftools and valgrind profile tools.

perftools

proftools has a cpuprofiler and a heap profiler. You need to just download it and ./configure, make, make install.

Then

export CPUPROFILE=/tmp/axis2_http_server.prof
export HEAPPROFILE=/tmp/axis2_http_server.hprof

where first one is where cpu profiler output file is created and the second one is where heap profiler output file is created. These files will be created when you run your program.

You need to build Axis2/C with -ltcmalloc and -lprofiler.

Just add those linker options into configure.ac CFLAGS.

Now you can your executable as normal. After that

For cpu profiling

./pprof –gv axis2_http_server /tmp/axis2_http_server.prof.

This will give you a nice graph. You must have installed dot, gv and perl5 installed in your system.

If you have installed kcachegrind in your system you can use it to have nice graphical view by

pprof –callgrind ./axis2_http_server /tmp/axis2_http_server.prof > axis2_http_server.callgrind

kcachegrind axis2_http_server.callgrind

For heap profiling

–callgrind ./axis2_http_server /tmp/axis2_http_server.hprof.0001.heap > axis2_http_server.callgrind

again you can use kcachegrind to have a nice gui view.

kcachegrind axis2_http_server.callgrind

Note:I found that proftools are only good for profileing Axis2/C client side because I could not find a way to capture profiling for request handling in server side. If you need to just profile deployment in Axis2/C then this can be used.

Callgrind

You can separetely install callgrind. But if you install valgring in dabian uisng apt-get install you can use it by using

valgrind –tool=callgrind < your program name>

I used callgrind to profile simple axis2 server as follows

valgrind –tool=callgrind ./axis2_http_server

Now make a request to the server.

Now you have a file generated in your current folder called something like callgrind.out.8307.

Use this as input to kcachegrind

kcachegrind callgrind.out.8307

Important: I did the following addition in my src/core/transport/http/receiver/http_svr_thread.c code towards the end of server while loop to exit the server after serving two requests. Instead if I use ctrl-c to signal the server to stop then profiling tool won’t be able to capture the requests for profiling.

if(counter == 1)
{
svr_thread->stopped = AXIS2_TRUE;
}
counter++;

}// end while loop

My conclusion is that both of these tools are good for profiling in a threaded environment. Problem with gprof is that it could not be used for profiling in a threaded environment. Although there is a hack to do this I saw a user say that it is not trustworthy.

If you are new to web services world

One of the best way to get you immediately get aware of the concepts

http://video.google.com/videoplay?docid=1710477770315021899&q=type%3Agoogle+engEDU

Ubuntu Story

My wife was asking me for some time to install linux on her machine. I thought of this. Well it take some time, not to install ubuntu. But to answer her non-stop questions on how to do this, how to do that. Last week. Behold she has installed ubuntu on her machine herself. She says, “best replacement for windows”.

Bike trip

Nice account of a bike trip. XT225 is very nice bike. I know this cause I have one.

http://www.xt225.com/trips/t1/trip.htm

Benchmark testing tools for Web Services

These days I am working on some benchmark testing of Axis2/C. While looking for good tools for the purpose I came across several good candidates. I would like to discuss about some of these tools.

Tools that impressed me and disscussing here are

ab(apache workbench), siege, hammerhead and javabench and http_load. I also tried tsung and curl-loader but could not get it work for me. Here it again prove that however good a software be it does not make appeal to average user if it does not provide a good impression on first try. What most developers expects from a good software is to get them immediately familier with it with some intuitive and working samples. To me it should provide a good <toolname> -h help menu, easy and upto the point samples first. A comprehensive documentations should follow next. tsung seems to be a good performance testing tool that provide a good distributed testing enviroment.

Since my perticular interest is on testing web services one of my main requirement is supporting http POST method. http_load seems to be a good tool but  it does not support POST. So I am not discussing it further here.

apache bench

The tool most impressed me is ab(apache bench) that ships with apache2. So naturally it is the tool I selected for my purpose.

ab -k -T “application/soap+xml; charset=UTF-8″ -p data/data.xml -n 1000 -c 32 http://localhost:80/axis2/services/Benchmark > results/result.txt

Here -k indicates whether to use keep-alive feature of http 1.1. You can use -H to add any custom http header you want.

-T options is for giving the content type.  Using -p option you can give your data file which contain basically the soap envelope you need to send. Use ab -h to see the meaning of all options.

Following is a result I got after executing the tool

Concurrency Level:      10
Time taken for tests:   0.084 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Keep-Alive requests:    100
Total transferred:      97910 bytes
Total POSTed:           77550
HTML transferred:       76900 bytes
Requests per second:    1195.69 [#/sec] (mean)
Time per request:       8.363 [ms] (mean)
Time per request:       0.836 [ms] (mean, across all concurrent requests)
Transfer rate:          1143.26 [Kbytes/sec] received
905.52 kb/s sent
2048.78 kb/s total

siege

There is a configuration file called siegerc.

siege –rc=./siegerc -H”Content-type: application/soap+xml; charset=UTF-8″ -H”Connection: Keep-Alive”

Almost all ouf your configurations can be put into this configuration file.

Following are some of the options available there

verbose = false

logging = false

protocol = HTTP/1.1

connection = keep-alive

concurrent = 25 #Number of concurrent requests

reps = 1000 #Number of repetitions

file = ./urls.txt

delay = 0 #If you are using for benchmarking value should be 0

benchmark = true

Note the file options above. It indicate to use the urls.txt file in the current folder as the url file. An entry in the urls.txt is as follows.

This entry is for a http POST request

http://localhost:80/axis2/services/Benchmark POST < data/echoDoubles-10.xml

Here is a typical output from siege.

Transactions:               25000 hits
Availability:              100.00 %
Elapsed time:               20.18 secs
Data transferred:           18.33 MB
Response time:                0.01 secs
Transaction rate:         1238.85 trans/sec
Throughput:                0.91 MB/sec
Concurrency:               10.68
Successful transactions:       25000
Failed transactions:               0
Longest transaction:            0.49
Shortest transaction:            0.00

siege is very good tool but to me it seems to be somewhat less capable than apache bench in its sieging capabilities.

hammerhead

hammerhead uses a configuration file and a scenario folder to execute.

hammerhead –conffile=/home/damitha/workspace/apache2_performance/hammerhead/test.conf -s20 -oresults

This tells hammerhead to use test.conf configuration file and run for 20 seconds.

In the test.conf you have an entry like following

Scenario_Directory /home/damitha/workspace/apache2_performance/hammerhead/scn

scn scenario directory can contain any number of .scn files.

An example .scn file is

NSample POST
HContent-type: application/soap+xml; charset=UTF-8
HConnection: Keep-Alive
RPOST /axis2/services/Benchmark

B<soapenv:Envelope xmlns:soapenv=”http://www.w3.org/2003/05/soap-envelope”><soapenv:Header /><soapenv:Body><ns1:echoDoubles xmlns:ns1=”http://www.extreme.indiana.edu/wsdl/Benchmark1″><ns1:input>0.0</ns1:input><ns1:input>1.0</ns1:input></ns1:echoDoubles></soapenv:Body></soapenv:Envelope>
T0

You can think the entries of the .scn file as key value pairs. The first letter in the each line(in block letter) represent the key. rest of the line represent the value. For example B reperesent the http POST message.

Here is sample output file for hammerhead.

Config File                  : /home/damitha/workspace/apache2_performance/hammerhead/test.conf
Machine                      : 127.0.0.1:80
: 127.0.0.1:80
: 127.0.0.1:80
: 127.0.0.1:80
: 127.0.0.1:80
: 127.0.0.1:80
: 127.0.0.1:80
Time                         : Fri Jul 11 11:11:52 2008
SEED =                       : 1215754872
Parent Process PID =         : 12090
Total Run Time (sec)         : 20
Sessions                     : 2
Session Sleep Time (msec)    : 0
Startup Lag Time (seconds)   : 1
Failures                     : 0
NoVerify                     : 0
Total Requests Served        : 10
Total Turnaround Time (msec) : 49990
Average Request Time  (msec) : 4999
Total Responses              : 10
Average Responses / sec      : 0.250000
Total Response Time (msec)   : 6
Average Response Time (msec) : 0
Scenario Throughput          : 10
Sequence Throughput          : 12
Run Time (sec)               : 40
Scenarios / sec              : 0.250000
Sequences / sec              : 0.300000
Content Length bytes         : 7690
Content Length bytes / sec   : 192
Read Length bytes            : 7690
Read Length bytes / sec      : 192

javabench

javabench is basically apache bench written in java. You can use it in the same way you use apache bench. However it is not as capable sending many requests per second.

You can download a complementary upload of javabench  from

http://wso2.org/files/java-bench-0.1.zip

After downloading you can run the tool by using the following script(modify it according to requirement)

Here is the script

for f  in ./*.jar
do
JAVABENCH_CLASSPATH=$JAVABENCH_CLASSPATH:$f
done

export JAVABENCH_CLASSPATH

java -classpath $JAVABENCH_CLASSPATH org.apache.http.contrib.benchmark.HttpBenchmark -k -T “application/soap+xml; charset=UTF-8″ -p data/data.xml -n 1000 -c 32 http://localhost:80/axis2/services/Benchmark > results/result.txt

Axis2/C 1.4 Version 1.4.0 Released.

It is great to see that Axis2/C “The C web services stack” has it’s 1.4.0 version released. Thanks Dinesh for the great effort.

You can download this release from http://ws.apache.org/axis2/c/download.cgi

Key Features
————

1. Support for one-way messaging (In-Only) and request response messaging (In-Out)

2. Client APIs: Easy to use service client API and more advanced operation client API

3. Transports supported: HTTP
* Inbuilt HTTP server called simple axis server
* Apache2 httpd module called mod_axis2 for server side
* IIS module for server side
* Client transport with ability to enable SSL support
* Basic HTTP Authentication
* Digest HTTP Authentication
* libcurl based client transport

4. Transports supported: HTTPS
* HTTPS Transport implementation using OpenSSL

5. Transports supported: TCP
* for both client and server side

6. Transport proxy support (HTTP)
* Proxy Authentication (Basic/Digest)

7. Module architecture, mechanism to extend the SOAP processing model.

8. WS-Addressing support, both the submission (2004/08) and final (2005/08) versions,
implemented as a module.

9. MTOM/XOP support.

10. AXIOM, an XML object model optimized for SOAP 1.1/1.2 messages;
This has complete XML infoset support.

11. XML parser abstraction
* Libxml2 wrapper
* Guththila pull parser support

12. Both directory based and archive based deployment models for deploying
services and modules

13. Description hierarchy providing access to static data of
Axis2/C runtime (configuration, service groups, services, operations and messages)

14. Context hierarchy providing access to dynamic
Axis2/C runtime information (corresponding contexts to map to each level of
description hierarchy)

15. Message receiver abstraction
* Inbuilt raw XML message receiver

16. Code generation tool for stub and skeleton generation for a given
WSDL (based on Java tool)
*  Axis Data Binding (ADB) support

17. REST support (more POX like) using HTTP POST, GET, HEAD, PUT and DELETE
*  Support for RESTful Services

18. Comprehensive documentation
* Axis2/C Manual

19. WS-Policy implementation called Neethi/C, with WS-SecurityPolicy extension

Major Changes Since Last Release.
——————————–

1. Fixed library version numbering
2. Made Guththila as default XML parser
3. Many bug fixes.
4. Memory leak fixes

C Web Services Demo Site

See here

Reliable Messaging with Mercury

There is this new implementation of Reliable Messaging from WSO2 called Mercury. You can download it from here. There is discussion going on as to whether Mercury should be sequal to Sandesha2 as Sandesha3 or not. Ppl who argue against it has basically following points

1. When adding features to current Mercury it will ultimately become as complex and as ugly as Sandesha2. In other words they have concerns whether Mercury solve existing Sandesha2 problems.

2. Sandesha2 is tested for performance and results are satisfactory and have doubt about Mercury performance.

3. Sandesha2 is feature rich.

4. Mercury still only provides only a very basic feature set and therefore they can’t adopt it at this stage.(RM 1.1 spec is not supported)

What those ppl accept in general is that

Sandeha2 is complex and need lot of improvement and that improving is not easy.

Ppl who argue in favour in Mercury have following points

1. Mercury architecture is based on a state model which is clean.

2. It would not be susceptible for race conditions and dead locks(I heard some ppl worry about that in Sandesha2)

3. Mercury is open for improvements and they beleive that it could be done in a cleanly fasion.

I am these days considering implementing Mercury/C for Axis2/C web services platform. When I had an initial look at the architecture diagrams and Mercury/Java code I understand that it is actually very clean and easy to adopt for writing Mercury/C, although I am not satisfied with the quality of the architecture documents that shipped with the Mercury release. It seems that nobody has reviewed it. For example it mention CSR but nowhere it has defined what is CSR(Of course Create Sequence Response). I must thank Amila Suriarachchi for coming up with this innovative design. I will continue to analyse Mercury architecture and most probably  will implement Mercury/C  which  I beleive could be completed in about 3 months time once started.

WSF/C Hackathon

For the past two and half years we developers at WSO2 were busy working on to add more and more features to WSF/C framework with the help of the community. After two years we can be happy with what we have acheived so far. WSF/C is now a feature rich, stable web services framework which also is the base for WSF/PHP, WSF/C++, WSF/Perl and WSF/Ruby. So it is the time to lookback and to see what we can do more to improve WSF/C. So the developers at WSO2 got together and discussed and worked on a plan to concentrate on the following.

- Improve the code quaulity by adding appropriate code comments, adding more descriptive loggings and reviewing the places where importand and/or complex logic involved.

- Get rid of the heaped junk here and there.

- Understand if we have missed anything.

- Identify any need for api changes that would lead to a next major release.

We started on doing this first on Axis2/C, the web services core of WSF/C and then plan to continue this on Rampart/C, Sandesha2/C, Savan/C and wsclient.

After two weeks of initial involvement it is the time for look back and evaluate the progress. Personally I’m quite satisfied about the progress. I consider this as a great opportunity to turn back and look at my own code on Axis2/C, a task I could not do for most of the time because of other involvements of adding more features.

There are basically two major paths a software can be developed.

1. Start developing the actual software after a rigorous and comprehensive architectural designs and discussions that involve software architects, users, developers etc. The design phase usually supposed to be comparatively longer than actual developement and by the time the design come to the developer, he has very descriptive details on how to proceed with the development. This is the model that is followed in most of the large scale software development companies in Sri Lanka. The life of the developer is very easy if everything goes as planned and if architectures involved are extrememely experienced and talented so that they could forsee what is going to be developed in exact details correctly.

2. Work in a kind of iterative development method where developer initiate the work and he uses his every ingenuity to achieve the final goal. He always have something working and continue on adding to it until it grows. Sometimes in the process he need to get rid of complete iterations that fails but provides direction for correct path. This often leads to very good quality software. But the success of this method relies greatly on the ablity and creativity of the developer.

What I believe is a hybrid of the above two to overcome the cons of both approaches.

I believe WSFC/Hackathon like sesssions constitute a greater part in a hybrid process like above.

wsclient and amazoneclient

As I have promised earlier I have written two articles on wso2 wsclient and amazoneclient. wsclient is a wget like command line tool which is designed to consume web services from command line. Today I added the capability to add custom soap headers by using the new option –soap-header LINE. This new option makes wsclient much more powerful tool since you can add ws* capablities through custom soap headers.

amazoneclient is also a command line tool which internaly use wsclient. This tool can be used to do online transactions with Amazon E-Commerce web services. These days I’m working on a WSO2 solutions demo site which is aimed to demonstrate the power of WSO2 WSF/C web services platform through some solutions. amazoneclient will be the first solution published on this site. This site will be launched within couple of weeks.

The articles are Calling Web Services from the Shell and Amazon ECommerce Client

Developer Guide needed for Axis2/C internals Revisited

In an earlier blog I emphasized the importance of having a developer guide for Axis2/C internals. I think a new article Overcoming Memory Related Issues in Axis2/C published in Wso2 Oxygen Tank by Manjula Peiris is a step towards that direction.

Useful Axis2/C Web Services Links

Useful Axis2/C web services links

Developer Guide needed about Axis2/C internals

See my post on the topic

Reliable Messaging with Sandesha2/C

See my post here

Karate Do and Self Defence

Nowadays there I have heard that as Self Defence there is no use of learning Karate. They say it is only a sport or a way of creating a healthy body and mind. But for me it can never be a good sport and it is the best form of self defence. With an opponent weilding a gun infront of you what self defence Karate can provide to you? they ask.

Before answering this question we must know the meaning of the phrase “Self Defence”. Defend the Self from whom?. Your opponent from whom you could defend yourself could be anything. It could be a flowing stream, a strong wind, a falling tree, a fierece dog, a burgler or it could be even a physical ailment and most of the time it is your own fear.

To defend from these opponents your self you must be always prepared mentally and physically. By practising Karate devotedly, diligently prepares you for this. It is moving meditation.

It may be true that you cannot avoid a bullet triggered at you in an alarming speed. But before that bullet is triggered at you, you have a thousand and one things to do to defend yourself if you have mastered Karate.

First you could avoid such confrontations by being not angered easily or by not making other ppl angered by you. You could avoid dangerous ppl and places by improving sense of awareness which is result of moving medition of Karate. Even then if you could not avoid such confrontations you could always hide or run before being confronted. To hide, to run effectively you need greater phsical ability. For example to instantly jump over an 8 feet high wall or nimbly climb up 15 feet tree require great phisical strength and some sense of correct technique which are all results of years of training(You don’t train to climb trees and jumping over walls while practsing Karate. But you are in a better possition than average ppl to successfully doing these when need arise.). Even if you cannot avoid the confrontation by these methods you can attack him in lightening speed before his finger touch the trigger(you are trained for such speed and power). Even if you could not avoid that then you can die bravely and unregrettedly which is the highest attainment the Karate traing could give you.

But realistically being confronted by a opponent with a gun is an incident that an ordinary person face not even once in a life time. But he is sure to be confronted by fear, decease and his own anger many times in life which could effectively devastate one’s self.

It is interesting to see that ppl who do not have time for practising this moving medition(or whatever form of exercise) at their younger age find ample time in thier old age to suffer from ailments which are unavoidable result of unhealthy mind and body.

I would like to produce following excerpts from the Master Text “Karate Do Kyohan” which has been my Bible for “self defence” for more than a decade.

* There is absolutely no need for concern about one-sided development of the body in Karate, and the fact that unform development may be considered to be on of the benefits of Karate.

* In most cases, only a minute or two is required to complete a Kata(my note:Predefined sequence of techniques that is soul of Karate) . Moreover, as one continues to practice, the movements become quicker and the training as a whole more vigours, so that one can get ample exercises from a relatively short period of time.

* No specific area, equipment, or even partner are necessary, for it can be performed in a garden, living room, hallway, at any time or place that one feels the desire to practise.

*Other contemporary karate experts such as masters Yamaguchi, Aragaki, chibana, Nakazato, Yahiku, Tokashiki, Sakihara, and chinen, have all lived to be over eighty. These examples are indicative of the role of Karate as a superier method of maintaining one’s health.

* It is not an exaggeration to assert that a man well trained in this form of defense may consider the whole body to be a weapon of awesomely effective offensive power.

* Although Karate does have throwing techiniques, it relies prinicipally on striking, kicking, and thrusting techiniques. These movements are much quicker and can escape the untrained eye. Block -attack combinations are executed simultaneously, and weaker individual, women or young boys, do have ample strength to control a more powerful opponent with them.

* An insight into this art, a mastery of its techniques a polishing of the virtues of courage, courtesy, integrity, humility, and self-control to make them the inner light to guide one’s daily actions:these require at the least ten or twenty years, if possible a lifetime of devotion to the study of this art.

* One who truely trains in this do and actually understands Karate-do is never easily drawn into a fight. One attack or a single kick determines life or death.

* True Karate, that is , Karate-do strives internally to train the mind to develop a clear conscience enabling one to face the world truthfully., while externally developing strength until one may overcome even ferocios wild animals. Mind and technique are to become one in true Karate.

* It is the small-minded individual who likes to brag upon acquiring some small skill, and those with little knowledge who carry on as if they were experts are childish. It is because of the large number of false martial artists in the world that the public tends either to ignore the martial artist or to consider him wild.

* For the Karate-do student, the most shameful trait is indecisiveness.

I hope these ideas let newcommers to do correctly

Amazon web services client using wsclient

These days I’m working on a command line web services client to consume Amazon AWS Web services.

The idea is to use this tool from command line easily to do transactions with amazon. It can search for items, then lookup for searched items for details and review the details of items and when finally decide to buy it and add to the amazon shopping cart.

The name I put for the client is amazoneclient(subjected to change).

User can get help for using the tool by

amazoneclient –help.

There are two kinds of options user can provide; general options and operation specific options.

For example

$amazoneclient –subscription_id=<your access key> –op=ItemLookup –op_args=”ItemId=B00008OE6I ResponseGroup=OfferFull”

–subscription_id is a general option. Item_id is a operation specific option for the operation ItemLookup.

wsclient internally use wsf/c wsclient as the web sevices client. I hope to finish the work on this tool within a week. I’ll update you with more information then.

Be Humble, Be Honest

I read this article about the life style of Iranian President. This is unbelievable cosidering the fact that Iran is much more wealthier country compared to Sri Lanka. Whatever their political view every politician in Sri Lanka should consider the facts in this article seriously provided they claim themselves as volunteers dedicated for the benefit of the country and it’s people. Not only politicans, we all should know what is the economy of our country(family) could afford and do accordingly.

http://lankaguardian.blogspot.com/2008/02/will-our-politicians-emulate-iranian.html

Soap Action and Addressing action

For me it has alway been a headache to understand and remember the complexity behind soap action and web services addressing action. As I have again started working on WSFC/wsclient a wget like command line tool specifically designed for consuming web services, I thought of writing down the facts from the corresponding authentic sources clearly for the benefit of at least me for future reference.

In WS-I version 1.1 it states

The SOAPAction header is purely a hint to processors. All vital information regarding the intent of a message is carried in the envelope.

– `A HTTP request MESSAGE MUST contain a SOAPAction HTTP header field with a quoted value equal to the value of the soapAction attribute of soapbind:operation, if present in the corresponding WSDL description.`

– `A HTTP request MESSAGE MUST contain a SOAPAction HTTP header field with a quoted empty string value, if in the corresponding WSDL description, the soapAction of soapbind:operation is either not present, or present with an empty string as its value.`

Again the `Web Services Addressing 1.0 – WSDL Binding` states clearly about the explicit association between wsa:action and SOAPAction as follows

`WS-Addressing defines a global attribute, wsaw:Action, that can be used to explicitly define the value of the [action] property for messages in a WSDL description. The type of the attribute is xs:anyURI and it is used as an extension on the WSDL input, output and fault elements. A SOAP binding can specify SOAPAction values for the input messages of operations. In the absence of a wsaw:Action attribute on a WSDL input element where a SOAPAction value is specified, the value of the [action] property for the input message is the value of the SOAPAction specified. Web Services Addressing 1.0 – SOAP Binding[WS-Addressing SOAP Binding] specifies restrictions on the relationship between the values of [action] and SOAPAction for SOAP 1.1 and SOAP 1.2.

The inclusion of wsaw:Action without inclusion of wsaw:UsingAddressing has no normative intent and is only informational. In other words, the inclusion of wsaw:Action attributes in WSDL alone does not imply a requirement on clients to use Message Addressing Properties in messages it sends to the service. A client, however, MAY include Message Addressing Properties in the messages it sends, either on its own initiative or as described by other elements of the service contract, regardless of the presence or absence of wsaw:UsingAddressing. Other specifications defining the value of [action] are under no constraint to be consistent with wsaw:Action.`

 

 

I do not like to interpret the above readings from the specifications with my own wording and confuse the reader or myself later. The best thing is to re-read the above and to refer if neccessary to the relevant sources to understand this clearly.

Making Logging with Axis2 C Fun !!!

I have this problem with Axis2 C logging system where I work with a branch of Axis2 C for the project I’m currently working on. In this project I have several services deployed in Axis2C. While developing these complex services I put whole lot of debugging log messages to track the development process. My problem is that when axis2c engine is started with log level of debug it prints whole bunch of engine/transport related debug messages so that my debug messages of the service which are my current interest are obscured. It is true that the engine/transport related debug messages are of importance but at this level of my debugging I can safely assume that engine/transport workes as intended and stable I have no interest in debugging them. My only interest is seeing the log messages clearly for the service I’m developing now.

In one of my services I periodically send soap messess reusing a service client. For each message I send, there are hundereds lines of repeated messages from engine/transport. Not only that, because of this within few minutes the files size become several megabytes.

The unlimited growth of log file without any log rotating mechanism is another issue for which I have put an jira proposing a simple log rotating mechanism for it.

When I discussed this issue with Dinesh he suggested that this should be handled at service level, may be by editing and inserting a parameter in services.xml. The solution I have provided below is not exactly that but his idea of handling at service level is taken.

I have provided a soultion for the problem at hand in the axis2c branch using for the project. It is to provide another logging function in addition to the existing set

AXIS2_LOG_DEBUG()

AXIS2_LOG_INFO()

AXIS2_LOG_WARNING()

AXIS2_LOG_ERROR()

AXIS2_LOG_CRITICAL()

The new logging function is AXIS2_LOG_SERVICE()

This function can be called from the service code you are writing. Then you can start the axis2c engine with the new log level AXIS2_LOG_LEVE_SERVICE. Then you will find that only the critical/errors from the engine/transport are printed to the log file and all the messages you put as AXIS2_LOG_SERVICE are also printed. Now it is very useful because you can easily highlight the problems at hand.

So suppose your service have following log messages

AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI, “Error log message”);
AXIS2_LOG_DEBUG(env->log, AXIS2_LOG_SI, “Debug log message”);

AXIS2_LOG_SERVICE(env->log, AXIS2_LOG_SI, “Custom Service Log”);

Now can you guess what messages are printed and when?

Answer is when started the sever with run level AXIS2_LOG_LEVEL_SERVICE only error/critical and service messages are printed, not the debug level log messages from engine/transport.

If you start the service with log leve debug then all the messages are printed.

Simple solution which save lot of energy right?

One other thing that is needed to make logging with axis2c fun is the ability to plug in some other logging mechanisms to Axis2C.  For example I have a requirement in my project  to log critical errors to syslog while printing other levels to axis2c log.  I’m thinking how can this be solved immediately.

About Apache Savan/C First Release

Last week I released Apache Savan/C version 0.90 version.

It can be downloaded from
http://ws.apache.org/savan/c/download.cgi
This release is supposed to be used with the Axis2C 1.1.0 distribution which is released recently.

Key Features:

1 Support for WS-Eventing Specification August 2004.
2 Support for Subscribe, Unsubscribe, Renew and GetStatus operations.
3 Support for subscription management endpoint which could be the
Event Source itself or separate subscription management endpoint.
4 Persistent storage for subscribers.
5 Support for both SOAP 1.1 and 1.2.
6 Samples to test scenarios.
7 Documentation
I have just finished writing an tutorial on Savan/C which will be soon published on Oxygen Tank. I also plan to write an article on my experience of using Savan C on a real project. Your early feedback on this implementation is most welcome.

The First Impression

I’m using WSF/C in a project and here is some useful tips that we used in out project that may help WSF/ C more attractive to the linux user.

There is a standard body for File System Hierarchy Standard for Unix-like operating systems. We followed that standard and make our components install to these standard places. For example our libraries go to /opt/wso2/lib, our logging goes to /var/log/opt/wso2/x.log etc. The idea is by making it to standard places one who uses it intuitively know where to look for certain files and also make it all easy to provide standard packages like rpm and deb packages. There are many other advantages as you can see in

http://www.pathname.com/fhs/pub/fhs-2.3.html

Other thing is decide on how to break up the distribution into several packages. Instead of distributing a one big rpm or deb package we can break it into several packages and the interested user will download and install only the components he is interested. He can always download and install other components as required. For example in providing rpm’s we can give util rpm, axiom rpm, core rpm and even apache module rpm, xmpp transport rpm etc. Also we can provided dev rpm’s test rpms etc. Of course we need to put a dependancy on util rpm in core rpm etc. Also the default installation should include minimum useful set of rpm’s(same apply for debs).

We can provide a set of test rpm’s which enable the user to immediately make sure that everything goes on well in a very short time. In the work I’m doing I have a collection of axis2c services for various purposes. In my tests I deploy these services in the engine, start the engine as an apache module and send some client requests against these services and then show the success/failure of these tests using CUnit at client side. All this is done by running a single CUnit test file.

rpm -i <wsfc package>

rpm -i <product component 1>

rpm -i <product component 2>

rpm -i <product component 3>

rpm -i <test suite>

./test_client.

That’s all the user need to do to test soe product components. This also make it easy to write good scripts to automate the things.

Great Team Work

I had the greatest teamwork experience during our off-season journey to Sri Pada couple of weeks back. Our team consisted of Kaushalya, Nandika, SanjayaR, Dinesh, Dushshantha, Amila, My brother in law(Upul) and Me. We choose the route along Kuruwita which is the longest(11.5km) and the most difficult of the three main roads leading to the the top. We set out early in Saturday morning hoping to comeback in Sunday evening and make it to the office by Monday morning. In the off-season the stoney steps are wowen with moss which cause a slippery serface slowing the progress. It was about 7pm when we make it to the top finally. We spent the night in a hired room with minimum facility of some mats on the cold floor. But we were lucky to watch the sun rise in the early morning which I must say is a very rare experience which only you can enjoy freely only in the off-season. But the problem in the off-season is that you are often prevented from seeing it by clouds and mist but in our case we were lucky. When we set out the return journey it was sunny all over the mountain range but at the middle of the journey heavy rain began to pour down. We were all equiped with rain coats so it did not hinder our progress, but the danger was awaiting us!.

Amila, Dinesh and Upul Aiya lead rest by about 20 minutes and they managed to cross Seetha Gangula before us. Seetha Gangula is a small river across the Sri pada path. It is about 30 meter’s wide and normally you can wade it with water upto your knees.

But by the time rest of us reach it, it began to overflow making it difficult to cross. Infront of our eyes the water level was rising rapidly so we decided to stay in the Ambalama nearby(Ambalama is the sinhala name to the resting place for people to use for rest in long journeys) . Luckily there was Ambalam in both side of the river. I think this was built on purpose because the rising of water is a usual thing to happen and people caught in it can stay on these Ambalama’s until the water level reduce.

Nature has done a nice separation between us on the two banks. All the matchbox and lighter was with us. Firewood was with them!!!. Raw Noodles we took with us to cook for lunch were with us but the pot to cook with was with them!!!. So SanjayaR, Kaushalya, Nandika, Dushshyantha and me who were trapped in this side had nothing to do, but pray for the Rain to stop. The time was 1pm. There was no sign of stopping the rain. There was a great chance that we may have to spend the night there. There were no signals to our phones so that we could not inform our families that we may get delayed. It was unimaginable to stay the night with no food and in the cold and dump Ambalama. We need a fire somehow!!!. All were hopeless for some time and everybody looked bewildered. But slowly but gradually the team started to get it’s spirit back thanks to Sanjaya who managed to make some fire by using small leftover’s of some firewood and papers. But how we are going to keep that fire going?. I remembered one of my previous similar experience and knew that wet logs can be make good wirewood if we can dry them for some time in fire. So Kaushalya and me collected a good amount of wet logs from around the Ambalama. That worked very well and after some strenuos effort we were having a good fireplace promising to give warmth for a whole night. Now we are warm next problem is food. Then Nandika came with that great innovation of making a pot using some left overs of a rectangular Aluminum sheet which he managed somehow to find from within the Ambalama. So we managed to cook the noodles using his primitive age cook pot. I have never enjoyed such delicious noodles !!!.

By the time the rain stoped it was about 8pm. The rain was so heavily pouring non-stop for about 8 hours!!!. For water level of the water to reduce it took about another two hours. By the time we crossed the river it was 10pm.

We managed to come to the office by Monday 2pm.

An Axis2 C design issue?

There was this discussion between Sanjaya Karunasena, Samisa and me regarding the application/session scope supported by Axis2 C engine when it is deployed as an Apache module.

The problem that initiated the discussion is as following. In Savan C(The WS Eventing Specification implementation of Axis2 C web services Frame Work) when a subscriber send a subscription message to a Data Source, that data source maintain the subscriber’s state in a hashmap stored in one of it’s service parameter. So the next time the subscriber send a Renew Subscription message or a GetStatus message or a Unsubscribe message to the data source, that subscriber’s identity is updated accordingly in the service parameter’s hashmap.

This scenario works as expected when axis2 C is deployed as an axis2c stanalone server. But when Axis2 C is deployed as an Apache Module this does not work. I will explain it as following. when a request comes to a Axis2 C service, the Apache web server serves it in a thread spawn from a certain process. The next reqeust from the same requester for the same service may be served by a different thread from the same process or a different thread from a completely different process spawned by Apache2 to increase the performance. When a new process is spawned by Apache the Axis2c module’s init module function is called for that process, resulting in a new Axis2 C configuration context. A new configuration context means a new serivce context!!!. Now all the state stored in the earlier process’s service parameter is lost in this new service context’s service. Because of this problem the subscription information stored in the hashmap of the datasource service’s parameter in the subscription request may be lost when another request comes regarding the state of the subscription.

To solve this problem what I did is to store the subscriber’s state to a database. But Sanjaya’s argument was this is a hack the developer has to provide because of the Axis2 C engine’s inability to maintain the application or session scope as it promise. But Samisa argued back saying that it’s wrong of Apache2 to init the Axis2 each time it serve the new request. But Sanjaya is saying that still we should be able to do some inter process communication to keep the session or application scope as required. He proposed to do a multicasting as it is done in gmond applications to propogate the state between the machines in which the gmond is running. He proposed that this same solution can be extended when it comes to load balancing support in Axis2 C.

It was also discussed to find out how other Apache module developer’s who need to maintain state between module instances do this. For example how PHP do this? The discussion ended agreeing to start a discussion thread on this.

Subscription manager for savan C

I have written a subscription manager for savan C implementation. According to the Ws eventing specification the subscription clients subscribe for a eventing topic which is a web service hosted somewhere. It can handle it’s own subscriptions or it can delegate the subscription management to another web service(subscription manager service).

In implementing this I used the following technique. If the subscription manager service and topic service reside in the same axis2c engine container then when a subscription is received to the topic service, the subscriber list of the subscription manager is updated by simply getting the subsription manager service from the configuration context by name. If the subscription service and topic service reside in two differnt axis2c engine instances then use soap to communicate between them through tcp transport.

I initially thought that whether the topic service and subscription manager service reside in the same engine or not should be transparent when I doing the coding, by using an internal transport(yet to be developed) . If the two serivce are in the same service container then it should automatically infer to use internal transport instead of tcp. But since I have the subscriber list as a service parameter value I could be content with the previous techinique I explained.

My subscription manager support following functions

add_topic

remove_topic

get_topic_list

add_subscriber

remove_subscriber

get_subscriber_list_for_topic