Feeds:
Posts
Comments

Archive for July, 2008

Axis2/C Performance

Axis2/C benchmark performance test results are published here

Read Full Post »

lighttpd module for Axis2/C

Dinesh has developed an Axis2/C module for lighttpd (pron. lighty) a light weight web server is designed and optimized for high performance environments. It claims to power Youtube. Also see.

I mentioned that Axis2/C version 1.5 reached 25K tps performance margin running on Apache2. I hope Axis2/C running as a lighttpd module will further extend this margin. Still there is some work to be done before it could be brought to production use. But the important thing is the module is working now.

Few words about Dinesh’s effort on the work. He started on this module two weeks before he left WSO2 for his higher studies in USA(And he married last month !!). In the middle of all his preparation for new life he had time and will to bring this work to a working state. At this time of his parting I would like to remind his great contributions to Axis2/C project. He workied on lot of ares’s in Axis2/C code. He initiated Guththila Stax parser, XMPP transport and contributed to wsclient command line tool to name a few. I think he is the one so far to be the release manager for most number of Axis2/C releases. Kudos Dinesh for your great work.

Read Full Post »

Axis2/C and Rampart/C is used as main technologies in an open source EC2 cloud computing implementation. See the main technologies used in this document and this. See this infoQ vidio for a presentation of Eucaplytus by its lead.

Read Full Post »

I would like to talk about our programming model when it comes to Axis2/C Apache2 module. As discussed in a earlier blog suppose two requests come to Axis2/C sequentially. Say the first request is served by one apache process and the next request is served by another process. Then problem is you cannot access the previous requests configuration context from your next request because each process has it’s own configuration context. We recently solved this problem using shared memory. But as should be expected this is extremely inefficient.

The other and more effiecient solution is using a database that could be shared among your processes. Sandesha2/C uses this approach. To solve concurrency issues when using this approach we can use Apache2 golbal mutex which could be used when accessing global resources like system files and databases. But the problem is how we could pass this global mutext from Axis2 apache module to be accessible from other parts of the code, for example from a Axis2/C module. We currently pass Apache2 pools through axis2_env_t environment. My understanding is that we could use a similar method to solve the problem.

Read Full Post »

Interesting bike journey, again using XT225 serow

Read Full Post »

According to my performance testing on Axis2/C 1.5(Soon to be released) I could reach 25K tps with keep alive on. When tested with a empty module instead of Axis2/C where no handling of soap data is done but just set a HTTP_OK and return it, reached 40Ktps mark(This is also similar to index page request as might be expected). This means there is still room for 15K improvement. Profiling tools show that mostly the improvements needs on parser.

Also when I experiments with using a localpool instead of the request pool of apache the performance degraded by about 1000 tps. This further shows that there could be further performance gains that could be achived perhaps by improving apache2 module. Has anybody thought of implementing Axis2/C module as a apache2 filter or using filters instead as an apache2 content generator handler?. In apache2 documents it says that

‘Any form processing application is normally be implemeneted as a handler-perticulary those that accept POST data.’

However in the same documentation it says that markup processing libraries better be implemented as filters or using filters.  It also says that parsers like expat and libxml2 which have parseChunk APIs work well with apache2 filters. Working with filters give direct access to bucket brigades which gives the control into the filter developer for data manipulations.

I heard somebody is developing a expat parser plugging for Axis2/C. It is a good thing to do a performance comparision with that also.

These directions are for future considerations only. Axis2/C is already ready for  performance hungry  deployments, may be ahead of other competing  web services engines. I have actually plans for doing some performance benchmarks using Axis2/C as a soap gateway in an EC2 cloud.

Read Full Post »

Now that I have done some performance tests(Results will be published soon. In summary Axis2/C 1.5 which will be released within next week achieved 25K tps benchmark) I thought of using a profiler to see which part of codebase need further attention for improvement. I profiled with google proftools and valgrind profile tools.

perftools

proftools has a cpuprofiler and a heap profiler. You need to just download it and ./configure, make, make install.

Then

export CPUPROFILE=/tmp/axis2_http_server.prof
export HEAPPROFILE=/tmp/axis2_http_server.hprof

where first one is where cpu profiler output file is created and the second one is where heap profiler output file is created. These files will be created when you run your program.

You need to build Axis2/C with -ltcmalloc and -lprofiler.

Just add those linker options into configure.ac CFLAGS.

Now you can your executable as normal. After that

For cpu profiling

./pprof –gv axis2_http_server /tmp/axis2_http_server.prof.

This will give you a nice graph. You must have installed dot, gv and perl5 installed in your system.

If you have installed kcachegrind in your system you can use it to have nice graphical view by

pprof –callgrind ./axis2_http_server /tmp/axis2_http_server.prof > axis2_http_server.callgrind

kcachegrind axis2_http_server.callgrind

For heap profiling

–callgrind ./axis2_http_server /tmp/axis2_http_server.hprof.0001.heap > axis2_http_server.callgrind

again you can use kcachegrind to have a nice gui view.

kcachegrind axis2_http_server.callgrind

Note:I found that proftools are only good for profileing Axis2/C client side because I could not find a way to capture profiling for request handling in server side. If you need to just profile deployment in Axis2/C then this can be used.

Callgrind

You can separetely install callgrind. But if you install valgring in dabian uisng apt-get install you can use it by using

valgrind –tool=callgrind < your program name>

I used callgrind to profile simple axis2 server as follows

valgrind –tool=callgrind ./axis2_http_server

Now make a request to the server.

Now you have a file generated in your current folder called something like callgrind.out.8307.

Use this as input to kcachegrind

kcachegrind callgrind.out.8307

Important: I did the following addition in my src/core/transport/http/receiver/http_svr_thread.c code towards the end of server while loop to exit the server after serving two requests. Instead if I use ctrl-c to signal the server to stop then profiling tool won’t be able to capture the requests for profiling.

if(counter == 1)
{
svr_thread->stopped = AXIS2_TRUE;
}
counter++;

}// end while loop

My conclusion is that both of these tools are good for profiling in a threaded environment. Problem with gprof is that it could not be used for profiling in a threaded environment. Although there is a hack to do this I saw a user say that it is not trustworthy.

Read Full Post »

Older Posts »