Feeds:
Posts
Comments

Posts Tagged ‘httperf’

In some of my previous blogs [1], [2], [3] I mentioned about some of the open source performance testing tools I have been using. Among them were httperf, Autobench, Apache Bench and Java Bench.

I used these tools heavily to performance test WSO2 open source products. While doing so I understood that the performance test tool, usually the load generator is not alone adequate to fulfill my performance test requirements. Load generation is only a part of my requirement. The bigger picture required a systematic approach to performance testing which, none of these tools alone could fulfill. So what are my requirements?

– Provide easy to use standard configuration system so that tests could be configured easily.

– A standard way to provide my input for the tests

– A standard way to collect output from my tests.

– Way to collect system statistics while running the tests

– Automate the whole process. Just provide input data, required configuration and run the test. That’s it. Then collect the results

– A standard way to organize tests and the results

Commercially avaiable performance testing tools provide some or all of these requirements but for a very high cost. The name of the framework I implemented with Eranda to fulfill above requirements, is Ravana. Ravana will be released during next couple of weeks licensed with GPL. Ravana will be released with helpful documentation that will get you familiar with Ravana immediately.

Using Ravana is simple. All your input, configuration, output for a particular test scenario will come  under one test scenario you define. You just need to define your input, configuration under that scenario and then run the test.

With Ravana you can use one of httperf, java bench or apache bench as the load generator. It also support a JMS load generator.

If you are too curious and need immediately try  Ravana, check it out from https://svn.wso2.org/repos/wso2/branches/commons/performance-tools/ravana2. The code in svn is pretty stable now.

[1] https://damithakumarage.wordpress.com/2009/03/15/benchmark-testing-with-httperf/

[2] https://damithakumarage.wordpress.com/2008/07/20/profiling-tools-that-i-used-to-profile-axis2c/

[3] https://damithakumarage.wordpress.com/2008/07/11/benchmark-testing-tools-for-web-services/

Advertisements

Read Full Post »

WSO2 wsclient can consume web services from your platforms command line shell. I have already introduced it in my previous entry titled Access security enabled web services from command line and in an article titled Calling web services from the shell

Today I discuss about generating your custom soap messages using wsclient. You may need pre-built custom soap messages for various purposes. For example your performance test tool may need pre-built soap messages in order to generate load for the target server, or you may need to customize each of your message sent to the target server and observe how server react.

Here is a sample usage of wsclient to send a message to target server at http://localhost:8280/services/EchoProxy

./wsclient –action echo –log-level debug –soap –no-mtom http://localhost:8280/services/EchoProxy < ./data/msg.xml

So your input file msg.xml contain

<m:buyStocks xmlns:m=”http://services.samples/xsd”&gt;
<m:order>
<m:symbol>IBM</m:symbol>
<m:buyerID>asankha</m:buyerID>
<m:price>140.34</m:price>
<m:volume>200000</m:volume>
</m:order>
<m:order>
<m:symbol>MSFT</m:symbol>
<m:buyerID>ruwan</m:buyerID>
<m:price>23.56</m:price>
<m:volume>803000</m:volume>
</m:order>
<m:order>
<m:symbol>SUN</m:symbol>
<m:buyerID>indika</m:buyerID>
<m:price>14.56</m:price>
<m:volume>500000000</m:volume>
</m:order>
</m:buyStocks>

Now we want instead of sending that message to the server, dump it to a file.

./wsclient –action echo –log-level debug –soap –no-mtom –soap-dump http://localhost:8281/services/EchoProxy < ./data/msg.xml > output.xml

The trick is –soap-dump option passed to the wsclient.

Here is the message dumped into a file by the wsclient

<soapenv:Envelope xmlns:soapenv=”http://www.w3.org/2003/05/soap-envelope”&gt;
<soapenv:Header xmlns:wsa=”http://www.w3.org/2005/08/addressing”&gt;
<wsa:To>http://localhost:8281/services/EchoProxy</wsa:To&gt;
<wsa:Action>echo</wsa:Action>
<wsa:MessageID>urn:uuid:da14cd20-6820-1df1-2c4f-000000000000</wsa:MessageID>
</soapenv:Header>
<soapenv:Body>
<m:buyStocks xmlns:m=”http://services.samples/xsd”&gt;
<m:order>
<m:symbol>IBM</m:symbol>
<m:buyerID>asankha</m:buyerID>
<m:price>140.34</m:price>
<m:volume>200000</m:volume>
</m:order>
<m:order>
<m:symbol>MSFT</m:symbol>
<m:buyerID>ruwan</m:buyerID>
<m:price>23.56</m:price>
<m:volume>803000</m:volume>
</m:order>
<m:order>
<m:symbol>SUN</m:symbol>
<m:buyerID>indika</m:buyerID>
<m:price>14.56</m:price>
<m:volume>500000000</m:volume>
</m:order>
</m:buyStocks>
</soapenv:Body>
</soapenv:Envelope>

You may argue that you can hand write these messages without going into hassle of downloading and using wsclient to do the task. But the real important usage comes when your target server expect ws secured soap messages. How do you encrypt and/or sign your messages?. wsclient come into rescue.
Say you need to encrypt and sign your message. Here is how you do it in wsclient.

./wsclient –log-level error –no-wsa –soap –no-mtom –sign-body –key $WSFC_HOME/samples/src/rampartc/data/keys/ahome/alice_key.pem –certificate $WSFC_HOME/samples/src/rampartc/data/keys/ahome/alice_cert.cert –recipient-certificate /home/damitha/projects/perftest-framework/wsclient/wso2carbon.pem –encrypt-payload –policy-file ./policy.xml –soap-dump http://localhost:8280/services/EchoProxy < ./data/message1k.xml > output.xml

I can take you one step further by showing how this could be useful when using httperf,  a populer opensource performance testing tool.

Say you need to load your server with signed and encryped timestamped messages with a nonce. Each of your messages should be different. That means your input file to httperf should not contain the same message.

My trick is to generate an input file with as much as different messages by using the following script, which uses wsclient

echo -n "/services/SignEncProxy method=POST contents=\"" > inputfile
./wsclient --log-level error --no-wsa --soap --no-mtom --sign-body --key $WSFC_HOME/samples/src/rampartc/data/keys/ahome/alice_key.pem --certificate $WSFC_HOME/samples/src/rampartc/data/keys/ahome/alice_cert.cert --recipient-certificate /home/damitha/projects/perftest-framework/wsclient/wso2carbon.pem --encrypt-payload --policy-file ./policy.xml --soap-dump http://localhost:8280/services/EchoProxy < ./data/message1k.xml > inputfile_temp
perl -e 'while (<>) { if (! /\|$/ ) { chomp; } print ;}' inputfile_temp >> inputfile

echo -n "\"" >> inputfile
echo "" >> inputfile
echo "" >> inputfile

for i in {1..10}
do
echo -n "/services/SignEncProxy method=POST contents=\"" >> inputfile
./wsclient --log-level error --no-wsa --soap --no-mtom --sign-body --key $WSFC_HOME/samples/src/rampartc/data/keys/ahome/alice_key.pem --certificate $WSFC_HOME/samples/src/rampartc/data/keys/ahome/alice_cert.cert --recipient-certificate /home/damitha/projects/perftest-framework/wsclient/wso2carbon.pem --encrypt-payload --policy-file ./policy.xml --soap-dump http://localhost:8280/services/EchoProxy < ./data/message1k.xml > inputfile_temp
perl -e 'while (<>) { if (! /\|$/ ) { chomp; } print ;}' inputfile_temp >> inputfile

echo -n "\"" >> inputfile
echo "" >> inputfile
echo "" >> inputfile

done
sed s/'"'/'\\\"'/g < inputfile > x
sed s/'Envelope>\\"'/'Envelope\>"'/g < x > y
sed s/'contents=\\"'/'contents="'/g < y > z
cat z > inputfile

rm x y z inputfile_temp

Above script would generate an httperf input file for http post with 11 different soap messages.
In the above example codes and scripts make sure that you replace paths with your own environment.

Read Full Post »

In a previous blog I discussed benchmark testing tools I used for some benchmark perposes. At that time I really used Apache Bench(ab) for the purpose. However when I am again doing some bench mark testing I stumbled upon a new tool(at least for me) httperf which seems to be more advanced and reliable than ab. Although there were ample guidance on how to use this for http GET requests I could not find any easy to read guide on how to use httperf with http POST requests. Following I try to explain how httperf could be used in testing my services.

httperf --server localhost --port 9090 --uri /axis2/services/weather
     --num-conn 4000 --num-call 16 --rate 64 --timeout 5 > report

Above command is a simple scenario where I request a HTTP get request on a servic running on context /axis2/services/weather on localhost.

It is important to understand the  options for the command.

1. –server – ip of the machine service is running

2. –port – port the service is running

3. –uri – The context path of the service on the server

4. –num-con – Number of test calls made to the service

5. –num-call – Number of calls per tcp connection.

6. –rate – Number of connections created per second to make requests to the service.

7. –time-out – This is the maximum time httperf wait for a successful response.

It is important to understand the correct balance between num-conn, num-call and rate options.

The maximum number of requests the httperf would generate = num-call * rate

Clearly your num-conn could be reasonably higher than this number. You could experiment between num-call and rate to produce a good load to the server. The actual number of requests to the server you will see in the generated report could be less than this number depending on the saturation of the server, client or network bandwidth. If such thing happen you need to reduce your num-call or rate options and find out at what point exactly saturation happen. If your num-conn is very large your server may exaust all servicing processes after some time.

Now let’s look at a more advance scenario. Here I am sending a http post request with some payload defined in the file name inputfile.

httperf --server localhost --port 80 --uri /axis2/services/weather --hog --method POST
    --add-header="Content-type:application/soap+xml;charset=UTF-8``\n``"
    --wsesslog=4000,0,inputfile --max-piped-calls 16 --rate 64 --timeout 5  > result

In adition to the payload you can also specify the http method and the server context uri in this input file. For more descriptions on writing this input file read [1]. Following is the content of the inputfile I used.

/axis2/services/weather method=POST contents="<soapenv:Envelope xmlns:soapenv=
\"http://www.w3.org/2003/05/soap-envelope\"><soapenv:Header xmlns:wsa=
\"http://www.w3.org/2005/08/addressing\"><wse:Identifier xmlns:wse=
\"http://schemas.xmlsoap.org/ws/2004/08/eventing\">
urn:uuid:57becf58-0fc3-1de1-24ad-0012f0b857ac</wse:Identifier>
<wsa:To>http://localhost:9090/axis2/services/weather</wsa:To>
<wsa:Action>http://schemas.xmlsoap.org/ws/2004/08/eventing/GetStatus</wsa:Action>
<wsa:MessageID>urn:uuid:14d256a2-0f95-1de1-2b1a-0012f0b857ac</wsa:MessageID>
</soapenv:Header><soapenv:Body><wse:GetStatus xmlns:wse=
\"http://schemas.xmlsoap.org/ws/2004/08/eventing\"></wse:GetStatus>
</soapenv:Body></soapenv:Envelope>"

The important additional options of the above command is
1.–wsesslog – This value for this option is a three comma separated parameters. First one is the number of request sessions you need to make to the server. Second is used to emulate a real user delay between sessions. Third is the inputfile.

2. –max-piped-calls – Number of calls need in one tcp connection(in http 1.1). This is used instead of num-call here.

[1] http://www.xenoclast.org/doc/benchmark/HTTP-benchmarking-HOWTO/node6.html

Read Full Post »