Web Testing the Easy Way

JBlitz web test tool brings you an easy to use, comprehensive testing solution for your websites and web applications.

JBlitz simplifies the task of web testing by giving you the nuts and bolts control over what gets tested and how it gets tested. This approach avoids the needs for complicated web conversation recording tools and scripting languages.

Whether you're using the JBlitz Professional edition or the JBlitz Direct edition, our approach has always been that you, as web developer or test engineer, are the best equipped to define what parts of your website or web application need testing and how they can best be tested.

This produces testing that is pinpointed. It tests dynamic content and ignores static content. It lets you define likely pathways through your site and lets you define the dynamic portions of your URLs to give an effective overall test.

JBlitz's great looking GUI front end gives you a quick and easy way to configure your tests. A snapshot from JBlitz Pro is shown below.

Performing a 'Dry Run' to test stock item removals

Test cases are clearly laid out. Each page is shown from left to right with any query string or POST data separately highlighted. Dynamic test data is also highlighted. Context sensitive options are shown in a right mouse popup menu. Various UI configuration options are available to let you tailor the display to suit your needs.

Another snapshot, this time from the integrated HTTP debugger is shown below.

Using the integrated HTTP debugger

Proper web testing for your website or web application should ideally comprise different types of tests. You'll probably be interested in validating the integrity of the data returned under different scenarios. You'll need assurance that your website or web application can handle the anticipated number of users. You'll want to know how fast it is, and you'll probably be interested in identifying any performance degradation that could occur after prolonged use and that may lead to costly downtime.

For each of these principal testing modes, we've summarized how JBlitz can help you:

Load testing can also be good at identifying resource usage problems on the server (e.g. memory or database connections) where sustained high loading can lead to poor performance or even server outages due to memory leaks or running out of connections.

Each installation of JBlitz allows you to configure multiple virtual users - as many as you can practically use - without extra cost. Ramping up the load on your server does not need to burden your pocket!

Text searches can be configured either to check for the presence of 'error' text or the absence of some expected 'success' text. They can be configured to search for test data passed to the server in a previous URL. Searches are either plain text searches or regular expression pattern matches. The latter can be used to perform complex multi-term searches ala Perl.

Further, more detailed, error checking can be carried out by using the customization API to write a response checker. This gives you the flexibility you need to check every detail of each response page and raise a custom error when the checks fail. See the customization help page for more information on writing a response checker.

JBlitz is different from other test tools in that we don't ask you to pay extra money for extra virtual users. We like to keep things simple and easy. With every purchased license, you can configure as many virtual users as you like (see note*). For the same flat fee, you can choose to test with 10s, 100s or 1000s of virtual users - its up to you!

Note: for the retail versions of JBlitz, you can configure up to 500 virtual users per test case (Pro) / web page (Direct) and up to 500 test cases / web pages. For the evaluation versions of JBlitz you are restricted to at most 3 virtual users per test case (Pro) / web page (Direct) and at most 5 test cases / web pages.

Check out the features list below and learn how testing with JBlitz can bring your organization real benefits today!

From the outset, JBlitz has been written to be feature rich and easy to use. That's why you won't find hard to learn test script languages being used or a complicated recording / playback approach.

Our motto is simplicity:

• Create a test configuration file for each testing scenario you need to cover
• Create test cases within each configuration that target each functional area
• Enter the target URLs that query the dynamic content or do the work on your site / app
• Define straightforward dynamic substitutions that vary each URL and make your site work hard!

And simplicity does not mean lack of flexibility either. You can cater for out of the ordinary test scenarios, perform detailed response page checking or integrate with third party software by using the customization API. Write ten lines of Java code that plugs straight into the JBlitz query engine and you've got the ultimate in custom behaviour.

Loading is applied using 'virtual users' which work and behave just like real ones! Each operates independently to navigate through a sequence of web pages on your site. JBlitz uses operating system threads to implement virtual users internally - this means that there is minimal 'coupling' between the users and, given sufficient hardware resources, an effective load is applied. Each virtual user maintains cookie and other state independently from each other as it navigates.

Every purchased installation of JBlitz carries a theoretical upper limit of 250,000 virtual users. A realistic upper limit for the number of virtual users that could be run together on one test machine will depend highly on the machine configuration, its operating system, memory and CPU amongst other things.

The evaluation version allows at most 20 virtual users per test case and at most 3 test cases.

Each test case is allocated a number of virtual users that run through the pages independently from each other. Each one maintains its own state and is, to all intents and purposes, unaware of any other.

When testing, multiple test cases are run together within JBlitz. This allows you to test common functions on your website as if they were being independently accessed by different sets of users. A typical scenario would be to test read-only access methods in one test case alongside read-write update methods in another. So, for instance, if your website backs onto a database that holds customer details and order information, you'd probably want to test read-only customer and order enquiries alongside read-write 'add an order' or 'remove an order' requests. You may also want to add test cases that perform database maintenance and administration tasks at the same time (if you plan to have 24/24 availability).

Each response received can be validated to ensure data integrity and the correct functioning of your website or web application. Validation might be as simple as searching for an expected piece of text in the response, or it can be as complex as calling out to your own Java code to examine the response in detail. The choice is yours.

Further help on error detection and error handling in JBlitz Pro is available.

Note: for the retail versions of JBlitz, you can configure up to 500 virtual users per test case (Pro) / web page (Direct) and up to 500 test cases / web pages. For the evaluation versions of JBlitz you are restricted to at most 3 virtual users per test case (Pro) / web page (Direct) and at most 5 test cases / web pages.

The customization API has proven to be a great success in giving customers the power to 'tweak' their test environments. This simple, open API allows you to write a few lines of Java code that can be used to verify each downloaded page or, at a deeper level, to enhance, modify or even replace parts of the JBlitz query engine. When you write a customization module, you get access to every URL, each request, each response and you can register for a full variety of events. With the customization API, you truly do have the power!

Amongst other things, the customization API allows you to:

• Verify each downloaded web page.
• Integrate with third party software.
• Modify the sequence in which web pages are downloaded from the server (impose your own arbitrary download sequence).
• Alter the socket, request method, URL, timeout or request message for any given request.
• Perform custom error checking based on the response page or headers and raise custom errors.
• Listen to events generated as they occur - e.g. virtual user started, virtual user stopped, hit, error etc.
• Add your own logging or debugging etc.

Further help on using the customization API is available.

Currently, the following basic substitution types are available:

• Integer. %i00 -> %i99, %j, %k, %l, %m, %n. These substitutions generate random integer replacements. They are designed for simple queries based on integer parameter values - for example, collectionSearch?artifactId=%j could be used to query artifacts from a catalog website with a varying catalog index.
• Prior. %p1 -> %p9. This special substitution is used to regenerate the value generated previously for another of the substitution types. Especially handy for sending the same piece of test data to the server again in a subsequent URL or within a response search for finding an expected piece of test data within the response.
• Floating point. %r00 -> %r99. This substitution generates random floating point replacements within a specified range.
• String. %s, %S. These two substitutions generate random character sequences. They are designed for simple queries based on string parameter values - for example, addComment?itemId=%j&commentText=%s could be used to add test comments onto items.
• Boolean. %t, %T. These two substitutions generate the boolean values 'true' and false' either in lower or upper case.
• File content. %x00 -> %x99. File content substitutions replace portions of URLs, request headers or POST data with contents from an arbitrary file on the local file system. They can be used directly as a convenient short-hand for POSTing a document. Options are also given to URL encode the document contents. Documents can recursively contain other dynamic substitution elements.
• Item. %y00 -> %y99. Item substitutions are free-format replacements where each substitution element is replaced with a randomly chosen item from a set of items. Each item can be a number, a string or whatever. Item substitutions can recursively contain other substitution elements.
• File Item. %z00 -> %z99. File item substitutions are similar to item substitutions except that the items of test data are read in from an external test data file. Similarly, file item substitutions can contain other substitution elements. 

File, item and file item substitutions can be used recursively. This means, for example, that you can get JBlitz to transmit an XML document that itself contains substitution elements making the document contents vary dynamically too.

You can effectively define further substitution elements using the customization API. A sample module exists demonstrating this - com.clan.jblitz.custom.samples.DynamicData. This module looks for the substitution element %zz within any URL and replaces it with test data generated from a static list of test strings. A similar approach can be employed to implement your own substitution types.

Further help on defining and using dynamic substitutions is available.

With the debugger, you can:

• Step a virtual user through a test case.
• Inspect and modify request messages issued and responses received.
• Set breakpoints for virtual users on any given web page download.
• Break on error so that the debugger is invoked only if an error is detected.
• Run any given test case on its own or all test cases together.

Debug mode gives you the control you need to pinpoint problems and troubleshoot your web application. Available with JBlitz Pro 5.1 only.

Further help on using the integrated debugger is available.

Secure web pages are shown on the main screen with a 'padlock' icon.

A right mouse menu gives access to the virtual user control. Running virtual users can be suspended and suspended ones can be restarted. A traffic light icon to the right displays the current status of each virtual user.

Further help on virtual user activity and control is available.

• Via cookies that are sent back from the web server to the client (JBlitz) and which are then added as a cookie header with each subsequent request made. Cookie based state maintenance is commonly used to support server side 'sessions' that allow server software to track the state of any given client.
• Via name-value pairs extracted from response pages and used as input to the next request. Two variants of this are supported. Firstly, hidden form data where <form> tags in the response page contain name-value pairs in "hidden" <input> tags. This is a common way of specifying name-value pairs to be sent with the next request. Secondly, name-value pairs in hyperlinks where the state is coded directly into the hyperlink to be used for the next request.

Both these mechanisms are supported by JBlitz.

Cookie based state maintenance is either enabled or disabled globally. Once enabled, cookies are automatically maintained for every configured test case / web page.

Reply based state maintenance can be enabled or disabled globally, but also needs to be configured for each web page that state is transferred to. This per-page configuration describes how the previous reply page should be parsed and how the extracted state data should be handled. Reply based state maintenance relies on equating the next page as specified in the test case with either the action of the <form> tag  or the href of the <anchor> tag (depending on what variant is configured).

Further help on state maintenance is available.

• hits - any response that is not an error.
• errors - any response that has been deemed an error by one of the error detection mechanisms described below.
• exception errors - any request that does not generate a response - for example, if no connection to the server can be made, a 'could not connect to server' exception error is raised.

Error are detected in one of the following ways:

1. By finding a given error or success string within the response page body (regular expression or plain text searches).
2. By finding a given error or success string in the response page headers (regular expression or plain text searches).
3. By finding a match between the response HTTP code and a configurable list of 'error' HTTP response codes.
4. By encountering a 'wait time' error (slow connection or response times).
5. By a custom response checker returning an error response for any given web page download.
6. By a customization module returning a custom error in the responseBodyReceived() or responseHeadersReceived() methods of the QueryExtension interface. See the customization help for details on how to incorporate custom error checking within JBlitz.

All errors and exception errors are recorded in the 'Error Inspector' window. JBlitz can be configured to take a variety of actions when an error is encountered - it can stop the test run entirely, stop the test run for that test case, just stop the virtual user that encountered the error, or do nothing.

Further help on error handling and error detection is available.

At a basic level, each HTTP request issued can be logged to the standard output device by using the command line parameter '-debug http'. This flag also issues debug messages related to what state maintenance options are being used for each request.

At the next level up, the Log Console window within JBlitz records log messages related to the activities of each virtual user during a test run. This window has some options to log either to the screen, to standard error or standard out, or to a file. You can also ask it to only log errors if you're not interested in successful requests.

At the highest level, the Response Recorder and the Error Inspector record summary information about every request made and every error detected respectively. Recording is memory based for speed, so a configurable number of the most recent records are maintained (between 100 and 5000). These two logs can be used during and after a test run to gain further information on how its proceeding. Each log shows a table with columns that can be sorted and re-arranged, and at the bottom of each window there is a filter that can be used to narrow down the list of records displayed.

• hits - hits versus time
• errors - errors versus time
• transactions (Pro only) - transactions are completed cycles through a test case
• download rate - the KBits / sec or MBits / sec download rate
• response time - average response times for each time interval
• connection time - average connection times for each time interval
• responsiveness - a heuristic that can be used to estimate how busy the server is

At the bottom of each graph is a 'legend'. The legend is used to configure what data is to be displayed on the graph. You can pick any given test case (Pro) / web page (Direct) or all combined. You can also choose to narrow the data down by picking individual virtual users within a test case / web page, or by picking a given single web page within a test case (Pro only). 

The legend allows you to add any number of lines to the graph so that you can compare them easily. There are also various options for the line display type (line graph, histogram etc) and refresh policy.

Further help on graphs is available.

On the main window, summary statistics are shown in the bottom left corner. This outlines the hit / error count and the current momentary download rate.

To the right of each test case (Pro) / web page (Direct), pertinent feedback information is shown including the download rate for that test case / web page and the hit / error count.

Under the third tab of the main tabbed pane titled 'Test Results', further summary information is displayed for the test run including an overall success indicator.

Detailed statistics are shown in the Statistics Manager window. This window provides information on the hit / error count, the transaction count (Pro only), the download rate and the connection / response times. All this information is fully filterable so that it can be viewed across all test cases (Pro) / web pages (Direct) or per test case / web page.

Not just a Windows application, JBlitz leverages Java's cross platform capabilities to give you a test solution for Linux and Solaris too. In fact, JBlitz goes just about anywhere that Java does.

To use the console mode, you'll need to have defined a runnable test configuration already. By default, JBlitz will try to run the last test configuration that was worked on (you can alternatively specify which configuration to use on the command line).

When JBlitz is running in console mode, logging options set up to log to a file or standard output will be honored. Test results gathered during the run will also be saved to the configured test results file on exit. JBlitz will exit once the test run has stopped. You'll probably want to configure JBlitz to stop the test run after some predefined time interval - there is no way to manually stop JBlitz when its running in console mode.

You can configure JBlitz to run until a certain number of hits have been encountered, a certain number of errors have been encountered, or a certain time period has expired. A fourth alternative is to configure JBlitz to run continuously for prolonged loading.

Note also that error detection settings may also cause JBlitz to terminate a test run.

Further help on run modes is available.

Choosing the right product for your test environment can be hard. To make things easier, we've laid out the major steps that a customer might typically follow when working with JBlitz.

JBlitz comes packaged into two distinct editions to target individual needs more precisely. Before you download anything, you'll need to decide which edition of JBlitz is most suited to your test environment:

JBlitz Professional Edition 5.1 is our advanced all round web test tool. Encompassing all the functionality of the Direct Edition, plus pathway testing, state management, cookie support and hidden form data support.

JBlitz Pro allows you to simultaneously test website transactions which take place over many pages. Here a prime example is the 'shopping cart'. Typically, this would involve a logon, purchase(s) and checkout. You enter this as a test case into JBlitz Pro, by specifying the web pages which comprise the transaction and how many virtual users you'd like to dedicate to it. Add other test cases too. Together they make up the load you want to place on your website. Let's say you have a back end database for storing the purchases made on your site. Typically, you'd want to exercise the principal server side components which add, query, update and delete purchases - only then could you be sure of its integrity in the real world.

JBlitz Direct Edition 4.2 is a streamlined version of the Professional Edition for users who only need to stress test individual server resources rather than pathways. JBlitz Direct is fully multi-threaded and can efficiently load your website. But it cannot simulate a user's visit to your site by following a pathway through a set of web pages, it is restricted to repeatedly downloading single pages.

Free trial downloads are available for both JBlitz Professional and JBlitz Direct. Each is packaged up as either a Windows installation program (comes fully self-contained) or a ZIP file for all platforms (requires separate download of a 1.4 or higher JVM from Sun).

We recommend to all our customers that they download, install and evaluate the trial version of JBlitz Pro or JBlitz Direct before making any purchase. We want our customers to be sure they are buying a tool that's right for them.

See the installation page for detailed information on installing and running JBlitz.

JBlitz owes it's ease and simplicity of use to our fundamental design decision not to use recording based test scripts. As developers, designers and test engineers, you know exactly how your site should work and exactly what potential vulnerabilities it has - where there may be bottlenecks, where there may be defects due to concurrent access etc. Hence, the best way of creating test scripts is by you entering them into the tool directly.

So, the regular approach to testing might go like this:

1. Identify the areas within your website or web enabled application that need to be tested. There will be portions of your website / app that don't necessarily need testing - for instance static HTML pages and the like.

2. Work out the broad areas that should be tested together. The tests for each broad area will be contained in separate test configurations. Test configuration are XML based files. JBlitz works with one test configuration file at a time.

3. JBlitz Pro - for each test configuration, define the test pathways through your website or web app. These pathways are sets of URLs that are requested in sequence from the server and normally equate to testing a narrow area of functionality. Create a test case within JBlitz for each pathway. You can enter multiple test cases within a test configuration. Test cases have some of their own settings and also share some common settings defined at the test configuration level.

4. JBlitz Direct - for each test configuration, define the test URLs for your website or web app. JBlitz Direct can only test individual URLs. You can enter multiple URLs within a test configuration. Test URLs have some of their own settings and also share some common settings defined at the test configuration level.

5. Define the dynamically varying parts of your URLs. If you have a URL such as '/orderProduct?productName=teapot', you'll want JBlitz to run this URL with different product types. You can do this by using dynamic substitutions (in this case a '%y' choice substitution would be good). These substitutions will vary your URLs across each virtual user and each request to simulate pseudo-random queries from your users.

6. Work out how state is to be maintained across your test case 'pathways' (JBlitz Pro only). There's support for cookies (ASP / JSP / servlets etc) and state maintenance via hidden form data and embedded hyperlinks. See the help section on state maintenance for further information.

7. Configure how many virtual users should be assigned to each test case (Pro) or web page (Direct). Each of these virtual users is modeled using a 'thread' within the tool, and each runs concurrently. You can assign anywhere from 1 to 500 per test case / web page. Remember that for higher numbers of virtual users, higher specification test machines will be required. You can also define a pause interval between each web page request.

8. Work out what sort of testing you want to accomplish. For endurance testing, you may want to configure JBlitz to run under continuous load for, say, 24 hours. For functional testing, you can configure JBlitz to run until an error is detected. For performance testing, you may want to define several test configurations with increasing numbers of virtual users and run each for a given period - say an hour. You can then load the test results for each in turn to see the relative performance. See the hints on how to handle different types of testing.

Remember that you can take advantage of the customization API if you encounter out of the ordinary situations. Maybe you need to raise errors conditionally - look for the presence of absence of different  strings for different pages ? It can all be done quite easily with the help of a customization module. Check it out.

All test related configuration information entered into JBlitz is saved into the XML based test configuration file. You can work with a suite of these files. Maybe a configuration for each broad functional area within your site / app ? Or one configuration for an endurance test and another for a performance test ? You are also free to edit and manage these files outside of JBlitz if you find that easier.

A good source of further technical information is contained in the help page and the frequently asked questions page.

Software testing, like software development, is an iterative process. You'll doubtless be running your tests and refining your test configurations in light of the results. After a few iterations, you should have some test configurations you're reasonably happy with. You can form a suite of them that you can use to validate any changes to your website / web application. With luck, you'll have already highlighted (and fixed) some bugs and defects too.

At this point, you should decide whether to upgrade your evaluation version to the registered version by buying a license.

Purchasing a license for the registered copy of JBlitz brings you many benefits:

• Full restriction-free functionality: no virtual user limitations; no restriction to 10 minutes run time. As many test cases as you can handle
• Email support and help: attentive and responsive support for your test environment when you need it. We're friendly, we're helpful, we can usually find a way to fix-up any problems
• No product expiry: free yourself from the hassle of re-downloading and re-installing
• Free upgrades: you'll get free minor release upgrades and, depending on your license, free major ones too!
• Inclusion in our development suggestion program. We're not so large yet that we can't take on board customers suggestions for future releases. Most of our development work is driven by our customers, so your say may well mean something.

You can purchase your restriction-free registered copy of JBlitz online at our Software Store. If you have special requirements, are a member of an educational or not-for-profit organization or have any other enquiries, please email one of our sales team who'll be happy to help you with a customized order.

Whether or not you decide to make a purchase, we wish you the very best of luck with your development and testing programs.

* In contrast to other products, there is no 'per virtual user' charge. You are free to configure up to 250,000 per installation. The ability to perform web testing with large numbers of virtual users will depend highly on the configuration of your test machines, their operating systems, memory and CPU speed amongst other things.