Gatling-Tool in SBT or Play : Sample Projects

Content

This post is a direct follow up of this one where I introduced a bit what I did in order to integrate Gatling in SBT and in Play2.
Where this post was more about bits and bytes necessary to accomplish the task, this one will talk about how to use this mess.

Gatling for SBT

Now, that we've a dedicated SBT plugin in hand we can create a sample project that uses it (I already created one here).
In this new project, we'll need to create a file plugins.sbt which will contains the reference to the gatling-sbt-plugin.
Actually, it's the classical way to add a plugin to a SBT project (and the easier and the "semanticest").
We're now prepared to configure our build by using the pieces provided by the plugin.
At the end, we'll be able to write a first test and launch it.

Project Build

First of all, we must create a directory for your project with this basic structure:

Looking at the structure, we can see a Build.scala that will define our project build, a build.properties that defines the SBT version with a single line : sbt.version=0.11.3
And the latest file in the project folder is plugins.sbt which... declares the plugin.
Let's put the Build.scala aside for now, and have a look at the content of the latter. 
Self descriptive isn't it? Yes, we've just told that SBT has to use our gatling plugin... that's all? Not yet.
Actually this line will provide us everything that have been declared in the plugin, such as the Gatling configuration keys, the command, and the basic SBT settings. 
Now, we're gonna us them within our Build.scala. 

Before going in specific details, note that we had to add my own repo (hosted at Cloudbees) in order to fetch the project... but you could also use the URI fetch provided by SBT... 
What are really important to point out is the allSettings declaration and the import of the GatlingPlugin. 
At first, we reuse the default one by using Project.defaultSettings which is being appended the gatling settings, using Gatling.gatlingSettings... that object comes from our plugin! 
This will add all relevant keys for the "gatling-test" Configuration with default values. 
At this stage we've almost finish our build definition... All that remain (and shouldn't... have to figure out any help is welcomed?) is to add two things:

• Declare the test framework to have access to the gatling classes and the custom Simulation :

  "be.nextlab" %% "gatling-sbt-test-framework" % "0.0.1-SNAPSHOT" % "gatling-test"

• Declare the command :

  commands ++= Seq(gatlingTakatak)

After having used everything in your project (don't forget to add the GatlingTest configuration...) you're so close to write your first test.

Gatling Conf

This section will cover the other folder src wherein reside the classical main and test folders.
But there is an other one, gatling-test that is unknown at this stage.
In fact, this folder will be the root for our Gatling tests, holding the configuration file (I provided a basic one in the sample app) and the future simulations that you'll write.

The first is dropped in conf/galing.conf, where the latter is simply simulations.

Write your first test

At this stage, I'll make an assumption that you either know the Gatling api... Nah, the Gatling DSL is sufficiently self descriptive (otherwise check the wiki).

As you can see, it's very very easy (that's why I like Gatling-Tool).
All we had to do, since we're benchmarking google, is to implemnent the apply method of a simple scenario within a class (with must have an empty constructor... a limitation for now) that must extend our custom Simulation (yeah! I know ... I'll choose another name for it that's why GSimulation is needed).
Note the dummy implementations of the interceptors whose aren't needed here.
This Simulation must be located somewhere under the simulations folder.

Now, we can launch it and see where are stored the results.

That's enough boy...
Let it shot the server and generate the result.
 ...
...
Done? ok.

Now, that all bullets have been shot, you can go in this folder gatling-project/target/gatling-test/result.
See? True! Every run will create a specific folder under this one, named run-${scenario-name}-time.
Make some jump in it and locate the index.html file, open it in your browser and see the magic happened. 
NB: check this page for information about the reports. 
So far, so good. But we're writing web apps and we want to stress them... we all know that the google server won't crash (that easy). 
Let's move to Play2 (or you can write your own app w/o Play2 using Spray or another Java framework maybe, then mimic the following).

Gatling for Play2

For this part, I also wrote an application that makes use of the "plugin" introduced in the previous post.
It's quite complicated, not yet polished and might be a good topic for a future post, because I tried to make an application based on Event Sourcing with some workflows and usages of pure functional structure like Lens, Writer, State, Monad and so forth.

But let's stick with the actual topic, if you fork the project and browse a bit its build configuration you'll see that it is near from the one we just talked about. But still that some particularities are presents...

Definition

First of all, since Play2 has its own convention for sources folders (app, test under the root) we'll redefine those folders for gatling as well.
In order to do that, we must adapt the following configuration keys:

Actually, yeah, we've just skipped the "src" folder. Anyway. 
So far, so good... 
I heard you... 
But, do you known that we're already done? 
Allez, let's write a test.

Write

If you started from scratch (without forking the sample app) you can now stress test your index page that will render the default welcome page defined in Play2.
See the below Gist for that, or its follower for a more complicated one using the app.


Look what I've done Dad! 
In the Simulations, all we had to do is to simply declare an available port on which the framework can start the server, and its needed FakeApplication to have access to the routing. 
Then when we had to build a request path, we've just had to do the same as in our templates, that is, use the routes object under the controller package. Amazing. 
Furthermore, we can also reuse our type checked Form to build our feeder. Waouw. 
But, at this stage, there still some boilerplate (parameters). Looser.

Launch

And now, it's time to stress test our server and see how it'd behave on production. (Ok, that's not 100% true, since Gatling is running on the same machine...)

For that, just do the following:

Wait. Again.

Now, go to you result directory (the path hasn't changed). Open in your browser and let the shine comes in.

Wrap Up

At this stage, with small and few injuries we can use Gatling-Tool without having to install or trick stuffs in order to have information about our web app.
This by using either a full SBT application or a Play2 based one.
I know that there still a lot to do, but at least the basic features are there, and I'll enjoy anyone forking it, scrapping it if necessary and let me know.

Last words

I hope you liked these posts, otherwise I'd like to apologize because I did all projects and the blogs on the train, half asleep every morning and evening, back and forth from my actual mission's workplace.

Now I'm gonna sleep in this train...
Oh no, I can't...
I've to write my book now.

Gatling made easy for SBT or/and Play2

Forewords

These last few weeks, I took some time to understand a level deeper how SBT works and what it can provide. Since this post is not related to this learning trip (which was along existing blogs and wikis), I'll jump directly to the idea this new understanding gave me.
A while ago I started (and paused) a work on Gatling-Tool to have it integrated with Play 2.0 (see these posts here and here), this work has been refactored to better integrate with SBT.

What we'll find in this post

In this post, I'm gonna give you all tools in order to stress test you Web based application, either built upon Play 2.0, either only using SBT. By directly starting writing your tests rather than having to configure stuffs or to start others... So this post is composed of two parts:

• How I built these tools (can be skipped -- definitively)
• How to use these tools (it's probably the useful part)

Gatling shot 3 aside projects... for goods

This part is related to what was necessary to have gatling-tool easily integrated with SBT, and after with Play2.

Gatling SBT Test Framework

This project (on github) has a sole and simple goal, to implement the Test Interface (see) used by SBT to integrate new Test Framework (like was done for ScalaTest, Specs, JUnit, and...). This project contains only 2 classes and one convenient trait.

GatlingBootstrap

This class is one of the helper that avoid boilerplates in the future, because it starts the GatlingConfiguration stuff needed by Gatling-Tool to execute. This class requires two things "the path to the gatling configuration file" and "the path to the folder where will be stored the stress results".

Simulation

This trait is another tool in hand that extends the com.excilys.ebi.gatling.core.Predef.Simulation provided by Gatling in order to add it two interceptors whose are pre and post. Those interceptors will be really useful in the future, in order to start/stop a server for instance (e.g. FakeServer in Play2).

GatlingTestFramework

This class is the real processor of the project. Actually, it's also the implementation of the interfaces declared by the SBT test framework which are Framework, Runner2 and some Fingerprints. Basically, what is done there are those tasks:

• Create the gatling bootstrap based on sys properties
• Declare fingerprint to discover Stress Test based on the parent class being be.nextlab.gatling.sbt.plugin.Simulation
• Create the stress tests instances (only classes are handled for now) by reflection
• Call the pre interceptor
• Execute the stress test using the gatling api
• Call the post interceptor
• Generate the reports using the gatlin api

Gatling SBT Plugin

This project (here on github) aims to provide the basics to use the test framework easily within an SBT project. 
Its intent is to have testers able to write stress tests directly, just after having imported the module in SBT. 


It basically provides an SBT Configuration (gatling-test) that extends the SBT's Test one and one Command (takatak). The other goodies are settings which are common for all such stress tested projects using gatling. 
Those settings are essentially the conf file path, the result directories, the libraries (gatling ones principally) and so on. The last important thing that it does is to add the gatling test framework to the already existing list of test frameworks supported out of the box by SBT.

Gatling Play2 Plugin

This misnamed project (no more a Play2 plugin, neither a SBT one... but it's ok for now) brings only one little thing. It comes with a base implementation the custom Simulation declared in the Gatling Test Framework.

This base implementation is Play2 specific while creating a fake server and starting it in the pre interceptor, stoping it in the post one.

This will ease further tests in the Play2 environment.

Save Point

So far so good, if you read this you can now navigate the small projects that were necessary to do what we'll discuss in the next blog...

This one is already too long...

Gatling and Play2.0: continued

This blog entry is a follow-up of this entry where I introduced a spike I did on Play 2.0 stress tested using Gatling-tool.

At the time writing the above entry, I had to quickly hack gatling to use Akka 2.0 as Play 2.0 uses it, and I didn't wanted to have clashes.

But, thanks to Stéphane Landelle, Gatling is now Akka 2.0 enabled (since 1.1.2).

So that, it was time to give the plugin's embryo a refresh. For that I used another project that aims testing the neo4j rest plugin for Play2.0 I'm also writing. In case of, I've also introduced what I did in this project in this post.

Content

Using the Gatling I first tested how I've been able to stress test:

• simple (get) urls
• mutating (post) urls using server form underground to compile excepted data.
• duration tests (stress testing on a given period basis)

What I liked so much, even in this embryos+, is the ease to create stress tests when coupled with Play2.0 functionalities and Gatling's DSL.

Foreword

The Gatling plugin I'm currently building is located on github here and is based on sbt to build it.

But it's based on the version 1.1.4-SNAPSHOT version of Gatling's libraries (due to some fixes the Stéphane did "for me", while he was at devoxx fr, isn't he gentle !!!). 

At the time writing, you'll have to build gatling and gatling-highcharts locally using maven (quick fast!).

How to

Set up

Having created a Play 2.0 app, you now have to powerful of sbt in hands (especially if you've installed the TypeSafe stack 2.0). So, to stress test your app, you'll have first to build to plugin I told above.

Plugin

First of all, clone/fork this library project on github plugin, after what, you'll just have to run sbt publish-local in the related folder. That's it, you now have the plugin in your local repo.

Project

In your Play 2.0 app, you now are able to define the plugin as `test` dependency, using the following

That's it...

Use

Specs

Personally, for testing I use org.specs2.Specification, for using it with the plugin (at the time writing) you'll have to create the following:

This listing is creating a fake server to enable urls to be tested, and some functions to deal with it.

To create the server, the plugin defines a Util object (be.nextlab.play.gatling.Util), which also defines rough helpers to be used in stress tests.

A full spec should look the following:

Simple Url

You saw, in the provided gist above, that Util also defines a way to simply defines a gatling Simulation (basically a Function0 that returns a scenario builder: Gatling DSL result).

Having that in hand, here is the fragment to stress test the root url:

As you can see, it's pretty simple, but nothing can really be checked in the body of the specs (I'm working on having relevant information to check).

But, at least you can run this test to hit 10 times the root url ramping the numbers of users by 2.

Running it (sbt test) you'll have a new gatling folder in your target folder that contains a  results directory where are located all stress results in an html report (with great charts)

And all you had to do is to define the request headers and the url...

Mutating the server

If you have controllers that mutates the server, you should have define POST urls, which are using the Form feature provided by Play 2.0.

Having did so, you'll be able to stress it very easily using Map, JsObject or in the best case your Model.

Let's say we have a controller controllers.Stuffs that uses case classes models.Stuff. The controller defines a stuffForm and a createStuff action.

Your stress test can now be defined like the following:

In the gist, you can see 5 points to note, they are key-clues to create reusable stress tests.

Nothing is really hardcoded, neither the path to the http end point nor the parameters data.

That's http stress testing using type checked requests. CoOl isn't it? Hey, man, we got back our lovin' type cheking (one of the best scala feature).

Heavy check, duration based

This part is more a Gatling feature highlighting.

This last example is an heavy test that uses looping over a configuration for a given period. This gives you how many users could use your application.

Such test might be shaped the following:

Note

For now, it's NOT an official plugin neither a gatling nor a play 2.0 one, but discussions are on the way for that... stay tuned on twitter or here.

Gatling-Tool Plugin for Play 2.0

Goal

This post will be a kind of write-up of what I'm trying to do now. And for what I already achieved some tasks.

Since I'll soon start products in my company that will be based on technologies like Scala, Play 2.0, Neo4J, MongoDB, Heroku and misc, and that I'm a bit control freak; I wanted to be sure that what I'll build will mach the requirements in terms of capacity.

This is where Gatling-Tool comes in the equation, this is a very powerful (and scala based) stress testing tool, which the name recall.

We're about to have some words on it, but first let me tell you that a GitHub ready to be fork is available with my first step into a gatling plugin, find it here.

Gatling Tool

Gatling Tool is a cute, smart and intuitive stress testing tool for web application, which a neat DSL for http request and asserts.

The DSL written in scala, and following the good conventions for it will aim anybody to be able to write stress tools. In a way that even non-programmer peoples with a basic understanding will be able to do basic stress tests with a good learning curve.

It's integration with browsers (Firefox, through a proxy) ease much more the work because you'll be able to register like macros (or like badboy does) your scenario to be repeated again and again. This is what is called the recorder.

Scenarios could be written with a custom external DSL, but they're also available as regular scala code (internal DSL) and there is where the coupling with Play 2.0 scala should pay.

Integration with Play2.0

Akka 2.0

Fact: since the RC-5, Play 2.0 comes with the Akka 2.0 support.

Fact: Gatling being it-self based on Akka (and they're right for that) but on a previous version for the stable version (logical because Akka 2.0 is pretty new).

So an integration must go through an update of the Gatling-Tool to the same version of Akka 2.0 in order to be able to use them correctly using the same project (testing phase, but still).

That's why I decided to fork the Gatling-Tool on GitHub (aaah the great world of open source), in order to switch the support of Akka from their 1.x to 2.0.

Even if it is true that I did it roughly (at first), it remains that it works and is the two needed projects for the following are:

• https://github.com/andypetrella/gatling
• https://github.com/andypetrella/gatling-highcharts

So fork, clone them and build them locally using maven using the classical mvn clean install.

You'll have a brand new version of gatling 1.1.0-SNAPSHOT.

NOTE: I had some difficult choices when doing the migration, some are breaking the runtime behavior (a bit) and I'll have to discuss them further with the Gatling team. I've already been contacted by Stéphane Landelle who told me that we was interested by the work since it was planned the 1.2 release.

So don't be afraid, the official release will match the needs soon. (But ping me if you want more info and help me.)

Typesafe Stack 2.0

I recommend you to install this brand new stack that integrates all stuffs that you'll need for scala development, including Play 2.0 project. 

Now, simply follows this link for further steps, and then create a play-scala project.

Sbt

Since Play 2.0 is using Sbt for building its project, and the custom gatling library we built is in our .m2 repo, we have first to add our local maven repo to the repositories list this way (updating your Build.scala):

Tada, now we have a Play 2.0 project having our custom gatling as dependency.

Plugin

Before going in further details with integrating Gatling as a Play 2.0 plugin, I'm gonna talk about an uncovered subject in Play 2.0 (or not easy to track); the Plugin feature.

Play 2.0 comes with a pretty easy simple Plugin integration, this through the specific file, in the conf folder, named play.plugins. This file is meant to contain one single line by defined plugin, shaped this way: {priority}:{plugin's class path}.

But what is a plugin finally, this is a classical class extending play.api.Plugin... simply. This Plugin trait only defines three methods which are:

• onStart: this adds a hook when the application starts, helpful for initializing objects.
• onStop: cleaning the fields.
• enabled: helpful to disable the plugin in some specific cases.

Another point, that I have to highlight is that it seems that such Plugin's constructor must have an argument being the application it-self.

Test only

Hey wait gatling should be available in tests only ?! Right!

The first way to achieve this is the easies also, while implementing isEnabled, you can use the application (remember it is part of the constructor) which has a method isTest that should toggle the plugin.

The second way is to create a Specification that starts a FakeServer (since we'll stress the entire Play 2.0 flows) and give it a FakeApplication which is defined with the Plugins you wish. 1000 words replaced by one Gist:

Gatling integration

Now you're wondering what the heck is that Gatling Plugin class, don't you?

Gatling Plugin

The Gatling plugin class, located in my repo under the test/gatling folder, is extending the play.Plugin class (which defines dummy implementation of the three methods), this way I can concentrate on the only method I need, onStart. 

Actually I need some initialization in order to use gatling, it needs some folder to be defined, including the one interesting use the more: results.

So, the Gatling#onStart method is creating ephemeres folders under the target directory (that can be cleaned) and also the needed Gatling configuration file. And that's it.

We can now stress test our app.

How?

Gatling is able to understand Scala written scenarios, those Scala script have only one constraint: being an instance of com.excilys.ebi.gatling.core.scenario.configuration.Simulation.

This trait is actually a Function0 and thus defines only one helpful method which is apply(). The latter method is the container for building the stress test that we want to execute.

What is very common now is that you can write Gatling scenarios using the Scala Type System being checked in your favorite IDE, and they will be compiled by Sbt it self when requested (and hot swapped ^^). Where the classical Gatling workflow is to compile them on the fly, using their internal routines.

Run 'em

Having simulations written (example here), you can now ask Gatling to run them by creating a gatling runner instance on them. I won't go into deep details because it is not the purpose here, but here is how you can do.

See 'em in action

That's the easiest part, entering the play environment using sbt in console, you can launch the tests by typing test.

What'll be done is:

1. enter the specification
2. create the fake application
3. load the additional plugin
4. create the gatling folder and conf
5. configure the gatling system
6. create the fake server on 3333
7. create the simulations
8. run them
9. generate reports on them (located in the target folder => look 'em in your browser and you'll see how they're cute)

Problems

There are problems for now when executing several tests, because streams are closed (while generating further reports), that comes from a choice that I've have to do and which is commented on github here. This is mainly related to a feature that is no more available in Akka 2.0 (for good reasons, I'd say).

To be continued

If you want to help me going further, don't hesitate to contact me on my mailbox, or comment this post or on twitter.

What I like to have in the future is :

• clear Specification for Gatling (preventing the need to define each time the server and plugin)
• website for enabling test one by one, or any, or...
• redirecting to results reports
• more
• and more