Installing and Configuring Behat with Mink and Sahi Headlessly

by Han

Acceptance testing is a valuable tool which should be alongside unit and functional testing in your toolbox. By testing the behavior of the application instead of just the logic in the code, you can find out problems with UI before your users do. It’s a much more concrete, “real world” analysis about the correctness of your application. Even better, you can get the full spectrum of stakeholders involved since it’s much less technical.

Being able to engage the whole team in the process was a big reason we chose Behat as our acceptance testing framework. The use of Gherkin, a human readable domain specific language for writing the tests, is very approachable even for non-technical users. This allows our product owner to translate requirements and acceptance criteria directly into tests, helping to keep everyone on the same page. The Behat project on GitHub is constantly updated as well, so we don’t have to worry about legacy code not working with the latest version of PHP. Lastly, since our specialty is symfony, the fact that Behat is also written in PHP using Symfony2 components just adds icing to the cake.

Having just made some improvements in the deployment of our Behat acceptance testing framework, we’d like to spread the love by sharing the technical details for those of you who are interested in setting it up but finding documentation sparse online.

We assume you’ve read up on what Behat is and how to use it – this blog post just goes over the installation process since it’s not quite as clear cut as the documentation states, and we’ll go over how we tweaked the setup to run tests via Firefox headlessly.

Because we’re testing a web application, we are using Mink as the browser abstraction API and Sahi as the browser driver to control Firefox. However, to get the maximum performance, we’re running the tests on a Linux server which lacks a physical display, so we needed to configure Xvfb, an X Windows virtual framebuffer, to provide a virtual screen.

This setup lets us run automated acceptance tests on a beefy server class box and still take advantage of a real browser like Firefox instead of doing the tests in Goutte, the text based browser simulator that lacks JavaScript support, which makes testing our Ajax heavy application difficult. However, we only enable Firefox via the @javascript annotation when it’s needed – if the feature we’re testing doesn’t require JavaScript, then Goutte is definitely the better option since it’s more resource efficient and faster.

Installation

Behat

We chose to install behat through PEAR. However, the default command pear install behat/behat yields version 1.0.0 while the latest is 2.1.2, so we specified the version number directly.

$ pear channel-discover pear.symfony.com

$ pear channel-discover pear.behat.org

$ pear install behat/behat-2.1.2

Mink

The same process was used to install Mink via PEAR. Again, we had to specify the Mink version number, whose latest at the time of writing was 1.1.1.

$ pear channel-discover pear.symfony.com

$ pear channel-discover pear.behat.org

$ pear install behat/mink-1.1.1

Sahi

Sahi’s installation was a bit more tricky. It’s packaged as a .jar install, which won’t run in a non-graphical environment. Luckily, there’s a .zip file on Sourceforge, so we grabbed the latest version. Lo and behold, after unpacking there was a shell script at ‘bin/sahi.sh’ which starts the proxy. We added the unzipped folder to a user directory, but a world readable directory like /usr/bin works well too.

Xvfb

Xvfb should be a package on Linux; a simple sudo yum install Xvfb or sudo apt-get install Xvfb should do the trick.

sfBehatPlugin

Of course, if you’re not using Symfony you should skip this step, but since we are, we installed the Symfony plugin as a submodule via git. In the .gitmodules file in the app root, add the following lines at the end:

[submodule "plugins/sfBehatPlugin"]

path = plugins/sfBehatPlugin

url = git://github.com/Behat/sfBehatPlugin.git

Then run git submodule init followed by git submodule update to grab the latest version and install it in your symfony project.

Configuration

Behat’s configuration file can be auto-generated by the sfBehatPlugin by running:

$ ./symfony behat:setup

This will create the features/ folder and a behat.yml config stub file.

If you’re not using the plugin, you can do it manually by issuing:

$ behat --init

Open up the behat.yml file to manually specify the base_url of the web application you’re testing.

If you’re starting the sahi proxy as a user process instead of as a service, it should be done via nohup, which is the ‘no hangup’ command on Linux. It allows you to continue running a process even after the user who issued the command has logged off.

$ nohup sh /path/to/sahi/bin/sahi.sh

Same thing with Xvfb:

$ nohup Xvfb :1 -ac &

Lastly, any user running the behat tests needs to specify the display by setting the DISPLAY environment variable:

$ export DISPLAY=:1

You should put this in the .bashrc or .login scripts so that it automatically loads on user login instead of remembering to issue it every time.

Lastly, you’ll want to add the Xvfb command as well as the sahi shell script to start on machine boot, preferably as a service, otherwise if your box restarts and you forget to start either sahi or Xvfb you’ll get errors.

Run firefox on the command line to see if Xvfb is working correctly. Although you won’t be able to view the display on the command line, there’s a neat trick to capture a screenshot so you can debug if anything does go wrong.

import -window root screenshot.png

The above command will take a screenshot of the virtual X window environment and save it in the file screenshot.png, which you can then remotely download and view. It’ll be grainy but it’s super useful!

Running Tests

Finally, everything should be all set up now! Whew, pat yourself on the back for getting this far. If luck is on your side, you should be able to issue a quick behat path/to/features and get a success message. From here on out, adding scenarios should be a breeze. Remember to prepend your scenarios with the @javascript annotation if you want it to run on Firefox via Sahi, otherwise it will run in Goutte by default.

Hope that helps, best of luck in your installation!

Posted in Development and tagged . Bookmark the permalink.


2 Responses to Installing and Configuring Behat with Mink and Sahi Headlessly

  1. cordoval says:

    wow nice. This means now it can be integrated further. I always liked sahi better than the others. The spec can get very coupled to a particular driver sometimes, and they will fail with other drivers, with sahi I have had less problems than with any other.

    Is the real thing. Thanks for the tips! we thought about this for long and now you did it thanks!

  2. Joe McGuire says:

    Great article thanks for sharing. A couple of small amends
    Including ‘&’ at the of the nohup command to launch sahi to become:
    $ nohup sh /path/to/sahi/bin/sahi.sh &

    Might also be worth mentioning the ‘import’ command to get screenshots is provided by ImageMagick which can be installed with:
    $ yum install ImageMagick

    Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Join Our Team in Shanghai

Now hiring PHP Developer in Shanghai