Dion Moult Seriously who ever reads this description.

Use Behat to recover your legacy web app

Your legacy project is going to the dogs. No problem, with this easy 7 line code patch you can now fix your entire codebase within a weekend!

 * @Given /^I will write this test in the future$/
public function iWillWriteThisTestInTheFuture()
    throw new PendingException();

Go you! :)

Architectural visualisation renders with Blender

It’s been a while since I’ve made a post. Although there are posts in the queue, I figured I might post this as it’s a quick one. Let’s see the pictures first.

Visualisation 1

… and the other …

Visualisation 2

Images done with Blender and Cycles. Piano in second render done by RegusTtef. These images are 50% of actual size, together these images took a day and a half. Second image is a panoramic shot.

The building is a proposal for the Sydney Museum of Profligate Steel Welders. The rest writes itself :)

Building REST APIs with auto-discoverable auto-tested code

For the past few months, one of the projects I’ve been working on with SevenStrokes involves building a REST API for a service. REST APIs are tricky things to get right: they’re deceptively simple to describe, yet play host to plenty of interesting topics to delve into. Such topics can be statelessness, resource scope, authentication, hypermedia representation and so on.

However I’m going to only talk about the very basics (which many people overlook), and demonstrate how the Richardson Maturity Model can help with automated testing and documentation. If you haven’t heard of RMM yet, I recommend you stop reading and go through it now (especially if you’ve built a REST-like API before).

Let’s say our REST API conforms to a level 3 RMM: we have a set of standardised verbs, querying logical resources, receiving standardised status codes, and being able to navigate the entire system via links. We’ve got a pretty good setup so far. All these items in the RMM help our REST API system scale better. However what is doesn’t yet help with is keeping our documentation up to date. This is vital, because we know that the holy grail for REST API is an auto-generated, always up-to-date, stylish documentation that promotes your site/product api. There’s a bunch of tools that help you do this right now, but I think they’re all rather half-baked and used as a bolt-on rather than a core part of your application.

To solve this, I’m going to recommend one more addition: every resource must have the OPTIONS verb implemented. When invoked, it will respond with the following:

  1. An Allow header, specifying all the other verbs available on the invoked resource.
  2. A response body, containing the verbs, and under them in the hierarchy of the body (in whatever format), a description of:
    • Their input parameters, including type, and required boolean
    • A list of example requests and responses, detailing what headers, parameters and body are included in the request, and what headers, status code and body is included in the response.
  3. A list of assumptions that are being made for each example scenario (if applicable)
  4. A list of effects on the system for each example scenario (if applicable)
  5. A list of links to any subresources with descriptions

Let’s see a brief example:

# OPTIONS /user/

    "GET": {
        "title": "Get information about your user",
        "parameters": {
            "foobar": {
                "title": "A description of what foobar does",
                "type": "string",
                "required": false
            [ ... snip ... ]
        "examples": [
                "title": "View profile information successfully",
                "request": { "headers": { "Authentication": "{usersignature}" } },
                "response": {
                    "status": 200,
                    "data": {
                        "id": "1",
                        "username": "username1",
                        [ ... snip ... ]
            [ ... snip ... ]
    [ ... snip ... ]
    "_links": {
        "self": {
            "href": "\/makkoto-api\/user"
        [ ... snip ... ]

Sound familiar? That’s right. It’s documentation. Better than that, it’s embedded documentation. Oh, and better still, it’s auto-discoverable documentation. And if that isn’t great enough, it’s documentation identical to the format of requests and responses that API clients will be working with.

Sure, it’s pretty nifty. But that’s not all! Let’s combine this with TDD/BDD. I’ve written a quick test here:

Feature: Discover
    In order to learn how the REST API works
    As an automated, standards-based REST API client
    I can auto-discover and auto-generate tests for the API

    Scenario: Generate all tests
        Given that I have purged all previously generated tests
        Then I can generate all API tests

That’s right. This test crawls the entire REST API resource tree (starting at the top-level resource, of course), invokes OPTIONS for each resource, and generates tests based on the documentation that you’ve written.

Let’s see a quick demo in action.

Auto-documentation for REST APIs in action

It’s a really great workflow: write documentation first, generate tests from it, and then zone in on your tests in detail. This ensure that your code, tests and documentation are always in sync.

I hope someone finds this useful :) For the curious, the testing tool is Behat, and output format used is application/hal+json, using the HAL specification for linking, and link URI templates.

Using Sahi, Mink and Behat to test HTML5 drag and drop file uploads

For those that don’t know, Behat is an excellent tool for testing the business expectations of an application. In other words, it’s a behavior-driven approach towards full-stack application acceptance testing. Mink is a browser abstraction layer, allowing you to easily control different browser emulators through a common interface. Combining the two together brings us a mean bag of tricks when it comes to testing web applications.

This morning I had set myself the task of writing the tests for a spiffy HTML5 drag and drop file upload script that is all the rage nowadays. Needless to say it took far longer than I had thought it would. Let’s get started.

Testable elements of the HTML5 drag and drop

Drag and drops work by triggering the drop event of an element. This drop event contains a list of files in a format defined by the HTML5 FileAPI. The Javascript can loop over these file objects and perform client-side file validation checks. This data is then posted via AJAX to another URL. After the server-side processing is done, we get a response object with the results, and we parse these to give feedback to the user whether the upload finally succeeded. As you can see, there are various places we can begin to test.

Attempt 1: Just test the AJAX POST

Because the data is finally POSTed via AJAX, one option is to just test that and leave the rest to manual QA. In fact, we can forego AJAX altogther, and use PHP with cURL to make the request and check the response. Easy. Actually, too easy – we’re ignoring what makes our app cool – the drag and drop!

Attempt 2: Test the legacy file input fallback

Bah. This isn’t why you’re reading this post. You know how to do this already. And anyway, you’ve probably already got a legacy test but now you want to test the spiffy HTML5 one. Moving on.

Attempt 3: Use Sahi to run your test

Hello Sahi! Sahi is a web test automation tool with fully fledged GUI. But more relevant is that it supports Javascript, unlike its faster headless relatives (yes, there’s PhantomJS, but I wouldn’t mind seeing what’s going on in a drag-and-drop widget).

Before we even hit Mink and Behat, try recording the events to turn into a Sahi script. You’ll quickly notice that Sahi (unsurprisingly) doesn’t properly record the event of dropping a file onto the page.

The issue here is that Sahi has no concept of files outside the emulated browser window. There’s a sneaky trick around this. In our Behat definition , we’ll run evaluateScript to dynamically add a file input field, then attach our image file to that field. Now we can grab the file object from that!

$session = $this->getSession();
$session->evaluateScript('$("body").after("<input type=\"file\" id=\"sahibox\">")');
$session->getPage()->attachFileToField('sahibox', '/home/dion/image.png');
myfile = $("#sahibox").get(0).files[0];

If we run the Javascript manually, it works fine. And it also creates a good opportunity to stop and peek at exactly what’s your File object built from. However in Sahi, we don’t have the file object. Why? Because input file field values cannot be manipulated by Javascript for security reasons. But then why does Sahi even provide a function for this? Because “Sahi intercepts the request in the proxy, reads the file off the file system and inserts it into the multipart request”. So Sahi just does a sneaky slide into the form submit at the end.

Taking a peek at Sahi’s setFile documentation, they note they have a _setFile2 function – which essentially converts the input field into a text field in the process. This isn’t going to work either, because we actually need the file object to test.

Finally, Sahi provides a third alternative to selecting files to uploads by emulating native events in the process of selecting a file. It’s at the bottom of their setFile documentation. It basically walks through the steps of opening up the file browse dialogue, typing in the file path with keystrokes … on and on until we get what we want. It’ll work!

Yes, it’ll work. But not nicely. It’s slow. It’s littered with _waits(). Wouldn’t it be nicer if we could create the file object ourselves rather than emulate browsing our filesystem?

Attempt 4: Grab a file object from an image already on the server

Aha! We’ve already got images in our app, let’s just try to upload one of those. We’ll need two things: an image source, and a way to create a file.

For an image source, we’ll grab one with an XMLHttpRequest() in Javascript. We need to make sure that this image source is within Sahi’s proxy, though. This is because otherwise we’d run into cross-domain issues. That’s fine, we’ll upload the Sahi logo as our test image.

To create a File, we’ll create a Blob instead. Files inherit from Blobs, and so we can swap them in an out. Right, let’s see.

var xhr = new XMLHttpRequest();
xhr.open( "GET", "http://sahi.example.com/_s_/spr/images/sahi_os_logo1.png", true );
xhr.responseType = "arraybuffer";
xhr.onload = function( e ) {
    var arrayBufferView = new Uint8Array( this.response );
    window.myfile = new Blob( [ arrayBufferView ], { type: "image/png" } );

Great! So window.myfile will be populated with our file object now. But a test that relies on the existence of a Sahi image? Nasty.

Attempt 5: Create our file object from a base64 string

Simple but effective and none of that extra request messing around. Let’s create an image first. I made a black 100px square image for testing. The simpler the image the better, as it’ll make your base64 string smaller. Now let’s turn that image into base64:

$ base64 image.png 

Great. Now as it turns out, the folks at Mozilla have already worked out how to decode a base64 string into Uint8Array. Steal their functions and we’re good to go :)

So our test script will:

  1. Convert a base64 image into a Uint8Array
  2. Use that Uint8Array to construct a Blob with the mimetype of image/png
  3. Set various metadata that file uploaders need, such as file name and last modified date
  4. Create a new list of files, and put our Blob in there
  5. Create a new “drop” event.
  6. Add our list of files to the dataTransfer attribute of that drop event
  7. Trigger our on-page element with the drop event
  8. Wait for the AJAX call and server-side processing to be done

And here is the full script in action from our Behat definition (with the base64 string snipped out because it’s very long):

$session = $this->getSession();
$session->evaluateScript('myfile = new Blob([base64DecToArr("...snip...")], {type: "image/png"})');
$session->evaluateScript('myfile.name = "myfile.png"');
$session->evaluateScript('myfile.lastModifiedDate = new Date()');
$session->evaluateScript('myfile.webkitRelativePath = ""');
$session->evaluateScript('sahiFileList = Array()');
$session->evaluateScript('e = jQuery.Event("drop")');
$session->evaluateScript('e.dataTransfer = { files: sahiFileList }');

Great! It’s testable!

A short and simple beginners look at Markdown

At SevenStrokes, we forego email support and go straight to a forum / discussion-based system based off Vanilla. This is great, because we can organise client discussions much better, focus discussions on certain topics, split and merge topics as they spin off from original topics, and through an intuitive interface that takes no time to learn. Best of all, we can escape from those badly formatted client emails with the annoying 10-line signature and get to the point. That’s the reason our discussion post formatting is based off Markdown.

Too bad it’s not obvious enough how to use Markdown.

I wrote this very short, basic, and purposely omitting details guide to What is Markdown? – I hope you like it :)

← Back in Time