Technical

Using Sahi, Mink and Behat to test HTML5 drag and drop file uploads

For those that don’t know, Behat is an excellent tool for testing the business expectations of an application. In other words, it’s a behavior-driven approach towards full-stack application acceptance testing. Mink is a browser abstraction layer, allowing you to easily control different browser emulators through a common interface. Combining the two together brings us a mean bag of tricks when it comes to testing web applications.

This morning I had set myself the task of writing the tests for a spiffy HTML5 drag and drop file upload script that is all the rage nowadays. Needless to say it took far longer than I had thought it would. Let’s get started.

Testable elements of the HTML5 drag and drop

Drag and drops work by triggering the drop event of an element. This drop event contains a list of files in a format defined by the HTML5 FileAPI. The Javascript can loop over these file objects and perform client-side file validation checks. This data is then posted via AJAX to another URL. After the server-side processing is done, we get a response object with the results, and we parse these to give feedback to the user whether the upload finally succeeded. As you can see, there are various places we can begin to test.

Attempt 1: Just test the AJAX POST

Because the data is finally POSTed via AJAX, one option is to just test that and leave the rest to manual QA. In fact, we can forego AJAX altogther, and use PHP with cURL to make the request and check the response. Easy. Actually, too easy – we’re ignoring what makes our app cool – the drag and drop!

Attempt 2: Test the legacy file input fallback

Bah. This isn’t why you’re reading this post. You know how to do this already. And anyway, you’ve probably already got a legacy test but now you want to test the spiffy HTML5 one. Moving on.

Attempt 3: Use Sahi to run your test

Hello Sahi! Sahi is a web test automation tool with fully fledged GUI. But more relevant is that it supports Javascript, unlike its faster headless relatives (yes, there’s PhantomJS, but I wouldn’t mind seeing what’s going on in a drag-and-drop widget).

Before we even hit Mink and Behat, try recording the events to turn into a Sahi script. You’ll quickly notice that Sahi (unsurprisingly) doesn’t properly record the event of dropping a file onto the page.

The issue here is that Sahi has no concept of files outside the emulated browser window. There’s a sneaky trick around this. In our Behat definition , we’ll run evaluateScript to dynamically add a file input field, then attach our image file to that field. Now we can grab the file object from that!

$session = $this->getSession();
$session->evaluateScript('$("body").after("<input type=\"file\" id=\"sahibox\">")');
$session->getPage()->attachFileToField('sahibox', '/home/dion/image.png');
myfile = $("#sahibox").get(0).files[0];

If we run the Javascript manually, it works fine. And it also creates a good opportunity to stop and peek at exactly what’s your File object built from. However in Sahi, we don’t have the file object. Why? Because input file field values cannot be manipulated by Javascript for security reasons. But then why does Sahi even provide a function for this? Because “Sahi intercepts the request in the proxy, reads the file off the file system and inserts it into the multipart request”. So Sahi just does a sneaky slide into the form submit at the end.

Taking a peek at Sahi’s setFile documentation, they note they have a _setFile2 function – which essentially converts the input field into a text field in the process. This isn’t going to work either, because we actually need the file object to test.

Finally, Sahi provides a third alternative to selecting files to uploads by emulating native events in the process of selecting a file. It’s at the bottom of their setFile documentation. It basically walks through the steps of opening up the file browse dialogue, typing in the file path with keystrokes … on and on until we get what we want. It’ll work!

Yes, it’ll work. But not nicely. It’s slow. It’s littered with _waits(). Wouldn’t it be nicer if we could create the file object ourselves rather than emulate browsing our filesystem?

Attempt 4: Grab a file object from an image already on the server

Aha! We’ve already got images in our app, let’s just try to upload one of those. We’ll need two things: an image source, and a way to create a file.

For an image source, we’ll grab one with an XMLHttpRequest() in Javascript. We need to make sure that this image source is within Sahi’s proxy, though. This is because otherwise we’d run into cross-domain issues. That’s fine, we’ll upload the Sahi logo as our test image.

To create a File, we’ll create a Blob instead. Files inherit from Blobs, and so we can swap them in an out. Right, let’s see.

var xhr = new XMLHttpRequest();
xhr.open( "GET", "http://sahi.example.com/_s_/spr/images/sahi_os_logo1.png", true );
xhr.responseType = "arraybuffer";
xhr.onload = function( e ) {
    var arrayBufferView = new Uint8Array( this.response );
    window.myfile = new Blob( [ arrayBufferView ], { type: "image/png" } );
};
xhr.send();

Great! So window.myfile will be populated with our file object now. But a test that relies on the existence of a Sahi image? Nasty.

Attempt 5: Create our file object from a base64 string

Simple but effective and none of that extra request messing around. Let’s create an image first. I made a black 100px square image for testing. The simpler the image the better, as it’ll make your base64 string smaller. Now let’s turn that image into base64:

$ base64 image.png 
iVBORw0KGgoAAAANSUhEUgAAAGQAAABkCAAAAABVicqIAAAACXBIWXMAAAsTAAALEwEAmpwYAAAA
B3RJTUUH3gIYBAEMHCkuWQAAAB1pVFh0Q29tbWVudAAAAAAAQ3JlYXRlZCB3aXRoIEdJTVBkLmUH
AAAAQElEQVRo3u3NQQ0AAAgEoNN29i9kCh9uUICa3OtIJBKJRCKRSCQSiUQikUgkEolEIpFIJBKJ
RCKRSCQSiUTyPlnSFQER9VCp/AAAAABJRU5ErkJggg==

Great. Now as it turns out, the folks at Mozilla have already worked out how to decode a base64 string into Uint8Array. Steal their functions and we’re good to go :)

So our test script will:

  1. Convert a base64 image into a Uint8Array
  2. Use that Uint8Array to construct a Blob with the mimetype of image/png
  3. Set various metadata that file uploaders need, such as file name and last modified date
  4. Create a new list of files, and put our Blob in there
  5. Create a new “drop” event.
  6. Add our list of files to the dataTransfer attribute of that drop event
  7. Trigger our on-page element with the drop event
  8. Wait for the AJAX call and server-side processing to be done

And here is the full script in action from our Behat definition (with the base64 string snipped out because it’s very long):

$session = $this->getSession();
$session->evaluateScript('myfile = new Blob([base64DecToArr("...snip...")], {type: "image/png"})');
$session->evaluateScript('myfile.name = "myfile.png"');
$session->evaluateScript('myfile.lastModifiedDate = new Date()');
$session->evaluateScript('myfile.webkitRelativePath = ""');
$session->evaluateScript('sahiFileList = Array()');
$session->evaluateScript('sahiFileList.push(myfile)');
$session->evaluateScript('e = jQuery.Event("drop")');
$session->evaluateScript('e.dataTransfer = { files: sahiFileList }');
$session->evaluateScript('$("#dropbox").trigger(e)');
$session->wait(1000);

Great! It’s testable!

Technical

Motion tracking with Javascript, HTML5 and a webcam

Why would you use the web for motion tracking? Simple. HTML5 Canvas is exciting. Javascript is (pretty) cool. Combined with a lazy afternoon, we can create an ultra simple hand motion tracking and colour recognition system.

This isn’t entirely true. It doesn’t track the hand, it tracks a bright blue bottle cap I found on the floor. Even more truthful is to say that it tracks anything bright blue. But enough chat, here’s a demonstration. Just click the small button with the dash in it to get started, grab something blueish and wave it in front of your camera. It won’t be as good as we got it since we adjusted it for specific lighting conditions, but it’s enough as a proof of concept.

We’ve also got pretty pictures to show. Here’s one of the quick n’ dirty strap we used to embed the bottle cap in.

php_hand_motion_tracking

And here is one of it in action.

php_hand_motion_tracking1

You can see the (bad, hackish, thrown together) code for it in my playground repository on GitHub.

Creative

Zygomatic Studios design: an experiment in one-page animated layouts

Recently I did a front-end proposal for Zygomatic Studios. They’re an animation company started up by Erik Kylen and I’ll be maintaining their website.

Given that I knew them, I had some freedom to experiment. For an animation firm, the website itself had to be showy graphically somehow. I ended up making the entire page animated on page-load: to present itself in a showy way but not interrupt the user whilst actually using the page. “Slick” was what I was going for.

Another idea I wanted to play with was the one-page site concept, which displayed the highlights of each “sub page”, which could then be expanded if interested.

You can check it out in my alpha playground.

Zygomatic

Designed with GIMP, with a little help from Blender. Personally quite happy with the experiment.

Creative

The web as an art medium

Almost three months ago, I had a course which got me thinking: what if the web was seen as yet another medium for artists? I’m referring specifically to artists, those who create without purpose and just for kicks to add life’s flavour, rather thanĀ designers, who have an objective or a problem they are trying to solve. This means I’m not talking about webdesign – I’m talking about Web Art.

I’m also not talking about plugins or embeddable content like Flash or 3D – I’m talking about pure HTML DOM and things which manipulate it.

This isn’t a new idea. Ever since Chrome reminded the market about the importance of script execution and rendering optimisation, there have been a lot of experiments out there. It’s probably unsurprising, then, that now is the perfect time for artists to invade and create a layer of class and “meaning” behind this eyecandy.

My feelings about stuck-up artists aside, I made a series of web toys. Two of these were appropriate for public viewing, and so here they are.

Trauma

Named after its inspiration, Trauma is an experimental point and click environment. Nothing special scriptwise.

Vector

Also named after its inspiration, vector is a box that does stuff (click on it). Animation in Blender, exported as a single image of all the frames, and jQuery scrolls through it.

Note that these have not been optimised, cleaned, or browser tested in any way. It works in Firefox, but be sure to give it time to load fully as there is no preloader.

Enjoy! (or not – apparently you can’t tell people what to think with art)