BunnyLandMark - a new Flash game benchmark

I learned a lot about Flash rendering with my BunnyMark benchmark, but I felt it was missing a few vital features that would make it a true test of game-specific performance. So I've come up with a brand new test: BunnyLandMark! Once again I created a version using display list Bitmaps, and a version using copyPixels blitting. Here are the new features:

• a large scrolling world - about 4000x4000 pixels. I chose this size as it is roughly the biggest bitmap you can make in Flash player 10. With a bit more work it is possible to make infinitely large worlds, but this size was big enough for my purposes.
• Perspective depth-sorting. Bunnies that are further into the scene are sorted to appear behind those that are closer to the viewer.
• More bunnies - as off-screen bunnies don't need to be drawn, the total number of bunnies can be increased.

And here are the results (INSTRUCTIONS: use ARROW KEYS or WASD to scroll!)

• The blitting demo can handle 90,000 total bunnies at 30fps, with about 1,900 on-screen bunnies at a time. Blitting was ridiculously fast! That's definitely more bunnies than you require! Depth sorting using blitting is definitely more efficient. Don't forget to scroll with the arrow keys.
• The display list demo can handle 22,000 total bunnies at 30fps, with about 450 on-screen bunnies at a time. So, in this case blitting is about 4 times faster than the display list, versus only about twice as fast on the previous test.

So here's my conclusion: depth sorting on the display list is pretty slow. In fact, before I came up with some optimisations, it was much, much slower and I couldn't even handle 2,000 total bunnies at 30fps. In my first iteration I used setChildIndex to order the display objects, ignoring those that are offscreen. But when you think about thousands of bunnies slopping about in the display list, you can guess that this was incredibly slow. I don't know how the display list works behind the scenes, but it doesn't like you changing depths. The trick I came up with was to loop through and removeChild every bunny, and addChild back only the ones I need on screen. This meant my displayList was much shorter and so all my display list operations were much faster. I still think 450 on-screen, depth-sorted entities at a time is going to be plenty for most games, but I can definitely imagine a few scenarios where it wouldn't be enough.

Ok, over to you now readers! Please test and report any differences in frame rate you get. I'm particularly interested to see if Mac Safari performs as woefully on this display demo as it did on the last. I have updated the source zip to contain all the bunny demos.

Display list vs. blitting - the results!

To get some actual evidence for my opinions on the joys of the Flash display list, I created two demos that I'm calling "BunnyMark", a test of rendering small bouncing bunny pngs with alpha transparency. Since first posting, lots of readers helped by testing on different browsers and operating systems, and I have updated this post with their results.

The results were quite interesting, and not quite what I expected. Blitting was really fast, although actually a little bit slower than I expected, but gave a consistent rendering speed across all platforms. Bitmaps were also pretty fast, although in Safari on Mac performed really badly. I emailed Tinic from the Flash player team about this issue, and he has said he will look into it. Ok so here are the results:

• The display list demo could render 4000 bunnies at 30 fps on my PC without slowing down. This was replicated by readers on all Mac and Windows browsers except for Mac Safari, where it was down to 10-20 fps. Based on this interesting blog post from Tinic Uro (suggested by Richard Leggett), this seems like it may be something to do with the recent adoption of the Core Animation APIs in Safari. The demos has a lot more layers stacked up than you would need for most games, so this performance drop is unlikely to affect a real game - although I will be following up with a new benchmark to test that hypothesis. Bitmaps faired very badly on Android - it couldn't even render 10 bunnies at 30fps.
• The blitting demo could render 6000 bunnies at 30 fps without slowing down on my PC, and people with faster machines have reported up to 11000 bunnies at 30 fps. Blitting was also much more effective on Android, where it got up to 600 bunnies at 30 fps, certainly enough performance for an arcade-style game. (Thanks to Philippe Elsass for the Android tests).

So in this example, blitting is about twice as fast. But as I hope you can see, realistically 3000 bunnies is still a lot more than you are going to need in most situations. You can download the source code and see if you can improve the performance of either demo. A couple of readers have recommended performance optimisations, for example suggesting I use a fixed-length vector and using lock() and unlock() on my bitmapData, but neither strategy noticeably improved performance on my machine.

I also wondered whether switching the wmode in the HTML can fix the Safari issues - it doesn't. If you want to try them: Opaque, Transparent, Direct, GPU (both Direct and GPU give 5 fps in Chrome on Windows!). This post from way back in 2008 may possibly shed some light on this topic:

"Just because the Flash Player is using the video card for rendering does not mean it will be faster. In the majority of cases your content will become slower." - Tinic Uro

Just a final note - I ran a similar test to this 2 years ago in Microsoft XNA and was able to get something like 50,000 bunnies going at HD resolution, and 60 fps. I think molehill is going to make this discussion somewhat irrelevant next year - GPU blitting will annihilate both of these approaches. The question will then be, can the display list also be speeded up by the GPU, or is it just too wacky and different to what graphics cards are designed to handle?