Wednesday, 10 November 2010

BunnyLandMark - a new Flash game benchmark



I learned a lot about Flash rendering with my BunnyMark benchmark, but I felt it was missing a few vital features that would make it a true test of game-specific performance. So I've come up with a brand new test: BunnyLandMark! Once again I created a version using display list Bitmaps, and a version using copyPixels blitting. Here are the new features:
  • a large scrolling world - about 4000x4000 pixels. I chose this size as it is roughly the biggest bitmap you can make in Flash player 10. With a bit more work it is possible to make infinitely large worlds, but this size was big enough for my purposes.
  • Perspective depth-sorting. Bunnies that are further into the scene are sorted to appear behind those that are closer to the viewer.
  • More bunnies - as off-screen bunnies don't need to be drawn, the total number of bunnies can be increased.
And here are the results (INSTRUCTIONS: use ARROW KEYS or WASD to scroll!)
  • The blitting demo can handle 90,000 total bunnies at 30fps, with about 1,900 on-screen bunnies at a time. Blitting was ridiculously fast! That's definitely more bunnies than you require! Depth sorting using blitting is definitely more efficient. Don't forget to scroll with the arrow keys.
  • The display list demo can handle 22,000 total bunnies at 30fps, with about 450 on-screen bunnies at a time. So, in this case blitting is about 4 times faster than the display list, versus only about twice as fast on the previous test.
So here's my conclusion: depth sorting on the display list is pretty slow. In fact, before I came up with some optimisations, it was much, much slower and I couldn't even handle 2,000 total bunnies at 30fps. In my first iteration I used setChildIndex to order the display objects, ignoring those that are offscreen. But when you think about thousands of bunnies slopping about in the display list, you can guess that this was incredibly slow. I don't know how the display list works behind the scenes, but it doesn't like you changing depths. The trick I came up with was to loop through and removeChild every bunny, and addChild back only the ones I need on screen. This meant my displayList was much shorter and so all my display list operations were much faster. I still think 450 on-screen, depth-sorted entities at a time is going to be plenty for most games, but I can definitely imagine a few scenarios where it wouldn't be enough.

Ok, over to you now readers! Please test and report any differences in frame rate you get. I'm particularly interested to see if Mac Safari performs as woefully on this display demo as it did on the last. I have updated the source zip to contain all the bunny demos.

15 comments:

Hugo Larcher said...

Yea your numbers seem about right on Safari, really nice benchmark, thanks.

Jon Howard said...

10 fps on display list version
The blit one flashes up some funniest then jumps to a play button. Bug?

Platform. HTC Desire

Jon Howard said...

'Funniest' should read 'bunnies'

Troy said...

Great benchmark. I'd love to see HTML5/Canvas achieve the numbers in the blitting example! ;-)

My own experience in Mockingbird showed the same thing: when it comes to depth-sorted bitmaps, you can't beat blitting.

dafkobyte said...

Here are my results:

--Environment--
Win 7 Pro (64-bit) - Intel Xeon 3.2 GHz (2 processors), 4GB RAM
Flash Player 10.1.85.3

--Browsers--
Google Chrome 7.0.517.44
Blitting - 28 FPS & 260 MB
Display List - 22 FPS & 200 MB

Firefox 3.6.12
Blitting - 25 FPS & 142 MB
Display List - 20 FPS & 83 MB

Safari 5.0 (7533.16)
Blitting - 27 FPS & 142 MB
Display List - 21 FPS & 83 MB

IE 9 (beta) 9.0.7930.16406
Blitting - 30 FPS & 141 MB
Display List - 30 FPS & 82 MB

MC said...

What if you rotate and scale the bunnies...? how blitting would perform then?

MC said...

My PC: Windows XP, 1000Gb RAM, 2,79 Ghz CPU, Browser: Firefox 3.6.12
Blitting: 18 FPS, 160 MB
Display List: 16 FPS, 100 MB

Iain said...

In terms of scaling/rotating etc, to do it with blitting you would have to pre-cache all the frames of rotation, which would then mean it renders as fast as if there were no transformation - however that's not really the point is it - the point is that with the display list you can add those effects with just 1 line of code.

Bruno Garcia said...

I believe after Molehill's release we'll have a third way of pushing bunnies even faster than software blitting!

borming said...

platform : MacOSX MBP 13

blit : 30 - 140
display : 30 - 80

Owen said...

Mac OS X 10.5.8 2.93Ghz Intel Core Duo 2

Both Chrome and Safari run at 30fps. Both browsers show an initial low frame rate (~20) for blitting, which climbs to 30 fps after a second or 2.
Memory blitting/display Chrome: ~161MB/101MB, Safari ~141MB/82MB

How about adding a slider/input box so we can set the number of bunnies?

Andreas Renberg said...

--- Environment ---
Linux Ubuntu 10.4 LTS (32 bit)
3.9 GB Memory
Two Mobile Intel processors @ 1.20GHz

--- FireFox 3.6.12 ---
FP 10,1,102,64 (non-debug)
blit: 21FPS, 13.4MB
display: 29FPS, 10.2MB

--- Chrome 8.0.552.28 beta ---
FP 10,1,103,19 (non-debug)
blit: 21FPS, 21.2MB
display: 29FPS, 18.0MB

--- Standalone player ---
FP 10,0,45,2 (debug)
blit: 16FPS, 16.5MB
display: 19FPS, 18.1MB


--- Notes ---

NOTE Only on the "display version": After I start moving the camera around, the FPS dips down to 20FPS. Even when I have stopped, the framerate stays at around 23FPS.


Why am I getting RADICALLY smaller values for memory use than everyone else?

Note that the memory use does increase slowly over time, so should I leave it open a while and see how far it gets?

Orion said...

This is great stuff Iain, thanks!

wonderwhy-er said...

Made a post exploring some related stuff today linking to benchmarks of yours, seems blogger does not have pingback/trackback things so posting like that http://blog.wonderwhy-er.com/adobe-flash-rendering-compromises/
Hope you will find it interesting and input is welcome :)

bas said...

Wonder how this will run with using the new Molehill features :-)