Friday, 4 February 2011

Molehill + Scaleform = hardware accelerated 2D in Flash?

This post is pretty much an open letter to Adobe, so if you know anyone who works there, please send it to them. I've had what I think is quite an interesting idea, and I think it's one that Adobe should have a long think about. The Flash player is about to add hardware accelerated 3D rendering with "molehill". The demos that have been released so far look great - amazing 3D scenes rendered with no impact on the CPU. All the rendering is handled by the GPU, leaving the CPU free to process your game logic, including physics, collision detection and AI - which allows for more advanced gameplay (and stops Flash-haters fixating on how much of their precious CPU the "bloated" Flash plug-in is consuming).

However, this will only apply to 3D games. In fact 2D games - which we must not forget are what has made Flash the number 1 gaming platform on the web - will see no benefit from this release. What's the message from Adobe here? That 3D is the future of gaming - 2D is not important, or at least 2D games already have good enough performance. But when you consider that native 2D iPhone and Android games have similar (if not better) performance than 2D Flash games on much inferior hardware, you can see there is plainly massive room for improvement. Now look at the iPhone and Android appstores, where there is a genuine choice between 2D and 3D games. The charts are dominated by 2D games like Angry Birds, showing that 2D is still the dominant style for casual games.

Ok, so what can we as game developers do about it? Well, we could make all our 2D games using bitmaps rendered on 3D planes, and so leverage the GPU rendering of "molehill". Now our CPU is free for game logic. But what have we lost? Well for starters the display list with it's powerful nesting features, filters, textfields, use of the Flash pro IDE for layout. In short everything that makes Flash what it is.

So what can Adobe do about it? Well, there already exists a technology called Scaleform that renders the Flash vector engine using GPU accelerated 3D by tessellating 2D graphics - effectively turning them into flat meshes of 3D polygons that the GPU is designed to render fast. This technology is used to make 2D user interfaces for console and PC games. Now, I'm no expert in scaleform or graphics hardware, but it seems to me that this technology or something very like it could feasibly be integrated into the Flash player, meaning whole Flash scenes and timelines could be rendered largely on the GPU. This would keep the Flash pro IDE as a relevant product, and take Flash beyond being the most popular platform for 2D games, to being the best platform for 2D games.

In 2008, along with many other developers, I asked Adobe for joypad support in Flash. At MAX last year, Adobe finally announced their plan to add this feature. I'm just as certain that I'm right that Flash needs hardware accelerated 2D.

This video shows how much power the GPU brings to Flash - wouldn't it be awesome to have that much power in our 2D games? Comments / reasons why I'm barking up the wrong tree are greatly appreciated! :)

24 comments:

JaaQ said...

I agree. I was talking about this only a few days ago with some fellow developers. Flash's display list is a very useful way of interacting with the stage, but it's also pretty heavy on the CPU and I've reached this ceiling quite a few times while creating games. Sure there's blitting techniques etc, but as GPU's grow obscenely stronger and stronger, I think it's imperative that Adobe makes this work!

Mark Burvill said...

Interesting idea. I'd never heard of Scaleform before so can't comment on that, but completely agree that some kind of support for enabling 2D display list sprites to take advantage of Molehill is a must!

bas said...

If I understood correctly from the AdobeTV video demo, the Molehill is just a low level API where third parties can build their frameworks upon. I guess it will be a matter of time before the first 2D frameworks will be popping up.

So while this doesn't actually have to be a task for Adobe, I do agree it would be nice if it came from them. If only for showing their commitment to 2D.

Richard Leggett said...

You could conceivably port mobile 2D OpenGL ES game engines like Cocos2D or Rokon, there's not an enormous amount of code in the latter just to get things up and running.

The differences will then be down to the exact implementation of the "surface view" the context on which you draw, I haven't looked into Molehill but I presume they'll have a special kind of visual class for this kind of direct drawing.

Then you'll need a way to perform the drawing on a separate thread ideally, again I imagine this has been dealt with in the Molehill APIs, and you'd create a few callback hooks to allow your game engine to run the logic/update/input loop ready for your next graphics loop.

matt said...

good call, couldn't agree more. molehill is great news for games but the time and effort involved in the production of a decent 3d project make it more relevant for larger studios and bigger budget projects. this kind of thing would be a huge benefit for everyone.

Anonymous said...

thibault mentioned this in November on bytearray, so just up to someone to write the engine...

Thibault Imbert wrote::

Hi Alex and Zed-K,

Concerning 2D, you will be able to actually use Molehill for 2D too in the context of sprite engines. You can totally render 2D objects using triangles too on the GPU and get crazy fast rendering for 2D too in this case.

So, I guess a lot of people will come up with 2D engines built on top of Molehill too ;)

Thibault
Tuesday, November 9, 2010 at 12:24 am #

Tahir Ahmed said...

This can seriously be the best thing happened to Flash.

Matan Uberstein said...

Agreed! Thanks for bringing this 'our' attention, I certainly was blinded by all the new 3D features that I forgot about the 2D scene. Adobe should implement this, for sure!

Gary Paluk said...

2D is the same as 3D only on a single plane, imagine a piece of paper, it can still exist in our 3D world. It shouldn't be too long until an initiative is under way to recreate the usual 2D API (Which is actually already 3D) within Molehill itself or even better frameworks, although I'm guessing people who are very familiar with Flash would appreciate a near duplicated API.

Lawrie said...

Totally agree - I was excited to see the molehill demos, but definitely see more uses for super-fast 2d in my work.
Whilst a 3rd party solution would be great - I'd love to see Adobe tie it into Molehill officially.

Tyler Wright said...

The added performance of our UI's and other 2D work would be invaluable. CacheAsBitmapMatrix, a mobile-only AIR feature is a start in the right direction.

If Adobe is reading, +1 from me! Think of the benefit to Flex... :)

AdobeFlash said...

Adobe IS reading ;) and yes, we have been thinking about this.

as3isolib.v2 said...

Iain I am glad you are saying what I have been thinking since starting on the as3isolib.v2. I have sent a few emails to Adobe to see about getting access with no response.

But seeing as you, others and I all want this, and the fact that Adobe is reading your post, I REALLY hope this comes along soon.

This is music to my ears. Keep banging them drums!!!

Anonymous said...

For years talking to Adobe is like talking to a silent wall. Flash is no longer a platform thats worth putting all your developer eggs in.

Im very wary of molehill, if the api and shader language are so different from directx/opengl/glsl then adobe will be no better than apple with obj-c.

The flash community got hit hard in 2010, its very much dead compared to what it used to be. Hopefully molehill will be a successful defibrillator.

Anonymous said...

Great idea... I'm with you...
I hope ADOBE is really listening...

I would also suggest you to post this idea to ideas.adobe.com

Cheers!
Flash ON!

mail said...
This comment has been removed by the author.
Mandeep said...

Hardware accelerated 2D in Flash would be great. Things like netbooks with decent integrated graphics, like the Nvidia Ion or AMD Brazos chips, would really benefit. I imagine tablets running desktop OS's would get great use out of this too.

Brendan said...

Adobe's Flash Platform Evangelist Mark Doherty posted a great video from Mobile World Congress (MWC) about Scaleform's new GPU accelerated Flash engine on iPhone, iPad, and Android smartphones and tablets.

Direct link:
http://www.flashmobileblog.com/2011/02/15/scaleform

Update: Scaleform was recently acquired by Autodesk
http://www.autodesk.com/scaleform

devu said...

People, if there is a GPU 3D here, there is 2D as well. Is a matter of time some 2D or ISO frameworks will show up. Nothing to do for Adobe here. Is about Flash community to make it happens now. Flash Player is ready.

Iain said...

@devu - while I think you're right, I see it as a big shame that developers who use the Flash IDE and traditional Flash way of working with MovieClips etc will miss out on the hardware accelerated goodness.

Mem's said...

http://www.hugoandcat.com/bunnymark/index.html

Like devu said, the player is ready. Adobe give us a kind of low level access.
So it's our role (The Flash community) to make some libraries (create or adapte) to make this oportunity a benefit.

hobbymerge said...

Apparently flash player 11 is said to have full GPU support based on OpenGL.The exact launch date is not exactly known.

Tony said...

Now, I’m no expert in scaleform or graphics hardware, but it seems to me that this technology or something very like it could feasibly be integrated into the Flash player, meaning whole Flash scenes and timelines could be rendered largely on the GPU.

Adobe Software said...

Above, you can see a 3D game using the Unreal engine and looking great. What’s interesting is that the control system, which floats in the air in 3d is actually created in Flash Professional.