Is Gamebryo Good Middleware?

Kyle Wilson writes nicely thought out articles on game development. He just wrote a piece, Defining Good Middleware, on Gamasutra (also on his site, Game Architect).

I’ve been asked how Gamebryo stacks up to his definition of good middleware. So, I’ll discuss that, plus throw in some additional thoughts. Read his article first.

How does Gamebryo stack up?

Good middleware lets you hook your own memory allocator.

Certainly. We ship an optimized pooled allocator for small objects, and a memory debugging/tracking/analysis system as well. Snapping on your own allocator is easy.

Getting this implemented in a code base is much easier to do early on! Major Kudos go to Shawn Kime who was the muscle man implementing this feature onto a game engine that had already shipped across a console generation and contained nearly 10,000 files.

Good middleware lets you hook your own I/O functions.
Yes, you can register your own functions for loading engine content. There are a few edge cases, e.g. on some platforms we use platform specific APIs sometimes to load certain file types.

Good middleware has extensible functionality.
We design for customer extensions. But, this is a significant task! Every feature must be considered for extensibility, and the right amount of flexibility selected. Some extensibility comes with a performance or memory cost. Most flexibility adds complexity to code.

Solid C++ design can be used in many areas, allowing customers to register custom derived objects with a factory system. The cost is polymorphism, and adding conditionals to code to handle unforeseen cases customers may introduce. Other areas can use template programming with policies, functors, visitors, etc.

Gamebryo 2.5 shipped with a massive overhaul of the geometry system, allowing customers to arbitrarily specify data formats, semantics, interleaving, and sharing. This offers significant benefits, but with a cost. Several fast paths must exist for code based on different formats of data. Some modules restrict the data layout to a known subset. Template programming is used to offer "perfect inner loops" at compile time, but increase the complexity of the code. And the exporting pipeline complexity is increased as client data format requests are normalized against known constraints of the runtime.

Extensibility can not be taken lightly, it is a significant design concern for every feature added to middleware.

Good middleware avoids symbol conflicts.
Gamebryo code has used the "Ni" prefix for a decade... legacy from the product name, "NetImmerse". Once you pick a method to encapsulate your symbols, you’ll find it time consuming to refactor, and a burden to customers to change as well. Big new features are coming for Gamebryo, though, and we are moving to using name spaces for those.

Good middleware is explicit about its thread safety.
Gamebryo has focused on concurrency for some time. Background loading is one of the most basic forms. We also demonstrate how to multithread other areas of the codebase, e.g. concurrent scene graph updates. And, we ship an extremely powerful stream processing solution, Floodgate, built to make high-performance cross-platform multithreading easy for our customers.

Good middleware fits into your data pipeline.
Generic and platform optimized assets are definitely supported. A data pipeline for a game is a complicated system however, and there’s more to it than just intermediate files and packaged level files. There is the need to format data precisely as needed for your game, and we offer rich configuration for data formatting. There’s also the need to manipulate, optimize, and augment data. For that, we offer a configurable plug-in based processing pipeline.

Good middleware is stable.
Gamebryo has a mature code base tended by a large team of engineers for a decade. It’s been battle tested in 200+ games. We ship only a couple of times per year, with a rigorous end cycle of automated and manual testing. And yet, it’s a massive code base, evolving to offer better performance and features, and exposing significant customer extensibility. There are still issues that come back to us, but that is why we have a support team.

The reality of the market is that customers prefer advanced features and performance over high stability. You want it all, but at the end of the day you have to choose where to put your time and effort. What does it take to have a 100% stable codebase? How much effort goes into flight control software stability? How much testing is the right amount for a game engine when there are opportunity costs?

Good middleware gives you source.
Absolutely. If you’re building a AAA game on a middleware engine there is no question, this is essential. That said, casual or low budget games can do amazing things with a binary only version.

This statement gathered quite a few comments on the original posts, a few quotes here:

"... you could equally argue that good middleware should be free (of course not), or that game developers should give the source of their game to game players so that they may fix bugs (of course not).
...
By providing source to their software, middleware manufacturers essentially give up their IP."
- Anonymous-1-Gamasutra

"... there are cases where it makes sense for a middleware company to release the source code. However, in those cases, the value of the middleware company is not in the source code."
- Anonymous-2-Gamasutra

As a consumer, I’d require source. Large games are too complicated and developed in too short a time to not be able to get into source. Additionally, final optimizations of games need to be able to optimize globally, across all source code.

As a middleware developer, giving away source and being exposed to thieves is difficult. People steal the source directly, indirectly, in whole, in part, in many ways. But, they do often face legal repercussions, and the rest is a cost of doing business and selling what people need. And, as pointed out in the comments, the real value of middleware is being able to use it, and that comes from the relationship with the company that supports it. Customers of Gamebryo get an excellent companion staff of engineers working with them to make their games succeed.

All that said, many have done great things with binary only copies of Gamebryo, for example check out this Coldwood Tech Demo.


There are other questions to ask about any piece of middleware: How much memory does it use? How much CPU time does it require? What’s the upgrade path for your current code and data? How does it interact with your other middleware? How good is the vendor’s support? How much does it cost?
Memory and CPU? Mostly that has to do with assets, and how customers configure them.

Upgrade path? Re-export your models, animations, particle systems from Max/Maya/XSI using the Gamebryo exporter. Some clean up will be required, and proprietary formats will take custom code to convert.

Interacting with other middleware is important! We specifically design for it in many sub-systems. E.g. Floodgate and how it shares resources on Playstation3. We also have a team of engineers assisting with and maintaining integrations with partner middleware companies.

Our support is widely acclaimed best in the business. I didn’t believe that statement until I heard it from current and previous customers over and over, studios that have used 2 or 3 of the major engines. I’ve never heard of a studio that used Gamebryo and anyone else who didn’t believe Gamebryo’s support was superior.

Gamebryo’s cost... ;) well... what are you developing? We have a portfolio of options. Talk to sales.

Motivation for Middleware

Kyle makes two strong points:

1. Middleware provides you with more code than you could write yourself for a fraction of what it would cost you to try. No matter how clever you are, that’s how the economics of the situation work. ...
   
   
2. Middleware offers structure. Middleware draws a line between the things that you have to worry about and the things you don’t. ... As games grow ever-larger and more complex it’s become incredibly valuable to be able to draw a line and say, The stuff on the other side of that line isn’t my responsibility, and I don’t have to worry about it.
   

I’m not going to rant (in this post) about why middleware makes sense. But, I am going to invite developers to do what they already do, better.

After shipping several console games, it became clear to me that the industry needed higher quality engine code, and that Middleware made sense. We’re hiring at Emergent, ramping up for a new wave of awesomeness, a new line of console hardware, and a better way to make games. If you’ve shipped games, drop me an email. Even if you just what to chat about what it is like to work on Middleware.

(Chrome engine picture used by flickr user lars hammar, Coldwood tech screen shot by Coldwood/Emergent)