Friday, 14 May 2010

Penumbra: Overture goes Open Source!

It has been in our minds for quite some time and now finally it has become reality: Penumbra:Overture and the HPL1 Engine are now open source! In case that is all you need to know, then head straight to:
http://GitHhub.com/FrictionalGames
to get your hands on it!

For more information just keep on reading.


Penumbra: Overture
First of all I would like to stress the fact that this open source release does not make Penumbra: Overture free in anyway. All assets (except a few that are part of the engine) are still under the same copyright as before. The thing that is free is the source code for the executable which is now released under the GPL version 3 licence.

The code for Penumbra: Overture is a continuation of the one used for the tech demo + some addition for the not so long lived Robo Hatch project. It also contains some code from Unbirth, giving it quite some history. This history means that the code is far from clean and as expected quite hackish in places. That said, it should have a few interesting bits, the major probably being the physicsal interaction system. This system is not the latest version in the Penumbra series and misses something like rotation. These features should be farily easily to add though.

It is also important to note that Penumbra: Oveture source will not run Black Plague or Requiem. AI for the infected, GUI elements, etc are all missing, but all needed to implement them is present in the engine code (in case anybody is up for the challenge).


HPL1 Engine
For my part, the biggest part of this release is the engine itself. This is engine that has powered all of the Penumbra games and it even includes the stuff used to create the 2D platformer Energetic. The engine code was started in December 2004 and was actively developed until early 2008. After that only smaller fixes where made to it.

The transition from 2D to 3D and the fact that it was my first stab at a full 3D engine, makes the code quite patchy (and downright horrible) in places. This is especially true for some old and low level parts like the sound and input handlers which have evolved anything but gracefully. Later parts are often cleaner and nicer, but the code is not without its share of quick-and-dirty-hacks.

When it comes to interesting features, I think the following are the most prominent:

  • Physics sound system. This is all the code that is used to play and tweak the sounds heard during physics interactions (bumping, sliding, etc). A lot of work has gone into the system and is the result of a combined effort between me and Jens (who does all sounds) for several years.
  • Stencil shadow volumes. The shadow system in the engine is quite robust and can work on pretty much any mesh, something that shadow volumes usually don't. It is far from state of the art these days (when shadow maps rules), but should provide nice info for the curious.
  • Serialize Class system. This is code used to easily save and load classes to disk. It is very useful when creating a save system for games.
The engine contains tons of more stuff (almost 100k lines of code), but these are the most interesting stuff I could think of right now. I am sure there is more for awaiting to those brave enough to explore its dark depths!

Finally, it is also worth noting that this is the latest and final version of the HPL1 engine. It is the beast that powers all games of the Penumbra series.


OALWrapper
This library is made almost entirely by our tool programmer Luis, who actually started out doing work for us on the OpenAL sound implementation. This library is simple and easy way of accessing the OpenAL functionality and was made since there is a strange lack of free sound libraries. This is also the only part of the open source release that we will continue to update and we hope that others will find it useful and contribute themselves!


Questions and Support
As we are a small company and already swamped with work, do not expect us to do full time support on this. We will try and help as much as possible, but we also hope that a community will form helping each other out.

For more discussion on the source code please go here:
http://frictionalgames.com/forum/forum-27.html

This is the place to go for more technical information and please ask any lengthy / technical question there instead this blog.


Credits
I must also stress that I am not very experience managing an Open Source project and most work in making it happen is due to Edward Rudd, who is also responsible for the Linux and Mac ports of Penumbra. So big thanks to him!


End notes
We are extremely interested in seeing what people will be able to do with this! Open source releases of other games have spawned very exciting stuff and we hope this release will do that too. This means we are very interested in what people are doing with it. So whether you plan to do a full blown mod, or just check the code for learning purposes be sure to tell us about it!


48 comments:

  1. This is awesome! I love seeing projects like these go open source. Perhaps this will spawn some First Person Shooters with interesting twists thanks to the physics system, or maybe some nice puzzle games.

    By the way, the code in the PenumbraOverture repository, I guess it's mostly there for reference? Will you get a working game if you use the game data from a purchased copy of Penumbra: Overture?

    ReplyDelete
  2. Well done, first sticking your neck out on the humble indie bundle with the other devlopers and now releasing the source !

    /bow

    ReplyDelete
  3. that's good news guys, you've done a great job

    ReplyDelete
  4. Really impressed that you took the decision to go open source.

    One thing I'd like to say is that I would've bought the Penumbra Collection on offer if I had access to all the different platform executables and not just one. Perhaps something to think about.

    Really thanks so much for participating in the Humble Indie Bundle, I wouldn't have discovered your games otherwise. And furthermore, well done for going open source with Penumbra Overture.

    ReplyDelete
  5. The only thing that actually bugs me about this release is the choice of revision control. Git isn't the greatest choice if you want to encourage developers from all platforms (being designed especially for Unix, it doesn't have the greatest performance on Windows, nor can it integrate very nicely with Windows IDEs...).

    Mercurial or Bazaar would have been better choices. Oh well, I guess its too late now...

    ReplyDelete
  6. @Pharaoh Atem: I disagree. Git *is* the best choice of revision control.

    If you're on Windows, grab TortoiseGit.
    http://code.google.com/p/tortoisegit/

    It's all GUI and integrated for you. Now quit your bitchin' ;-)

    ReplyDelete
  7. > Git *is* the best choice of revision control.

    No it is not.

    ReplyDelete
  8. Yeah Windows Git isn't that bad anymore.. I agree that it used to be. But I myself have used msysgit and tortoisegit in a windows VM to track code changes for some .NET C# SQL Stored Procedures. And I know there is at least one IDE integration piece for Visual Studio. Though I don't really use the integrated IDE version control stuff anymore I just break out and use either gitg or GitX (depending on if I'm in Linux or Mac OS X).

    ReplyDelete
  9. This is really great, and I hope some community projects come from this like for the Quake-Based engines that John Carmack released.

    ReplyDelete
  10. Thank you very much for this. As a software engineer myself I find this very interesting. I can learn a lot from your code. Thanks!

    ReplyDelete
  11. Hurray !

    lets see if we can fix some penumbra glitches with ati cards, working on intel cards and even port it on android !! :)

    ReplyDelete
  12. Thanks so much for this, it really makes me proud to of bought the Humble Indie Bundle. I really hope that this Engine gets put to great use!

    ReplyDelete
  13. This is a great way for developers to go! Thank you so much and like the commenter above, I am proud to have bought the Humble Indie Bundle.

    ReplyDelete
  14. Git on windows was a pain 3 years ago; now I'm using it quite happily :)

    ReplyDelete
  15. Say what you want about git, it is far less annoying and far easier to use than gnu rcs.

    ReplyDelete
  16. Any plan to open-source the older games?

    ReplyDelete
  17. I agree with Manny. It would be cool to see this ported to the Android. I'm not sure how possible it would be, though, since your only real input is the touch screen.

    Thanks for open sourcing this, too. I'll have to take a look at the code.

    ReplyDelete
  18. Thanks, your engine is amazing!

    ReplyDelete
  19. leighman:
    Fiend: Already got entire source out at one point, but I assume it is gone now. Could put up if I can find on my harddrive. Awful code though :)
    Energetic: Not sure if I got the code anywhere.
    In any case, the code for these projects are pretty damn bad :P So not sure they would be helpful.

    ReplyDelete
  20. To everyone at Frictional Games: GOOD JOB! WELL DONE! Thank you for opening up your game and engine like this!

    ReplyDelete
  21. T___H___A___N____K____S!!!

    ReplyDelete
  22. P.S. I knew the name of the engine before - but I newer gave it much thought: could it be that the name "HPL" has something to do with a certain famous American horror author?

    ReplyDelete
  23. I'm curious as to whether I'd get a working copy of the game by compling the source found in the git repo?

    Or does the code in the repo exclude all the game data?

    Also, thanks for putting this in the humble bundle. I'm glad to have bought it.

    ReplyDelete
  24. I am happy to see that you hit you monetary mark as well as the fact that the source code is now open source, now its up to the community to do something interesting with it, thank you for sharing with the open source community.

    ReplyDelete
  25. Thanks, this is awesome! :) Can you please provide a document describing how to build Penumbra: Overture, combining the open-source code with the resources from a purchased game? This would be really good. I guess it's not too difficult, and I hope this is something you intend to be possible.

    ReplyDelete
  26. Varun:
    You still need the full game to get the data.

    ReplyDelete
  27. sswam:
    Just build executable and place in redist folder. Simple as that! :)

    ReplyDelete
  28. These are great news. Too bad, I'm quite busy right now...!. : )

    ReplyDelete
  29. Well, it's "Open Source" but it's definitely not Free Software, please don't associate GNU with Open Source. GNU represents Free Software as in User Freedom, GNU does not represent Open Source. Linus Torvalds and friends represent Open Source. The non-profit "Free Software Foundation", the non-profit "GNU Project", and the author of the GPL Richard Stallman, represent GNU and Free Software, please either take down the GNU image or change your terminology and fully liberate Penumbra, otherwise what you are doing isn't very helpful for our community.

    You can fully release all software source code, and still make money.

    ReplyDelete
  30. InaTux:
    I would argue that this release means still means user freedom,etc. Users can make patches and so on. Also, the engine is very much free with some parts even under zlib.

    We aim to please however and have now changed to a new pic! :)

    ReplyDelete
  31. Thomas:
    I am simply interested in why the developers didn't release the entire game. It can be released and still not run unless the CD is present, or it requires a code in the game retail box (this is tricky but can be done with a small part of the game being a binary blob that cannot be changed.) I think if any useful part of the game is still proprietary then it doesn't do much good.

    Believe me it does some good -- like if this release allows mods to be easily made -- but it doesn't do enough to brag about it. And I might be unclear on the amount released, and what could be released in the future.

    Also, the new image implies you think GNU is evil, which is contradictory. I think people will be confused. Wouldn't it be better to use the GNU head and just say "Free/Open Source" or "FOSS", or the easiest way would to just use Tux.

    Though, Tux is the mascot for the Linux Kernel, not the GNU operating system, which is what Penumbra is available on, that's semantics.

    Also, it would be cool to see the id Software approach applied to Penumbra, i.e. releasing source for each game a few years after it was put on the market.

    ReplyDelete
  32. InaTux:
    Wrote a long reply, and blogger managed to screw it all up. So I am afraid you will get a short reply:

    - We are releasing everything needed to build Overture. Nothing is excluded.

    - We are doing the same as Id. Actually even better! OALWrapper is under Zlib and is activly updated by us!

    - The Gnu is a monster-gnu and more fitting for a horror blog like this one :)

    ReplyDelete
  33. Loving the new pic =D

    ReplyDelete
  34. InaTux:
    So all is clear. Perhaps this sentence had you confused:
    "AI for the infected, GUI elements, etc are all missing [...]"

    Note that nothing is missing from overture, but it simply means that the Black Plague and Requiem executables contains stuff not present in Overture exe. If this source release works out well, then perhaps we will release the source for BP and Requiem too!

    ReplyDelete
  35. Thomas:
    Than that's great! I really don't know why companies don't just release their older work. It doesn't benefit competitors, as the newer stuff is the cooler stuff and is what people want.

    My opinion on the matter is of two things:

    It is okay to make money on software, actually absolutely essential for good quality software. Games are toys not tools, so they are less important, thus we need the source less. But I am one of those people who don't use software that I can't verify is safe, although I have played Penumbra in the past :) And I will now likely buy the collection once I examine the source code.

    By the by, porting games to GNU/Linux is all the buzz right now, apparently the developers of Osmos had/having problems porting their game. http://linuxlock.blogspot.com/2010/05/why-games-dont-get-ported-to-linuxa.html

    Now would be a great time for the developers to express their opinions on porting the game. It's a great way to get publicity for this, I will even give it publicity one way or another.

    ReplyDelete
  36. InaTux, let me explain a few things:
    "I really don't know why companies don't just release their older work. It doesn't benefit competitors, as the newer stuff is the cooler stuff and is what people want."

    It's not true, because (1) there are always people willing to buy an older game, and (2) the new stuff may be cooler on the surface, but this is just makeup - under the hood, the programming principles and techniques don't change at the same pace. Not even close. For example, the BSP tree algorithm that some games use today, was used in the days of the original DOOM.

    So, when someone releases the SDK, one can only say thanks, especially if the code is documented.
    The SDK is not useful only for the people creating mods, but also to anyone learning game programming.


    That's why Thomas was careful, and kind enough to warn us:

    "This history means that the code is far from clean and as expected quite hackish in places.


    But, take a note:

    "That said, it should have a few interesting bits, the major probably being the physical interaction system. This system is not the latest version in the Penumbra series and misses something like rotation. These features should be farily easily to add though."

    The Penumbra's physics system is quite the competitive feature, one of the more important features of the game, and one that many modern games don't do so well.
    So it was quite brave to release this to public domain, even if it's incomplete.

    ReplyDelete
  37. This discussion seems pointless to me. Correct me if I'm wrong: the whole HPL1 engine was released, it is now free and open-source and it's in the hands of the community. However, HPL1 is not Penumbra; Penumbra is built on top of HPL1. So Penumbra is not free or open-source; HPL1 is.

    Anyway, I say thanks even though I'll probably not use it; but I'm sure lots of people out there who are thinking about becoming game developers will have a great and fun time digging through it. And that'll probably mean great new games in the future, so you did a nice favor for all of us.

    ReplyDelete
  38. Messias:
    The executable part of Penumbra: Oveture is open source too :) It is "just" the assets that are not.

    Some people have started poking at, and already interesting stuff is popping. Like adding wii-mote controls:
    http://www.youtube.com/watch?v=d4heV1s8M0E

    ReplyDelete
  39. Anonymous (19 May 2010 22:46):
    Some people develop one thing and continue to use it in newer products, that's true. If a company authors a game engine, and 5 years later that company is still making games based on the same -- and not much improved -- engine, releasing source code would be a problem for them. But this is simply not the case for most companies.

    Example: Compare id Tech 2 to id Tech 4, there are improvements that competitors can gain from, that's why id Software hasn't released it. Once id Tech 5 is out, it would probably be okay to release it (which Carmack said that he plans to petition Zenimax to release the id Tech 4 source upon the release of Rage).

    id Software releasing these engines doesn't appear to be hurting id Software, it only seems to be helping them with their image of a good gaming company.

    Thomas:
    To clarify, most of the Free Software community and I appreciate the liberation of Penumbra, even if some of it remains proprietary (most of us using GNU/Linux are using the proprietary kernel drivers after all).

    And the Wii-mote controls are awesome.

    I am curious though...

    Is Penumbra now as FOSS as -- let's say -- Urban Terror?
    (which I believe also has proprietary assets)

    And what do you exactly mean by "assets"?
    (I believe in Urban Terror the proprietary assets are some textures, scripts, and some utilities for controlling the id Tech 3 engine as well as their modifications to the engine.)

    ReplyDelete
  40. Thomas:
    Afraid I have never heard of Urban Terror, so I cannot comment on it.

    Assets: All that is not in exe except for some engine specific resources. If you check the licenses it should be clear.

    ReplyDelete
  41. Wow, this is amazing news, what a great thing to do for your fans and the community, Thank you

    ReplyDelete
  42. Everyone should stop comparing any part of the game industry to id Software and John Carmack. Of course He can afford to release all of His source code a few years later... He is fucking John Carmack. He has been the sole creator of most computer graphics breakthroughs since the early 90s. He is not your average game developer. And His hobby is rocket science... seriously.

    I think most developers do not release their code for the following reasons:
    1) It is embarrassing. No programmer is proud of every line of code they write in a game. There are always a few bits of elegant, mindblowing, engenious code, but most of it is not so. Games are filled with dirty hacks and workarounds to just get things to work right and to meet milestones.
    2) It removes the value and magic of the game. You would no enjoy slimjims if you saw how they were made.
    3) Fuck You. It's my code. I worked damn hard to get here and I'm not giving you shit. Fuck You. Fuck You.

    Thanks for releasing this!!!! You put all the non-open developers to shame by bearing your soul like this.

    ReplyDelete
  43. I'm waiting for release of all the game as free (libre) software, not only engine parts

    ReplyDelete
  44. Hi guys!
    Can you explain me how the dragging things with mouse works in your games?
    I use AddForce. And I have the problem that the thing rotates around the cursor.

    ReplyDelete

Note: only a member of this blog may post a comment.